devcloud-go/redis

Introduction

Currently, the Redis supports three modes.single-read-write,local-read-single-write and double-write

single-read-write

local-read-single-write

double-write

Quickstart：

use yaml configuartion file

package main

import (
    "context"
    "time"

    "github.com/huaweicloud/devcloud-go/redis"
)

func main()  {
    ctx := context.Background()
    client := redis.NewDevsporeClientWithYaml("./config_with_password.yaml")
    client.Set(ctx, "test_key", "test_val", time.Hour)
    client.Get(ctx, "test_key")
}

use code(recommend)

package main

import (
    "context"
    "time"

    goredis "github.com/go-redis/redis/v8"
    "github.com/huaweicloud/devcloud-go/redis"
    "github.com/huaweicloud/devcloud-go/redis/config"
)
func main() {
    servers := map[string]*config.ServerConfiguration{
        "server1": {
            Hosts:      "127.0.0.0:6379",
            Password:   "XXXX",
            Type:       config.ServerTypeNormal,
            Cloud:      "huawei cloud",
            Region:     "beijing",
            Azs:        "az0",
        },
    }
    configuration := &config.Configuration{
        RedisConfig: &config.RedisConfiguration{
            Servers: servers,
        },
        RouteAlgorithm: "single-read-write",
        Active:         "server1",  
    }

    client := redis.NewDevsporeClient(configuration)
    ctx := context.Background()
    client.Set(ctx, "test_key", "test_val", time.Hour)
    client.Get(ctx, "test_key")
}

Yaml configuration file

props:
  version: v1
  appId: xxx
  monitorId: sdk_test
  cloud: huaweicloud
  region: cn-north-4
  azs: az1 
etcd: # Optional
  address: 127.0.0.1:2379,127.0.0.2:2379,127.0.0.3:2379
  apiVersion: v3
  username: XXXX
  password: XXXX
  httpsEnable: false
redis:
  redisGroupName: xxx-redis-group
  username: xxx # for redis 6.0
  password: yyy
  nearest: dc1
  connectionPool:
    enable: true
  servers:
    dc1:
      hosts: 127.0.0.1:6379
      password: password
      type: normal  # cluster, master-slave, normal
      cloud: huaweicloud  # cloud
      region: cn-north-4  # region id
      azs: az1  # azs
      pool: # Optional
        maxTotal: 100
        maxIdle: 8
        minIdle: 0
        maxWaitMillis: 10000
        timeBetweenEvictionRunsMillis: 1000
    dc2:
      hosts: 127.0.0.1:6380
      password: password
      type: master-slave  # cluster, master-slave, normal
      cloud: huaweicloud  # cloud
      region: cn-north-4  # region id
      azs: az1  # azs
      pool: # Optional
        maxTotal: 100
        maxIdle: 8
        minIdle: 0
        maxWaitMillis: 10000
        timeBetweenEvictionRunsMillis: 1000
routeAlgorithm: single-read-write  # local-read-single-write, single-read-write, double-write
active: dc1

Double-write

Redis also supports double-write modes, including memory double-write and file double-write, depending on asyncRemotePool.persist. true: file double-write; false: memory double-write

redis:
  redisGroupName: xxx-redis-group
  username: xxx # for redis 6.0
  password: yyy
  nearest: dc1
  asyncRemoteWrite:
    retryTimes: 4
  connectionPool:
    enable: true
  asyncRemotePool:
    persist: true
    threadCoreSize: 10
    taskQueueSize: 5
    persistDir: dataDir/
  servers:
    dc1:
      hosts: 127.0.0.1:6379
      password:
      type: normal  # cluster, master-slave, normal
      cloud: huaweicloud  # cloud
      region: cn-north-4  # region id
      azs: az1  # azs
      pool: # Optional
        maxTotal: 100
        maxIdle: 8
        minIdle: 0
        maxWaitMillis: 10000
        timeBetweenEvictionRunsMillis: 1000
    dc2:
      hosts: 127.0.0.1:6380
      password:
      type: normal  # cluster, master-slave, normal
      cloud: huaweicloud  # cloud
      region: cn-north-4  # region id
      azs: az1  # azs
      pool: # Optional
        maxTotal: 100
        maxIdle: 8
        minIdle: 0
        maxWaitMillis: 10000
        timeBetweenEvictionRunsMillis: 1000
routeAlgorithm: double-write  # local-read-single-write, single-read-write, double-write
active: dc2

Fault injection

Redis also supports the creation of services with fault injection. The configuration is similar to that of MySQL.

func DCRedis(etcdAddrs, redisAddrs []string) *redisconfig.Configuration {
    servers := make(map[string]*redisconfig.ServerConfiguration)
    for i, addr := range redisAddrs {
        stri := strconv.Itoa(i + 1)
        servers["ds"+stri] = &redisconfig.ServerConfiguration{
            Hosts:    addr,
            Password: "XXXX",
            Type:     redisconfig.ServerTypeNormal,
            Cloud:    "huawei cloud",
            Region:   "beijing",
            Azs:      "az1",
        }
    }
    configuration := &redisconfig.Configuration{
        RedisConfig: &redisconfig.RedisConfiguration{
            Servers: servers,
        },
        RouteAlgorithm: "single-read-write",
        Active:         "ds1",
        Chaos: &mas.InjectionProperties{
            Active:     true,
            Duration:   50,
            Interval:   100,
            Percentage: 100,
            DelayInjection: &mas.DelayInjection{
                Active:     true,
                Percentage: 100,
                TimeMs:     1000,
                JitterMs:   500,
            },
            ErrorInjection: &mas.ErrorInjection{
                Active:     true,
                Percentage: 30,
            },
        },
    }
    return configuration
}

Alternatively, add the following configuration to the configuration file:

chaos:
  active: true
  duration: 50
  interval: 100
  percentage: 100
  delayInjection:
    active: true
    percentage: 100
    timeMs: 1000
    jitterMs: 500
  errorInjection:
    active: true
    percentage: 20

Testing

package commands_test needs redis 6.2.0+, so if your redis is redis 5.0+, you need to execute

ginkgo -skip="redis6"

See more usages of ginkgo in https://github.com/onsi/ginkgo

Description of Configuration Parameters

Configuration

Parameter Name	Parameter Type	Value range	Description
props	PropertiesConfiguration	For details,see the description of the data structure of PropertiesConfiguration	Mas monitoring configuration,which is used together with etcd
etcd	EtcdConfiguration	For details,see the description of the data structure of EtcdConfiguration	Etcd configuration.If it is configured, it will be pulled from the remote end
redis	RedisConfiguration	For details,see the description of the data structure of RedisConfiguration	RedisServer configuration
routeAlgorithm	string	single-read-write,local-read-single-write,double-write	Routing algorithm
active	string	The value can only be dc1 or dc2	Activated Redis
chaos	InjectionProperties	For details,see the description of the data structure of InjectionProperties	Fault Injection Configuration

PropertiesConfiguration

Parameter Name	Parameter Type	Value range	Description
version	string	-	Project version number
appId	string	-	Project name
monitorId	string	-	Monitoring group name
cloud	string	-	Project deployment cloud group
region	string	-	Project deployment region
azs	string	-	Project deployment AZ

EtcdConfiguration

Parameter Name	Parameter Type	Value range	Description
address	string	-	Etcd address
apiVersion	string	-	Etcd interface Version
username	string	-	Etcd username
password	string	-	Etcd password
httpEnable	bool	-	Specifies whether HTTPS is enabled for Etcd

RedisConfiguration

Parameter Name	Parameter Type	Value range	Description
nearest	string	The value can only be dc1 or dc2	Indicates the local Redis
asyncRemoteWrite.retryTimes	int	-	Number of retries of asynchronous remote write operations
connectionPool.enable	bool	true/false	Indicates whether to enable the connection pool
asyncRemotePool	AsyncRemotePoolConfiguration	For details,see the description of the data structure of AsyncRemotePoolConfiguration	Configure the asynchronous write thread pool
servers	map[string]ServerConfiguration	The key is dc1/dc2.for details about a single dimension,see the description of the data structure of ServerConfiguration	RedisServer connection configuration of dc1 and dc2

AsyncRemotePoolConfiguration

Parameter Name	Parameter Type	Value range	Description
threadCoreSize	int	-	Basic size of the thread pool
persist	bool	true/false	Indicates whether the command is persistent.No:The command is fast.Yes:The speed is lower than that of non-persistent
taskQueueSize	int	-	Number of buffer queues
persistDir	string	Default root directory "/"	Redis persistent file directory

ServerConfiguration

Parameter Name	Parameter Type	Value range	Description
hosts	string	-	RedisServer IP address
password	string	-	RedisServer password
type	string	cluster,master-slave,normal	RedisServer Type
cloud	string	-	RedisServer cloud
region	string	-	Region to which the RedisServer belongs
azs	string	-	AZ to which RedisServer belongs
pool	ServerConnectionPoolConfiguration	For details,see the description of the data structure of ServerConnectionPoolConfiguration	Connection pool configuration

ServerConnectionPoolConfiguration

Parameter Name	Parameter Type	Value range	Description
maxTotal	int	-	Maximum number of active objects
maxIdle	int	-	Maximum number of objects that can remain in the idle state
minIdle	int	-	Minimum number of objects that can remain in the idle state
maxWaitMillis	int	-	Maximum wait time when no object is returned in the pool
timeBetweenEvictionRunsMillis	int	-	Idle link detection thread,detection interval,in milliseconds.A negative value indicates that the detection thread is not running

InjectionProperties

Parameter Name	Parameter Type	Value range	Description
active	bool	true/false	Whether the fault injection function is enabled
duration	int	-	Fault injection duration,in seconds
interval	int	-	Fault injection interval,in seconds
percentage	int	0-100	Injection failure probability
delayInjection.active	bool	true/false	Delay injection switch
delayInjection.percentage	int	0-100	Delayed Fault Effective Probability
delayInjection.timeMs	int	-	Indicates the delay base,in milliseconds
delayInjection.jitterMs	int	-	Indicates the jitter amplitude of the delay, in milliseconds
errorInjection.active	bool	true/false	Abnormal injection switch
errorInjection.percentage	int	0-100	Abnormal Fault Effective Probability

# Packages

# README