Redis cluster connector

This connector is only for use with Redis cluster databases and will not work with single-node databases.

For single-node databases, please use the regular Redis connector.

Related pages

• Redis and TLS

• Redis types

• Filter types

• Examples

Cluster considerations

When running Gallium Data with a clustered Redis database, you will need at least as many instances of Gallium Data as there are nodes in your Redis database. It's OK to have more, but the extra instances will not be used unless you grow the Redis cluster.

If you are not sure how many nodes you have in your Redis cluster, you can connect to one of the nodes using redis-cli and execute the command:

cluster nodes

It will print out something like:

a84d32ef...88932e5 122.168.188.131:6379@16379 slave acdcec0...88d6b37 0 1651051034551 2 connected

acdcec01...88d6b37 122.168.23.177:6379@16379 master - 0 1651051034000 2 connected 5461-10922

f1ac5645...c96a59a 122.168.101.7:6379@16379 slave 735df643...1cd00c5 0 1651051033543 3 connected

4ea546c4...7f140c0 122.168.101.68:6379@16379 myself,master - 0 1651051032000 1 connected 0-5460

735df643...1cd00c5 122.168.221.72:6379@16379 master - 0 1651051034046 3 connected 10923-16383

afaf1872...52cd87c 122.168.5.79:6379@16379 slave 4ea546c4...7f140c0 0 1651051032537 1 connected

In this example, there are 6 nodes in the cluster (their IP and ports numbers are in bold). You would therefore need 6 instances of Gallium Data to work with this cluster.

Instead of multiple instances of Gallium Data, you can also use multiple connections in a single instance of Gallium Data, but of course that defeats the purpose of having a cluster, since that single instance of Gallium Data becomes a single point of failure. However, for development and testing, this can be a useful option.

If you do not have enough Gallium Data instances/connections, this connector will not be able to work properly and will fail with an error message: There are not enough Gallium Data nodes to cover the entire cluster

Dynamic mappings

Most of the time, you can let Gallium Data do the mapping between Redis nodes and Gallium Data nodes.

Gallium Data will do that by contacting one of the Redis servers specified in the Redis servers parameter, obtain a list of nodes in the Redis cluster, and assign each one to one of the Gallium Data nodes specified in the Cluster members parameter.

To do so, each instance of Gallium Data must be given an instance ID, starting with 1 and going up to the number of instances, without any gaps. So if you have 6 Gallium Data nodes, they need to have instances IDs from 1 to 6.

This instance ID is usually specified as an environment variable on the Docker command line, e.g.:

docker run -d -p 6391:6391 -e instance_id=1 etc...

but it can also be specified in the options file for each node.

The main advantage of dynamic mappings is that you don't need to list all the nodes in the Redis cluster -- just a couple to get started. In addition, if your Redis cluster changes, you can have the Gallium Data instances rescan it (see below) without restarting or reconfiguring anything, provided that there are enough Gallium Data instances to cover all Redis nodes.

Static mappings

In some cases, you may want to have more control and specify which Redis node is assigned to which Gallium Data node.

This can be useful for optimization reasons, for instance if you want to keep nodes as close together as possible.

Another reason to do this is if you turn on the Use TLS with server parameter. There is a quirk/bug in Redis clusters that can cause it to respond with the wrong port if you connect to it over TLS. In that case, you will typically have to use static mappings.

The main advantage of static mappings is that you keep complete control over which Redis node is assigned to which Gallium Data node.

Connecting from a client

Once everything is in place, you can connect from a Redis client to any of the Gallium Data nodes instead of the Redis nodes.

How does it work?

The RedisCluster connector works by mapping one Gallium Data server to each Redis server.

The Gallium Data servers do not communicate with one another, they communicate only with one Redis server each, and the Redis clients.

When using dynamic mappings, each Gallium Data server uses the following algorithm:

1. when the first Redis client connects, it contacts one of the Initial Servers to get the list of nodes (using the CLUSTER NODES command)

2. it sorts the Redis nodes by ID

3. it assigns one Gallium Data server to each Redis node by using the Gallium Data server's instance ID (instance ID 1 gets the first Redis server, and so on) -- this is why the instance IDs must start at 1 and be consecutive

4. it uses its own instance ID to figure out which Redis server it is assigned to, and connects to it

When using static mappings, the nodes are mapped explicitly by you.

From that point on, the Gallium Data server will act like a regular proxy, with a few differences.

Redirect

When a Redis client asks for the value of a key, for instance with:

GET MyKey

the Redis server will respond with the value if it has the key in question. If it does not have that key, however, it will determine which Redis server has that value and respond with a message pointing the client to that server.

When Gallium Data detects that redirect message, it will automatically change it to point to the Gallium Data instance assigned to that Redis server. The Redis client will then (typically) connect to that Redis server through its Gallium Data instance.

Cluster commands

When a Redis client executes a CLUSTER command (such as CLUSTER NODES, CLUSTER SLOTS, etc...) that returns a response containing the address of Redis servers, these addresses will be automatically changed to the addresses of the corresponding Gallium Data servers.

Changing the Redis cluster

If the Redis cluster changes in any way by adding or removing cluster members, or adding or removing replicates, the Gallium Data servers must immediately be informed of that fact. This can be done in one of two ways.

Option 1: reconfigure the Gallium Data servers

In this option, you can change the connection settings in the Gallium Data connections to reflect the changes in the cluster. You should do this if the Redis Servers parameter has changed, otherwise the Gallium Data servers will keep trying to connect to non-existent servers.

When using static mappings, this is the only valid option.

Option 2: rescan the cluster

If the initial servers have not changed, and you are using dynamic mappings, and you have enough Gallium Data instances to cover all the cluster nodes, you can issue the command galliumdata scancluster to each Gallium Data server, and it will re-read the list of nodes from one of the Redis servers and reconfigure itself accordingly. For instance, using redis-cli:

162.18.1.69:6379> galliumdata scancluster

OK 6 nodes found in cluster

This will take effect immediately. This must be done on each Gallium Data node. Existing connections from clients may need to be re-established.

This allows you to reconfigure your Redis cluster without having to restart your Gallium Data instances.

You can get a list of the current cluster mappings by running the GALLIUMDATA STATUSCLUSTER command from any Redis client connected to Gallium Data:

162.18.1.69:6379> galliumdata statuscluster

OK Cluster mappings (dynamic):

Redis server 202.168.188.131:6379 is mapped to Gallium Data 202.168.66.69:6391

Redis server 202.53.92.91:6379 is mapped to Gallium Data 187.3.17.6:6391

Redis server 187.3.11.114:6379 is mapped to Gallium Data 187.3.17.41:6391

This is an internal command that is processed by Gallium Data and never reaches the Redis servers.