Redis cluster connector

This connector is only for use with Redis cluster databases and will not work with single-node databases.

For single-node databases, please use the regular Redis connector.

Cluster considerations

When running Gallium Data with a clustered Redis database, you will need at least as many instances of Gallium Data as there are nodes in your Redis database. It's OK to have more, but the extra instances will not be used unless you grow the Redis cluster.

If you are not sure how many nodes you have in your Redis cluster, you can connect to one of the nodes using redis-cli and execute the command:

cluster nodes

It will print out something like:

a84d32ef...88932e5 122.168.188.131:6379@16379 slave acdcec0...88d6b37 0 1651051034551 2 connected

acdcec01...88d6b37 122.168.23.177:6379@16379 master - 0 1651051034000 2 connected 5461-10922

f1ac5645...c96a59a 122.168.101.7:6379@16379 slave 735df643...1cd00c5 0 1651051033543 3 connected

4ea546c4...7f140c0 122.168.101.68:6379@16379 myself,master - 0 1651051032000 1 connected 0-5460

735df643...1cd00c5 122.168.221.72:6379@16379 master - 0 1651051034046 3 connected 10923-16383

afaf1872...52cd87c 122.168.5.79:6379@16379 slave 4ea546c4...7f140c0 0 1651051032537 1 connected

In this example, there are 6 nodes in the cluster (their IP and ports numbers are in bold). You would therefore need 6 instances of Gallium Data to work with this cluster.

Instead of multiple instances of Gallium Data, you can also use multiple connections in a single instance of Gallium Data, but of course that defeats the purpose of having a cluster, since that single instance of Gallium Data becomes a single point of failure. However, for development and testing, this can be a useful option.

If you do not have enough Gallium Data instances/connections, this connector will not be able to work properly and will fail with an error message: There are not enough Gallium Data nodes to cover the entire cluster

Dynamic mappings

Most of the time, you can let Gallium Data do the mapping between Redis nodes and Gallium Data nodes.

Gallium Data will do that by contacting one of the Redis servers specified in the Redis servers parameter, obtain a list of nodes in the Redis cluster, and assign each one to one of the Gallium Data nodes specified in the Cluster members parameter.

To do so, each instance of Gallium Data must be given an instance ID, starting with 1 and going up to the number of instances, without any gaps. So if you have 6 Gallium Data nodes, they need to have instances IDs from 1 to 6.

This instance ID is usually specified as an environment variable on the Docker command line, e.g.:

docker run -d -p 6391:6391 -e instance_id=1 etc...

but it can also be specified in the options file for each node.

The main advantage of dynamic mappings is that you don't need to list all the nodes in the Redis cluster -- just a couple to get started. In addition, if your Redis cluster changes, you can have the Gallium Data instances rescan it (see below) without restarting or reconfiguring anything, provided that there are enough Gallium Data instances to cover all Redis nodes.

Static mappings

In some cases, you may want to have more control and specify which Redis node is assigned to which Gallium Data node.

This can be useful for optimization reasons, for instance if you want to keep nodes as close together as possible.

Another reason to do this is if you turn on the Use TLS with server parameter. There is a quirk/bug in Redis clusters that can cause it to respond with the wrong port if you connect to it over TLS. In that case, you will typically have to use static mappings.

The main advantage of static mappings is that you keep complete control over which Redis node is assigned to which Gallium Data node.

Connecting from a client

Once everything is in place, you can connect from a Redis client to any of the Gallium Data nodes instead of the Redis nodes.

Parameters

This connector takes the following parameters:

Connection name

Can be any name you want.

Active

If checked, then this connection will be active any time Gallium Data is running, meaning that the local port will be open and listening for requests.

If you uncheck this (and then click Publish), then this connection is "commented out" -- it's as if it didn't exist, but you can still keep it around and re-activate it later.

Type

The type of database for this connection. This is set to "RedisCluster" when you create the connection and cannot be changed afterwards.

Redis servers

A list of Redis servers in the Redis cluster, in the form:

The address part can be a host name (e.g. redis7.acme.com), an IP4 address (e.g. 202.168.66.69), or an IP6 address (e.g. [fe80::7654:7895:4321:1aa3]) if your network supports IP6. Note that IP6 addresses must be in square brackets.

The address is followed by a colon, and the port number for the Redis service.

If you use dynamic mappings, these servers will be used by Gallium Data to discover all the nodes in the cluster. You therefore do not need to list all the Redis nodes: just a couple is typically enough.

When the first connection is opened by a client, Gallium Data will try to connect to the first server and get the list of nodes in the Redis cluster. If that fails, it will try the next server, until it is successful or runs out of servers, in which case the client connection will fail with an error.

If you use static mappings, this should be just one server, which this instance of Gallium Data will use as its Redis server.

Cluster members (dynamic mappings)

If you use dynamic mappings (which is the most common situation), this should be a list of all the Gallium Data servers that will front the Redis cluster. You must have at least as many instances of Gallium Data as there are nodes in the Redis cluster. The format is the same as the Redis Servers parameter, e.g.

202.168.66.69:6391

187.3.17.6:6391

[fe80::7654:7895:4321:1aa3]:6391

The address can be in IP4 or IP6 address (don't forget the square brackets for IP6).

Cluster members (static mappings)

If you use static mappings (less common), this should be a list in the format:

For example:

202.168.188.131:6379 = 202.168.66.69:6391

202.53.92.91:6379 = 187.3.17.6:6391

187.3.11.114:6379 = [fe80::7654:7895:4321:1aa3]:6391

This mapping will be used to translate certain responses from Redis to contain the address of the Gallium Data nodes instead of the address of the Redis nodes.

If you take a look at the output from the cluster nodes command, all these nodes should be listed here, with their corresponding Gallium Data nodes.

Use TLS with server

If selected, the connection to the server will use TLS. Obviously, if this option is selected, all the nodes in the Redis cluster are expected to be using TLS.

If you select this option, you will most likely have to use static mappings.

Trust server certificate

Whether to accept the SSL certificate from the Redis server at face value, or not.

If this is false, and Use TLS with server is selected, then the server's certificate will be verified, either by being signed by a well-known certificate authority, or by verifying it with the chain of certificates provided in the project's Trust entry.

If this is true, then the Redis server's certificate will be accepted without verification. This is useful for debugging and casual usage, but is discouraged in production because it is not secure.

Local address

The address on which to listen for requests. This is typically only relevant if your server has more than one address. By default, Gallium Data will listen on all addresses (which is equivalent to 0.0.0.0).

Local port

Required. The port number on which Gallium Data will be listening to requests from database clients. If you set this to a port other than 6379, your Redis clients will need to specify that port.

Use TLS with clients

If selected, clients will be required to connect using TLS, and any non-TLS connections will be immediately rejected.

Require TLS client authentication

If selected, clients will be required to provide a valid certificate.

How does it work?

The RedisCluster connector works by mapping one Gallium Data server to each Redis server.

The Gallium Data servers do not communicate with one another, they communicate only with one Redis server each, and the Redis clients.

When using dynamic mappings, each Gallium Data server uses the following algorithm:

when the first Redis client connects, it contacts one of the Initial Servers to get the list of nodes (using the CLUSTER NODES command)
it sorts the Redis nodes by ID
it assigns one Gallium Data server to each Redis node by using the Gallium Data server's instance ID (instance ID 1 gets the first Redis server, and so on) -- this is why the instance IDs must start at 1 and be consecutive
it uses its own instance ID to figure out which Redis server it is assigned to, and connects to it

When using static mappings, the nodes are mapped explicitly by you.

From that point on, the Gallium Data server will act like a regular proxy, with a few differences.

Redirect

When a Redis client asks for the value of a key, for instance with:

GET MyKey

the Redis server will respond with the value if it has the key in question. If it does not have that key, however, it will determine which Redis server has that value and respond with a message pointing the client to that server.

When Gallium Data detects that redirect message, it will automatically change it to point to the Gallium Data instance assigned to that Redis server. The Redis client will then (typically) connect to that Redis server through its Gallium Data instance.

Cluster commands

When a Redis client executes a CLUSTER command (such as CLUSTER NODES, CLUSTER SLOTS, etc...) that returns a response containing the address of Redis servers, these addresses will be automatically changed to the addresses of the corresponding Gallium Data servers.

Changing the Redis cluster

If the Redis cluster changes in any way by adding or removing cluster members, or adding or removing replicates, the Gallium Data servers must immediately be informed of that fact. This can be done in one of two ways.

Option 1: reconfigure the Gallium Data servers

In this option, you can change the connection settings in the Gallium Data connections to reflect the changes in the cluster. You should do this if the Redis Servers parameter has changed, otherwise the Gallium Data servers will keep trying to connect to non-existent servers.

When using static mappings, this is the only valid option.

Option 2: rescan the cluster

If the initial servers have not changed, and you are using dynamic mappings, and you have enough Gallium Data instances to cover all the cluster nodes, you can issue the command galliumdata scancluster to each Gallium Data server, and it will re-read the list of nodes from one of the Redis servers and reconfigure itself accordingly. For instance, using redis-cli:

162.18.1.69:6379> galliumdata scancluster

OK 6 nodes found in cluster

This will take effect immediately. This must be done on each Gallium Data node. Existing connections from clients may need to be re-established.

This allows you to reconfigure your Redis cluster without having to restart your Gallium Data instances.

You can get a list of the current cluster mappings by running the GALLIUMDATA STATUSCLUSTER command from any Redis client connected to Gallium Data:

162.18.1.69:6379> galliumdata statuscluster

OK Cluster mappings (dynamic):

Redis server 202.168.188.131:6379 is mapped to Gallium Data 202.168.66.69:6391

Redis server 202.53.92.91:6379 is mapped to Gallium Data 187.3.17.6:6391

Redis server 187.3.11.114:6379 is mapped to Gallium Data 187.3.17.41:6391

This is an internal command that is processed by Gallium Data and never reaches the Redis servers.