Embedded Files
This tutorial will show you how to run Gallium Data on your machine, along with Redis. Nothing will be installed permanently, it’s all done with containers, so you can throw everything away when you’re done.
There are several versions of this tutorial:
this one, which uses Redis as a database,
The concepts are the same, pick the database you're most familiar with.
This tutorial will take about 10 minutes, depending on your download speed. It can be run on anything that runs Docker.
Step 1: Make sure Docker is running
Step 1: Make sure Docker is running
For this tutorial, we will use Docker (1). To verify that Docker is indeed available:
⇨ run the following command from a command line:
docker version
You should see some output similar to this:
Client: Docker Engine - Community
Cloud integration 1.0.22
Version: 20.10.12
API version: 1.41
etc...
The exact version numbers are not important -- the important thing is that Docker needs to be running.
If you get an error message, you'll need to get Docker up and running before you can continue with this tutorial. Fortunately, there are lots of resources that can help you with that.
We’ll be starting three Docker containers — there are easier ways of doing this using e.g. Docker Compose or Kubernetes, but for this tutorial we want to make sure every component is visible and clearly understood.
We’ll be running all containers in their own Docker network, so let’s create that network.
⇨ Run the following command:
docker network create gallium-net
The response should be a long string of letters and numbers, which you can safely ignore, something like:
9ef282f6d3cce819a etc...
If you get an error, it may be because you've run another Gallium Data tutorial before, in which case you can ignore the error and carry on.
Docker is now ready, let’s move on to the next step.
Step 2: Start the three containers
Step 2: Start the three containers
If this is your first time running this tutorial, note that it will download about 900MB of container images, which can take a while on slower connections.
1 - Start the Redis database
1 - Start the Redis database
⇨ Run the following from a command line:
docker run -d --rm --name gallium-redis --network gallium-net galliumdata/redis-tutorial:3
This may take a minute as Docker downloads the image and starts it up.
This image is simply the standard Redis image from Redis Labs, preloaded with a bit of data.
There is nothing special about this image: Gallium Data can run with any Redis database (2.0 and above), including in the cloud (AWS, Azure, etc...)
2 - Start Gallium Data
2 - Start Gallium Data
⇨ Run the following from a command line:
docker run -d --rm --name gallium-data --network gallium-net -p 8089:8080 -e repository_location=/galliumdata/repo_redis galliumdata/gallium-data-engine:1.7.0-1448
Again, this may take a minute. This is the standard Gallium Data image, with a demo repository, which is set up for this tutorial. In the real world, you will typically use additional options to create your own repository.
3 - Start the database client
3 - Start the database client
Finally we’ll also need a database client. Here we’ll be using RedisInsight, but any other Redis client would work equally well (like Another Redis Desktop Manager or redis-gui).
⇨ Run the following from a command line:
docker run -d --rm --name gallium-redisinsight --network gallium-net -p 3015:8001 redislabs/redisinsight:latest
This is the standard RedisInsight image, with options to connect to Gallium Data.
We now have three Docker containers running, speaking to each other as follows:
Step 3: Take a look at the database
Step 3: Take a look at the database
⇨ Connect to RedisInsight at: http://localhost:3015/add/?name=Gallium&host=gallium-data&port=6379&redirect=true
⇨ Select the Browser in the left nav bar
The database contains a few objects, which we'll use in this tutorial.
Everything is now in place, now let's see Gallium Data in action.
Step 4: Let's change how the database works
Step 4: Let's change how the database works
A simple demonstration of the power of Gallium Data can be seen with just a basic filter that will modify certain types of requests.
⇨ Connect to Gallium Data at: http://127.0.0.1:8089
⇨ Log in
⇨ Open the project named Tutorial for Redis
⇨ Expand the Request filters area
⇨ Open the filter named Limit SCAN queries to Canada
The parameters are set so that it executes whenever a simple query is run for all keys, such as scan 0 match *
regex:SCAN
regex:\d+
regex:MATCH
*
⇨ Select the Code tab
The code changes the fourth component of the query, from an asterisk (indicating all keys), to Canada:* (meaning only those keys that start with "Canada:")
context.packet[3].string = "Canada:*";
⇨ Select the Active checkbox
⇨ Click Publish (top)
⇨ Go back to RedisInsight and click the refresh button at the top of the list
You now only see the objects whose key starts with "Canada:"
Feel free to experiment with the code. You can change the filter in Gallium Data to only see another country, or you can make the filter do whatever you want.
Any time you make a change, you'll need to click Publish (top right) in Gallium Data to deploy your changes.
You can use the log object to print out debugging messages, which you can see in the Logs page of Gallium Data, or by running the following in a command line:
docker logs -f gallium-data
[Hint: hit ctrl-c to regain control of the command line when you're done]
You can also use the debugger to suspend execution and step through your code.
Step 5: Changing data
Step 5: Changing data
Now let’s make Gallium Data modify data that's coming back from Redis.
⇨ Look at the value of the Canada:Main Cities object:
Let's say we don't want our database users to see Vancouver, and instead replace it with Red Deer.
⇨ Go back to the Gallium Data admin app
⇨ Expand the Response filters area
⇨ Click on the response filter called Hide Vancouver
You'll see that it has some parameters that activate this filter only when the current request starts with:
LRANGE Canada:Main Cities
⇨ Click on the Code tab
The code is very simple -- it goes over the objects in the result and, if it finds Vancouver, it replaces it with Red Deer:
for (var c of context.packet) {
if (c.string === "Vancouver, BC") {
c.string = "Red Deer, AB";
break;
}
}
⇨ Select the Active checkbox
⇨ Click Publish (top)
⇨ Go back to RedisInsight and hit the Refresh button (upper right)
Note that Vancouver is now replaced with Red Deer:
Nothing has changed in the database -- we're only transforming the data as it goes from Redis to the database client.
This is obviously a simple example -- we could get very fancy here. We could change data depending on any number of factors, we could generate random data, we could hide certain objects, we could insert objects into the result set, and so on. We have complete control, and we can make the database behave in ways that would normally be almost impossible, all without any changes to the database or the database client.
What have we seen?
What have we seen?
In this tutorial, you got a glimpse of how Gallium Data can intercept the traffic between database client and database server, and modify this traffic. This enables you to:
change the behavior of existing applications and databases without changing either the application or the database
catch unwanted or inefficient requests and change or reject them
tailor database responses to your exact needs, with a level of precision that would be almost impossible otherwise
monitor all traffic to and from the databases and react to whatever events are relevant to you
have a control point in front of your databases, which can run customized logic for your specific needs
Now, the question is: how will you use it?
What to do next
What to do next
We encourage you to take Gallium Data for a spin with your own database(s). It's always more interesting to work with your own data than with demo data. Take a look in Gallium Data (Repository -> Project -> Connections) to create a new connection or change the existing one.
The tutorial project contains a few other filters, but they are not active. You can take a look at them and try to activate them:
Various logging filters showing requests and/or responses -- see the output in the Logs page of Gallium Data, or with:
docker logs -f gallium-data
A request filter to implement a simple RANDOM command in Redis, along with a response filter that adds this new command to the list of valid commands returned by the COMMAND command
Gallium Data is free, so you can use it as much as you want, on your machines, servers, in the cloud, wherever.
Consult the documentation for all the gritty details, such as how to use the debugger, or the API for the objects Redis uses in its protocol.
Cleanup
Cleanup
Once you're done with this tutorial, and you want to remove everything that was installed,
⇨ Execute the following commands from a command line:
docker stop gallium-redis
docker stop gallium-redisinsight
docker stop gallium-data
docker network rm gallium-net
This will stop all the Docker containers started during this tutorial.
If you also want to remove the Docker images:
docker rmi redislabs/redisinsight:latest
docker rmi galliumdata/gallium-data-engine:1.3.1-1248
docker rmi galliumdata/redis-tutorial:3
This will remove everything installed by this tutorial.
We'd love to hear from you -- good or bad! Please drop us an email and let us know if you have any questions or comments.
Footnotes
Footnotes
(1) - Gallium Data is just a container, so it runs in anything that can run a container: Docker, podman, Kubernetes, containerd, etc... You can run this tutorial using something other than Docker, but you'll have to translate the command line options. On recent Windows versions, you can run Docker in WSL if you prefer.
Google Sites
Report abuse