MySQL Tutorial

This tutorial will show you how to run Gallium Data with MySQL on your machine. Nothing will be installed permanently, it’s all done with Docker containers, so you can throw everything away when you’re done.

There are several versions of this tutorial: 

• this one, which uses MySQL as a database,

• an equivalent tutorial for PostgreSQL,

• one for MongoDB,

• one for Microsoft SQL Server,

• one for IBM DB2

• one for Cassandra,

• and one for Vertica.

The concepts are the same, pick the one you're most familiar with.

This tutorial will take about 10 minutes, depending on your download speed.

You should see some output similar to this:

Client: Docker Engine - Community
Cloud integration  1.0.33
Version:           24.0.2
API version:       1.43
etc...

The exact version numbers are not important -- the important thing is that Docker needs to be running. 

If you get an error message, you'll need to get Docker up and running before you can continue with this tutorial. Fortunately, there are lots of resources that can help you with that.

We’ll be starting two Docker containers — there are easier ways of doing this using e.g. Docker Compose or Kubernetes, but for this tutorial we want to make sure every component is visible and clearly understood.

We’ll be running both containers in their own Docker network, so let’s create that network. 

The response should be a long string of letters and numbers, which you can safely ignore, something like:

9ef282f6d3cce819a etc...

If you get an error, it may be because you've run another Gallium Data tutorial before, in which case you can ignore it.

Docker is now ready, let’s move on to the next step.

Step 2: Start MySQL and Gallium Data

If this is your first time running this tutorial, note that it will download about 800MB of container images, which can take a while on slower connections.

1 - Start the MySQL database server

Feel free to experiment with the code in the Gallium Data filter. Any time you make a change, you'll need to click Publish (top right) to deploy your changes.  For instance, a simple-minded way to change the limit to all queries might be:

let sql = context.packet.query;
let re = /\blimit\s+\d+,\s*\d+/gim;
let match = re.exec(sql);
if (match) {
    context.packet.query = sql.substring(0, match.index) + "limit 0,10" +
            sql.substring(match.index + match[0].length);
    log.debug("Changed limit in SQL: " + context.packet.query);
}

We can also reject queries based on any desired condition, for instance:

if (new Date().getDay() === 0) {  // 0 == Sunday, 1 == Monday, etc...
    context.result.success = false;
    context.result.errorMessage = "You can't query products on Sunday";

    context.result.errorCode = 12345;

    log.debug("Rejecting query because it's Sunday");
}

Any changes take effect as soon as you hit the Publish button. You can use the log object to print out debugging messages, which you can see in the Logs page of Gallium Data, or by running the following in a command line:

docker logs -f gallium-data

Hit ctrl-c to regain control of the command line when you're done.

Step 5: Changing data

Now let’s make Gallium Data modify data that's coming back from the MySQL server.

You will see a number of customers:

mysql> select * from customers;

+----+-----------------------+------------------------------+---------+

| id | first_name            | last_name                    | country |

+----+-----------------------+------------------------------+---------+

|  1 | Andrea                | Albinioni                    | AR      |

|  2 | Bruno                 | Botticelli                   | BE      |

|  3 | Charles               | Corleone                     | CA      |

|  4 | Daniella              | Durango                      | DK      |

|  5 | Eric                  | Edmunds                      | ES      |

 [ ... 28 more rows here ... ]

+----+-----------------------+------------------------------+---------+

33 rows in set (0.00 sec)

Let's say we don't want our database users to see the full first name of the customers : we can easily make that happen with a response filter.

⇨ Go back to the Gallium Data admin app
⇨  Expand the Response filters area
⇨ Click on the response filter called Show customers initials 

You'll see that it has some parameters that activate this filter for result sets that include data from the gallium_demo.customers table.

⇨ Click on the Code tab

The code is a single line of code, which replaces the value for the first_name column with its initial.

⇨ Select the Active checkbox

⇨ Click Publish (top)

⇨ Go back to the MySQL command line and re-run the query (use up-arrow)

Note that you now only get the initials for all customers.

This is obviously a simple example -- we could get very fancy here. We could change data depending on any number of factors, we could generate random data, we could hide certain rows and/or columns, we could even insert rows into the result set. We have complete control, and we can make the database behave in ways that would normally be almost impossible, all without any changes to the database server or the database client.

A few things to try at this point, if you feel like it:

Hide a row using the remove() method:

if (context.packet.country === 'DK') {
  context.packet.remove();
}

If you deploy this code and re-run your query, you will see that customer 4 (Daniella Durango) no longer appears in the result set. This type of simple hiding can also be done without code, just with the configuration parameters of the filter, but you get more flexibility with code.

Insert a synthetic row:

if (context.packet.id === 2) {

    let newRow = context.packet.clone();

    newRow.id = -2;

    newRow.first_name = 'Gallium';

    newRow.last_name = "Data is fun!";

    // Insert before the original row

    newRow.sequenceId--;

    context.packets.addPacket(0, newRow);

}

If you deploy this code and re-run your query, you will see a new row appear before customer 2. Again, no changes are made to the database, only to the result set received by the database client.

What have we seen?

In this tutorial, you got a glimpse of how Gallium Data can intercept the traffic between database client and database server, and modify this traffic. This enables you to:

• change the behavior of existing applications and databases without changing either the application or the database

• catch unwanted or inefficient requests and change or reject them

• tailor database responses to your exact needs, with a level of precision that would be almost impossible otherwise

• monitor all traffic to and from the databases and react to whatever events are relevant to you

• have a control point in front of your databases, which can run customized logic for your specific needs

Gallium Data has a number of pre-defined filters, but it also makes it easy to create your own filters and be as sophisticated as you want.

Now, the question is: how will you use it?

What to do next

We encourage you to take Gallium Data for a spin with your own database(s). It's always more interesting to work with your own data than with demo data.

The tutorial project contains several other filters, but they are not active. You can take a look at them and try to activate them:

• Time of day connection filter

• Various logging filters showing requests and/or responses -- see the output in the Logs page of Gallium Data, or with:

docker logs -f gallium-data

Gallium Data is free for end users, so you can use it as much as you want, on your machines, servers, in the cloud, wherever, as long as you do not redistribute it (you'll need a license for that).

Consult the documentation for all the gritty details, such as how to use the debugger, or the API for various types of database packets.