Programmatic changes to the repository

Most users of Gallium Data use the built-in user interface to set up connections, create filters, etc...

Sometimes, though, it may be more convenient to automate some of these tasks. This can be done by programmatically pushing a new version of the repository to the Gallium Data server using its REST API.

Background

Everything in Gallium Data, such as projects, connections, filters, JavaScript code for filters, etc... is ultimately stored in a set of files called a repository. 

A repository is a hierarchy of folders containing various files and subfolders. The structure of a repository is explained in this page, and should be reasonably intuitive to anyone who has used Gallium Data.

When you are using Gallium Data's UI, you are modifying a copy of the repository that is stored in the browser. When you click Publish, that copy of the repository is pushed to the Gallium Data server, and your changes take effect immediately.

Pushing (and pulling) the repository is only possible if the Gallium Data instance was started with the REST port open -- see the configuration page for details. By default, this port is open, but, for security reasons, it is usually closed in production.

Getting the repository

You will always start from an existing repository, even if it is empty. The simplest way to get the repository is using the UI (using the "hamburger menu" at the top right of the UI, option Download repository).

You can also use a tool such as curl, e.g.:

curl "http://localhost:8089/zip/repository" --output repo.zip

You may need to change the host name and the port number as required, depending on how you deployed Gallium Data. If you have trouble figuring this out, take a look at the network traffic in your browser's debugger when you download the repository.

This will download a zip file, which contains the entire repository.

You can unzip that file using whatever tool you choose, for instance, if you have zip installed, you can run:

unzip repo.zip

You will then be able to browse the contents of the repository, and understand its structure. For instance, if you have a database connection named Test MSSQL, there will be a file named <project-name>/connections/Test MSSQL.json.

Similarly, if you have a request filter called Change query, there will be two files in the directory <project-name>/request_filters/Change query, one called request_filter.json, which contains the definition of the filter, and another file called code.js, which contains the JavaScript code for the filter (if any).

Modifying the repository

All these files and directories can be modified as needed, as long as you keep the general structure of the repository.

For instance, you can rename a project by renaming the directory that contains it. You can also rename a filter by changing the name of its directory. You can create new filters by creating new directories under (e.g.) request_filters, each of which will be expected to have a JSON file and a (possibly empty) JavaScript file with the names described above.

You should be careful when editing JSON files, as it is possible to change something in a way that Gallium Data will not understand. For anything that is not obvious, it's a good idea to download the repository, then make the change using the UI, then download the new version of the repository, and compare the old and the new versions.

Pushing the repository to the server

When you have made whatever changes you needed in the repository, you can zip it up (the name of the zip file does not matter), e.g.:

zip -r repo2.zip repo

where repo2.zip is the name of the zip file to create, and repo is the name of the top directory for the repository.

The resulting zip file can then be uploaded to the Gallium Data instance with a POST, such as:

curl -X POST -H "Content-Type: application/zip" --data-binary '@repo2.zip' http://localhost:8089/zip/repository

If you use curl, the response should be:

{"status":"success"}

The new repository takes effect immediately -- depending on the complexity of your changes, it could take a second or two.