Gallium Data is configured using three main mechanisms:
command line options
options file
repository
A word about files and directory paths
If you run Gallium Data as a Docker container, keep in mind that the files and directory paths you specify must be from the perspective of the Docker container. You will often map directories or files from the host file system into the Docker container. Make sure that the options you specify to Gallium Data correspond to the location of the files or directories as seen from within the Docker container.
Command line options
The following command-line options are accepted by the server.
repository-location=<directory path>
Required. Specifies the location of the repository in which all definitions are stored. The repository is a tree of directories and files, including JSON files, JavaScript files, Markdown files, and a few others. This directory contains everything you do in Gallium Data: projects, connections, filters, etc... It is therefore common to map this directory to your local file system, and to keep it under source control (e.g. Git, PerForce, Subversion, etc...)
backup-location=<directory-path>
Optional but recommended. Specifies a directory in which the repository will be backed up before every publish. This allows you to easily undo any changes, since backups can easily be restored from the Backups page in the admin application.
If this option is specified, Gallium Data will create up to four levels of subdirectories: year, month, day, and time every time the repository is published. Each backup is a complete copy of the repository as it was at that moment in time. Most repositories are relatively small, but over time this can grow and should be pruned from time to time, which can be done from the admin app, or directly in the file system.
meta-repo-location=<directory-path>
Optional. For advanced users only. Specifies where the meta-repository can be found. The meta-repository describes how adapters and filters behave.
rest-port=<number>
Optional. Specifies which port to use to expose the REST API used by the admin app. To use the admin app, you must enable either this option or the rest-port-ssl option (or both). In production, the REST API is usually turned off.
rest-port-ssl=<number>
Optional. Specifies which port to use to expose the REST API (over SSL) used by the admin app.
rest-keystore=<jks-file-path>
Required if rest-port-ssl is specified. This should point to a JKS file containing the private key and associated certificates require to run a TLS service.
rest-keystore-pw=<password>
Required if rest-keystore is specified and points to a JKS file that is password-protected.
web-base=<directory-path>
Optional. For advanced users only, who need to customize the admin app.
rest-password=<password>
Optional. If you want to restrict access to the admin app, you can assign a password using this option, and you will be prompted to log in when starting the admin app. The user name is always "admin".
rest-address-range=<regular-expression>
Optional. If you want to restrict access to the admin app, you can specify a regular expression using this option. Any attempts at using the REST API or the admin app will validate the caller's IP address against this regular expression. This is a Java regular expression.
For instance, if you want to allow only machines on your network, you might use something like: 209\.88\.151\..+
instance-id=<number>
Required only if you run multiple instances of Gallium Data in a cluster, like with the RedisCluster connector.
node-modules-dir=<directory-path>
Optional. Use the specified directory as the root for Node's require mechanism. In the Docker image, the default value is /galliumdata/node_modules
node-globals=<js-file-path>
Optional. For advanced users only. Points to a JavaScript file containing the global definitions required by installed packages.
node-modules-replacements=<replacements>
Optional. For advanced users only. A comma-separated list of JavaScript package names and their replacements. This is used for JavaScript packages that are not included in the JavaScript engine, such as fs and buffer. For more information, see the GraalVM documentation.
node-modules-download-dir=<directory-path>
Optional. If specified, JavaScript libraries specified in the repository will be downloaded to this directory. By default, the system temporary directory is used. In the Docker image, the default value is /galliumdata/npm_downloads
npm-registry-url=<URL>
Optional. Specifies which registry to use when installing JavaScript libraries, The default is http://registry.npmjs.org/
ivy-settings=<file-path>
Optional. Use this option to change the settings of the Java library subsystem, which allows you, for instance, to use a private Maven repository instead of the default Maven Central. See the Java library page for more details.
If, like most people, you run Gallium Data as a Docker container, these options can be specified using Docker's -e option, e.g.:
docker run -d \
-e repository-location=/repo \
-e rest-port=8080 \
-v /users/jdoe/projects/repo:/repo \
galliumdata/gallium-data-engine:<version>
Note how /repo is mapped from the host file system. This is common when working on connections and filters because it allows you to access the repository easily, especially for source control.
Options file
All the command-line options can also be specified in a file, which is then passed to Gallium Data as a single command-line option:
settings-location=<path-to-file>
Optional. Should point to a text file containing options as described above.
Again, remember that all the paths specified here are from the perspective of the Docker container.
Example options file
# Required: the location of the repository
repository-location=/galliumdata/repo
# Optional: the location where backups should be created. If not specified, backups are not created.
backup-location=/galliumdata/backups
# Required: the location of the meta-repository.
meta-repo-location=/galliumdata/metarepo
# Optional: the port for the REST/web interface. If neither this nor rest-port-ssl is specified,
# no HTTP service will be started.
rest-port=8043
# Optional: the port for the SSL REST/web interfaces. If neither rest-port nor rest-port-ssl
# is specified, then there will be no REST/web interface, which is common in production.
rest-port-ssl=8044
# If rest-port-ssl is specified, then you should specify where to find the key and certificates
# (and the password for the JKS file, if necessary).
rest-keystore=/galliumdata/KeyStore.jks
rest-keystore-pw=GalliumData
# Optional: where the web files are to be found.
web-base=/galliumdata/web
# Optional: a password required for basic auth for the web and REST interfaces.
#rest-password=Password1
# Optional: a Java regular expression specifying which IP4 or IP6 address(es) are allowed to use
# the web and REST interfaces.
#rest-address-range=(127\.0\.0\.1)|(0:0:0:0:0:0:0:1)
# Optional unless running in a cluster
#instance-id=1
# Optional: the location for Node.js modules
node-modules-dir=/galliumdata/node_modules
# Advanced: where globals are declared for Node modules
#node-globals=/somewhere/globals.js
# Advanced: substitution modules for the ones that aren't built it, e.g. fs
#node-modules-replacements=fs:node-fs-extra,assert:tapsert
# Optional: when installing new JavaScript libraries, the directory in which to download the tarball
node-modules-download-dir=/galliumdata/npm_downloads
Repository
The repository is where Gallium Data stores all its definitions: projects, connections, filters, etc... The easiest way to edit it is using the admin app, but (if you are careful) you can also edit the repository directly. This can be handy when doing automated deployments, for instance. Every time you publish in the admin app, the repository is updated (and a backup is created, if the backup option is enabled).
The structure of the repository is relatively straightforward. The top directory contains:
repository.json: describes the repository, including which Java and JavaScript libraries should be made available
projects: contains one directory per project
projects/<project-name>:
projects/<project-name>/project.json : describes the project. The only thing you might edit here is the "active" attribute.
projects/<project-name>/comments.md : description of the project in Markdown format.
projects/<project-name>/crypto: contains private key and certificate-related files for the project.
projects/<project-name>/crypto/crypto.json : can specify the algorithm used for the private key (if present), and the password for the JKS file (if present).
projects/<project-name>/crypto/key.pem : the private key used for SSL/TLS to database clients, and related certificates.
projects/<project-name>/crypto/trust.pem : the certificate(s) that should be trusted from the database server(s).
projects/<project-name>/crypto/keystore.jks : optional - a Java JKS file containing keys and certificates. Creating a JKS file requires a bit of work, but it does provide you with a lot of flexibility.
projects/<project-name>/connections : contains one JSON file per connection in this project.
projects/<project-name>/connections/<connection-name>.json : describes a connection, its type, and parameter values.
projects/<project-name>/connections/<connection-name>.md : a description of the connection, in Markdown format..
projects/<project-name>/connection_filters : contains one directory for each connection filter in the project
projects/<project-name>/connection_filters/<filter-name> :
projects/<project-name>/connection_filters/<filter-name>/connection_filter.json : describes the filter and its parameter values.
projects/<project-name>/connection_filters/<filter-name>/code.js : the JavaScript code for the filter, if any.
projects/<project-name>/connection_filters/<filter-name>/comments.md : a description of the filter, in Markdown format.
The structure is the same for the other three types of filters:
projects/<project-name>/request_filters :
projects/<project-name>/response_filters :
projects/<project-name>/duplex_filters :