SGA is a daemon for executing and monitoring CSBase jobs. This implementation is made of the following components:
SGA is a daemon for executing and monitoring CSBase jobs.
## sga-daemon
## Implementation
This implementation is made of the following components:
### sga-daemon
The core SGA daemon. To use it, you will also need a driver.
## sga-driver-posix
### sga-driver-posix
A SGA daemon driver for running jobs locally on POSIX operating systems. Tested on Linux and Cygwin.
## sga-driver-pbs
### sga-driver-pbs
A SGA daemon driver for running jobs on Torque PBS clusters. This driver is written using `sga-exec`, so it can run either locally (at the cluster master machine) or manage the cluster remotely over SSH.
## sga-driver-slurm
### sga-driver-slurm
A SGA daemon driver for running jobs on Slurm clusters. This driver is written using `sga-exec`, so it can run either locally (at the cluster master machine) or manage the cluster remotely over SSH.
To enable SSH tunnelling to a Slurm server, see [next section](#sga-exec).
## sga-exec
### sga-exec
An extensible library for abstracting local and remote execution of commands, to be used by SGA drivers.
...
...
@@ -40,7 +44,7 @@ If the SSH tunnelling needs authentication, add your `id_rsa` file located in th
To use `sga-exec` when writing your own driver, the rule of thumb is to avoid Lua's standard `io.*` and `os.*` routines.
## ssh-datatransfer
### ssh-datatransfer
A SGA daemon could use a ssh data transfer mechanism to copy input and executable files to execute on remote host sandbox. To enable it adds the following configuration to your `sgad.cfg` file:
...
...
@@ -60,6 +64,17 @@ The `ssh_private_key_path` value is the private key path in the CSGrid server ma
## Installation
### Docker
The SGA Rest Daemon can be deployed using a Docker image.
For more information see [README](./Docker/README.md).
### Kubernetes
The SGA Rest Daemon can be installed via Helm Chat. The installation process can be seen at [README](./kubernetes/posix/helm/README.md).
### Manual
Requirements:
+ gcc 4.8.5
...
...
@@ -93,8 +108,6 @@ Alternatively, use the following options of the script to choose which drivers s
-`--pbs`: Install the PBS driver
-`--slurm`: Install the Slurm driver
## Execution
Run the installed SGA daemon using command `sgad`, and providing a SGA configuration file.
The version of SGA can be found in `sga-daemon-\*.rockspec` file. The same version is also found in each component: `sga-driver-posix`, `sga-driver-pbs`, `sga-driver-slurm `and `sga-exec`. One can execute the following command to list the SGA version:
To check the installed version of the SGA REST daemon,
execute the following command:
```console
$${SGAD_HOME}/bin/luarocks list sga-daemon
```
## Release a new version
## Development
To release a new SGA version run the following script passing as parameter the release version. Only versions with numbers are permitted (e.g. 1.6.2):
### Automatic tests
All tests implementation are in directory `test`.
```console
$./release-prepare 1.6.2
$cd test
```
The command will automatic change the version attribute in *.rockspec files and it will create a new tag with the release version. After that, it will change rockspec files setting the version attribute to the development version ("scm").
After the tag creation, the CI/CD pipeline will be automatically launched. The last stage on the pipeline is a step to build and publish the SGA's Docker image.
## Docker
Before running tests for the first time,
it is necessary to install the Lua runtime environment required to execute the SGA REST daemon,
as well as the Node.js packages required to run the test suite.
The Docker image only works with the POSIX driver.
The SGA Rest Daemon can be installed via Helm Chat. The installation process can be seen at [README](./kubernetes/posix/helm/README.md).
To release a new SGA version run the following script passing as parameter the release version. Only versions with numbers are permitted (e.g. 1.6.2):
```console
$./release-prepare 1.6.2
```
The command will automatic change the version attribute in `*.rockspec` files and it will create a new tag with the release version. After that, it will change rockspec files setting the version attribute to the development version ("scm").
After the tag creation, the CI/CD pipeline will be automatically launched. The last stage on the pipeline is a step to build and publish the SGA's Docker image.