Commit b7fd6cf7 authored by Renato Figueiro Maia's avatar Renato Figueiro Maia
Browse files

Inclusão de instruções de desenvolvimento no README.

[SOMA-7020][SOMA-6957]
parent ab24002a
## Docker
The Docker image only works with the POSIX driver.
### Build
```console
$ docker build -f Docker/Dockerfile -t soma/sga .
```
### Run
```console
docker run -d \
--rm \
-p 40100:40100 \
-v "${WORKING_DIR}/data/projects":/sgad/data/projects \
-v "${WORKING_DIR}/data/algorithms":/sgad/data/algorithms \
-e CSBASE_SERVER="http://csgrided:40500" \
-e SGAD_HOST="${HOSTNAME}" \
-e SGAD_PORT="40100" \
-e SGAD_NAME="tempestade" \
--network=host \
--privileged \
soma/sga
```
# SGA
SGA is a daemon for executing and monitoring CSBase jobs. This implementation is made of the following components:
SGA is a daemon for executing and monitoring CSBase jobs.
## sga-daemon
## Implementation
This implementation is made of the following components:
### sga-daemon
The core SGA daemon. To use it, you will also need a driver.
## sga-driver-posix
### sga-driver-posix
A SGA daemon driver for running jobs locally on POSIX operating systems. Tested on Linux and Cygwin.
## sga-driver-pbs
### sga-driver-pbs
A SGA daemon driver for running jobs on Torque PBS clusters. This driver is written using `sga-exec`, so it can run either locally (at the cluster master machine) or manage the cluster remotely over SSH.
## sga-driver-slurm
### sga-driver-slurm
A SGA daemon driver for running jobs on Slurm clusters. This driver is written using `sga-exec`, so it can run either locally (at the cluster master machine) or manage the cluster remotely over SSH.
To enable SSH tunnelling to a Slurm server, see [next section](#sga-exec).
## sga-exec
### sga-exec
An extensible library for abstracting local and remote execution of commands, to be used by SGA drivers.
......@@ -40,7 +44,7 @@ If the SSH tunnelling needs authentication, add your `id_rsa` file located in th
To use `sga-exec` when writing your own driver, the rule of thumb is to avoid Lua's standard `io.*` and `os.*` routines.
## ssh-datatransfer
### ssh-datatransfer
A SGA daemon could use a ssh data transfer mechanism to copy input and executable files to execute on remote host sandbox. To enable it adds the following configuration to your `sgad.cfg` file:
......@@ -60,6 +64,17 @@ The `ssh_private_key_path` value is the private key path in the CSGrid server ma
## Installation
### Docker
The SGA Rest Daemon can be deployed using a Docker image.
For more information see [README](./Docker/README.md).
### Kubernetes
The SGA Rest Daemon can be installed via Helm Chat. The installation process can be seen at [README](./kubernetes/posix/helm/README.md).
### Manual
Requirements:
+ gcc 4.8.5
......@@ -93,8 +108,6 @@ Alternatively, use the following options of the script to choose which drivers s
- `--pbs`: Install the PBS driver
- `--slurm`: Install the Slurm driver
## Execution
Run the installed SGA daemon using command `sgad`, and providing a SGA configuration file.
```console
......@@ -151,55 +164,56 @@ eval $(${instdir}/bin/luarocks path --bin)
sgad "$@" 2>&1 | tee -a "${logfile}"
```
## Check SGA Version
The version of SGA can be found in `sga-daemon-\*.rockspec` file. The same version is also found in each component: `sga-driver-posix`, `sga-driver-pbs`, `sga-driver-slurm `and `sga-exec`. One can execute the following command to list the SGA version:
To check the installed version of the SGA REST daemon,
execute the following command:
```console
$ ${SGAD_HOME}/bin/luarocks list sga-daemon
```
## Release a new version
## Development
To release a new SGA version run the following script passing as parameter the release version. Only versions with numbers are permitted (e.g. 1.6.2):
### Automatic tests
All tests implementation are in directory `test`.
```console
$ ./release-prepare 1.6.2
$ cd test
```
The command will automatic change the version attribute in *.rockspec files and it will create a new tag with the release version. After that, it will change rockspec files setting the version attribute to the development version ("scm").
After the tag creation, the CI/CD pipeline will be automatically launched. The last stage on the pipeline is a step to build and publish the SGA's Docker image.
## Docker
Before running tests for the first time,
it is necessary to install the Lua runtime environment required to execute the SGA REST daemon,
as well as the Node.js packages required to run the test suite.
The Docker image only works with the POSIX driver.
```console
$ ../install.sh --force --posix --pbs --slurm $(pwd)/lua_runtime
$ npm install
```
### Build
After that,
we can run the tests using the following command
```console
$ docker build -f Docker/Dockerfile -t soma/sga .
$ npm run test
```
### Run
Whenever changing the test implementation,
use the following command to automatic reformat the JavaScript code to adhere to the current standard coding style.
```console
docker run -d \
--rm \
-p 40100:40100 \
-v "${WORKING_DIR}/data/projects":/sgad/data/projects \
-v "${WORKING_DIR}/data/algorithms":/sgad/data/algorithms \
-e CSBASE_SERVER="http://csgrided:40500" \
-e SGAD_HOST="${HOSTNAME}" \
-e SGAD_PORT="40100" \
-e SGAD_NAME="tempestade" \
--network=host \
--privileged \
soma/sga
$ npm run lint
```
## Kubernetes
### Release a new version
The SGA Rest Daemon can be installed via Helm Chat. The installation process can be seen at [README](./kubernetes/posix/helm/README.md).
To release a new SGA version run the following script passing as parameter the release version. Only versions with numbers are permitted (e.g. 1.6.2):
```console
$ ./release-prepare 1.6.2
```
The command will automatic change the version attribute in `*.rockspec` files and it will create a new tag with the release version. After that, it will change rockspec files setting the version attribute to the development version ("scm").
After the tag creation, the CI/CD pipeline will be automatically launched. The last stage on the pipeline is a step to build and publish the SGA's Docker image.
## Credits
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment