Skip to main content

Pipeline YAML Reference

This is the reference for the configuration of config.yaml files. The entire build, simulation and deployment process can be specified by the configuration file (config.yaml). The configuration file is stored in the directory .bbx/config.yaml in the the root folder of your repo. They are written in YAML syntax. If you are not familiar with YAML, please see this guide.

runners

Required. The runners that are used in this pipeline. Runners provide the execution environment for your jobs to run in. Multiple different runners maybe used in the same pipeline so that the optimal choice is used for a particular job.

KeyRequiredTypeDescription
<runner_id>YMapThe ID of a runner.

runners.<runner_id>

Required. Each runner must have an ID, which is unique. <runner_id> must start with a letter and only contain alphanumeric characters.

KeyRequiredTypeDescription
imageYMapThe runner image used for this runner.

runners.<runner_id>.image

Required. The runner image used for a particular runner. Images provide the OS, the specification, the pre-installed libraries and tools for a runner. Names of runner images, as well as additional information may be found in the runner store.

Example

runners:
first-runner:
image: work1-virtualbox:5000/ubuntu-vitis-2020-1:latest

jobs

Required. A workflow is made of one or more jobs. Each job is provided with a runner environment and will by default run in parallel with another job, unless a depends field is present. BeetleboxCI does not enforce any limit on the number of jobs that may run in parallel (unless there is lack of hardware resources on the machine to run a job).

KeyRequiredTypeDescription
<jobs_id>YMapThe ID of jobs.

jobs.<jobs_id>

Required. Each job must have an associated ID, which is unique. The job ID is used as key and its map is used as configuration data. <jobs_id> must start with a letter and only contain alphanumerical characters.

KeyRequiredTypeDescription
current_working_directoryNStringThe current working directory of the runner (job container). If it is not set, the default value of ~/ will be used.
deviceNStringThe name of the device from the device registry to be used in the job to be run.
dependsNMapIdentifies the name of the jobs that must successfully complete before this job is run.
inputNMapArtifacts that are to be retrieved from the artifact store for use by the job.
outputNMapThe files that are to be saved from the job environment and stored as artifacts in the artifact store.
privilegedNBooleanSpecifies whether the runner (job container) for the job runs with elevated privileges. This allows the runner to directly access hardware devices on the computer, such as USB and serial ports. Any job that requires communication with a registered device requires a privileged runner.
resource_specNStringThe amount of resources to allocate to this job.
runnerYStringThe name of the runner to be used for this job.
stepsYListSteps are the tasks that a job is to perform.

jobs.<jobs_id>.current_working_directory

Sets the current working directory of the image. When running jobs, Git repositories that are cloned will be automatically placed in this directory. If not set, the default value of ~/ will be used.

jobs:   
first-job:
current_working_directory: /tools/Xilinx/Vivado/2020.1/

This example will set the current working directory to the Vivado install path and will place the Git repository at this location.

jobs.<jobs_id>.depends

Depends are used to determine what jobs need to successfully complete for this job to start execution.

Example

jobs:   
first-job:
second-job:
depends:
- first-job
third-job:
depends:
- first-job
- second-job

In this example, first-job must complete successfully before second-job may start. third-job must wait for both first-job and second-job.

jobs.<jobs_id>.device

The name of the device from the device registry to be used in the job to be run.

Example

jobs:
run-hw:
device: raspberry-pi

jobs.<jobs_id>.input

input indicate the folder and files that are to be downloaded onto the runner before the job executes.
KeyRequiredTypeDescription
artifactYListThe artifacts that are to be downloaded onto the runner.
jobs.<jobs_id>.input.artifact

Used to indicate the artifacts that are to be downloaded from the artifact store onto the runner before job execution.

Example

jobs:   
first-job:
input:
artifact:
- example1.zip
- example2.zip
jobs.<jobs_id>.node_selector

Use to indicate which node of the cluster a particular job should run on. The node_selector value is a dict which much match the labels on a particular node, for a job to be placed on that node. To view the labels of a particular node, run the command kubectl describe node [name-of-node]. The labels will be listed in the form key=value, but in the config.yaml, it must be written as key: value. If this field is not specified, the job will automatically be placed onto any of the available nodes in the cluster.

Format

jobs:   
first-job:
node_selector:
key: value

Example

jobs:   
first-job:
node_selector:
kubernetes.io/hostname: name-of-node

jobs.<jobs_id>.output

This is used for folders and files that are to be uploaded onto the artifact store once the runner has finished execution.

KeyRequiredTypeDescription
artifactYListThe artifacts that are to be downloaded onto the runner.
jobs.<jobs_id>.output.artifact

Identifies the artifacts that are to be uploaded onto the artifact store once the runner has finished execution.

Example

jobs:   
first-job:
output:
artifact:
- example1.zip
- example2.zip

jobs.<jobs_id>.resource_spec

The amount of resources to allocate to this job. Defaults to small if unspecified.

KeyCPURAMType
micro0.51GiBString
small1.04GiBString
medium2.08GiBString
large4.016GiBString
xlarge8.032GiBString
1.5xlarge12.048GiBString
2xlarge16.064GiBString
3xlarge24.096GiBString
4xlarge32.0128GiBString

Example

jobs:   
first-job:
resource_spec: micro

jobs.<jobs_id>.runner

Required. The runner that the job is to execute on. The runner provides the OS, specification, pre-installed libraries and tools for the job.

Example

jobs:   
first-job:
runner: first-runner

jobs.<jobs_id>.steps

Required. Each executes a series of tasks known as steps, which are able to execute commands on the runner. Each step runs in the same environment and has full access to the tools and filesystem of that runner. There are two types of steps: run and transfer. Every step must have one (but not both) of these types specified.

KeyRequiredTypeDescription
runYMapSpecify a run type step.
transferYMapSpecify a transfer type step.
jobs.<jobs_id>.steps.run

Invoke one or more command-line programs through providing the command string. By default, commands are executed in a bash shell.

KeyRequiredTypeDescription
nameYStringThe name of the step.
commandYStringThe specific command to be run.
on-deviceNBooleanDetermines whether the command will be executed directly in the device terminal or not.
passNStringFor on-device steps only: The step immediately passes if this string is detected in the terminal output.
failNStringFor on-device steps only: The step immediately fails if this string is detected in the terminal output.
jobs.<jobs_id>.steps.run.name

Required. The name of the step to be displayed.

Example

jobs:   
hello_world:
steps:
- run:
name: Say Hello
jobs.<jobs_id>.steps.run.command

Required. The commands to be executed. By default command is executed on a bash shell.

Example

jobs:   
hello_world:
steps:
- run:
name: Say Hello
command: echo "Hello World"
jobs.<jobs_id>.steps.run.on-device

Required. Determines whether the command will be executed directly in the device terminal or not.

Example

jobs:   
hello_world:
device: raspberry-pi
steps:
- run:
name: Say Hello
command: echo "Hello World"
on-device: True
jobs.<jobs_id>.steps.run.pass

Required. For on-device steps only: The step immediately passes if this string is detected in the terminal output.

Example

jobs:   
hello_world:
steps:
- run:
name: Say Hello
command: echo "Hello World"
on-device: True
pass: Hello World
jobs.<jobs_id>.steps.run.fail

Required. For on-device steps only: The step immediately fails if this string is detected in the terminal output.

Example

jobs:   
hello_world:
steps:
- run:
name: Say Hello
command: echo "Hello World"
on-device: True
fail: Error
jobs.<jobs_id>.steps.transfer

Invoke one or more command-line programs through providing the command string. By default, commands are executed in a bash shell.

KeyRequiredTypeDescription
nameYStringThe name of the step.
SOURCENStringThe location from where files are transferred. The SOURCE may also be hardcoded in the device configuration.
DESTINATIONNStringThe location to where files are transffered. The DESTINATION may also be hardcoded in the device configuration.
jobs.<jobs_id>.steps.transfer.name

Required. The name of the step to be displayed.

Example

jobs:   
hello_world:
device: raspberry-pi
steps:
- transfer:
name: Copy files
jobs.<jobs_id>.steps.transfer.SOURCE

Required. The location from where files are transferred. The SOURCE may also be hardcoded in the device configuration. The example shows files being copied from a runner (on the server) to a device.

Example

jobs:   
hello_world:
device: raspberry-pi
steps:
- transfer:
name: Copy files
SOURCE: /PATH/TO/SOURCE/FOLDER/*
DESTINATION: root@192.168.1.92:/PATH/TO/DESTINATION/FOLDER

jobs.<jobs_id>.steps.transfer.DESTINATION

Required. The location to where files are transfered. The DESTINATION may also be hardcoded in the device configuration. The example code files being copied from a device to the runner (on the server)

Example

jobs:   
hello_world:
device: raspberry-pi
steps:
- transfer:
name: Copy files
SOURCE: root@192.168.1.92:/PATH/TO/SOURCE/FOLDER/*
DESTINATION: /PATH/TO/DESTINATION/FOLDER

jobs.<jobs_id>.volumes

Use to indicate if any directories from the node which a job runs on should be mounted as a read-only volume inside the job container. This is useful for cases where jobs must access large amounts of data or very large (>50GB) software programs.

Example

jobs:   
random-job:
volumes:
- mount:
name: volume1
path: /tmp/firstvolume
- mount:
name: volume2
path: /tmp/secondvolume
- mount:
name: volume3
path: /opt/thirdvolume

workflows

Required. A workflow is used to orchestrate and manage jobs.

KeyRequiredTypeDescription
<workflow_id>YMapThe ID of a workflow.

workflows.<workflow_id>

Required. The unique id of a workflow. <workflow_id> must start with a letter and only contain alphanumeric characters.

KeyRequiredTypeDescription
jobsYListThe jobs that are to be run in this workflow

workflows.<workflow_id>.jobs

Required. The jobs that are to be run in this workflow.

workflows:
workflow-example:
jobs:
- job-example-1
- job-example-2

workflows.<workflow_id>.triggers

The jobs that are to be run in this workflow.

KeyDescription
manualThe workflow will run when the user presses the run button in the webapp. If no triggers are specified, the workflow can still be run manually using the run button.
pushThe workflow will run whenever there is a push to the branch of the repository which the config file is in.
scheduleThe workflow will run periodically according to the schedule specified by the cron string. The cron string uses the standard cron syntax.
workflows:
workflow-example:
jobs:
- job-example-1
- job-example-2
triggers:
- manual
- push
- schedule:
cron: "* * * * *"

Reserved keywords

There are certain strings that should not be used in a BeetleboxCI config file, as these are used internally by BeetleboxCI. Using these keywords in your config file may cause errors or unpredictable behaviour. The keywords are listed in the table below:

Key
__line__