Devices

The main feature that differentiates BeetleboxCI from other CI/CD platforms is the support for external devices such as FPGAs and embedded systems. BeetleboxCI has functionality to communicate with external devices through user-specified commands and a record of the devices are stored in a device registry. This includes transferring files to/from devices and running commands directly on devices. This page gives a detailed account of the devices feature and the functionality that it provides.

Registering a new device

In the Devices page in the BeetleboxCI web app, the Register Device button will take you to the text editor where you can enter the configuration data for the new device.

Editing an existing device

In the Devices page, the edit device button under actions will take you to the text editor where you can edit the configuration data for the existing device.

Device Configuration Editor

The device configuration data is entered and stored in a YAML format. The following example shows a simple device being defined. In order to register a device, you need to define a devices: section and add some information to it. Each device requires a unique name, and in the example below the device has been named raspberry-pi.

devices:
 raspberry-pi:

Connection

Each device requires a mandatory connection field. The connection states the command used to issue commands to the device. In this instance, the data in the connection field sets up an SSH connection to the device. Please note that we do not recommend inputting passwords through sshpass where security is important!

devices:
 raspberry-pi:
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92

Prompt

Each device requires a mandatory prompt field. The prompt is used to monitor the terminal within the device. In this instance, # represents the prompt in the Linux terminal of the device.

devices:
 raspberry-pi:
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "#"

Transfer

Optionally, a transfer field can be included to allow files to be transferred to and from the device.

In this example, an SCP command is defined for transferring files, from the SOURCE folder on the runner and the DESTINATION folder on the device.

devices:
 raspberry-pi:
  transfer: bash -c "sshpass -p raspberry scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /PATH/TO/SOURCE/FOLDER pi@192.168.1.92:/PATH/TO/DESTINATION/FOLDER"
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "#"

In this example, an SCP command is defined for transferring files, from the SOURCE folder on the device to the DESTINATION folder on the runner that is running the job.

devices:
 raspberry-pi:
  transfer: bash -c "sshpass -p raspberry scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r /PATH/TO/SOURCE/FOLDER pi@192.168.1.92:/PATH/TO/DESTINATION/FOLDER"
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "#"

While the transfer command is defined in the two examples above, the transfer will still need to be invoked as part of a step when a job is being run. Defining it in the device configuration alone is not sufficient to make any file transfer happen.

This third example shows that the transfer command can actually use variables for the SOURCE and DESTINATION. This requires the SOURCE and DESTINATION to be defined in the pipeline configuration file that includes this device. You can find more information in the Pipeline Configuration section further down this page.

devices:
 raspberry-pi:
  transfer: bash -c "sshpass -p raspberry scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r $SOURCE $DESTINATION"
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "#"

Status

The user can optionally include config data to monitor the status of the device. The monitoring is done by spawning a runner (container) in BeetleboxCI (in a similar manner to how jobs are executed) and using it to communicate with the device. When setting this up, a status field is added to the device, and some additional fields within that. The setup field is optional and may be used if some commands need to be run beforehand before it can monitor the status of the device. Most often, this would be the case if certain software or drivers are required for the runner to communicate with the device. The command field contains the actual command(s) that are executed on the device. The pass field is the string that must be detected in the terminal output to confirm that the device has given the expected response. On the devices page, the refresh button can be clicked under the actions section, in order to refresh the status of a device.

The interval field is optional and the user may set this in cron format to periodically monitor the status of the device. The example below sets the status of the device to be checked every hour.

devices:
 raspberry-pi:
  transfer: bash -c "sshpass -p raspberry scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r $SOURCE $DESTINATION"
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "$"
  status:
      setup: |
        apt-get install sshpass -y >/dev/null 2>&1
      command:
        bash -c 'echo Hello World'
      pass:
        "Hello World"        
      interval:
        0 * * * *

Pipeline Configuration with Devices

In order to use the devices we define, we need to create pipelines that include these devices.

Consider one of the devices that were defined earlier (raspberry-pi), where the variables $SOURCE and $DESTINATION were used. If you specifically use the words $SOURCE and $DESTINATION in the transfer field of the device, you can pass values into these variables when you actually run a job.

devices:
 raspberry-pi:
  transfer: bash -c "sshpass -p raspberry scp -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -r $SOURCE $DESTINATION"
  connection: sshpass -p raspberry ssh -o "UserKnownHostsFile=/dev/null" -o "StrictHostKeyChecking=no" pi@192.168.1.92
  prompt: "#"

The example below shows the definition of a job within a pipeline config file, which copies files from a runner (on the server) to the a Raspberry Pi device. The transfer step specifies a SOURCE and a DESTINATION, and the locations defined there are substituted into the $SOURCE and $DESTINATION variables used in the transfer field of the device. The code below transfers all the files in /PATH/TO/SOURCE/FOLDER/ to /PATH/TO/DESTINATION/FOLDER within the device itself.

jobs:   
 hello_world:
  device: raspberry-pi
  steps:
   - transfer:
      name: Copy files
      SOURCE: /PATH/TO/SOURCE/FOLDER/*
      DESTINATION: root@192.168.1.92:/PATH/TO/DESTINATION/FOLDER

This is a another example of a pipeline configuration code snippet showing the SOURCE and DESTINATION being specified when transferring files. Note that there is a prior step installing sshpass, which this example uses to communicate with the device through SSH. The previous example in fact assumed that the runner had all the necessary software to communicate with the device. In reality, it will often be the case that the user will need to install software and/or drivers on the runner for it to communicate with specific devices. After the prerequisite software is installed, the file input-file.tar.gz is transferred to the device. Note that input-file.tar.gz is an artifact that is sent in to this job , and by default the artifacts are placed in the current_working_directory.

jobs:
 run-hw:
  current_working_directory: /PATH/TO/CURRENT/WORKING/DIRECTORY
  device: raspberry-pi
  input: 
   artifact: 
    - input-file.tar.gz
  steps:
   - run:
      name: Setup
      command: apt-get install sshpass
   - transfer:
      name: Transfer
      SOURCE: /PATH/TO/CURRENT/WORKING/DIRECTORY/input-file.tar.gz
      DESTINATION: root@192.168.1.92:/PATH/TO/DESTINATION/FOLDER

When running jobs that include a device, you can explicitly define particular steps to be executed in the terminal of the device. This requires the connection field of the device config to be correctly defined, as the runner will use this to issue commands to the device. Otherwise, the commands of all steps are executed in the runner's internal terminal. When steps are executed on the device, if no errors are observed, the step will pass.

jobs:   
 hello_world:
  device: raspberry-pi
  steps:
   - run:
      name: Say Hello
      command: echo "Hello World"
      on-device: True

BeetleboxCI allows the user to specify certain occurrences where a step running on a device will immediately pass or fail if a certain string is detected in the terminal output upon issuing a particular command. For example, the command below prints Hello World to the terminal, and the pass condition is that Hello World is detected in the terminal output. The fail condition is that rror is detected in the terminal output (this matches both "Error" and "error"). If neither of the conditions are met and no errors occur while the commands are run, the step will be assumed to have succeeded.

jobs:   
 hello_world:
  device: raspberry-pi
  steps:
   - run:
      name: Say Hello
      command: echo "Hello World"
      on-device: True
      pass: Hello World
      fail: rror