Volumes

A voume is a directory mount from the node which a container is running on. The user defines any volumes to be mounted in the .bbx/config.yaml file.

Overview

The user may require containers to run very large software or have access to large datasets. Kubernetes, the platform that BeetleboxCI is built on, is optimised for smaller containers. Running multiple jobs simultaneously with large containers (>50GB) may become prohibitive in terms of disk usage. We recommend users to avoid using or creating containers larger than 50GB in size. In order to prevent containers becoming too large, BeetleboxCI allows users to mount directories from a node directly into a container, rather than having everything stored within the container.

Defining Volumes within Jobs

To use volumes within jobs, the user can include the volumes field, and then list the directories from the node to be mounted within the container. The container will use the exact same path for the directories inside the container, as on the node.

jobs:   
 random-job:
  volumes:
   - mount:
      name: volume1
      path: /tmp/firstvolume
   - mount:
      name: volume2
      path: /tmp/secondvolume  
   - mount:
      name: volume3
      path: /opt/thirdvolume        

Caveats

All user-defined volumes are read only, in order to avoid corrupting data on the node where the container lives.

It must be noted that while you can mount folders from the /tmp directory on a node, mounting the /tmp directory directly is not supported—doing so will cause the job to fail. The /tmp directory within job containers is used by BeetleboxCI to store files, and mounting a volume in that directory will set that directory as read only, and prevent BeetleboxCI from creating temporary files in that directory.

Other directories on the node may be mounted as long as they have sufficient read permissions.