BeetleboxCI Enterprise Installation
BeetleboxCI Enterprise is an on-premises CI solution for customers who have large scale computational needs for their CI applications. BeetleboxCI is capable on running on bare metal or on a virtual machine, and it operates on a Kubernetes cluster. Kubernetes is used to schedule user-defined jobs.
Mininum requirements for installing BeetleboxCI Enterprise:
- An x64/x86_64/AMD64/Intel 64 system
- At least 4 CPUs
- At least 16GB RAM
Recommended:
- An x64/x86_64/AMD64/Intel 64 system
- At least 12 CPUs
- At least 32GB RAM
Introduction
This guide describes the process of installing the enterprise version of BeetleboxCI.
Services
BeetleboxCI exposes a number of services:
Frontend web application accessed through a browser. (Port 32767)
A local container registry to store images to run jobs. (Port 5000)
An NFS server to allow other nodes to access shared files (Port 2049)
beetleboxci-db - Stores information about users, jobs etc.
beetleboxci-redis-server - Manages the queue of jobs to be run.
beetleboxci-redis-worker-main - Orchestrates user defined jobs.
beetleboxci-redis-worker-poll - Monitors pipelines for changes.
beetleboxci-redis-scheduler - Handles cron jobs.
beetleboxci-nginx - Proxy that handles web server requests.
beetleboxci-web - Backend web server.
metrics-server - Monitors performance of user-defined jobs.
registry - Docker registry for storing custom container images.
Ports
The application exposes a number of ports:
53 - TCP/UDP - Kubernetes DNS server
179 - TCP - Calico CNI
443 - TCP - Metrics server
2049 - TCP - Network File System
2379 - TCP - etcd API
2380 - TCP - etcd internal communication
5000 - TCP - Docker registry
5432 - TCP - PostgreSQL
6443 - TCP - Kubernetes API server
6379 - TCP - Redis
10250 - TCP - Kubelet API
10251 - TCP - Kube-scheduler API
10252 - TCP - Kube-controller-manager API
32767 - TCP - Web app traffic
Platform
BeetleboxCI Enterprise is designed to be deployed onto a server with a Kubernetes cluster.
Scripted Installation
BeetleboxCI provides a set of scripts through which you can install the application on a server from scratch if your server is running Ubuntu Server 22.04.1. These scripts set up kubernetes for you as well as installing BeetleboxCI using a helm chart.
It is likely that many server setups will require manual installation.
Installing BeetleboxCI
You can download the latest version of the installer here: BeetleboxCI Enterprise.
Before you begin, please ensure that you know the hostname, fully qualified domain name (FQDN) and IP address of each of the servers that will form the cluster.
The hostname may only contain lowercase alphanumeric characters, hyphens and dots. You can find the commands to change the hostname of your machine here: Changing the hostname. Please note that other applications that depend on the current hostname may stop working if the hostname is changed.
On a Linux machine:
- Hostname: The hostname can be obtained by typing the command
hostname
into a Linux terminal. - Fully Qualified Domain Name: The fully qualified domain name would be managed by the network administrator in your organisation if the server is part of a company intranet. It may be of a similar form to
hostname.companyname.com
. For machines not on a network domain, the FQDN would usually behostname.local
. - IP Address: The IP address can be obtained by typing
ifconfig
into a terminal or by runninghostname -I | awk '{print $1}'
, which will usually list several network adapters. The IP address to use is the one that corresponds to the main network interface (Ethernet/WiFi) of your server.
Extract the tarball using a terminal and enter the folder that gets extracted:
tar -xzvf BeetleboxCIv*.tar.gz
cd dist/
tar -xzvf BeetleboxCIv*.tar.gz # Extract the tarball inside this one too
cd BeetleboxCIv*Open the
000_params_example.sh
parameter file and edit the following parameters (you may copy this file to a different location and rename it for future reference):BeetleboxCI application parameters:
Parameter Description CONTROL_NODE_FQDN Fully qualified domain name of the server where the BeetleboxCI application is installed BBX_DEPLOY_NODE Hostname of the server where the BeetleboxCI application is installed BBX_DEPLOY_NODE_IP IP address of the server where the BeetleboxCI application is installed Network File System (NFS) parameters:
Parameter Description NFS_SERVER_FQDN Fully qualified domain name of the server where the NFS is located NFS_SERVER_NAME Hostname of the server where the NFS is located NFS_SERVER_IP IP address of the server where the NFS is located NFS_SERVER On multi-node clusters, we recommend setting this to the same value as the NFS_SERVER_FQDN. On single node clusters, this can be set to the IP of the server if the IP is static or otherwise set to the same value as NFS_SERVER_FQDN. Container Registry parameters:
Parameter Description REGISTRY_FQDN Fully qualified domain name of the server where the container registry is hosted REGISTRY_NODE_NAME Hostname of the server where the container registry is hosted REGISTRY_IP IP address of the server where the container registry is hosted REGISTRY_URL On multi-node clusters, we recommend setting this to the same value as the REGISTRY_FQDN. On single node clusters, this can be set to the IP of the server if the IP is static or otherwise set to the same value as REGISTRY_FQDN or REGISTRY_NODE_NAME. If you are only using a single node cluster, then the BeetleboxCI application, the NFS and the Container Registry will share the same parameter values. During the installation, the parameter file will be copied to the Beetlebox install path (default:
/mnt/beetleboxci
)Run the 00_init_control_node.sh script. You will be prompted for the path of the parameter file which you edited earlier. This script installs the Kubernetes software and sets up a cluster on your machine.
./00_init_control_node.sh
Now you can restart your machine. Restarting is required for the containerisation backend to function correctly, otherwise the following scripts will not work correctly.
Run the following command to check that all the modules are running correctly — they may take a few minutes to stabilise, so you may need to run this command again:
kubectl get po --all-namespaces
Run the
10_install_nfs.sh
script. This installs the software needed for a network file system to run on the machine../10_install_nfs.sh
Run the
11_create_nfs.sh
script. You will be prompted to enter the path of the parameter file again. This sets up a network file system on your machine to allow communication between containers in BeetleboxCI../11_create_nfs.sh
Run the
40_create_certs.
script. You will be prompted to enter the path of the parameter file. This creates the SSL certificates needed to set up the local container registry../40_create_certs.sh
Run the
41_create_registry.sh
script. You will be prompted to enter the path of the parameter file../41_create_registry.sh
Run the
60_install_app.sh
script. You will be prompted to enter the path of the parameter file. This builds and starts the BeetleboxCI application on the Kubernetes cluster. During the installation, the parameter file will be copied to the Beetlebox install path (default:/mnt/beetleboxci
)./60_install_app.sh
To access the application within the (virtual) machine where you installed BeetleboxCI, open a web browser and navigate to http://127.0.0.1:32767. If you need to access the BeetleboxCI web application from another machine (on the same local network) other than the control plane node, then you will need to replace 127.0.0.1 with the IP address or hostname of the control plane node.
If you are accessing the app for the first time, you will be prompted to create a superuser account. You can log in once you create the superuser account.
Upgrading BeetleboxCI
You can download the latest version of the installer here: BeetleboxCI Enterprise.
The tarball contains a self-extracting executable that can be used to upgrade to a newer version, and the steps to upgrade are below:
Extract the tarball using a terminal and enter the folder that gets extracted:
tar -xzvf BeetleboxCIv*.tar.gz
cd dist_self_extracting/Run the
./BeetleboxCIv*.run
file. You will be prompted to enter the path of the parameter file. You can now locate this parameter file in the Beetlebox installation directory (default:/mnt/beetleboxci
)../BeetleboxCIv*.run
Run the following command to check that all the modules are running correctly — they may take a few minutes to stabilise, so you may need to run this command again:
kubectl get po --all-namespaces
You are now ready to use the latest version of BeetleboxCI.
Manual Installation
In some cases, you may already have a server with Kubernetes installed on it, and in other cases, the exact dependencies required by the automated scripts may not be available. In this case, please get in touch with us to discuss.