Jenkins Build Slave
In a Jenkins based CI/CD pipeline the Jenkins slaves are the work horses. A slave machine is configured to perform variety of tasks – compilation of application source code, executing playbooks and running a wide range of automation tasks.
The Kurian Jenkins slave AMI is preloaded with most of the popular tools that are required to run such automations tasks:
- Git Client
- Java OpenJDK 8 as the default Java version
- OpenJDK releases from 9 through 14.
- C/C++ compilers
- Go compiler
- Python 2.7 and 3.x
- AWS CLI
All the tools are installed following official guidelines so you don’t need to deal with any custom environment and you can build on the baseline build platform available on this AMI.
The build slaves spun up from this AMI work the best with Jenkins server created from Kurian’s AMI for Jenkins Server. AMIs are available for all the major Linux distributions.
All the compilers, interpreters and other tools that are normally used by build and automation jobs are installed on the AMI.
- OpenJDK is the default version of Java available on the AMI.
- The OpenJDK versions 9 and above are installed under
/opt/java. For example, release 9 is available under
/opt/java/jdk-9. For the actual path and related info check under
/opt/javaand point JAVA_HOME to appropriate path as needed.
g++ are available on the AMI and the related standard header files.
Latest version of Go is installed and available in the system path. Run
go version to see the version info.
The default version of Python available on the AMI is Python 3. Python 2.7 is also available on the AMI and it may have to be accessed as
Please note the Python paths are slightly different on each Linux distribution as we didn’t make any attempt to standardize to avoid customization.
Latest version of Node.js executable is available in the system path.
Git client is installed and available in the system PATH as
The system environment variable
ANT_HOME points to the install location of Apache Ant.
Latest version of Maven is installed and it can be accessed as
mvn from the system path.
Gradle and Yarn
Latest stable releases of
Yarn are installed. The system environment variable
GRADLE_HOME points to the install location.
A recent version of Ansible is available on the AMI. To the see the installed version and the Python interpreter it uses, run
The AWS CLI command
aws can be accessed as
/usr/local/bin/aws depending on the Linux distribution.
The latest, stable version of Docker is available on the AMI for Docker based builds.
A recent version of the
kubectl command-line tool is installed for Jenkins jobs that need to access Kubernetes clusters.
For SSH connectivity the headless user
jenkins is setup with the home directory
/var/jenkins. This user has passwordless sudo access on the system.
How It Works
In a Jenkins based CI/CD pipeline, the Jenkins jobs can be configured to run on a slave host, and that is the best practice. On the Jenkins server side the build slaves are called nodes. A node can be configured to work with Jenkins server different ways and for the details look at the section on configuration.
Click on the links below to take you to Amazon Marketplace for spinning up EC2 nodes.
- Jenkins Build Slave AMI for Amazon Linux 2018
- Jenkins Build Slave AMI for Amazon Linux 2
- Jenkins Build Slave AMI for CentOS 7
- Jenkins Build Slave AMI for CentOS 8
- Jenkins Build Slave AMI for Red Hat Enterprise Linux 7
- Jenkins Build Slave AMI for Red Hat Enterprise Linux 8
- Jenkins Build Slave AMI for Fedora 31
- Jenkins Build Slave AMI for Debian 10
- Jenkins Build Slave AMI for Ubuntu 16.04
- Jenkins Build Slave AMI for Ubuntu 18.04
- Jenkins Build Slave AMI for openSUSE 15
- Jenkins Build Slave AMI for SUSE Linux Enterprise Server 12
- Jenkins Build Slave AMI for SUSE Linux Enterprise Server 15
Kurian releases latest, stable versions of open source software on popular Linux distributions. If you need a configuration that is not currently available in marketplace, send us your requirements to firstname.lastname@example.org. Please include the following info in the email:
- Name of Software to be on the AMI with version numbers.
- Linux distribution and the version.
- The EC2 instance type you plan to spin up from the AMI (optional).
A host can be configured as a Jenkins slave by running
agent.jar on it and connecting the agent service to the Jenkins server. The connectivity could be initiated from Jenkins server or from the slave. There are multiple methods available and few popular methods are summarized here. The Kurian Jenkins Build Slave is compatible with any method you prefer.
After spinning up the EC2 node using this AMI use one of the following methods to configure it as a build node on the Jenkins server. The Jenkins servers built from Kurian AMIs are the best suited for building CI/CD infrastructure.
Launch agent on slave node via SSH
Generate a SSH key pair without passphrase. (It’s OK to have a passphrase actually but the setup would little easier without one. It’s up to your local security norms.)
An SSH credential to access slave node from Jenkins server must be setup using the private key generated in the last step. For this, navigate to Jenkins -> Credentials -> System and click
SSH Username with private keyas
- In the
Private Keyfield, use
Enter directlyoption to add the content of private key.
In the new node creation window, choose these options:
Remote root directory.
Launch agents via SSHas
- Use private IP or public IP as
Hostdepending on how Jenkins server to slave node SSH access is configured.
- Select the credential created in the previous step (or existing) as
Manually trusted key Verification Strategyas
Host Key Verification Strategy.
Launch agent on slave by connecting it to the server
This allows the agent running on a slave node to be connected to the Jenkins master whenever it is ready. This means that the agent need not be reachable from the Jenkins server; the agent just needs to be able to reach the server. If you have enabled security via the Configure Global Security page, you can customize the port on which the Jenkins master will listen for incoming agent connections.
- In the
Configure Global Securitypage, under
TCP port for inbound agentsselect Fixed or Random port for inbound access from the agent. Make sure network access is provisioned for this inbound access accordingly.
- Create a new node with
Launch methodselected as
Launch agent by connecting it to the master. When clicked on the newly created node, you will be presented with multiple options as in the picture below:
- SSH into the slave node and as user
$ cd /var/jenkins $ curl http://JENKINS-SERVER-IP:8080/jnlpJars/agent.jar -o agent.jar $ echo SECRET > SECRET-FILE $ java -jar /var/jenkins/agent.jar -jnlpUrl http://JENKINS-SERVER-IP:8080/computer/NODE-NAME/slave-agent.jnlp -secret @/var/jenkins/SECRET-FILE -workDir "/var/jenkins"
Verify that the agent is able to connect to Jenkins server successfully. To run the agent process on the slave node
agent.jar can be run as a background process as follows:
$ nohup java -jar /var/jenkins/agent.jar -jnlpUrl http://JENKINS-SERVER-IP:8080/computer/NODE-NAME/slave-agent.jnlp -secret @/var/jenkins/SECRET-FILE -workDir "/var/jenkins" &
Automatically provisioned build nodes using Amazon EC2 plugin
For detailed steps, refer this official documentation: https://plugins.jenkins.io/ec2/. Make sure that it’s installed on the Jenkins server. This plugin is preinstalled in Kurian’s Jenkins Server AMIs.
These are the important points to note:
- Add a new provisioner by selecting Jenkins -> Configure Clouds -> Amazon EC2
- Have a Jenkins Credential item to store an AWS keys set that has privileges to spin up EC2 nodes and configure security groups. Select that as
Amazon EC2 Credentials.
EC2 Key Pair's Private Keymust correspond to the SSH key pair that will be used for communicating between Jenkins server and build slave node.
AMI IDmust be the id of Kurian Jenkins Build Slave AMI.
Security group namesshould have AWS security group(s) that opens SSH accessing from Jenkins server to slave node on port 22.
Remote usermust be
Remote FS rootmust be
/var/jenkins. Both these are preconfigured already on the AMI.
User Datato inject the public key corresponds to
EC2 Key Pair's Private Key. Use following command-line:
#!/bin/bash echo "PUBLIC-KEY" >> ~jenkins/.ssh/authorized_keys chown jenkins:jenkins ~jenkins/.ssh/authorized_keys chmod 600 ~jenkins/.ssh/authorized_keys
PUBLIC-KEYwith actual content of the public key.
Security groups for slave hosts must be configured to open the network inbound connection from Jenkins server.
Remote SSH access for
jenkinsuser on the slave nodes must be ensured, such as configuring SELINUX settings on Red Hat Linux distributions.
Connection Strategywould depend on how the network access between Jenkins server host and the slave node is provisioned. The best practice is using
Private IPas that is more secure.
As a best practice, in-place update of a Jenkins slave node is discouraged. The
agent.jar file is downloaded from or pushed by (depending on the configuration method used) Jenkins server, and make sure the
agent.jar runs on the slave node is compatible.
Maintenance and Troubleshooting
Kurian can help with configuring Jenkins for your specific needs. Contact us with details to email@example.com
Remote Access to EC2 Node
The communication between Jenkins master and slave host is over ssl and port 22 must be opened on the slave host for the Jenkins master.
The headless user is needed on the slave host for the communication between Jenkins master and slave. On the AMI
jenkins has been setup for that purpose.
In addition, following are the standard AWS defined users available when a node is provisioned:
- Amazon Linux: ec2-user
- Red Hat Linux: ec2-user
- SuSE Linux: ec2-user
- Fedora: fedora
- CentOS: centos
- Debian Linux: admin
- Ubuntu: ubuntu