Running a multi-arch cluster with Rancher/Docker

Just a heads up — this no longer works! We’ll have to go back to the drawing board to update.

I just bought three raspberry pi’s to attempt to cheaply increase the capacity of my home cluster (I’m building something awesome 😃 )… This is a quick write up on what I did and how I did.

I currently run rancher as my orchestration tool of choice. But first, I have to get docker installed on my pi’s.

I followed this guide by hypriot to get started.

Once I got the pi up and running, it was a bit of a challenge to get rancher-agent built for arm. Arm support in rancher is still experimental, so you can follow these instructions or download the pre-built images listed at the bottom.

Solution

If you don’t want to build all the required containers from scratch, you can just follow these simpler instructions on your arm device:

docker pull withinboredom/agent-instance:v0.8.3
docker pull withinboredom/agent:v1.0.2
docker tag withinboredom/agent:v1.0.2 rancher/agent:v1.0.2

Now that you’ve gotten everything downloaded, go ahead and copy the command rancher gives you to add a custom host.

Within a few seconds you should see it appear in your infrastructure tab and you can deploy containers to it. _When you see the network agent_ come online, jump back to your machine and run this, after it fails.

docker tag withinboredom/agent-instance:v0.8.3 rancher/agent-instance:v0.8.3

And bam, everything should work — well, anything that is compiled on an arm processor!

You may need to follow these instructions to be able to view logs and enter containers from the web:

docker exec -it rancher-agent bash
# Then inside the container, run this
cp /usr/bin/nsenter  /var/lib/cattle/bin/nsenter

cadvisor is still broken for the moment … still trying to figure that out.

Build from scratch

On your arm device, we’ll need to build a few pieces from scratch. Depending on how powerful device you have, this may take awhile. First things first, you need to build s6-libs. This is pretty straight forward:

git clone https://github.com/imikushin/s6-builder
cd s6-builder
./build.arm.sh
docker run -it rancher/s6-builder:v2.2.4.3_arm /opt/build.sh

Once you’ve built s6 for arm, you now need to build the rancher-instance:

# We need to build agent-instance
git clone https://github.com/rancher/agent-instance.git
cd agent-instance/
git remote add imikushin https://github.com/imikushin/agent-instance.git

# You can change the version number to reflect what you actually need
git checkout v0.8.3
git fetch imikushin
git merge imikushin/arm --no-ff
./build-image.sh
cd ..

# Now we need to build the actual agent
git clone https://github.com/rancher/rancher.git
cd rancher/agent
git remote add imikushin https://github.com/imikushin/rancher.git
git fetch imikushin
git merge imikushin/multiarch-hosts
# Change the from line at the top of the Dockerfile to armhf/ubuntu:14.04
./build-image.sh

# Now tag the resulting images for release (You can skip this)
docker tag rancher/agent-instance:v0.8.3_arm withinboredom/agent-instance:v0.8.3
docker tag rancher/agent:v1.0.2_arm withinboredom/agent:v1.0.2
docker push withinboredom/agent-instance:v0.8.3
docker push withinboredom/agent:v1.0.2

# And tag for running
docker tag rancher/agent-instance:dev rancher/agent-instance:v0.8.3

# Now go execute the steps from the Solution section above