This blog is to share my experience on installing OpenShift 3.7 on Atomic Host. Since I am using OpenShift Container Platform (supported by Red Hat), there are 2 options for installation. They are the RPM install which is on Red Hat Enterprise Linux (RHEL) and containerized install which is using Atomic Host (Container OS). In version 3.7, installation can be done via a container on Atomic Host, or from an Linux bastion host.
My test used a Linux host to install OpenShift on Atomic host using AWS which is one of the ways to get an Atomic instance provisioned. Atomic host are provisioned via private AMI image for cloud provider account, the AMI image is ami-e9494989 for my test. Here is where you need to register to get access for importing the the private AMI here https://www.redhat.com/en/technologies/cloud-computing/cloud-access.
There are many ways to automate the steps for installation which is not my blog is about. I want to test out how easy or hard to installation OpenShift on a container OS, so I want to test all the steps for the installation manually.
Setting Up on a Cloud Provider
There are few things we need to setup on AWS. A wildcard entry and a public master hostname are required prior to the installation. I used Route53 for adding the A records for both of the requirements.
Per my test, I also had to add a tag to all Atomic instances with key as KubernetesCluster and the value of the key can be anything. The value of the KubernetesCluster key, will be used for parameter openshift_clusterid in the ansible inventory file. Without this tag on the Atomic instances, I will not be able to register the OpenShift node with the cloud provider.
Setting Up Bastion host
Bastion host is a Linux host (RHEL) to run automation scripts to prepare and install OpenShift on all Atomic hosts. This is one of the option to install on Atomic host. I like this option because I can reuse the same bastion host to install more that one cluster.
The step to prepare the bastion host is straight forward.
subscription-manager register
subscription-manager attach --pool=
subscription-manager repos --disable="*"
subscription-manager repos \
--enable="rhel-7-server-rpms" \
--enable="rhel-7-server-extras-rpms" \
--enable="rhel-7-server-ose-3.7-rpms" \
--enable="rhel-7-fast-datapath-rpms"
yum install atomic-openshift-utils -y
Preparation before installation.
Preparation steps are available at https://docs.openshift.com/container-platform/latest/install_config/install/host_preparation.html.
1. Generate SSH key on Bastion host via 'ssh-keygen' as root
2. Distribute the SSH key too all hosts (master and node) using the following command from bastion host:
ssh-copy-id -i ~/.ssh/id_rsa.pub
3. Create a hosts.prepocp file which include all the hostnames for the cluster.
Example is shown below.
[nodes]
ip-172-31-7-15.us-west-2.compute.internal
ip-172-31-5-243.us-west-2.compute.internal
4. Create a ansible-play (openshiftprep.yml) to automate the host preparation.
An Example is: here https://github.com/piggyvenus/examples/blob/master/installAnsibleSample/v3.7/atomic/openshiftprep.yml
5. Execute the ansible playbook which will register, update Atomic host and configure docker on to the added device (/dev/xvdb)
ansible-playbook -i hosts.prepocp openshiftprep.yml
Create Ansible Hosts file for OpenShift Advance Installation
Since we will need to create an inventory file (often refer to ansible hosts file) for OpenShift installation, here is an example of OpenShift Advance Installation for Atomic host on AWS: https://raw.githubusercontent.com/piggyvenus/examples/master/installAnsibleSample/v3.7/atomic/hosts.atomic.template
Download this file and update the corresponding information for installation. Then, save this file as /etc/ansible/hosts.
There are a few lesson learned here as for creating the ansible hosts file. I added the following parameters to get successful installation:
openshift_release=v3.7.23
openshift_image_tag=v3.7.23
openshift_pkg_version=-3.7.23
openshift_clusterid=<value of key KubernetesCluster from AWS Atomic instance>
Setting up Atomic Host For OCP Installation
I learned that there are few extra steps which I had to add to prepare the Atomic installation before OpenShift Installation. These are the steps that I used in my test. I am not an Atomic expert. It does what I wanted it to do.
Besides adding disk for docker, I also need extra disk space for root partition. The OOTB Atomic instance from AWS has only 3GB root partition which is not enough for OpenShift installer. I have to do the following to get my docker and root partition configure to the way I wanted it. My goal is to extend my root partition to have extra disk space and configure docker using the added volume that I attached to the instance.
The preparation script did configure docker to use /dev/xvdb. After running the previous ansible playbook, the following steps were to use to extend my root partition.
ansible all -m shell -a "lvextend -L+50G /dev/mapper/atomicos-root"
ansible all -m shell -a "xfs_growfs /"
ansible all -m shell -a "df -h"
Next is to reboot all hosts via the following command.
systemctl reboot
The following step is to configure docker and startup docker after all hosts were rebooted from the bastion host.
1. Download this ansible playbook
https://raw.githubusercontent.com/piggyvenus/examples/master/installAnsibleSample/v3.7/atomic/openshiftprep2.yml
2. Run ansible playbook using the same hosts.prepocp file as shown below.
ansible-playbook -i hosts.prepocp openshiftprep2.yml
OpenShift Containerized Installation
Once the ansible host (/etc/ansible/hosts) is updated, installation can be started by executing the following command.
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml
note: if the inventory file is /etc/ansible/hosts, no need to specify with "-i" option.
If there is no error after this step, you can access the OpenShift console via https://<your-public-master-hostname>:8443/ and login as any username and password.
Setting up Persistence for Registry for Non-Production
There are many options to setup the persistence for Registry on AWS. Since I only have 1 registry for the single master cluster, I decided to use what gp2 which is configured as default storageclass after the installation. Here are the steps I setup storage for OpenShift insternal registry. I used ReadWriteOnce as access mode because AWSElasticBlockStore volume plugin only support ReadWriteOnce (https://kubernetes.io/docs/concepts/storage/persistent-volumes/)
1. ssh to the Atomic master host
2. /usr/local/bin/oc login -u system:admin
3. oc project default
4. run the following:
oc create -f - <<EOF
{
"apiVersion": "v1",
"kind": "PersistentVolumeClaim",
"metadata": {
"name": "registry-volume-claim",
"labels": {
"deploymentconfig": "docker-registry"
}
},
"spec": {
"accessModes": [ "ReadWriteOnce" ],
"resources": {
"requests": {
"storage": "20Gi"
}
}
}
}
EOF
2. oc volume deploymentconfigs/docker-registry --add --name=registry-storage -t pvc --claim-name=registry-volume-claim --overwrite
Setting up Metrics with Dynamic storage
Since the dynamic provisioning is configured, I used the default gp2 storageclass to configure metics as well. Here are the steps.
1. Add following in /etc/ansible/hosts file on bastion host
openshift_metrics_install_metrics=true
openshift_metrics_hawkular_hostname=hawkular-metrics.<your wildcard suffix>
openshift_metrics_cassandra_storage_type=dynamic
openshift_metrics_image_version=v3.7.23
2. Run the metrics playbook to setup metrics from bastion host
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/openshift-metrics.yml
If the playbook failed, simply uninstall metrics component by setting openshift_metrics_install_metrics=false and re-run the metric playbook.
Setting up Logging with Dynamic Storage
Logging can be configured via ansible playbook as well. I am using the default gp2 storageclass since it provides dynamic provision for the Persistence Volume. Here are the steps.
1. Add following in /etc/ansible/hosts file on bastion host
openshift_logging_install_logging=true
openshift_logging_image_version=v3.7.23
openshift_logging_es_pvc_dynamic=true
openshift_logging_es_pvc_size=30Gi
openshift_logging_es_cluster_size=1
openshift_logging_es_memory_limit=1Gi
2.Run the logging playbook to setup logging from bastion host
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/openshift-logging.yml
If the playbook failed, simply uninstall logging component by setting openshift_logging_install_logging=false and re-run the logging playbook.
Installing from a Container on an Atomic Host Option
Instead of using a bastion host to execute ansible playbook to install on an Atomic host. You can execute the following command to install OpenShift from a container. Here are the steps I tested.
1. atomic install --system \
> --storage=ostree \
> --set INVENTORY_FILE=/root/hosts \
> registry.access.redhat.com/openshift3/ose-ansible:v3.7
Getting image source signatures
Copying blob sha256:9cadd93b16ff2a0c51ac967ea2abfadfac50cfa3af8b5bf983d89b8f8647f3e4
71.41 MB / ? [----------------------------------=-------------------------] 7s
Copying blob sha256:4aa565ad8b7a87248163ce7dba1dd3894821aac97e846b932ff6b8ef9a8a508a
1.21 KB / ? [=------------------------------------------------------------] 0s
Copying blob sha256:7952714329657fa2bb63bbd6dddf27fcf717186a9613b7fab22aeb7f7831b08a
146.93 MB / ? [---------------------------------------------=------------] 16s
Copying config sha256:45abc081093b825a638ec53a19991af0612e96e099554bbdfa88b341cdfcd2e6
4.23 KB / 4.23 KB [========================================================] 0s
Writing manifest to image destination
Storing signatures
Extracting to /var/lib/containers/atomic/ose-ansible-v3.7.0
systemctl daemon-reload
systemd-tmpfiles --create /etc/tmpfiles.d/ose-ansible-v3.7.conf
systemctl enable ose-ansible-v3.7
2. systemctl start ose-ansible-v3.7
3. journalctl -xfu ose-ansible-v3.7
Hope this will help someone to have a successful OpenShift containerized installation.
OpenShift Lessons Learned 