Red Hat OpenShift on Amazon (ROSA) is GA!

I have previously blogged about the pre-GA ROSA, and now it is GA. I decided to write up my GA experience on ROSA.

Let’s get started here.

Enable ROSA on AWS

After logging into AWS, enter openshift in the search box on the top of the page.

Click on the “Red Hat OpenShift Service on AWS” Service listed.

It will then take you to a page as shown below and click to enable the OpenShift service.

Once it is complete, it will show Service enabled.

Click to download the CLI and click on the OS where you run your ROSA CLI. It will start downloading to your local drive.

Set up ROSA CLI

Extract the downloaded CLI file and rosa add to your local path.

tar zxf rosa-macosx.tar.gz
mv rosa /usr/local/bin/rosa

Setting AWS Account

I have set up my AWS account as my IAM user account with proper access per the documentation. There is more information about the account access requirements for ROSA. It is available here.

I have configured my AWS key and secret in my .aws/credentials.

Create Cluster

Verify AWS account access.

rosa verify permissions

Returns:

I: Validating SCP policies...
I: AWS SCP policies ok

Verify the quota for the AWS account.

rosa verify quota --region=us-west-2

Returns:

I: Validating AWS quota...
I: AWS quota ok

Obtain Offline Access Token from the management portal cloud.redhat.com (if you don’t have one yet) by clicking Create One Now link

Go to https://cloud.redhat.com/openshift/token/rosa, and you will have to log in and prompt to accept terms as shown below.

Click View Terms and Conditions.

Check the box to agree the terms and click Submit.

Copy the token from cloud.redhat.com.

rosa login --token=<your cloud.redhat.com token>

Returns:

I: Logged in as 'your_username' on 'https://api.openshift.com'

Verify the login

rosa whoami

Returns:

AWS Account ID:               ############
AWS Default Region:           us-west-2
AWS ARN:                      arn:aws:iam::############:user/username
OCM API:                      https://api.openshift.com
OCM Account ID:               xxxxxyyyyyzzzzzwwwwwxxxxxx
OCM Account Name:             User Name
OCM Account Username:         User Name
OCM Account Email:            name@email.com
OCM Organization ID:          xxxxxyyyyyzzzzzwwwwwxxxxxx
OCM Organization Name:        company name
OCM Organization External ID: 11111111

Configure account and make sure everyone setup correctly

rosa init

Returns

I: Logged in as 'your_username' on 'https://api.openshift.com'
I: Validating AWS credentials...
I: AWS credentials are valid!
I: Validating SCP policies...
I: AWS SCP policies ok
I: Validating AWS quota...
I: AWS quota ok
I: Ensuring cluster administrator user 'osdCcsAdmin'...
I: Admin user 'osdCcsAdmin' already exists!
I: Validating SCP policies for 'osdCcsAdmin'...
I: AWS SCP policies ok
I: Validating cluster creation...
I: Cluster creation valid
I: Verifying whether OpenShift command-line tool is available...
I: Current OpenShift Client Version: 4.7.2

Creating cluster using interactive mode.

rosa create cluster -i
I: Interactive mode enabled.
Any optional fields can be left empty and a default will be selected.
? Cluster name: [? for help] 

Enter the name of the ROSA cluster.

? Multiple availability zones (optional): [? for help] (y/N) 

Enter y/N.

? AWS region:  [Use arrows to move, type to filter, ? for more help]
  eu-west-2
  eu-west-3
  sa-east-1
  us-east-1
  us-east-2
  us-west-1
> us-west-2 

Select the AWS region and hit <enter>.

? OpenShift version:  [Use arrows to move, type to filter, ? for more help]
> 4.7.2
  4.7.1
  4.7.0
  4.6.8
  4.6.6
  4.6.4
  4.6.3

Select the version and hit <enter>.

? Install into an existing VPC (optional): [? for help] (y/N)

Enter y/N.

? Compute nodes instance type (optional):  [Use arrows to move, type to filter, ? for more help]
> r5.xlarge
  m5.xlarge
  c5.2xlarge
  m5.2xlarge
  r5.2xlarge
  c5.4xlarge
  m5.4xlarge

Select the type and hit <enter>.

? Enable autoscaling (optional): [? for help] (y/N)

Enter y/N.

? Compute nodes: [? for help] (2)

Enter the numbers of workers to start.

? Machine CIDR: [? for help] (10.0.0.0/16)

Enter the machine CIDR or use default.

? Service CIDR: [? for help] (172.30.0.0/16)

Enter the service CIDR or use default.

? Pod CIDR: [? for help] (10.128.0.0/14)

Enter the pod CIDR or use default.

? Host prefix: [? for help] (23)

Enter the host prefix or use default

? Private cluster (optional): (y/N) 

Enter y/N.

Note:

Restrict master API endpoint and application routes to direct, private connectivity. You will not be able to access your cluster until you edit network settings in your cloud provider. I also learned that you would need one private subnet and one public subnet for each AZ for your existing private VPC for the GA version of ROSA. There will be more improvement to provide for the private cluster in the future release.

Returns:

I: Creating cluster 'rosa-c1'
I: To create this cluster again in the future, you can run:
   rosa create cluster --cluster-name rosa-c1 --region us-west-2 --version 4.7.2 --compute-nodes 2 --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23
I: To view a list of clusters and their status, run 'rosa list clusters'
I: Cluster 'rosa-c1' has been created.
I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
I: To determine when your cluster is Ready, run 'rosa describe cluster -c rosa-c1'.
I: To watch your cluster installation logs, run 'rosa logs install -c rosa-c1 --watch'.
Name:                       rosa-c1
ID:                         xxxxxxxxxxyyyyyyyyyyyaxxxxxxxxx
External ID:
OpenShift Version:
Channel Group:              stable
DNS:                        rosa-c1.xxxx.p1.openshiftapps.com
AWS Account:                xxxxxxxxxxxx
API URL:
Console URL:
Region:                     us-west-2
Multi-AZ:                   false
Nodes:
 - Master:                  3
 - Infra:                   2
 - Compute:                 2 (m5.xlarge)
Network:
 - Service CIDR:            172.30.0.0/16
 - Machine CIDR:            10.0.0.0/16
 - Pod CIDR:                10.128.0.0/14
 - Host Prefix:             /23
State:                      pending (Preparing account)
Private:                    No
Created:                    Mar 30 2021 03:10:25 UTC
Details Page:               https://cloud.redhat.com/openshift/details/xxxxxxxxxxyyyyyyyyyyyaxxxxxxxxx
 

Copy the URL from the Details Page to the browser and click view logs to see the status of the installation.

When ROSA is completed, you will see the similar page as below.

You will need to access the OpenShift cluster.

Configure Quick Access

Add cluster-admin user

rosa create admin -c rosa-c1

Returns:

I: Admin account has been added to cluster 'rosa-c1'.
I: Please securely store this generated password. If you lose this password you can delete and recreate the cluster admin user.
I: To login, run the following command:

   oc login https://api.rosa-c1.xxxx.p1.openshiftapps.com:6443 --username cluster-admin --password xxxxx-xxxxx-xxxxx-xxxxx

I: It may take up to a minute for the account to become active.

Test user access

$ oc login https://api.rosa-c1.xxxx.p1.openshiftapps.com:6443 --username cluster-admin --password xxxxx-xxxxx-xxxxx-xxxxx
Login successful.

You have access to 86 projects, the list has been suppressed. You can list all projects with 'oc projects'

Using project "default".

Configure Identity Provider

There are options for identity providers. I am using Github in this example.

I am not going to explain how to get identity to provide set up here. I did that in last blog. I will walk through the step to configure ROSA using Github.

rosa create idp --cluster=rosa-c1 -i
I: Interactive mode enabled.
Any optional fields can be left empty and a default will be selected.
? Type of identity provider:  [Use arrows to move, type to filter]
> github
  gitlab
  google
  ldap
  openid

Select one IDP

? Identity provider name: [? for help] (github-1)

Enter the name of the IDP configured on the ROSA

? Restrict to members of:  [Use arrows to move, type to filter, ? for more help]
> organizations
  teams

Select organizations

? GitHub organizations:

Enter the name of the organization. My example is `sc-rosa-idp`

? To use GitHub as an identity provider, you must first register the application:
  - Open the following URL:
    https://github.com/organizations/sc-rosa-idp/settings/applications/new?oauth_application%5Bcallback_url%5D=https%3A%2F%2Foauth-openshift.apps.rosa-c1.0z3w.p1.openshiftapps.com%2Foauth2callback%2Fgithub-1&oauth_application%5Bname%5D=rosa-c1&oauth_application%5Burl%5D=https%3A%2F%2Fconsole-openshift-console.apps.rosa-c1.0z3w.p1.openshiftapps.com
  - Click on 'Register application'

Open a browser and use the above URL to register the application and cope client ID

? Client ID: [? for help] 

Enter the copied Client ID

? Client Secret: [? for help]

Enter client secret from the registered application.

? GitHub Enterprise Hostname (optional): [? for help] 

Hit <enter>

? Mapping method:  [Use arrows to move, type to filter, ? for more help]
  add
> claim
  generate
  lookup

Select claim

I: Configuring IDP for cluster 'rosa-c1'
I: Identity Provider 'github-1' has been created.
   It will take up to 1 minute for this configuration to be enabled.
   To add cluster administrators, see 'rosa create user --help'.
   To login into the console, open https://console-openshift-console.apps.rosa-c1.xxxx.p1.openshiftapps.com and click on github-1.

Congratulation! IDP configuration is completed.

Login in with IDP account

Open a browser with the URL from the IDP configuration. Our example is: https://console-openshift-console.apps.rosa-c1.xxxx.p1.openshiftapps.com.

Click github-1

Click Authorize sc-rosa-idp

Overall, it is straightforward to get started on creating a ROSA cluster on AWS. I hope this will help you in some ways.

Reference

Red Hat OpenShift on Amazon Documentation

Running ROSA on an Existing Private VPC

Pre-GA ROSA test

Installing OCP 3.7 on Azure

My goal of this blog is to share my experience on installing OCP 3.7 on Azure. With the information provided here, hope I can save you some time and have a successful installation on Azure. The exercise was to install OCP on Azure and enable dynamic provisioning as well. Here is the list that you may want to pay more attention when you plan your installation.

Use Azure instance name as the value of openshift_hostname in your inventory file

Assuming you are using OpenShift advanced installation on Azure, we will need to configure openshift_hostname in the inventory file to match the instance name in your Azure portal. The azure instance name is the name of your virtual machine.

Host Configuration

When creating virtual machines in your resource group, it will depends if you are using Azure domain or custom domain of yours. If you are not using the Azure domain, you can set your domain on you VM via the following command:

hostnamectl set-hostname <FQDN of the host>

Setup NetworkManager

Once the ansible inventory file was created, execute the following steps to make sure the Network Manager is working properly.

ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/openshift-node/network_manager.yml

Configure resolv.conf on all hosts

export DOMAIN=`domainname -d`

ansible all -m shell -a "systemctl restart NetworkManager"

ansible all -m shell -a "nmcli con modify eth0 ipv4.dns-search $DOMAIN"

ansible all -m shell -a "systemctl restart NetworkManager"

Create service principal

This is a required step to setup Azure dynamic provision storage. There are few option for creating service principal. An example is shown below.  Please see https://github.com/microsoft/openshift-container-platform for more details around it.

az ad sp create-for-rbac -n <friendly name> --password <password> --role contributor --scopes /subscriptions/<subscription_id>

Location in azure.conf is not optional for managed and unmanaged disk

If you are using managed disk option, you would want to make sure location is also in your azure.conf.

Here is the example of /etc/azure/azure.conf.

tenantId: xxxxxxxxxx
subscriptionId: xxxxxxxxxxxxxx
aadClientId: xxxxxxxxxx
aadClientSecret: xxxxx
aadTenantId: xxxxxxxxxx
resourceGroup: xxxx
cloud: AzurePublicCloud
location: westus

Did not need to remove the node

The registration is automated. Once the installation completed, adding /etc/azure/azure.conf on all nodes

Update the master-config.yml with the following information:

kubernetesMasterConfig:
  apiServerArguments:
    cloud-config:
    - /etc/azure/azure.conf
    cloud-provider:
    - azure
    runtime-config:
    - apis/settings.k8s.io/v1alpha1=true
    storage-backend:
    - etcd3
    storage-media-type:
    - application/vnd.kubernetes.protobuf
  controllerArguments:
    cloud-provider:
    - azure
    cloud-config:
    - /etc/azure/azure.conf

restart atomic-openshift-master-api and atomic-openshift-master-controllers on the master via following commands:

systemctl restart atomic-openshift-master-api

systemctl restart atomic-openshift-master-controllers

Update all node-config.yaml as below

kubeletArguments:
  cloud-config:
  - /etc/azure/azure.conf
  cloud-provider:
  - azure

restart atomic-openshift-node on the nodes via following command

systemctl restart atomic-openshift-node

While you are starting up the atomic-openshift node, you can watch the log by running on the node

journalctl -u atomic-openshift-node -f

You should see the node get remove and re-register back with the cloud provider. There is no need to delete the node. If  the below command whiling restart the atomic-node-service, you will see the specific node get removed and added back to the cluster after sometime.

oc get nodes -w

Creating the correct storage type for your environment

You can validate your disk option before creating storageclass. If the type of storageclass does not match, the pod will throw error as it start to mount to the volume.

At the VM creation, the managed and unmanaged disk options are available.  I learned that we will need location string in the /etc/azure/azure.conf for managed disk option. It looks like RWX is not an option for Azure disk volume plug-in. There are the storageclass for unmanaged disk (top) and managed disk (bottom) I tried.

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: azure-storageclass
annotations:
  storageclass.beta.kubernetes.io/is-default-class: "true"
provisioner: kubernetes.io/azure-disk
parameters:
  storageAccount: xxxxxxxx

 

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: slow2
provisioner: kubernetes.io/azure-disk
parameters:
  storageaccounttype: Standard_LRS  
  kind: Managed
  location: westus

 To set the storageclass as default

oc annotate storageclass azure-storageclass storageclass.beta.kubernetes.io/is-default-class="true"

Now, you should be able to install an OpenShift cluster on Azure with Azure storage as default storageclass.