Azure Red Hat OpenShift 4 (ARO 4) integrate with Azure Active Directory

I happened to test out ARO 4 with Azure Active Directory integration. The Azure documentation is good, but I had to change a few while testing the steps. I am sharing my experience here and hope someone will find it useful.

Setting the requirements

Install or update Azure CLI
brew update && brew install azure-cli
Make sure you have permission to create resources in the resource group. I logged in as a global administrator when I am testing this.

Setup the environment variables
$ cat aro-env
LOCATION=centralus. # the location of your cluster
RESOURCEGROUP=aro-rg # the name of the resource group where you want to create your cluster
CLUSTER=poc #cluster-id of the ARO 4 cluster
$ source aro-env
Log in Azure
az login
Create a Resource Group
az group create \
--name $RESOURCEGROUP \
--location $LOCATION
Add DNS zone
If you don’t have a DNS zone already, you can use this step.
  1. Login Azure Portal
  2. Type: “DNS Zones” in the search box on the top and click on “DNS Zones”
  3. Click “+Add” on the top
  4. Select the newly created resource group
  5. Enter your domain
  6. Select the location
  7. Create “Review+Create”

Notes:

  • I am using a domain name outside of the Azure. You will need to add the NS records from the overview page of the DNS zone to your domain.
  • Request increase of quota from Azure portal. ARO requires a minimum of 40 cores.
Register Resource Provider
az account set --subscription
az provider register -n Microsoft.RedHatOpenShift --wait
az provider register -n Microsoft.Compute --wait
az provider register -n Microsoft.Storage --wait
Create a Virtual Network
az network vnet create \
--resource-group $RESOURCEGROUP \
--name aro-vnet \
--address-prefixes 10.0.0.0/22
Create an empty subnet for master nodes
az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--name master-subnet \
--address-prefixes 10.0.0.0/23 \
--service-endpoints Microsoft.ContainerRegistry
Create an empty subnet for worker nodes
az network vnet subnet create \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--name worker-subnet \
--address-prefixes 10.0.2.0/23 \
--service-endpoints Microsoft.ContainerRegistry
Disable private endpoint policy
az network vnet subnet update \
--name master-subnet \
--resource-group $RESOURCEGROUP \
--vnet-name aro-vnet \
--disable-private-link-service-network-policies true
Once the above steps are done. You don’t have to redo the steps if you are going to reuse the names and resources.

Create Cluster

Please make sure you log in to Azure and environment variables are set.

Information that we need for creating a cluster
  • Get a copy of the pull secret from cloud.redhat.com. If you don’t have a user name created, please just register as a user for free.
  • Create an ARO cluster using the following command. Please apply to appropriate values.
    Some values were used in the example are explained as shown below.
    • aro-vnet – the name of virtual network
    • master-subnet – the name of master subnet
    • worker subnet – the name of worker subnet
    • ./pull-secret.txt – the path and pull secret where is located
    • aro.ocpdemo.online – custom domain for the cluster
az aro create \
--resource-group $RESOURCEGROUP \
--name $CLUSTER \
--vnet aro-vnet \
--master-subnet master-subnet \
--worker-subnet worker-subnet \
--pull-secret @./pull-secret.txt \
--domain aro.ocpdemo.online

The information from the JSON output of the above command can be useful if you are not familiar with OpenShift 4. You can find your API server IP, API URL, OpenShift console URL and ingress IP. You will need the API, and ingress IP for the next step.

{- Finished ..
"apiserverProfile": {
"ip": "x.x.x.x",
"url": "https://api.aro.ocpdemo.online:6443/",
"visibility": "Public"
...
},
"consoleProfile": {
"url": "https://console-openshift-console.apps.aro.ocpdemo.online/"
},
....
"ingressProfiles": [
{
"ip": "x.x.x.x",
"name": "default",
"visibility": "Public"
}
....

Post ARO Installation

Adding two A records for api and *.apps in the DNS zone
  1. Login to Azure portal
  2. Go to DNS zone
  3. Click onto the domain for the ARO cluster
  4. Click “+ Record Set” on the top menu to create an A record and add values to Name and IP. You will need to repeat this step for both api and *.apps A records.
    • Name: api or *.apps
    • IP: the *.apps/ingress IP is from the output of the creation of the ARO
  5. The below screenshot shows the DNS zone configuration and adding 2 A records.

Test ARO Cluster

Getting Kubeadmin credential
az aro list-credentials \
--name $CLUSTER \
--resource-group $RESOURCEGROUP
The command will return the kubeadmin credential.
Log in OpenShift Console
Open a browser and go to the OpenShift console or look for “consoleProfile” from the JSON output from ARO creation
https://console-openshift-console.apps.<DNS domain>/
The login user is kubeadmin and the password is the credential from the last command. Congrats!! The ARO installation is completed!

Azure Active Directory Integration

Getting oauthCallBackURL
  • Download OpenShift command line tool from console.
Download the OpenShift Command Lind Interface (CLI) from there. Once you extract it and add to the PATH. You can move on to the next step.
  • Login to ARO via OC CLI
$ oc login -u kubeadmin -p <password> https://api.<DNS domain>:6443/

$ oauthCallBack=`oc get route oauth-openshift -n openshift-authentication -o jsonpath='{.spec.host}'`

$ oauthCallBackURL=https://$oauthCallBack/oauth2callback/AAD
Note: AAD is the name of the identity provider when configuring OAuth on OpenShift

Creating Application on Azure Active Directory
az ad app create \
  --query appId -o tsv \
  --display-name poc-aro-auth \
  --reply-urls $oauthCallBackURL \
  --password '<ClientSecret>'
Note: Please note that the above command returns the registered Application Id (AppId) which you will need it when configuring the OAuth on OpenShift.
Get tenant Id
az account show --query tenantId -o tsv
Note: Please note that you will need the tenant Id for the OAuth configuration on OpenShift
Create manifest file
cat > manifest.json<< EOF
[{
"name": "upn",
"source": null,
"essential": false,
"additionalProperties": []
},
{
"name": "email",
"source": null,
"essential": false,
"additionalProperties": []
}]
EOF
Update the Azure Active Directory with a manifest
az ad app update \
--set optionalClaims.idToken=@manifest.json \
--id <AppId>
Update Application permission scope
az ad app permission add \
--api 00000002-0000-0000-c000-000000000000 \
--api-permissions 311a71cc-e848-46a1-bdf8-97ff7156d8e6=Scope \ 
--id <AppId>
Grant admin consent
  1. login Azure portal
  2. Go to Azure Active Directory
  3. Click App Registrations
  4. Click “All Application” and search for newly create application name
  5. Click onto the display name of the application
  6. Click view API permissions
  7. Click on the “check” to grant admin consent for directory
Add service principal
$ az ad sp create-for-rbac --role Contributor --name poc-aro-sp
You will need the “appId” from the output of the above command and that is the appId for the service principal
$az role assignment create --role "User Access Administrator" \
--assignee-object-id $(az ad sp list --filter "appId eq '<service-principal-appid>'" \
| jq '.[0].objectId' -r)
$az ad app permission add --id <appId> \ 
--api 00000002-0000-0000-c000-000000000000 \ 
--api-permissions 824c81eb-e3f8-4ee6-8f6d-de7f50d565b7=Role
This will output the follow command as shown below.
$az ad app permission grant --id <appid> --api 00000002-0000-0000-c000-000000000000
I also grant the admin consent for the API permission for the service principal.
Create secret for identity provider on OpenShift
oc create secret generic openid-client-secret-azuread \
--namespace openshift-config \
--from-literal=clientSecret=<your password>
Create YAML for identity provider for AAD
apiVersion: config.openshift.io/v1
kind: OAuth
metadata:
  name: cluster
spec:
  identityProviders:
  - name: AAD
    mappingMethod: claim
    type: OpenID
    openID:
      clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
      clientSecret:
        name: openid-client-secret-azuread
      extraScopes:
      - email
      - profile
      extraAuthorizeParameters:
        include_granted_scopes: "true"
      claims:
        preferredUsername:
        - email
        - upn
        name:
        - name
        email:
        - email
      issuer: https://login.microsoftonline.com/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Note:
  • The clientID is the AppId of your registered application.
  • Issuer URL is https://login.microsoftonline.com/<tenant id>.
  • The clientSecret is using the secret (openid-client-secret-azuread) that you created from the previous step.
Alternatively, you can obtain the clientID and tenant id from Azure Portal.
  • Login Azure Portal
  • Click Home
  • Click Azure Active Directory
  • Click App registrations on the left menu
  • Click all applications tab
  • Type the application that you just created in the search area
  • Click onto the application (my application is poc-aro-auth)
  • Under Overview, the information is shown as “Application (client) ID” and Directory (tenant) ID” as in the image below.
Update OpenShift OAuth Configuration
oc apply -f openid.yaml
Login OpenShift console via AAD
It will redirect you to Azure login page

Troubeshoot

Tip #1: If you are getting error, you can login as kubeadmin and check the logs from oauth-openshift pods under openshift-authentication project.

Tip #2: if you are creating a new registered application to try, make sure you clean up the user and identity.

Reference

Azure OpenShift 4 documentation

ARO and Azure Active Directory integration

One thought on “Azure Red Hat OpenShift 4 (ARO 4) integrate with Azure Active Directory

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s