Running Virtual Machine on ROSA HCP

Out of curiosity, I want to see if I can run a virtual machine on my ROSA HCP cluster.

Create ROSA HCP

OCP 4.16.2 is now available on ROSA HCP. I created ROSA HCP cluster 4.16.2 for this test. Since I am following the ROSA documentation to create the ROSA HCP cluster, I share my commands here on how I create the cluster for this test. Please refer to the “Reference” section for the details.

$ rosa create account-roles --hosted-cp
$ export ACCOUNT_ROLES_PREFIX=ManagedOpenShift
$ rosa create oidc-config --mode=auto --yes
$ export OIDC_ID=xxxxxxxx
$ export OPERATOR_ROLES_PREFIX=demo
$ rosa create operator-roles --hosted-cp --prefix=$OPERATOR_ROLES_PREFIX --oidc-config-id=$OIDC_ID --installer-role-arn arn:aws:iam::${AWS_ACCOUNT_ID}:role/${ACCOUNT_ROLES_PREFIX}-HCP-ROSA-Installer-Role
$ rosa create cluster --sts --oidc-config-id 2ci6ntk6g92bq7qm21pvhfff1fp07li1 --operator-roles-prefix demo --hosted-cp --subnet-ids $SUBNET_IDS

After my cluster installation completes, log into Red Hat Hybrid Cloud Console to configure access for the cluster.

• Go to https://console.redhat.com/openshift, the ROSA HCP cluster should show “Ready” state.

• Click on the cluster name -> click on the “Access control” tab -> select htpasswd as the IDP to add a user

• Click Add after entering the user information

• Click “Add user” to add a cluster-admin as shown below

• Go to the Network tab -> click “open console” and log in to the ROSA HCP cluster.

Install OpenShift Virtualization Operator

• Once you log in as cluster admin to the OpenShift console -> Click Operators -> OperatorHub -> click OpenShift Virtualization -> Click “Install”
• Click the “Installed Operators” on the left nav -> make sure the status show as “Succeed” for OpenShift Virtualization Operator.

• Click “OpenShift Virtualization” -> OpenShift Virtualization Deployment -> Create HyperConverged CR using the YAML as shown below.

apiVersion: hco.kubevirt.io/v1beta1
kind: HyperConverged
metadata:
  name: kubevirt-hyperconverged
  namespace: openshift-cnv
  annotations:
    deployOVS: "false"
  labels:
    app: kubevirt-hyperconverged
spec:
  applicationAwareConfig:
    allowApplicationAwareClusterResourceQuota: false
    vmiCalcConfigName: DedicatedVirtualResources
  certConfig:
    ca:
      duration: 48h0m0s
      renewBefore: 24h0m0s
    server:
      duration: 24h0m0s
      renewBefore: 12h0m0s
  evictionStrategy: LiveMigrate
  featureGates:
    alignCPUs: false
    autoResourceLimits: false
    deployKubeSecondaryDNS: false
    deployTektonTaskResources: false
    deployVmConsoleProxy: false
    disableMDevConfiguration: false
    enableApplicationAwareQuota: false
    enableCommonBootImageImport: true
    enableManagedTenantQuota: false
    nonRoot: true
    persistentReservation: false
    withHostPassthroughCPU: false
  infra: {}
  liveMigrationConfig:
    allowAutoConverge: false
    allowPostCopy: false
    completionTimeoutPerGiB: 800
    parallelMigrationsPerCluster: 5
    parallelOutboundMigrationsPerNode: 2
    progressTimeout: 150
  resourceRequirements:
    vmiCPUAllocationRatio: 10
  uninstallStrategy: BlockUninstallIfWorkloadsExist
  virtualMachineOptions:
    disableFreePageReporting: false
    disableSerialConsoleLog: true
  workloadUpdateStrategy:
    batchEvictionInterval: 1m0s
    batchEvictionSize: 10
    workloadUpdateMethods:
    - LiveMigrate
  workloads: {}

Create Bare Metal MachinePool (with IMDSv2)

• Go to https://console.redhat.com/openshift and access the ROSA HCP cluster -> click “MachinePools” -> click “Add Machine Pool”

• Enter a name, select the subnet, select m5zn.metal as the instance type, and add a label (type=metal). You will need to use the same label when creating VMs in the later step.

Making sure the bare metal EC2 instance is up

When the machine pool was first created, I saw the metal node was terminated. I enabled the IMDSv2 on the metal node, and the node can start.

Update Notes (09/2024): using ROSA CLI 1.2.43+, you can use ROSA CLI to create the machine pool with the flag --ec2-metadata-http-tokens=required. Then you will enable IMDSv2 at the creation. An Example of the command to create a machine pool via ROSA CLI is shown below.

rosa create machinepool --cluster=rosa-hcp --name=virt-mp   --replicas=1  --instance-type=m5zn.metal --ec2-metadata-http-tokens=required

Create a VM

Once the bare metal node is up, and the OpenShift virtualization operator is installed and configured successfully. You are ready to create a VM.

• Go to the OpenShift console, select “Overview” under Virtualization on the left menu -> click “Create VirtualMachine”

• Create a new project and give a new of your project

• Click “Template catalog” -> Fedora VM

• Click “Customize VirtualMachine”

• Click YAML tab and add nodeSelector with Label “type: metal”

• Click “Create VirtualMachine”
• The VirtualMachine should be in running in a few minutes.

Reference:

• Installing ROSA HCP: https://docs.openshift.com/rosa/rosa_hcp/rosa-hcp-sts-creating-a-cluster-quickly.html
• Mobb blog about deploying OpenShift virtualization on ROSA: https://cloud.redhat.com/experts/rosa/ocp-virt/basic/
• ROSA virtualization documentations: https://docs.openshift.com/rosa/virt/install/preparing-cluster-for-virt.html
• Download ROSA CLI: https://docs.redhat.com/en/documentation/red_hat_openshift_service_on_aws/4/html/rosa_cli/rosa-get-started-cli#rosa-get-started-cli