Create a CentOS Virtual Machine Using #Azure Resource Manager (ARM)

Creating a CentOS VM Using ARM

Docker and Open Source projects are getting lots of attention, so I decided that it was time for me to build a Linux Virtual Machine on Microsoft Azure. This post is all about creating an Azure Resource Manager Template for a CentOS Virtual Machine with two stripped Data Disks. This template should be used as a starting point and may require some tweaking to meet your needs. Feel free to share your thoughts by using the comment section.

Finding a CentOS VM Image

Using our favorite PowerShell tools, we can discover details about pre-built CentOS 7.1 Azure Virtual Images.

Switch-AzureMode -Name AzureResourceManager

$location = 'westus'

Get-AzureVMImagePublisher -Location $location

$publisherName = 'OpenLogic'

Get-AzureVMImageOffer -Location $location `
                      -PublisherName $publisherName
$offer = 'CentOS'

Get-AzureVMImageSku -Location $location `
                    -PublisherName $publisherName `
                    -Offer $offer `
    | Select-Object -Property 'Skus'

$sku = '7.1'
                                 
Get-AzureVMImage -Location $location `
                 -PublisherName $publisherName `
                 -Offer $offer `
                 -Skus $sku

$verison = '7.1.201504'

Get-AzureVMImageDetail -Location $location `
                       -PublisherName $publisherName `
                       -Offer $offer `
                       -Skus $sku `
                       -Version $verison

Building the ARM Template

Starting from the Ubuntu Template in Visual Studio we can use the CentOS Virtual Image information, found in the previous section of this post, to create a CentOS 7.1 Standard_A3 Virtual Machine. To regroup the data disks as a RAID0 volume we can use the Create Ubuntu vm data disk raid0 template as a guide. Then we can make small changes to reach our goal.

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "newStorageAccountName": {
            "type": "string",
            "metadata": {
                "description": "Unique DNS Name for the Storage Account where the Virtual Machine's disks will be placed."
            }
        },
        "adminUsername": {
            "type": "string",
            "metadata": {
                "description": "User name for the Virtual Machine."
            }
        },
        "adminPassword": {
            "type": "securestring",
            "metadata": {
                "description": "Password for the Virtual Machine."
            }
        },
        "dnsNameForPublicIP": {
            "type": "string",
            "metadata": {
                "description": "Unique DNS Name for the Public IP used to access the Virtual Machine."
            }
        },
        "OSVersion": {
            "type": "string",
            "defaultValue": "7.1",
            "allowedValues": [
                "7.1"
            ]
        }
    },
    "variables": {
        "addressPrefix": "10.0.0.0/16",
        "dataDiskSize": "100",
        "imageOffer": "CentOS",
        "imagePublisher": "OpenLogic",
        "location": "West US",
        "nicName": "myVMNic",
        "OSDiskName": "osdisk",
        "publicIPAddressType": "Dynamic",
        "scriptUrl": "https://briseboispackages.blob.core.windows.net/linux/centos-vm-disk-utils-0.1.sh",
        "storageAccountType": "Standard_LRS",
        "subnetName": "Subnet",
        "subnetPrefix": "10.0.0.0/24",
        "subnetRef": "[concat(variables('vnetID'),'/subnets/',variables('subnetName'))]",
        "virtualNetworkName": "MyVNET",
        "vmName": "msbriseboislinux",
        "vmSize": "Standard_A3",
        "vmStorageAccountContainerName": "vhds",
        "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]"
    },
    "resources": [
        {
            "type": "Microsoft.Storage/storageAccounts",
            "name": "[parameters('newStorageAccountName')]",
            "apiVersion": "2015-05-01-preview",
            "location": "[variables('location')]",
            "tags": {
                "displayName": "StorageAccount"
            },
            "properties": {
                "accountType": "[variables('storageAccountType')]"
            }
        },
        {
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/publicIPAddresses",
            "name": "[parameters('dnsNameForPublicIP')]",
            "location": "[variables('location')]",
            "tags": {
                "displayName": "PublicIPAddress"
            },
            "properties": {
                "publicIPAllocationMethod": "[variables('publicIPAddressType')]",
                "dnsSettings": {
                    "domainNameLabel": "[parameters('dnsNameForPublicIP')]"
                }
            }
        },
        {
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/virtualNetworks",
            "name": "[variables('virtualNetworkName')]",
            "location": "[variables('location')]",
            "tags": {
                "displayName": "VirtualNetwork"
            },
            "properties": {
                "addressSpace": {
                    "addressPrefixes": [
                        "[variables('addressPrefix')]"
                    ]
                },
                "subnets": [
                    {
                        "name": "[variables('subnetName')]",
                        "properties": {
                            "addressPrefix": "[variables('subnetPrefix')]"
                        }
                    }
                ]
            }
        },
        {
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Network/networkInterfaces",
            "name": "[variables('nicName')]",
            "location": "[variables('location')]",
            "tags": {
                "displayName": "NetworkInterface"
            },
            "dependsOn": [
                "[concat('Microsoft.Network/publicIPAddresses/', parameters('dnsNameForPublicIP'))]",
                "[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]"
            ],
            "properties": {
                "ipConfigurations": [
                    {
                        "name": "ipconfig1",
                        "properties": {
                            "privateIPAllocationMethod": "Dynamic",
                            "publicIPAddress": {
                                "id": "[resourceId('Microsoft.Network/publicIPAddresses',parameters('dnsNameForPublicIP'))]"
                            },
                            "subnet": {
                                "id": "[variables('subnetRef')]"
                            }
                        }
                    }
                ]
            }
        },
        {
            "apiVersion": "2015-05-01-preview",
            "type": "Microsoft.Compute/virtualMachines",
            "name": "[variables('vmName')]",
            "location": "[variables('location')]",
            "tags": {
                "displayName": "VirtualMachine"
            },
            "dependsOn": [
                "[concat('Microsoft.Storage/storageAccounts/', parameters('newStorageAccountName'))]",
                "[concat('Microsoft.Network/networkInterfaces/', variables('nicName'))]"
            ],
            "properties": {
                "hardwareProfile": {
                    "vmSize": "[variables('vmSize')]"
                },
                "osProfile": {
                    "computername": "[variables('vmName')]",
                    "adminUsername": "[parameters('adminUsername')]",
                    "adminPassword": "[parameters('adminPassword')]"
                },
                "storageProfile": {
                    "imageReference": {
                        "publisher": "[variables('imagePublisher')]",
                        "offer": "[variables('imageOffer')]",
                        "sku": "[parameters('OSVersion')]",
                        "version": "latest"
                    },
                    "osDisk": {
                        "name": "osdisk",
                        "vhd": {
                            "uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/',variables('vmStorageAccountContainerName'),'/',variables('OSDiskName'),'.vhd')]"
                        },
                        "caching": "ReadWrite",
                        "createOption": "FromImage"
                    },
                    "dataDisks": [
                        {
                            "name": "datadisk1",
                            "diskSizeGB": "[variables('dataDiskSize')]",
                            "lun": 0,
                            "vhd": {
                                "Uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/vhds/',variables('vmName'),'dataDisk1' ,'.vhd')]"
                            },
                            "caching": "None",
                            "createOption": "Empty"
                        },
                        {
                            "name": "datadisk2",
                            "diskSizeGB": "[variables('dataDiskSize')]",
                            "lun": 1,
                            "vhd": {
                                "Uri": "[concat('http://',parameters('newStorageAccountName'),'.blob.core.windows.net/vhds/',variables('vmName') ,'dataDisk2','.vhd')]"
                            },
                            "caching": "None",
                            "createOption": "Empty"
                        }
                    ]
                },
                "networkProfile": {
                    "networkInterfaces": [
                        {
                            "id": "[resourceId('Microsoft.Network/networkInterfaces',variables('nicName'))]"
                        }
                    ]
                }
            }
        },
        {
            "type": "Microsoft.Compute/virtualMachines/extensions",
            "name": "[concat(variables('vmName'), '/azureVmUtils')]",
            "apiVersion": "2015-05-01-preview",
            "location": "[variables('location')]",
            "dependsOn": [
                "[concat('Microsoft.Compute/virtualMachines/', variables('vmName'))]"
            ],
            "properties": {
                "publisher": "Microsoft.OSTCExtensions",
                "type": "CustomScriptForLinux",
                "typeHandlerVersion": "1.2",
                "settings": {
                    "fileUris": [
                        "[variables('scriptUrl')]"
                    ],
                    "commandToExecute": "bash vm-disk-utils-0.1.sh -s"
                }
            }
        }
    ]
}

Publishing the ARM Template

Prepare the Template Parameter File

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "newStorageAccountName": {
            "value": "mslinuxbrisebois"
        },
        "adminUsername": {
            "value": "brisebois"
        },
        "dnsNameForPublicIP": {
            "value": "briseboisdns"
        }
    }
}

Then deploy the Azure Resource Manager Template and Parameters using PowerShell.

Set-AzureSubscription -SubscriptionName (Get-AzureSubscription -Current).SubscriptionName `
                      -CurrentStorageAccountName 'briseboispackages'

Switch-AzureMode AzureResourceManager

New-AzureResourceGroup -Name 'mslinuxbrisebois' `
                       -Location 'westus' `
                       -TemplateFile 'LinuxVirtualMachine.json' `
                       -TemplateParameterFile 'LinuxVirtualMachine.param.dev.json' `
                       -Force -Verbose

# New-AzureResourceGroup VERBOSE Log

VERBOSE: 10:15:18 AM - Created resource group 'mslinuxbrisebois' in location 'westus'
VERBOSE: 10:15:19 AM - Template is valid.
VERBOSE: 10:15:21 AM - Create template deployment 'LinuxVirtualMachine'.
VERBOSE: 10:15:24 AM - Resource Microsoft.Storage/storageAccounts 'mslinuxbrisebois1' provisioning status is running
VERBOSE: 10:15:34 AM - Resource Microsoft.Network/publicIPAddresses 'briseboisdns' provisioning status is running
VERBOSE: 10:15:36 AM - Resource Microsoft.Network/virtualNetworks 'MyVNET' provisioning status is running
VERBOSE: 10:15:46 AM - Resource Microsoft.Network/virtualNetworks 'MyVNET' provisioning status is succeeded
VERBOSE: 10:15:49 AM - Resource Microsoft.Network/publicIPAddresses 'briseboisdns' provisioning status is succeeded
VERBOSE: 10:15:51 AM - Resource Microsoft.Network/networkInterfaces 'myVMNic' provisioning status is succeeded
VERBOSE: 10:15:58 AM - Resource Microsoft.Storage/storageAccounts 'mslinuxbrisebois1' provisioning status is succeeded
VERBOSE: 10:16:03 AM - Resource Microsoft.Compute/virtualMachines 'msbriseboislinux1' provisioning status is running
VERBOSE: 10:18:16 AM - Resource Microsoft.Compute/virtualMachines 'msbriseboislinux1' provisioning status is succeeded
VERBOSE: 10:18:21 AM - Resource Microsoft.Compute/virtualMachines/extensions 'msbriseboislinux1/azureVmUtils' provisioning status is running
VERBOSE: 10:20:55 AM - Resource Microsoft.Compute/virtualMachines/extensions 'msbriseboislinux1/azureVmUtils' provisioning status is succeeded

This template will use the following script inspired by vm-disk-utils-0.1.sh for Ubuntu to create a RAID0 out of all the unformated data disks. A small modification has been applied for this to work on CentOS.

#!/bin/bash

# The MIT License (MIT)
#
# Copyright (c) 2015 Microsoft Azure
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
# 
# The above copyright notice and this permission notice shall be included in all
# copies or substantial portions of the Software.
# 
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
# SOFTWARE.
#
# Script Name: centos-vm-disk-utils.sh
# Author: Trent Swanson - Full Scale 180 Inc github:(trentmswanson)
# Version: 0.1
# Last Modified By: Alexandre Brisebois
# Description:
#  This script automates the partitioning and formatting of data disks
#  Data disks can be partitioned and formatted as seperate disks or in a RAID0 configuration
#  The scrtip will scan for unpartitioined and unformatted data disks and partition, format, and add fstab entries
# Parameters :
#  1 - b: The base directory for mount points (default: /datadisks)
#  2 - s  Create a striped RAID0 Array (No redundancy)
#  3 - h  Help 
# Note : 
# This script has only been tested on CentOS 7.1 and must be root

help()
{
    echo "Usage: $(basename $0) [-b data_base] [-h] [-s]"
    echo ""
    echo "Options:"
    echo "   -b         base directory for mount points (default: /datadisks)"
    echo "   -h         this help message"
    echo "   -s         create a striped RAID array (no redundancy)"
}

log()
{
    # Un-comment the following if you would like to enable logging to a service
    #curl -X POST -H "content-type:text/plain" --data-binary "${HOSTNAME} - $1" https://logs-01.loggly.com/inputs/<key>/tag/es-extension,${HOSTNAME}
    echo "$1"
}

if [ "${UID}" -ne 0 ];
then
    log "Script executed without root permissions"
    echo "You must be root to run this program." >&2
    exit 3
fi

#A set of disks to ignore from partitioning and formatting
BLACKLIST="/dev/sda|/dev/sdb"

# Base path for data disk mount points
DATA_BASE="/datadisks"

while getopts b:sh optname; do
    log "Option $optname set with value ${OPTARG}"
  case ${optname} in
    b)  #set clsuter name
      DATA_BASE=${OPTARG}
      ;;
    s) #Partition and format data disks as raid set
      RAID_CONFIGURATION=1
      ;;
    h)  #show help
      help
      exit 2
      ;;
    \?) #unrecognized option - show help
      echo -e \\n"Option -${BOLD}$OPTARG${NORM} not allowed."
      help
      exit 2
      ;;
  esac
done

get_next_md_device() {
    shopt -s extglob
    LAST_DEVICE=$(ls -1 /dev/md+([0-9]) 2>/dev/null|sort -n|tail -n1)
    if [ -z "${LAST_DEVICE}" ]; then
        NEXT=/dev/md0
    else
        NUMBER=$((${LAST_DEVICE/\/dev\/md/}))
        NEXT=/dev/md${NUMBER}
    fi
    echo ${NEXT}
}

is_partitioned() {
    OUTPUT=$(partx -s ${1} 2>&1)
    egrep "partition table does not contains usable partitions|failed to read partition table" <<< "${OUTPUT}" >/dev/null 2>&1
    if [ ${?} -eq 0 ]; then
        return 1
    else
        return 0
    fi    
}

has_filesystem() {
    DEVICE=${1}
    OUTPUT=$(file -L -s ${DEVICE})
    grep filesystem <<< "${OUTPUT}" > /dev/null 2>&1
    return ${?}
}

scan_for_new_disks() {
    # Looks for unpartitioned disks
    declare -a RET
    DEVS=($(ls -1 /dev/sd*|egrep -v "${BLACKLIST}"|egrep -v "[0-9]$"))
    for DEV in "${DEVS[@]}";
    do
        # The disk will be considered a candidate for partitioning
        # and formatting if it does not have a sd?1 entry or
        # if it does have an sd?1 entry and does not contain a filesystem
        is_partitioned "${DEV}"
        if [ ${?} -eq 0 ];
        then
            has_filesystem "${DEV}1"
            if [ ${?} -ne 0 ];
            then
                RET+=" ${DEV}"
            fi
        else
            RET+=" ${DEV}"
        fi
    done
    echo "${RET}"
}

get_next_mountpoint() {
    DIRS=$(ls -1d ${DATA_BASE}/disk* 2>/dev/null| sort --version-sort)
    MAX=$(echo "${DIRS}"|tail -n 1 | tr -d "[a-zA-Z/]")
    if [ -z "${MAX}" ];
    then
        echo "${DATA_BASE}/disk1"
        return
    fi
    IDX=1
    while [ "${IDX}" -lt "${MAX}" ];
    do
        NEXT_DIR="${DATA_BASE}/disk${IDX}"
        if [ ! -d "${NEXT_DIR}" ];
        then
            echo "${NEXT_DIR}"
            return
        fi
        IDX=$(( ${IDX} + 1 ))
    done
    IDX=$(( ${MAX} + 1))
    echo "${DATA_BASE}/disk${IDX}"
}

add_to_fstab() {
    UUID=${1}
    MOUNTPOINT=${2}
    grep "${UUID}" /etc/fstab >/dev/null 2>&1
    if [ ${?} -eq 0 ];
    then
        echo "Not adding ${UUID} to fstab again (it's already there!)"
    else
        LINE="UUID=\"${UUID}\"\t${MOUNTPOINT}\text4\tnoatime,nodiratime,nodev,noexec,nosuid\t1 2"
        echo -e "${LINE}" >> /etc/fstab
    fi
}

do_partition() {
# This function creates one (1) primary partition on the
# disk, using all available space
    _disk=${1}
    _type=${2}
    if [ -z "${_type}" ]; then
        # default to Linux partition type (ie, ext3/ext4/xfs)
        _type=83
    fi
    echo "n
p
1


t
${_type}
w"| fdisk "${_disk}"

#
# Use the bash-specific $PIPESTATUS to ensure we get the correct exit code
# from fdisk and not from echo
if [ ${PIPESTATUS[1]} -ne 0 ];
then
    echo "An error occurred partitioning ${_disk}" >&2
    echo "I cannot continue" >&2
    exit 2
fi
}
#end do_partition

scan_partition_format()
{
    log "Begin scanning and formatting data disks"

    DISKS=($(scan_for_new_disks))

    if [ "${#DISKS}" -eq 0 ];
    then
        log "No unpartitioned disks without filesystems detected"
        return
    fi
    echo "Disks are ${DISKS[@]}"
    for DISK in "${DISKS[@]}";
    do
        echo "Working on ${DISK}"
        is_partitioned ${DISK}
        if [ ${?} -ne 0 ];
        then
            echo "${DISK} is not partitioned, partitioning"
            do_partition ${DISK}
        fi
        PARTITION=$(fdisk -l ${DISK}|grep -A 1 Device|tail -n 1|awk '{print $1}')
        has_filesystem ${PARTITION}
        if [ ${?} -ne 0 ];
        then
            echo "Creating filesystem on ${PARTITION}."
    #        echo "Press Ctrl-C if you don't want to destroy all data on ${PARTITION}"
    #        sleep 10
            mkfs -j -t ext4 ${PARTITION}
        fi
        MOUNTPOINT=$(get_next_mountpoint)
        echo "Next mount point appears to be ${MOUNTPOINT}"
        [ -d "${MOUNTPOINT}" ] || mkdir -p "${MOUNTPOINT}"
        read UUID FS_TYPE < <(blkid -u filesystem ${PARTITION}|awk -F "[= ]" '{print $3" "$5}'|tr -d "\"")
        add_to_fstab "${UUID}" "${MOUNTPOINT}"
        echo "Mounting disk ${PARTITION} on ${MOUNTPOINT}"
        mount "${MOUNTPOINT}"
    done
}

create_striped_volume()
{
    DISKS=(${@})

    if [ "${#DISKS[@]}" -eq 0 ];
    then
        log "No unpartitioned disks without filesystems detected"
        return
    fi

    echo "Disks are ${DISKS[@]}"

    declare -a PARTITIONS

    for DISK in "${DISKS[@]}";
    do
        echo "Working on ${DISK}"
        is_partitioned ${DISK}
        if [ ${?} -ne 0 ];
        then
            echo "${DISK} is not partitioned, partitioning"
            do_partition ${DISK} fd
        fi

        PARTITION=$(fdisk -l ${DISK}|grep -A 2 Device|tail -n 1|awk '{print $1}')
        PARTITIONS+=("${PARTITION}")
    done

    MDDEVICE=$(get_next_md_device)    
    
    mdadm --create ${MDDEVICE} --level 0 --raid-devices ${#PARTITIONS[@]} ${PARTITIONS[*]}

    MOUNTPOINT=$(get_next_mountpoint)
    echo "Next mount point appears to be ${MOUNTPOINT}"
    [ -d "${MOUNTPOINT}" ] || mkdir -p "${MOUNTPOINT}"

    #Make a file system on the new device
    mkfs -t ext4 "${MDDEVICE}"

    read UUID FS_TYPE < <(blkid -u filesystem ${MDDEVICE}|awk -F "[= ]" '{print $3" "$5}'|tr -d "\"")

    add_to_fstab "${UUID}" "${MOUNTPOINT}"

    mount "${MOUNTPOINT}"
}

check_mdadm() {
    rpm -ql mdadm >/dev/null 2>&1
    if [ ${?} -ne 0 ]; then
        yum clean all
        yum update
        yum install mdadm -y
    fi
}

# Create Partitions
DISKS=$(scan_for_new_disks)

if [ "$RAID_CONFIGURATION" -eq 1 ]; then
    check_mdadm
    create_striped_volume "${DISKS[@]}"
else
    scan_partition_format
fi

Once the Azure Resource Manager Template completed its execution, I opened an SSH session using Putty and verified that our data disks have been stripped and mounted.

We are now ready to start working with this brand new CentOS Virtual Machine.

Resources

• Creating Azure VMs with ARM PowerShell cmdlets
• How to Use SSH with Linux on Azure
• How to Reset a Password or SSH for Linux Virtual Machines
• Azure Quickstart Templates