Cloud bursting with HP CloudSystem Matrix infrastructure orchestration: Matrix-to-Matrix, HP Matrix KVM Private Cloud, HP Cloud Services, and Amazon EC2

ManualsBrandsHP ManualsSoftwareHP Matrix Operating Environment w/Insight Control 24x7 Supp 1 Server Lic

Technical white paper

Cloud bursting with

HP CloudSystem Matrix

infrastructure orchestration

Matrix-to-Matrix, HP Matrix KVM Private Cloud, HP Cloud Services, and Amazon

EC2

Contents

Overview ........................................................................................................................................................ 3

What is CloudSystem bursting? ................................................................................................................... 3

Matrix-to-Matrix ........................................................................................................................................ 3

HP Matrix KVM Private Cloud ..................................................................................................................... 3

HP Cloud Services ..................................................................................................................................... 3

Amazon EC2 ............................................................................................................................................ 3

CloudSystem Matrix bursting concepts and capabilities ................................................................................. 3

Inventory............................................................................................................................................ 4

Server types ....................................................................................................................................... 4

Allocation .......................................................................................................................................... 4

Account access ................................................................................................................................... 4

Provisioning ....................................................................................................................................... 4

Prerequisites ............................................................................................................................................. 4

Egress ports ....................................................................................................................................... 5

Overview of CloudSystem configuration ............................................................................................................. 6

Required cloud provider configuration ................................................................................................................ 7

Obtain an account .................................................................................................................................... 7

Obtaining the secret access key ........................................................................................................... 7

Define key pairs ........................................................................................................................................ 7

Enabling Cloud Capacity .................................................................................................................................. 8

Edit the infrastructure orchestration properties file .......................................................................................... 8

Copy sample configuration files .................................................................................................................. 8

Sample files ....................................................................................................................................... 8

Define capacity pools ................................................................................................................................ 8

Create mxpassword entries for provider account access ................................................................................ 9

Configure capacity pools ........................................................................................................................... 9

Create the server type configuration file ..................................................................................................... 11

Create image filters ................................................................................................................................. 11

Configure the SSL certificate ..................................................................................................................... 11

Restart the infrastructure orchestration service ............................................................................................. 12

Using the Matrix KVM cloud configuration dialog ....................................................................................... 12

Provisioning to an external cloud in Matrix infrastructure orchestration ................................................................. 14

Launch infrastructure orchestration designer ................................................................................................ 14

Create a template ................................................................................................................................... 14

Launch infrastructure orchestration console ................................................................................................. 17

Create a pool ......................................................................................................................................... 17

Create a new service using the template .................................................................................................... 18

Summary of content (38 pages)

PAGE 1
Technical white paper Cloud bursting with HP CloudSystem Matrix infrastructure orchestration Matrix-to-Matrix, HP Matrix KVM Private Cloud, HP Cloud Services, and Amazon EC2 Contents Overview........................................................................................................................................................3 What is CloudSystem bursting? ...................................................................................................................3 Matrix-to-Matrix ..
PAGE 2
Access the cloud server ............................................................................................................................19 Troubleshooting .............................................................................................................................................20 Appendix A: Provisioning to an Amazon VPC ................................................................................................... 22 Limitations ............................................
PAGE 3
Overview What is CloudSystem bursting? Cloud bursting is a feature of HP CloudSystem Matrix that enables enterprises to provision public and private infrastructure resources seamlessly.
PAGE 4
Inventory • Resources accessible via provider accounts are represented as Cloud Resources inside CloudSystem Matrix. Cloud Resources can be added to a server pool in the same way as any other VM Host or ESX Resource Pool. • Capacity limits are enforced by the provider and limit-related errors can be reported during provisioning. The cloud provider account holder must apply to raise resource limits, if necessary, to meet the needs of services to be provisioned.
PAGE 5
Egress ports If a firewall is used between the Matrix system and a cloud provider, egress traffic must be allowed on these ports: • Matrix-to-Matrix: the port specified in the API endpoint (usually 8774) • HP Matrix KVM Private Cloud: 443 • HP Cloud Services: 35357 and 443. • Amazon EC2: 80 or 443, depending on whether an HTTP or HTTPS API endpoint is used.
PAGE 6
Overview of CloudSystem configuration The following sections describe the configuration steps that must be performed in CloudSystem before you can provision to a cloud provider. These steps are similar for all providers, with any provider specific considerations listed in separate sections. The configuration steps in CloudSystem are: 1. Edit the infrastructure orchestration properties file 2. Copy sample configuration files 3. Define capacity pools 4. Configure capacity pools 5.
PAGE 7
Required cloud provider configuration Obtain an account Create, or use an existing, cloud provider account. You will need to specify associated user credentials when configuring cloud capacity pools for the provider. For public cloud providers, use their web-based self-service user interfaces. For other providers, contact the provider or system administrator.
PAGE 8
Enabling Cloud Capacity Perform the following steps on the Central Management Server (CMS) where HP Matrix infrastructure orchestration is installed. Edit the infrastructure orchestration properties file 1. Open the IO properties file in the default location at: C:\Program Files\HP\Matrix infrastructure orchestration\conf\hpio.properties 2.
PAGE 9
Edit the capacity pool definition, which appears as a Cloud Resource in the Matrix infrastructure orchestration designer, self service portal, and console. The following is a sample element, with associated child elements, for a HP Matrix KVM Private Cloud pool. PAGE 10
# Tenant name for authentication with the cloud provider provider.tenant.name= # Access Key ID from the API Keys for the HP Cloud Services account # (see https://console.hpcloud.com/account/api_keys) provider.access.key.id= # Use the command "mxpassword -a -x =" to create # an mxpassword entry, where is the value specified here and # is the Secret Key associated with the Access Key ID above. provider.secretkey.mxpassword.
PAGE 11
The @min-xxx shortcut above is not appropriate for use in these cases: o An XML flavor file is not used, and one or more server types must be excluded from auto-selection. (For example, those with special properties such as a cluster type or specialized CPU access like Amazon's t1.micro.) o An XML flavor file is not used, and server types introduced in the future must not be selected automatically without an explicit choice to do so.
PAGE 12
The following instructions are for Internet Explorer. If you use another web browser, the steps will differ. 1. Enter the following URL in your IE browser: https://www.hpcloud.com or https://aws.amazon.com. 2. Select File→Properties. 3. Click Certificates and then the Certification Path, and select the root of the path. 4. Click the View Certificate button. 5. Select the Details tab in the newly opened window. 6. Click Copy to File… to start the Certificate Export Wizard, and click Next. 7.
PAGE 13
4. o Creates a password entry in mxpassword. o Refreshes the resource inventory so that the new server, software, and network pool resources appear. In the Configure KVM cloud pool dialog, specify the following input: Resource Name Name of the cloud capacity pool to be created. This name becomes the in the configuration filenames (with whitespace removed). This resource name will appear as a Cloud Resource in Matrix infrastructure orchestration.
PAGE 14
Provisioning to an external cloud in Matrix infrastructure orchestration The following sections describe how to create a template and create a service to deploy to a Cloud Resource. Launch infrastructure orchestration designer To access HP Matrix infrastructure orchestration designer using a browser, navigate to the following URL and log in: https://:51443/hpio/designer/. (Clicking New or Edit on the Templates tab in the infrastructure orchestration console also launches the designer.
PAGE 15
3. Right click the server group to configure it. a. Enter a name for the server group. b. Specify the number of servers, processors per server, and memory size, OR Specify the number of servers, and in Cloud Server Type, enter a valid server/instance recognized by the cloud provider (for example, standard.small for HPCS or m1.small for EC2, found in the server type configuration file). If specified, this server type is used for server creation, and the processor and memory settings are ignored.
PAGE 16
4. Select the Networks tab. a. Enter a hostname for the service. A hostname is required by IO, although it is not necessarily used to customize cloud servers. b. Select a network if you connected more than one network icon to the virtual server group. c. For each network interface, specify DHCP for the assignment type. 5. On the Firewall tab, select Required, Optional, or None, according to the provider requirements.
PAGE 17
o Provision to a preconfigured subnet on an Amazon Web Services VPC (see "Appendix A: Provisioning to an Amazon VPC"). o Have the flexibility to do either of the above, using template customization when the service is created A validation error is displayed if a selected network is not available from a source that matches the software selection source. 9.
PAGE 18
c. 3. Select a Cloud Resource from the left table, and click the right arrow to move the resource to the new pool, then click Save. Click Modify Users and assign users/groups to the new pool. Create a new service using the template 1. Click the Templates tab. Select the template previously created and click Create Service. 2. On the Create Service screen: a. Click Options. b. Enter a service name. c. Enter a hostname completion string. d.
PAGE 19
3. Click the Requests tab to view the progress and status of the create service request. Access the cloud server For applicable providers, access the cloud server using SSH. The administrator must have previously registered with one or more named key pairs in the cloud provider account. The administrator gives the key pair, including the actual keys, to users who are authorized to provision resources using the Cloud Resource.
PAGE 20
The Resource ID matches the Instance ID shown on the cloud provider console (in this example, the AWS Management Console). Troubleshooting • You can find useful information in the default location of: C:\Program Files\HP\Matrix infrastructure orchestration\logs\hpio-controller.log • You can enable HTTP tracing in the hpio-controller.log file by following the instructions in the following location, and then restarting the hpio service. C:\Program Files\HP\Matrix infrastructure orchestration\conf\log4j.
PAGE 21
• The hpio-controller.log lists HPIO diagnostic messages which can include configuration and connectivity errors. Failure to use ioassist to verify configuration and connectivity results in diagnostic messages like the following:  com.hp.hpio.sbapi.tasks.cloud.ListCloudImagesTask.executeTask - Error getting Cloud Images: java.util.concurrent.ExecutionException: com.hp.hpio.cloud.ec2.exception.
PAGE 22
Appendix A: Provisioning to an Amazon VPC CloudSystem can provision instances to an Amazon VPC in a manner similar to EC2. • A VPC must be pre-provisioned, with subnets, gateways, and routing tables configured as necessary. For more information, see the Amazon Virtual Private Cloud FAQs on the following website: http://aws.amazon.com/vpc/faqs/ • A single cloud capacity pool can be used to provision to EC2 or to a specified VPC, but not both.
PAGE 23
Appendix B: Sample files Matrix-to-Matrix sample files Image filter file
PAGE 24
HP Cloud Services https://console.hpcloud.com PAGE 25
# Optional file containing server type definitions. If not specified, server # type definitions are loaded directly from the provider. Custom server type # definitions can be used to exclude the use of some server types and to # normalize processor count and memory size values. #provider.servertype.file= # A prioritized list of server types to use when the logical server does not # specify a server type.
PAGE 26
HP Matrix KVM Private Cloud sample files Image filter file
PAGE 27
sample-hpcs-pool Sample HP Cloud Services capacity pool HP Cloud Services https://console.hpcloud.com PAGE 28
subnet.dns.domain=MatrixKVM.domain # Name of the key pair to use to access the provisioned servers default.keypair.name=cloud-user # A filter file that controls which images appear in the software inventory provider.image.filters.file=MatrixKVM-image-filters.xml # Optional file containing server type definitions. If not specified, server # type definitions are loaded directly from the provider.
PAGE 29
HP Cloud Services sample files Image filter file Image status values are described in the HP Cloud Compute Developer Guide at https://api-docs.hpcloud.com/hpcloudcompute/1.0/hpcloud-compute-1.0.pdf.
PAGE 30
1 1 standard.small 2 2 standard.medium 4 2 standard.large 8 4 standard.
PAGE 31
sample-MatrixKVM-pool Sample HP Matrix KVM Controller HP Matrix KVM Controller false PAGE 32
# A prioritized list of server types to use when the logical server does not # specify a server type. The first server type that meets the minimum # processor count and memory size for the logical server is selected. # # A keyword shortcut can be used to include all server types (limited by # provider.servertype.file). The keyword @min-processors prioritizes first by # minimum processor count and then by minimum memory size.
PAGE 33
Amazon EC2 sample files Image filter file
PAGE 34
Instance type definition file
PAGE 35
m1.xlarge 15 8 x86_64 m2.xlarge 17.1 6.5 x86_64 m2.2xlarge 34.
PAGE 36
Capacity pool definition file – EC2 adapter
PAGE 37
# An optional value used to pre-populate the DNS domain value in EC2 subnets subnet.dns.domain=us-east-1.amazonaws.com # Name of the EC2 key pair to use to access the provisioned servers default.keypair.name= # A filter file that controls which AMIs appear in the software inventory provider.image.filters.file=ec2-image-filters.xml # A file that defines the instance types to use and their characteristics provider.instancetype.file=ec2-instance-types.
PAGE 38
For more information The following cloud bursting white papers and the latest versions of manuals and other white papers for Matrix OE and related products can be downloaded from the following website: http://www.hp.com/go/matrixoe/docs • HP CloudSystem Matrix Bursting Connector HP CloudSystem Matrix version 7.2 (and later) includes a bursting connector component which, when enabled, can be used to provide compute services to other CloudSystem Matrix systems.