Last updated August 19, 2009 23:24, by Tim Bray
Feedicon  

"Hello Cloud" - An illustrative walk-through of the Sun Cloud API


This document illustrates by example the process of building a Virtual Data Center to support a simple web application. Let's assume that this application does storage and retrieval of large media files; the required infrastructure is:

  • A firewall appliance for connecting to the Internet
  • A private network connecting the firewall to other systems
  • A Linux web server running a LAMP stack
  • A Solaris database machine running MySQL for application persistence
  • A WebDAV server for the media files

Here's a schematic of the effect we want to achieve.

This walk-through uses the pure REST version of the API exclusively. Quite likely, many users would prefer to script this in a higher-level language or even to use a Web or compiled GUI.

All the resources in the Virtual Data Center (machines, clusters, networks, storage volumes) are "Resources" in the Web sense: that is to say, they are identified by URIs, and operated on by HTTP requests, chiefly GET and POST. Whenever you GET one of these resources, you receive a representation encoded using JSON.

This walk-through should make it obvious that the cloud interface doesn't constrain the design of the URI space. URIs used for retrieving, creating, and controlling the resources are provided by the system; clients can make no assumptions as to their internal syntax.

Of course, it is necessary to bootstrap this process. A single URI must be provided, that of the Cloud. In this example, let's assume that a customer named "XRGY" is building the VDC; initially, all that is known is the VDC's URI: xrgy.cloud.sun.com.

Finding Virtual Data Centers

Retrieving a representation of the XRGY Cloud will yield the names and representations of any Virtual Data Centers in that cloud.

To server:

 GET /
 Host: xrgy.cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.Cloud+json
 X-Compute-Client-Specification-Version: 0.1

From Server:

 HTTP/1.1 200 OK
 Content-Type: application/vnd.com.sun.cloud.Cloud+json
 Content-Length: nnn
 
 {
   "implementation_version": "597",
   "vdcs": [
     {
       "name": "XRGY Virtual Data Center",
       "uri": "/vdc"
     }
     {
       "name": "R&D sandbox"
       "uri": "/sandbox"
     }
   ],
   "uri": "http://xrgy.cloud.sun.com/",
   "specification_version": [
     "0.5"
   ]
 }

This reveals that XRGY has access to two Virtual Data Centers. For the purpose of this walk-through we'll use the production VDC whose URI is "/vdc".

Examining the Virtual Data Center

Now we fetch a representation of the VDC, to ascertain what resources are available and can be created.

To server:

 GET /vdc
 Host: xrgy.cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.Vdc+json
 X-Compute-Client-Specification-Version: 0.1

From server:

 HTTP/1.1 200 OK
 Content-Type: application/vnd.com.sun.cloud.Vdc+json
 Content-Length: nnn
 
 { 
   "name" : "XRGY Virtual Data Center",
   "uri" : "http://xrgy.cloud.sun.com/vdc",
   "vm_templates" : "http://cloud.sun.com/resources/template-cat.json",
   "addresses" : [
     {
       "name": "144.34.100.199",
       "uri": "/addresses/144.34.100.199",
       "ip_address": "144.34.100.199"
     }
   ],
   "cluster" : {
     "name" : "ROOT",
     "uri" : "/vdc/",
     "tags" : [ ],
     "volumes" : [ ],
     "clusters" :  [
     ]
     "tags" : [ ],
     "controllers" : [
       "start" : "/vdc/ops/start",
       "stop" : "/vdc/ops/stop",
     ]
     "vnets" : [
       {
         "name": "vnet1",
         "uri": "/vnets/10.31.145.0",
         "netmask": "255.255.255.0",
         "network": "10.31.145.0"
       }
     ],
     "vms": [
       {
         "name" : "Firewall"
         "uri": "/vdc/m~FW01",
         "run_status" : "HALTED",
         "description" : "Firewall appliance; ports 80 and 22 open.",
         "hostname" : "FW01",
         "os" : "Solaris 10",
         "cpu" : 1800,
         "memory" : 2,
         "boot_disk" : 10,
         "data_disk" : 80,
         "temp_disk" : 10,
         "params" : { },
         "tags" : [ ],
         "back_up": "/vdc/m~FW01/ops/back-up",
         "attach": "/vdc/m~FW01/ops/attach",
         "detach": "/vdc/m~FW01/ops/detach",
         "backups" : [ ],
         "interfaces" : [
           {
             "mac_address": "00:16:3E:08:00:91",
             "ip_address": "144.34.100.199",
             "public_address": "/addresses/144.34.100.199",
             "nic": "eth0"
           },
           {
             "vnet": "/vnets/10.31.145.0",
             "mac_address": "00:16:3E:08:00:92",
             "ip_address": "10.31.145.254",
             "nic": "eth1"
           }
         ]
         "controllers": {
           "start": "/vdc/m~FW01/ops/start",
         }
       }
     ]
   }
 }

A close look reveals that the VDC is mostly empty. It comes with a few preconfigured resources:

  • A "cluster", an administrative container for grouping virtual machines. Every VDC has one built-in top-level cluster. Clusters can contain other clusters, virtual machines, and private networks.
  • A firewall-appliance virtual machine, in the top-level cluster.
  • A private network in the top-level cluster, to which the appliance is connected.
  • A single public IP address, also connected to the firewall.

Note that these objects are equipped with URIs which may be used to add more resources to the VDC, or control the operation of resources that are there. While the URIs in the example follow an obvious pattern, that is an artifact of this implementation; another Cloud service provider might us an entirely different URI-space design. Clients should treat the URIs as opaque strings.

However, note that almost all the embedded URIs are relative references, beginning with "/". The usual practices for absolutizing them apply; standard HTTP libraries will take care of this.

Virtual Machine templates

Our first step is to examine the templates available use for creating virtual machines. The JSON representation of the VDC above includes a field named "vm_templates"; its value is a URI by which a catalog of templates may be retreived.

To server:

 GET /resources/template-cat.json
 Host: cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.VMTemplates+json
 X-Compute-Client-Specification-Version: 0.1

From server:

 HTTP/1.1 200 OK
 Content-Type: application/vnd.com.sun.cloud.VMTemplate+json
 Content-Length: nnn
 
 [
   {
     "name" : "Firewall appliance",
     "uri" : "/resources/vmtemplates/901"
     ... other fields omitted for readability ...
   },
   {
     "name" : "2G Dual-core Debian LAMP x64",
     "uri" : "/resources/vmtemplates/001"
     ...
   },
   {
     "name" : "2G Quad-core OpenSolaris x64",
     "uri" : "/resources/vmtemplates/002"
     ...
   }
   {
     "8G Quad-core OpenSolaris MySQL x64",
     "uri" : "/resources/vmtemplates/003"
     ...
   }
   ...
 }

A virtual-machine template has many fields, similar to those observed in the representation of the firewall appliance above; we've omitted them to make this walk-through more readable.

Adding Servers

In this setup, we will use the firewall appliance that's already there, a Debian LAMP-stack VM, and a larger OpenSolaris VM for MySQL. We could group these into multiple clusters for administrative convenience, access control, and programmability, but for this walk-through we will bypass that and put everything in the root cluster.

To create a machine in a cluster, we need to post enough of the VM rpresentation to that cluster's URI to get the job done; in this case, a name and description for the new VM, and a reference to the template we want to base it on.

To server:

 POST /vdc/
 Host: xrgy.cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.Status+json
 Content-Length: nnn
 Content-Type: application/vnd.com.sun.cloud.VM+json
 X-Compute-Client-Specification-Version: 0.1
 
 {
   "name" : "Database"
   "from_template" : "http://cloud.sun.com/resources/vmtemplates/003",
   "description" : "MySQL host",
   "tags" : [ "sql" ]
 }

Operations such as this which change the state of the cloud may be time-consuming. Furthermore, the latency may vary from implementation to implementation and from invocation to invocation. Rather than wait for the eventual completion of the operation, the server returns a Status resource, as follows:

 HTTP/1.1 202 Accepted
 Content-Type: application/vnd.com.sun.cloud.Status+json
 Content-Length: nnn
 
 {
   "op": "new-VM",
   "progress": 0,
   "target_uri": "/vdc/m~001",
   "status_uri": "/vdc/status?op=new-vm&vm=001"
 }

There are several points of interest about the response. First, the HTTP status code 202, which says that the create-VM request has been accepted, implying that it will complete at some later time.

The response body includes a representation of a Status resource, which contains four fields:

  • op identifies the type of operation for which this is the status; this helps make the Status resource self-describing.
  • progress is a 0-100 range, where 100 means the task is complete.
  • The meaning of the target_uri field depends on the op field. Since in this case we're requesting VM creation, this field gives the URI to use use for the new VM once it exists.
  • status_uri is the URI to use to retrieve this status resource; it's designed to be polled.

Let's assume we poll the status_uri a few times and eventually note that the progress value has hit 100. At that point we can use the target_uri value to access the new VM we created.

To server:

 GET /vdc/m~001
 Host: cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.VM+json
 X-Compute-Client-Specification-Version: 0.1

From server; for readability, we omit many of the VM's fields where their values are not particularly relevant to this deployment - the Firewall example in the VDC representation above is an example of a complete VM representation:

 HTTP/1.1 200 OK
 Content-Type: application/vnd.com.sun.cloud.VM+json
 Content-Length: nnn
 
 {
   "name" : "Database"
   "uri": "/vdc/m~001",
   "run_status" : "HALTED",
   "description" : "MySQL host",
   "tags" : [ "sql" ]
   "hostname" : "c1vm001",
   "interfaces" : [ ]
   "attach": "/vdc/m~001/ops/attach",
   ... other fields omitted for readability ...
 }

Note that the newly-added server is not connected to any networks, since its interfaces field is unpopulated.

The request that would be used to create the Solaris web server would be straightforwardly similar. After these two requests had completed, the vms field of the root cluster would have representations of three VMs.

Virtual Networks

We want to connect our servers to the Firewall server, but not directly to the outside world. For this purpose, we use a Virtual Network (VNet). Our VDC's root cluster came pre-configured with one named "vnet1", so we will use this. In a more complex configuration we might want further isolation of the Web tier from the Database tier, and use two VNets. VNets may be created by POSTing a VNet representation to the URI of the containing VDC.

Connecting the machines to the network

Given that the Firewall is already connected to our VNet and a Public Address, we need only connect the Database and Web-Server servers to the VNet. Examining the representation of the Database machine reveals a field named attach whose value is "/vdc/m~001/ops/attach". The only piece of information the system needs to make the connection is the URI, the unique identifier of the VNet.

To server:

 POST "/vdc/m~001/ops/attach
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Content-Length: nnn
 Content-Type: application/vnd.com.sun.cloud.VNet+json
 X-Compute-Client-Specification-Version: 0.1
  
 {
   "uri" : "http://xrgy.cloud.sun.com/vnets/10.31.145.0"
 }

From server:

 HTTP/1.1 202 Accepted
 Content-Type: application/vnd.com.sun.cloud.Status+json
 Content-Length: nnn
 
 {
   "op": "attach",
   "progress": 100,
   "target_uri": "/vdc/m~001",
   "status_uri": "/vdc/status?op=attach&vm=001&net=10.31.145.0"
   "status": 201
 }

Note that although the return code is 202 with a Status resource, the progress value of 100 signals that the attach operation has completed, and the HTTP status code of 201 in the status field indicates that the VM creation was successful. Thus, the interfaces field in the representation of the Database would be something like this:

 "interfaces" : [
   {
     "vnet": "/vnets/10.31.145.0",
     "mac_address": "00:16:3E:08:00:92",
     "ip_address": "10.31.145.254",
     "nic": "eth0"
   }
 ]

This signals that the Database machine is connected to the VNet on interface "eth0" at IP address 10.31.145.254.

Similarly, it would be straightforward to connect the Web-server machine to the same VNet.

Creating and Attaching a Storage Volume


For the purposes of this application, we will use a WebDAV server to store the media objects, with metadata and links stored in the MySQL database. A POST of a Volume representation to a VDC's URI is a request to create a new storage volume within that VDC. Since most of the fields are set by the server, we only supply the name. The server knows that this is a request to create a Volume by examining the Content-Type header.

To server:

 POST /vdc
 Host: xrgy.cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.Status+json
 Content-Length: nnn
 Content-Type: application/vnd.com.sun.cloud.Volume+json
 X-Compute-Client-Specification: 0.1
 
 {
   "name": "media-store"
 }

As with all other update operations, this returns "202 Accepted" and a Status resource, which gives the URI to be used for the new Volume, in this case "/vdc/volumes/001". Once the progress field reaches 100, we can retrieve it.

To server:

 GET /vdc/volumes/001
 Host: cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Accept: application/vnd.com.sun.cloud.Volume+json
 X-Compute-Client-Specification-Version: 0.1

From server:

 HTTP/1.1 200 OK
 Content-Type: application/vnd.com.sun.cloud.Volume+json
 Content-Length: nnn
 
 {
   "name" : "public_data",
   "uri" : "http://xrgy.cloud.sun.com/vdc/volumes/001",
   "tags" : [ ],
   "webdav" : "https://storage.cloud.sun.com/uc913042/public_data",
   "snapshots" : [ ],
   "snapshot" : "http://xrgy.cloud.sun.com/vdc/snapshot?volume=001"
 }

At this point, the volumes field in the representation of the VDC would be something like this:

   "volumes" : [
     {
       "name" : "public_data",
       "uri" : "http://xrgy.cloud.sun.com/vdc/volumes/001",
       "tags" : [ ],
       "webdav" : "https://storage.cloud.sun.com/uc913042/public_data",
       "snapshots" : [ ],
       "snapshot" : "http://xrgy.cloud.sun.com/vdc/snapshot?volume=001"
     }
   ],

Starting and Stopping

At this point, we have created two servers, a WebDAV-accessible storage volume, and made the necessary network connections. However, the VDC is not yet actually in operation, since none of the computers are running.

The representations of the VM and Cluster objects contain start controller fields, for use in starting individual VMs or a whole cluster at a time, respectively.

In this minimal setup, it is only necessary to start the root cluster to have the infrastructure on the air. For this type of operation, the only operation in the request body is a human-readable note to be used for logging and auditing purposes.

To server:

 POST /vdc/ops/start
 Host: xrgy.cloud.sun.com
 Authorization: Basic xxxxxxxxxxxxxxxxxxx
 Content-Length: nnn
 Content-Type: application/json
 X-Compute-Client-Specification-Version: 0.1
 
 {
   "note": "Start servers!"
 }

From server:

 HTTP/1.1 202 Accepted
 Content-Type: application/vnd.com.sun.cloud.Status+json
 Content-Length: nnn
 
 {
   "op": "start-cluster",
   "progress": 0,
   "target_uri": "/vdc/",
   "status_uri": "/vdc/status?op=start"
 }

This will cause the run_state of all the machines in the cluster to change to STARTING, and then, when the value of progress in this status resource goes to 100, all of them will be STARTED.

Conclusion

This completes the initial walk-through, illustrating the basics of creating a fairly minimal virtual data center. Many capabilities of the API have been omitted for brevity. For further reference, consult the specification pages on this wiki which detail the resource models and all the operations available on them.

Some capabilities omitted in this walk-through include:

  • Cloning and copying: These features are operations at both the Cluster and VM levels. They represent the most straightforward and efficient way of configuring homogeneous machines for operation in one tier of of a Web application.
  • Clustering: Clusters of VMs are useful in cloning/copying operations, for enforcing access control, for modeling geographically distributed groups of machines, and for simplifying automated operational scripting. A typical real-world VDC would probably have multiple clusters; for example, one for the firewall(s), one for the App Server tier, and one for the Database tier.
  • Backups: Backup and restore operations are available at the VM level and may be automated through this interface.
  • Startup parameters. A machine, when started, needs to be instructed as to its role and workflow. This information is made available on a virtual filesystem which is mounted at boot time at /cloud; the VMs' startup scripts, after booting has completed, execute /cloud/startup. The contents of /cloud may be controlled programmatically through the params field of the VM resource.

Copyright © Sun Microsystems, 2009. This work is licensed under Creative Commons Attribution 3.0 Unported License

  • Mysql
  • Glassfish
  • Jruby
  • Rails
  • Nblogo
Terms of Use; Privacy Policy;
© 2013, Oracle Corporation and/or its affiliates
(revision 20140418.2d69abc)
 
 
Close
loading
Please Confirm
Close