News Archive

How Heat Orchestrates your OpenStack Resources

One of the great things about orchestration is that it automatically figures out the operations it needs to perform. So whenever you deploy (or modify) your infrastructure you need not write a script to do so; you simply describe the infrastructure you want and the orchestration engine does the rest. I am often asked about how this works in Heat, so here is a quick run-down.

The first thing to note is that the order in which you define resources in your template is completely ignored. Instead, Heat builds a dependency graph based on the relationships between resources expressed in the template. Dependencies are inferred in three ways.

Firstly, any resource whose body uses Ref to refer to another resource is deemed to be dependent on that resource. So if one resource (say, a Nova server) has a property that refers to another resource (say, a Cinder volume), then those resources will always be created in the correct order.

A similar story applies to Fn::GetAtt. Say you have a database server and a separate application server in your template. You might use {"Fn::GetAtt": ["DBServer", "PrivateIp"]} in the UserData property of the application server to get the IP address of the database server. Of course the IP address is not known until the database server has been created, so this too adds a dependency to ensure things always happen in the right order.

Finally, you can use the DependsOn entry in a resource definition to explicitly add a dependency on another resource.

In the Grizzly (2013.1) version of Heat, resources were created serially. We simply performed a topological sort of the dependency graph and started the resources in that order. In the current development stream and the upcoming Havana (2013.2) release, however, Heat will create resources in parallel. Each resource will be created as soon as all of its dependencies are complete.

No discussion of dependency ordering would be complete without a word on what ‘complete’ means. Heat considers the creation of a resource complete as soon as the underlying OpenStack API reports that it is. So in the case of Nova, for instance, a server resource will be marked complete as soon as Nova has finished building it—i.e. as soon as it starts booting. The stack itself is marked complete as soon all of its constituent resources are.

To allow you to wait for a server to boot and be configured, CloudFormation provides (and Heat supports) a WaitCondition resource type. Calling the cfn-signal script from within the server signals success or failure. You want the timeout period to start when the server starts booting, so you’ll indicate that the wait condition DependsOn the server. If another resource requires software on the server to be configured before it can be created, simply mark that it DependsOn the wait condition.

Putting all of that together, here is a quick example that creates a Nova server and waits for it to boot before marking the stack complete:

Resources:
  MyWaitHandle:
    Type: AWS:CloudFormation::WaitConditionHandle

  MyServer:
    Type: AWS::EC2::Instance
    Properties:
      InstanceType: m1.small
      ImageId: F18-x86_64-cfntools
      UserData:
        Fn::Base64:
          Fn::Join:
          - "\n"
          - - "#!/bin/bash -v"
            - "PATH=${PATH}:/opt/aws/bin"
            - Fn::Join:
              - ""
              - - "cfn-signal -e 0 -r Done '"
                - {"Ref" : "MyWaitHandle"}
                - "'"

  MyWaitCondition:
    Type: AWS::CloudFormation::WaitCondition
    DependsOn: MyServer
    Properties:
      Handle: {"Ref": "MyWaitHandle"}
      Timeout: 300
Tags: