The behavior without this patch is that the performance of the vm_clone
operation in unacceptably slow for VMware vCenter deployments with
multiple hundreds of virtual machines.
Performance is unacceptable because the vm_clone operation makes
multiple API calls to list _all_ of the VM's in the inventory. This
patch eliminates the need to list all VM's by adding path and folder
filters to limit our API calls to subtrees of the VMware inventory.
= API Changes =
* New datacenters request that caches the Datacenter objects for the
life of the process.
* New clone() method on the server model that returns a server model of
the new VM even if it is not yet done cloning.
* Ability to limit collections to inventory paths by passing the
* 'folder' filter to the servers collection. For example:
`conn = Fog::Compute[:vsphere];
conn.servers('path' => '/Datacenters/DC1/vm/Templates')`
this filter will greatly reduce the number of SOAP API calls by
limiting the server models in the collection to only those in the
Templates inventory folder. Note, this is not recursive yet.
= Tests =
Tests have been updated. The vm_clone request no longer takes an
instance_uuid because we cannot actually use this to search the
inventory efficiently. Instead, the vm_clone request now requires a
path attribute to allow Fog to search only a subset of the inventory.
Update `Fog::Compute::Vsphere` tests to reflect the way the
`Fog::Compute::Vsphere#convert_vm_mob_ref_to_attr_hash` method currently
converts a `RbVmomi::VIM::ManagedObject` object to a hash.
Without this patch, tests related to converting an instance of
`RbVmomi::VIM::ManagedObject` to a hash will raise an exception:
NoMethodError: undefined method `collect!' for Hash; causing the test to
fail.
This patch solves the problem by mocking `RbVmomi::VIM::ManagedObject`
with a new `MockManagedObject` class, which provides a `collect!`
method, and the `_ref` and `parent` attributes.
This patch renames fake_vm to fake_vm_mob_ref in order to provide a
more descriptive name for what's actually being tested.
The `Fog::Compute::Vshpere::Mock` class has been updated to require the
rbvmomi library, which prevents an `NameError` exception from being raised
due to the `Fog::Compute::Vsphere::Shared::RbVmomi` constant not being
initialized.
The `Fog::Compute::Vshpere::Mock` class has been updated with a
`get_folder_path` method, which prevents a `NoMethodError` exception
from being raised due to the `get_folder_path` method being undefined.
Without this patch, `Fog::Compute::Vsphere#convert_vm_mob_ref_to_attr_hash`
method produces unhandled exceptions during VMware cloning and listing
operations. The root cause of these exceptions are based on the fact
that some VMware virtual machine attributes: hypervisor name, and
macaddress, are not available until the cloning process has finished.
These exceptions can be triggered when external events take place within
the VMware infrastructure such as end-users cloning machines via some
other VMware management tool.
This patch solves the problem by catching any exceptions that occur
during attribute lookups for both the hypervisor name and the virtual
machine macaddress, and setting them to nil.
This patch also removes the host attribute from the hash generated by
the `Fog::Compute::Vsphere#convert_vm_mob_ref_to_attr_hash` method. The
hypervisor attribute is added instead, which is an alias to host.
This patch changes the behaviour of the
`Fog::Compute::Vsphere#convert_vm_mob_ref_to_attr_hash` method by
catching exceptions for missing attributes, and setting them to nil.
This patch adds a new `path` attribute to the vm_mob_ref hash returned
by the `convert_vm_mob_ref_to_attr_hash` method. In order to support the
new `path` attribute, this patch also adds a new `get_folder_path` method
to the `list_virtual_machines` vsphere module.
This patch changes how the `list_all_virtual_machine_mobs` method
searches for virtual machines. Previously the search was based on
children of the `dc.vmFolder` object. This makes the false assumption
that all children are virtual machines.
Now the search is based on the entire inventory of the `dc.vmFolder`
object, which gives us more control over what we consider a virtual
machine.
This patch also adds a new `find_all_in_inventory` method; a generic
method that can be used to search an inventory for any VMware object
exposed by RbVmomi. Using `find_all_in_inventory` we are able to
traverse VMware folders and pickout specific VMware object, in this case
`RbVmomi::VIM::VirtualMachine`
This patch reduces the number of round trips to the vSphere API by
using the `collect!` method on the ManagedObject, vm_mob_ref, which
retrieves most of the properties in one request.
For the remaining properties: hypervisor and mac_addresses, we still
need to make additional requests.
Overall this patch provides a nice speed improvement for the
`convert_vm_mob_ref_to_attr_hash` method.
Without this patch the vm_clone requiest would not find a Managed Object
Reference when a UUID that is not a string is passed as the
instance_uuid option. This is a problem because an unhelpful "undefined
method `parent' for nil:NilClass" would be thrown to the application.
This patch throws a more helpful Fog::Compute::Vsphere::NotFound
exception if the Virtual Machine template is not found.
The tests have been updated to reflect this expectation.
Without this patch it was difficult to figure out from the outside what
vSphere server Fog connected to. The application using Fog should be
able to easily print out the connection information without breaking the
encapsulation Fog provides.
This patch makes the connected vSphere server hostname and API username
an attribute of the connection instance.
The tests have been updated to validate these attribute accessor
methods.
This massive commit refactors all of the request methods on the
Fog::Compute[:vsphere] instance to return simple hashes. The behavior
before this commit returned full vmware object references which was a
problem because it was difficult to unit test.
With this patch, it is much easier to add and maintain Mock
implementations of the request methods. This makes adding behavior
tests for the server model much easier.
In addition, test coverage using Shindo has been added. Previously
there was little test coverage of the behavior.
To run the tests:
shindont tests/vsphere/
Without this patch it is very difficult to reload the model of a VM in
the process of being cloned. All we have is the vmware managed object
reference ID string and the name of the VM.
This patch adds a number of improvements:
First, the model of a VM being cloned can reload itself after the VM
finishes cloning, even though we don't have an instance_uuid until the
clone completes.
Because the model can reload itself, it's now possible to do something
like:
c = Fog::Compute[:vsphere]
new_vm_name = "test"
c.vm_clone(:instance_uuid => "abc123", :name => new_vm_name)
my_new_vm = c.servers.find { |vm| vm.name == new_vm_name }
my_new_vm.wait_for { uuid }
puts "New VM is ready! (It has a UUID)"
my_new_vm.wait_for { ipaddress }
puts "New VM is on the network!"
Without this patch, a VM model could not reload itself with an id of
'vm-123', reloading the model only works if the ID is a UUID.
In addition, a number of the attributes of the server model have been
adjusted to be nil values when the VM is in the process of cloning.
This makes it easier to use wait_for conditionals in blocks.
Without this patch, we could not call the reload method
of a server model instance. This is because the Fog library
would try to call the get method using the existing identity
(instance_uuid) as the key.
This patch implements the get method by instantiating a new
model instance from a set of attributes obtained from the
vmware managed object instance.
The logic reflects that inside of the backspace servers
collection implementation.
As a side effect of this patch we also get wait_for.
For example:
server = Fog::Compute[:vsphere].servers.last
server.start
server.wait_for { ipaddress }
With the introduction of the vm_clone API request, a server
may be returned in the list which does not have a complete
configuration yet. This is because the system is still in
the process of being cloned.
This is a problem because Fog would throw an undefined
method error when calling the config method of the vm managed
object instance.
This patch fixes the problem by checking if the config method
returns something and only sets attributes that are known to be
available for a cloning VM.
Without this patch we have no way to clone a new VM
from a template. This patch adds the vm_clone request
which takes an instance uuid as the source template
to clone from an a name parameter which is the new VM's
name.
The clone operation is handled asynchronously as a vSphere
task object. The clone request does not return a handle
for this task, which may make it difficult to monitor
the progress of the clone operation from Fog. A future
enhancement may be to clone and return the task object
itself to monitor progress.
Without this patch we did not have a convenient way to find
a template by instance UUID. We could only find virtual
machines by instance UUID, which may or may not be a template.
This patch adds a specific request to obtain the VMware managed
object instance of of a specific template. This is necessary
plumbing for the clone request.
Without this patch we have no way to completely destroy
a server instance. This patch adds a vm_destroy API request
and implements the destroy method on the server model.
The vSphere API requires the server to be in a poweredOff state.
The model action and the request do not verify this is the case
before issuing the command.
This patch implements the start, stop and reboot methods for the Server
model instances.
These server model methods share common names with the AWS server model.
This patch also implements the API requests required to control the
power state of a VMware Virtual Machine.
The requests default to issuing shutdown and reboot commands to the
guest operating system itself. However, if force is set to true for
power_off and reboot, then the VM is powered off or reset at the virtual
hardware layer.
This patch adds two request methods to the compute service for VMware.
First, finding a VM by it's own UUID (from the vmx file) is supported.
This UUID is not guaranteed to be unique so this patch also implements
finding by instance UUID which is guaranteed to be unique.
The server models will primarily use these requests to obtain VM
managed objects to issue commands against.
Without this patch the change set currently has no framework for testing
the Vsphere provider. This patch doesn't actually test anything, but
does add the skeleton code to being testing mocked versions of other
code.
Particularly, we're going to take the approach of completely mocking the
@connection instance variable of the Compute instance. This will allow
us to set expectations and return things from the underlying vmware API
library.
This patch adds a request list_virtual_machines which is responsible for
making an API connection and returning a raw "response" object from the
API.
Model instances of a Server (compute resource) are returned as a
collection through the "all" method. The Fog framework calls all on the
instance of the collection.
Without this patch, no actual API calls are being made through the Fog
layer to the underlying rbvmobi later and ultimately to the vSphere
target API.
This patch adds a simple current_time request which is similar to a
"ping" The layers and API are exercised fully using this simple API
call to retrieve the current time on the remote system.
This provides:
>> Fog::Compute[:vsphere].current_time
Tue Aug 30 20:46:27 UTC 2011
>> Fog::Compute[:vsphere].requests
[:current_time]
Without this patch we were blindly trusting the remote end of our API
connection is who they claim to be. This is an insecure state because
we leave ourselves open to a man in the middle attack.
This patch adds a vsphere_expected_pubkey_hash setting for the Vsphere
provider. This setting is expected to be the SHA256 hex digest string
of the PEM encoded text of the RSA public key.
The first time an end user connects this string is displayed to them in
the error message. They need simply copy and paste it into ~/.fog to
securely connect to the remote end.
For example:
:vspherebadpw:
:vsphere_server: vc01.acme.lan
:vsphere_username: api_login
:vsphere_password: badpassword
:vsphere_expected_pubkey_hash: 431dd...
This patch adds a compute service to fog setting the stage to model
VMware virtual machines using Fog. The patch adds support for:
rdebug -- fog vsphere
>>> connection = Fog::Compute.new(:provider => :vsphere)
The connection to the VMware API is implemented along with
authentication using an username and password. The connection is not
fully secured with this patch because no validation of the SSL
certificate is implemented.
Raw API requests are working with this patch, but none of the API
requests have associated Fog models or collections.
