Detect Xen guests more reliable

merged to opscode/master. Thanks!

Holger, I just noticed that we don't have a CLA from you. We'll need this before we can merge your patches. Do you mind taking a few minutes to complete one, please?

http://wiki.opscode.com/display/chef/How+to+Contribute

Sent my CLA yesterday. Now waiting for the revert of the revert

This patch has been merged to opscode/master and will be included in the next release. Thank you for contributing to Ohai!

Bryan, I have made some changes to the plugin as the original OHAI-253 commit was not accounting for certain guest types. Tested on Xen hosts running Ubuntu 8.04/Xen 3.2 with Hardy guests and Debian 6/Xen 4.0.1 with Squeeze and Hardy guests.

Pull request #38.

Thank you.

Thanks for loading in the original 253 commit. As I said in another comment, after I took a look at the code, it was still missing some cases. I have since fixed that I believe.

https://github.com/opscode/ohai/pull/38

https://github.com/josephreynolds/ohai/commits/OHAI-253
https://github.com/josephreynolds/ohai/commits/OHAI-253-Rebase

After update to ohai 0.6.6-1 my debian squeeze xen guest detecting as xen host.

$ ls /proc/xen
balloon capabilities privcmd xenbus

/proc/xen/capabilities — empty
/dev/xen/evtchn — exist

But it's guest.

I can show any information about this instance, if you ask.

Bugreport: http://tickets.opscode.com/browse/OHAI-299

Tests for specific cases would be awesome. To do this we need to know:

1) distribution and version of that you're running (e.g. Ubuntu 8.04)
2) Exact and complete outputs of related files.

From this produce specs that match specific use cases:

# Xen Guest running Debian 6 (Squeeze)
  it "should detect that we are a Xen Guest running Debian 6" do
    # whatever
  end

  # Xen Guest Running Ubuntu 8.04 (Lucid)
  it "should detect that we are a Xen Guest running Ubuntu 8.04" do
    # whatever
  end

I would have expected these be the same though.

I haven't used Xen, but from my research /dev/xen/evtchn is support for the event channel in the kernel (or a module), part of paravirtualization support for the dom0. From what I can tell, you don't need it on the domU, but nothing prevents you from having it.

http://wiki.xen.org/xenwiki/XenParavirtOps#head-cac8537aeb7e0b959fa0953560e5986c7c6875b7

So the reason I chose to look for evtchn is that ,afaik, /dev/xen/evtchn is created by the Xenstore daemon. It's exclusive purpose, as I understand, is to facilitate access between the hypervisor and the guest. A machine cannot be a host unless the Xenstore daemon is running, and I do not believe that the daemon is used for anything else.

That being said it would be nice to be able to test for a variety of different configurations. In order to do this, the xen part of the plugin may need to become a little longer as the different Dom0 configurations affect how to determine that a machine is a guest.

For the time being would it be preferable to add in a check for the /proc/xen/capabilities file with control_d in it as well as looking for evtchn? There was also a method of determining guests by checking for the wallclock (which isn't available in all versions of the xen kernel) which was removed some time ago. Perhaps all methods should go in until a silver bullet is found ( something like virt-what, http://people.redhat.com/~rjones/virt-what/).

So the reason I chose to look for evtchn is that ,afaik, /dev/xen/evtchn is created by the Xenstore daemon.

It appears that this is created by CONFIG_XEN_DEV_EVTCHN in the kernel and needs to exist for xend/xenstored to even start.

You need to have Xen event channel (evtchn) driver included in your dom0 kernel for xend to be able to start. If Xen event channel support is compiled as a module, use "lsmod" to check that you have xen-evtchn (or xen_evtchn) driver module loaded. It not, use "modprobe" to load it. If Xen event channel support is statically built-in to the dom0 kernel then you don't need to load any modules.

http://wiki.xen.org/xenwiki/XenParavirtOps#head-6a64160b847b6fbd780350181436269c5a0a5b11

It appears that this is created by CONFIG_XEN_DEV_EVTCHN in the kernel and needs to exist for xend/xenstored to even start.

My orginal assertion was based off this piece of info:

The Xen Store Daemon is started in xenstored_core.c. It creates a device file ("/dev/xen/evtchn") in case such a device file does not exists and it opens it. (see : domain_init() ,file tools/xenstore/xenstored_domain.c).

found here http://wiki.xensource.com/xenwiki/XenIntro#head-4e27782aeef9debb00d812d9041a8e180ca414f0

That is why I chose evtchn is as the indicator of whether a machine is a host. As far as I know, there is no reason for that file to be created unless you're running xend, which only a host, or host capable machine would run.

See OHAI-299 for further discussion.

The Xen Store Daemon is started in xenstored_core.c. It creates a device file ("/dev/xen/evtchn") in case such a device file does not exists and it opens it. (see : domain_init() ,file tools/xenstore/xenstored_domain.c).

This appears to be out of date.

XenIntro (last edited 2009-02-12 16:31:12 by TimPost)

In Xen 4.0.0 domain_init() calls xc_evtchn_open() which in 'tools/libxc/xc_linux.c' does create this if it does not exist. However, in later versions this is not the case. In commit e645730ee5f8cc5cd6bef73318606768a41baf41 [1] they changed this practice with:

tools: assume that special Xen devices have been created by the platform

Remove all the magic surrounding the special Xen devices in Linux specific code whereby we attempt to figure out what the correct major:minor number is and check the the existing device has these numbers etc. In 2010 we really should be able to trust that the platform has created the devices correctly or provide correct configuration settings such that they are without resorting to tearing down the platform configured state and rebuilding it.

tools/hotplug/Linux/xen-backend.rules already contains the necessary udev rules to create /dev/xen/evtchn and friends in the correct place.

The above rule file creates '/dev/xen/evtchn' when the 'evtchn' kernel driver is loaded. Thus, any kernel with this driver built in, or if this module is loaded, will have this device.

I'm for more than one approach like virt-what does for Xen, but the only thing I can see evtchn telling us is possibly something about paravirtualization.

[1] http://xenbits.xen.org/gitweb/?p=xen.git;a=commit;h=e645730ee5f8cc5cd6bef73318606768a41baf41

Would like to mention that Ohai (0.6.8.pre) does not detect a Linode's DomU, at least with their latest kernel (3.0.4-x86_64-linode21). /proc/xen/capabilities is not available.

There's some info available in /sys. Still researching a reliable way to detect it there.

So far, so good. I've found a couple of ways:

1.

/sys/hypervisor/type will read xen

/sys/hypervisor/uuid will read "00000000-0000-0000-0000-000000000000" for Dom0 and random UUID for a DomU (Apparently, this is not true for Citrix XenServer, but you'll have a hard time installing Ohai there anyway)

2.

/sys/bus/xen/devices is empty in the Dom0 but it's populated with devices in a DomU

How do you feel about resorting to one of this methods if /proc/xen/capabilities isn't there/empty?

Looks like OHAI-299 will fix this too.

I'm all for trying a couple different methods. In OHAI-299 I fell back on the 'independent_wallclock' method if 'capabilities' doesn't exist. We can keep trying methods further down that block, I'd just like to be careful to not misidentify.

virt-what [1] has a nice tests directory with copies of output from various systems. These agree for your first use case, where populated. They lack a copy of the 'devices' file.

btm@btm-mbp-dev:~/src/virt-what/tests [master] $ cat ./rhel5-xen-domU-pv/sys/hypervisor/uuid
e7a4b45e-2172-c900-4ba4-dd8796ce14cb
btm@btm-mbp-dev:~/src/virt-what/tests [master] $  cat ./rhel5-xen-dom0/sys/hypervisor/uuid
00000000-0000-0000-0000-000000000000

[1] git clone git://git.annexia.org/git/virt-what.git

Matthew Kent noted on IRC that he "couldn't find a case where independent_wallclock would exist but /proc/xen wouldn't"