The amount of information returned in windows::kernel is 10 times all other data on the node, creating un-necessary but critical performance issues

I tried to disable windows::kernel via the client.rb config, but platform.rb:find_platform_and_version fails if languages,kernel, and windows::kernel aren't included

Ohai::Config[:disabled_plugins] = [

# The following plugins are required for Chef to work
# "hostname","windows::hostname","windows::platform",

# "network", #needed for ipaddress
# ,"windows::network",  # needed for ipaddress

#  "languages",        # find_platform_and_version fails if languages,kernel, and windows::kernel aren't included
#  "kernel",           # needed for platform_and_version
#  "windows::kernel"   # needed for find_platform_and_version in lib/chef/platform.rb

  "c", "cloud", "ec2", "rackspace", "eucalyptus", "command", "dmi", "dmi_common", "erlang", "groovy", "ip_scopes", "java", "keys",
  "lua", "mono", "network_listeners", "ohai_time", "passwd", "perl", "php", "python", "uptime", "virtualization", "windows::cpu",
  "windows::filesystem", "windows::virtualization"
]

https://gist.github.com/2475733 # try this on a node with windows, with and without removing the kernel.modules and kernel.pnp_drivers and compare the size, it's a huge difference for me. I may try and figure out a way to do something similar in a recipe to the automatic attributes before it gets saved back to the server if we don't feel this needs to be changed in ohai.

I can confirm that using the following in a recipe removes the extra data:

# This stuff can take up 10 times the amount of data
# as the entire rest of the node... we don't need it 

['pnp_drivers', 'modules'].each do |extraneous_info|
  node.automatic['kernel'].delete extraneous_info
end

I suggest putting kernel_pnp_drivers and kernel_modules into different ohai plugins so they can be easily disabled (and probably so by default.

I am actually working on this. I haven't had a chance to test everything yet. Life has interfered. My thought was to break the kernel plugin up so the second half has its own plugin but still producing the same node attributes. While I don't find them useful, someone may already be using them for something and I don't want this to break their envs or be noticeable.

Currently the main kernel does this:

case languages[:ruby][:host_os]
when /mswin|mingw32|windows/
require_plugin "windows::kernel"

I figured to add a second line that requires windows::kernel_drivers (or something like that)

This is dependent on the require losing out to a plugin being disabled or an additional bit of logic to only require it if the disabled plugins array doesn't contain it. I haven't done this part yet, although I've split the plugin and tested both new ones successfully.

I've started testing on EC2 today. I have a win2008r2 AMI that I'm working with. Interestingly, the first time measurements I ran were not the same dramatic times I see when running on VMs at the client where I first saw this. Tomorrow I'll fire up a VMWare Player and Virtualbox instances of Windows7 as that's what I have on hand. I'll see if the time to return ohai data is any different. I'll also ask my client to start up a fresh Win2008 VM so I can pull fresh time stats from there. But for now, this is what I see when I run stock Ohai and then again with the modules/pnp rivers commented. It's less than 1.5 seconds difference.

Win2008r2 EC2 default ohai
PS C:\Users\Administrator> Measure-Command {ohai}
Days : 0
Hours : 0
Minutes : 0
Seconds : 11
Milliseconds : 655
Ticks : 116554904
TotalDays : 0.000134901509259259
TotalHours : 0.00323763622222222
TotalMinutes : 0.194258173333333
TotalSeconds : 11.6554904
TotalMilliseconds : 11655.4904

-------------------------------------------
Win2008r2 EC2 ohai with php/modules commented

PS C:\Users\Administrator> Measure-Command {ohai}

Days : 0
Hours : 0
Minutes : 0
Seconds : 10
Milliseconds : 41
Ticks : 100415756
TotalDays : 0.000116221939814815
TotalHours : 0.00278932655555556
TotalMinutes : 0.167359593333333
TotalSeconds : 10.0415756
TotalMilliseconds : 10041.5756

Pull request for Ohai windows kernel submitted.

I'm still not sure this is going to fix the ohai slowdown problem, but it's a nice separation of files regardless.

Sascha, Thank you for taking the time to help make Ohai better! We require a Contributor License Agreement (CLA) from contributors to help protect all Ohai users. It is easy to complete one online. Instructions to do so are on our wiki, which is linked below. Once your account is updated, you will have access to 'resolve' tickets that indicates to the code review team that your contribution is ready for review.

http://wiki.opscode.com/display/chef/How+to+Contribute

Oops. Apparently I'm logging in and working with the wrong email as I set up the CLA in April. I'll get the other email address set up and then change the ticket. I must have missed the part where it says to change the status code on the ticket. I thought it just meant "update the ticket with a commented status."

Marking resolved for triage. Also, pull request: https://github.com/opscode/ohai/pull/81/files

Marking for merge.

Merged to master, thanks Sascha.

kermit flail! My first contribute.

Sascha Bates | [1]sbates@brattyredhead.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo | On 8/3/12 3:29 PM, Bryan McLellan (JIRA) wrote:

[ [2]http://tickets.opscode.com/browse/OHAI-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Bryan McLellan closed OHAI-357.
-------------------------------

Fix Version/s: 6.14.2

Merged to master, thanks Sascha.

The amount of information returned in windows::kernel is 10 times all other data on the node, creating un-necessary but critical performance issues
---------------------------------------------------------------------------------------------------------------------------------------------------

Key: OHAI-357
URL: [3]http://tickets.opscode.com/browse/OHAI-357
Project: Ohai
Issue Type: New Feature
Components: Network
Affects Versions: 0.6.12
Environment: chef 10.8 msi on windows
Reporter: Hippie Hacker
Priority: Critical
Fix For: 6.14.2

Given a standard windows node
And chef installed via the 10.8 msi
When chef-client runs
Then I want only useful data populated into chef
However currently:
node.kernel.pnp_drivers + node.kernel.drivers =~ 300k
everything else < 30k
With an extra 300k of key+value pairs being populated into couchdb, solr, exander per node, it's killing my production environment performance.
Having to xfer 330k any time you want to just look at a node with automatic attributes on (from the chef server) doesn't help things either.
–
This message is automatically generated by JIRA.
For more information on JIRA, see: [4]http://www.atlassian.com/software/jira

----------------------------------------------------------------------------------------
[1] sbates@brattyredhead.com
[2] http://tickets.opscode.com/browse/OHAI-357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
[3] http://tickets.opscode.com/browse/OHAI-357
[4] http://www.atlassian.com/software/jira