"gem install chef" fails due to dependency conflicts

Pull request submitted:
https://github.com/opscode/chef/pull/626

This is affecting me too, on Chef versions going back to 10.16.4.

This is preventing me from bootstrapping new instances, which is kind of a big deal.

I'm requesting that this be fixed in the 10.x line as well, since I haven't yet had time to update my cookbooks to accomodate the breaking changes in Chef 11.

I also think that a proper resolution to this bug would be to have all Chef dependencies locked to a specific version. This isn't the first time something like this has happened, an update to moneta had the same effect on Chef bootstraps.

I can see that similar requests have been Won't Fixed, but I think that should be reconsidered. Operational stability should be a top priority for a tool like Chef.

I'd like to echo Brandon Adams' sentiment here - version conflicts seem to happen often with the gems that Chef depends on. Locking versions could help.

This is a known issue.

http://jtimberman.housepub.org/blog/2013/02/06/chef-and-net-ssh-dependency-broken/

There's a 11.2.0.rc.1 gem available on RubyGems.org:

sudo gem install chef --pre

Builds are in progress for Chef 11 client packages. We'll get Chef 10 updated soon too.

I updated net-ssh-multi 1.1.2 earlier to resolve this issue. It should resolve this issue as it requires net-ssh 2.6.5.

See my comment below.

This affects 10.14 (0.14) equally - what is the ETA on a clean vendor-approved fix for this ?

This is a very severe bug. Like Brandon, I am also not able to install 10.16.4, which means I can't bootstrap new instances either.

Chef 11.2.0 will ship momentarily. A 10.x release should be out within the hour.

Bryan can you confirm there will be a fix for 10.14.x that does not include a forced upgrade to 10.16/10.18 ?

Thanks.

Thanks Delano, that allowed me to get it to work. I'm still not able to straight `gem install chef -v 10.16.4`, but with a little gem-jiggling, I'm able to get it to work.

For anyone else who is in my exact same version number situation, here's what's currently working for me:

gem sources -c
rvm --force gemset delete tmp1
rvm gemset use tmp1 --create
gem install net-ssh -v 2.2.2
gem install net-ssh-gateway -v 1.1.0
gem install net-ssh-multi -v 1.1
gem install chef -v 10.16.4

You're welcome.

The remaining issue is likely that one (or more) of the offending gems is already installed locally. I updated the blog post to reflect that ( http://solutious.com/blog/2013/02/06/net-ssh-gem-code-signed/ ).

Cheers Delano Mandelbaum, thanks for following up with us!

One more thing on locking to a specific version vs. just locking to a major version – updating the Chef version we use takes up a huge chunk of my day.

I have eleven autoscaling launch configurations that have to be updated. For each one I have to update the Chef version in a CloudFormation template, then update a CloudFormation stack in a staging environment, launch a couple of instances with the new launch configuration, confirm that everything works, then update the stack in production and do the same confirmation. We're about to add a twelfth launch configuration, and will only add more as we break more chunks of our application off into separate services.

We simply cannot pin library versions in Rubygems.

If you look at Chef as a Ruby application which uses other Ruby libraries, it might appear that life would be easier if we tests a specific set of libraries and then fixed like, like running a Rails application with a Gemfile.lock.

However, Chef is not an isolated Ruby application. It is also a library and there is a vibrant ecosystem of tools built around it. If Chef pinned specific versions, than all the other applications that use Chef as library are required to use that exact same version. All of the libraries they use are also required to use the same version as well. The level of agreements and compromises would be huge and we would end up stuck on old versions of libraries forever because it would be too much work to get everyone to upgrade in lockstep.

Here's what happens when a Chef constraint doesn't match one in Vagrant: https://github.com/mitchellh/vagrant/issues/326
Here's a JSON constraint breaking cucumber-chef: https://github.com/Atalanta/cucumber-chef/issues/96
Here is where a constraint is breaking foodcritic: https://github.com/acrmp/foodcritic/issues/14

It's everyones responsibility who publishes gems to do our best to be up-front about the level of breaking changes in a release. Many people use a slightly modified version of semver to do so. As we find breakage we have to work with each other to resolve it and/or set constraints to exclude those versions. Because we're all only loosely working together, breakage will happen. The best and recommended ways to avoid having this affect production systems to to use the Omnibus packages which contain specific versions of gems, or to maintain a local rubygems server.

This is a bad news, but thanks for trouble-shooting the issue. One question. Is there any Omnibus bootstrap template somewhere we can reference?

The chef-full template installs using Omnibus and has been the default bootstrap template since 0.10.10. See CHEF-2815

Cool

I've found that the RPMs used by the Omnibus installer work well for addressing this problem. I tested by pointing rubygems.org to localhost in my /etc/hosts and saw that the RPM install succeeded.

For those using bootstrapping on AWS with an autoscaling launch configuration, just add a line like this instead of installing the Chef gem:

rpm --nosignature -i https://opscode-omnitruck-release.s3.amazonaws.com/el/6/x86_64/chef-10.18.2-2.el6.x86_64.rpm

--nosignature suppresses warnings about GPG keys. Modify the path to the RPM depending on which version you wish to install.

The Omnibus packages already have installed gems in them, so you don't have to worry about the versions changing between installations as long as your using the same version and iteration. If you used chef_gem to install a gem that required a higher version of a gem that chef used, it could produce an issue, but it'd be a very rare case.

The same fixed solved this as CHEF-3156.