We simply cannot pin library versions in Rubygems.
If you look at Chef as a Ruby application which uses other Ruby libraries, it might appear that life would be easier if we tests a specific set of libraries and then fixed like, like running a Rails application with a Gemfile.lock.
However, Chef is not an isolated Ruby application. It is also a library and there is a vibrant ecosystem of tools built around it. If Chef pinned specific versions, than all the other applications that use Chef as library are required to use that exact same version. All of the libraries they use are also required to use the same version as well. The level of agreements and compromises would be huge and we would end up stuck on old versions of libraries forever because it would be too much work to get everyone to upgrade in lockstep.
Here's what happens when a Chef constraint doesn't match one in Vagrant: https://github.com/mitchellh/vagrant/issues/326
Here's a JSON constraint breaking cucumber-chef: https://github.com/Atalanta/cucumber-chef/issues/96
Here is where a constraint is breaking foodcritic: https://github.com/acrmp/foodcritic/issues/14
It's everyones responsibility who publishes gems to do our best to be up-front about the level of breaking changes in a release. Many people use a slightly modified version of semver to do so. As we find breakage we have to work with each other to resolve it and/or set constraints to exclude those versions. Because we're all only loosely working together, breakage will happen. The best and recommended ways to avoid having this affect production systems to to use the Omnibus packages which contain specific versions of gems, or to maintain a local rubygems server.