Chef Encrypted Data Bag Error due to Different YAML Engines

Fixed in linked Pull Request.

original ticket: CHEF-3328

Can the syck parser read data bags created by the psych parser? If it cannot, people who have created data bags on Ruby 1.9.3 prior to this patch will be unable to read their data bags after this release.

syck has been removed from trunk, we think this won't be released until Ruby 2.0. However, we will need to make a plan on how to get from syck to psych.

commit 5571c7315e118b339c6b6590e666dfda68a7327d
Author: tenderlove <tenderlove@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
Date:   Wed Aug 22 17:43:16 2012 +0000

    * ext/syck: removed. Fixes [ruby-core:43360]
    
    * test/syck: removed.
    
    * lib/yaml.rb: only require psych, show a warning if people try to set
      the engine to syck.
    
    git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@36786 b2dd03c8-39d4-4d8f-98ff-823fe69b080e

Warning message will be: "syck has been removed"

http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/43360
http://bugs.ruby-lang.org/issues/6163

At the very least, we need to determine that syck can load psych created data bags.

Further though, it's possible a user is using YAML elsewhere, perhaps sending data to an application with Chef, and forcing their parser to syck may break them.

In the end, much more thought is needed.

To put all the info in one place:

Until we figure out a safe course of action, users affected by this should add this:

YAML::ENGINE.yamler = 'syck' if RUBY_VERSION > '1.9'

...to the config files everywhere they use Ruby 1.9.3.

As Bryan notes above, we're not sure if this will break you if you're using YAML-serialized data to communicate with other applications, such as configuring the database.yml for a rails app; be sure to test that your recipes work correctly if you use YAML outside of encrypted data bags.

We were able to work-around the EncryptedDataBag failures we experienced by fixating the YAML::ENGINE in each developers knife.rb file to match what was load by our servers (running omnibus embedded ruby).

That seemed to be syck for the newer, 10.12 omnibus servers.

We also noticed this between Ruby 1.9.2 installs (rvm enforced Ruby 1.9.2p180 on our chef workstations and omnibus 1.9.2p290 on our servers)

One incompatibility encoding with syck and decoding with psych is any string containing a newline. For example:

>> require 'yaml'
=> false
>> def encode_18(s)
>>     old = YAML::ENGINE.yamler
>>     YAML::ENGINE.yamler = 'syck'
>>     yy = s.to_yaml
>>     YAML::ENGINE.yamler = old
>>     yy
>>   end
=> nil
>> 
?>   def decode_19(s)
>>     old = YAML::ENGINE.yamler
>>     YAML::ENGINE.yamler = 'psych'
>>     r = YAML.load(s)
>>     YAML::ENGINE.yamler = old
>>     r
>>   end
=> nil
>> decode_19(encode_18("a\n"))
Psych::SyntaxError: couldn't parse YAML at line 4 column 0
	from /Users/seth/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/psych.rb:148:in `parse'
	from /Users/seth/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/psych.rb:148:in `parse_stream'
	from /Users/seth/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/psych.rb:119:in `parse'
	from /Users/seth/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/psych.rb:106:in `load'
	from (irb):13:in `decode_19'
	from (irb):17
	from /Users/seth/.rbenv/versions/1.9.2-p290/bin/irb:12:in `<main>'
>> 

Rather than permanently modifying the YAML parser, I think we can make a more targeted fix and temporarily fall back to decoding with syck if parsing with psych fails.

This might also be a good time to consider adding a new non-YAML based format for the values with code that would write in the new format, but read old yaml style(s).

Opened Opscode ticket OC-4542 for scheduling.

Fixed by CHEF-3392