With debug logging level set long running commands do not show their output in real time

Enclosed patches also now will show the STDOUT and STDERR messages interleaved in the correct order, rather than just showing all the STDOUT messages followed by all the STDERR messages.

Patches here: http://github.com/mlandauer/chef/tree/chef-173

I would appreciate it if people could have a good hard look at the patches as the logging is pretty core to things and also there might be subtle error situations which are not handled properly. Also, there of course cross-platform differences which could cause problems.

The most troublesome change I had to make is here:
http://github.com/mlandauer/chef/blob/6ccd3662817ca2fe29ee17fb4bb0a21a908e1111/chef/lib/chef/mixin/command.rb#L159

setting args[:waitlast] to true (if it isn't overridden)

Using select for this is the right thing.

The issue that caused the existence of "waitlast" was CHEF-75, and was summarized in a blog post at http://blog.opscode.com/2009/02/sometimes-grandchildren-are-pesky.html .

Our solution has another bug, though, which is that if the command you are running generates a significant amount of output (more than 4k, most likely, but it's OS specific) we'll wind up blocking anyway (the write end will block while it waits for someone to drain the buffer.)

I'm going to come up with an integration test for this case, so we can merge your changes, plus a final fix for the pesky grandchild problem. Reach out to me if you want to help out with the larger fix, otherwise expect me to put some elbow grease behind this over the weekend.

This is fixed in CHEF-238.

We still use the IO.select logic that Matthew's patch contained, but we have made it the standard path for all use of run_command and popen4. We've also enhanced it to ensure that it is also not going to be subject to the grandchild bug.

One side-effect of the approach taken in CHEF-238 is that, while Matthews patch made things appear to come out in the proper order some of the time, we don't even try to mimic how the console would behave: you get STDOUT, then STDERR.