I wrote the simplest cookbook I could think of to demonstrate this issue: https://github.com/andreacampi/reval
Running this with Chef 10.12 (e.g. with `kitchen test`) creates two files `/tmp/working` and `/tmp/broken`.
Running it with Chef 10.14.4 only creates `/tmp/working`.
In a nutshell, recipe_eval doesn't actually evaluate the block when called from within a resource, e.g. a `ruby_block`.
The sample cookbook calls recipe_eval from an LWRP; I'm not sure 100% that's strictly relevant but that's the easiest way I know of.
I suspect it's an interaction with why-run, whereby the `converge_by` call in recipe_eval appends the block but it's too late to actually call it.
Some background: I noticed because this breaks the `application`, which calls one of its callbacks (`before_deploy`) within a ruby_block.
That's likely unnecessary but I need more time to verify it with the different cookbooks that depend on it.