Deploy should create the directories it needs if they don't exist

Details

  • Type: Improvement Improvement
  • Status: Closed Closed
  • Priority: Minor Minor
  • Resolution: Fixed
  • Affects Version/s: 0.7.12
  • Fix Version/s: 0.10.6, 10.14.0
  • Component/s: Chef Client
  • Labels:
    None

Description

Deploy needs the :deploy_to directory to exist before it will work. This makes some sense in that the user should have the config dir pre-populated, therefore the deploy_to dir would already exist. In conjunction with making the config dir, erm, configurable (CHEF-609) and the fact that not all apps will necessarily need configurations, the deploy provider should create the deploy_to dir if it doesn't exist.

Activity

Ascending order - Click to sort in descending order
Hide
Permalink
Dave Myron added a comment -

It would also be useful if the "typical" shared dirs were created at the same time too. The symlinking currently assumes they exist so deploying the app will appear to succeed but will be in a bad state with invalid links to nonexistent directories (such as "shared/log"). Making this list have a sane default and be configurable like the *_before_symlink properties would be perfect. I believe that sane default would be %w(log config system pids)

Show
Dave Myron added a comment - It would also be useful if the "typical" shared dirs were created at the same time too. The symlinking currently assumes they exist so deploying the app will appear to succeed but will be in a bad state with invalid links to nonexistent directories (such as "shared/log"). Making this list have a sane default and be configurable like the *_before_symlink properties would be perfect. I believe that sane default would be %w(log config system pids)
Hide
Permalink
jtimberman added a comment -

This is listed as an open issue on the Deploy Resource page. Assigning a fix version so it gets some attention.

Show
jtimberman added a comment - This is listed as an open issue on the Deploy Resource page. Assigning a fix version so it gets some attention.
Hide
Permalink
Daniel DeLeo added a comment -

bulk updating fix versions to next release

Show
Daniel DeLeo added a comment - bulk updating fix versions to next release
Hide
Permalink
Avishai Ish-Shalom added a comment -

https://github.com/avishai-ish-shalom/chef/tree/CHEF-630

Show
Avishai Ish-Shalom added a comment - https://github.com/avishai-ish-shalom/chef/tree/CHEF-630
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

Looks good. We should probably not try to create the directory if it already exists. If we create the directory, we want to set the owner and group to the corresponding attributes from the resource. If the user wants these directories to have different permissions they should create them prior with the directory resource.

Show
Bryan McLellan [Opscode] added a comment - Looks good. We should probably not try to create the directory if it already exists. If we create the directory, we want to set the owner and group to the corresponding attributes from the resource. If the user wants these directories to have different permissions they should create them prior with the directory resource.
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

This patch has been merged to opscode/master and will be included in the next release. Thank you for contributing to Chef!

Show
Bryan McLellan [Opscode] added a comment - This patch has been merged to opscode/master and will be included in the next release. Thank you for contributing to Chef!
Hide
Permalink
Cameron Johnston added a comment -

I believe this patch has broken the deploy resource in the way it is used by my organization, namely that this fix does not seem to take into account the possibility that one will want to symlink regular files as well as directories.

e.g. specifying

symlinks "local_settings.py" => "local_settings.py"

as a parameter for a deploy resource results in an error like this one:

(haystack::default line 139) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /opt/needle/haystack/shared/local_settings.py: File exists - /opt/needle/haystack/shared/local_settings.py

This use case works under 0.10.4.

Show
Cameron Johnston added a comment - I believe this patch has broken the deploy resource in the way it is used by my organization, namely that this fix does not seem to take into account the possibility that one will want to symlink regular files as well as directories. e.g. specifying symlinks "local_settings.py" => "local_settings.py" as a parameter for a deploy resource results in an error like this one: (haystack::default line 139) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /opt/needle/haystack/shared/local_settings.py: File exists - /opt/needle/haystack/shared/local_settings.py This use case works under 0.10.4.
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

I don't seem to be able to reproduce this. Could you provide the resource that you're using or provide a basic resource to reproduce?

Show
Bryan McLellan [Opscode] added a comment - I don't seem to be able to reproduce this. Could you provide the resource that you're using or provide a basic resource to reproduce?
Hide
Permalink
Cameron Johnston added a comment - - edited

Here's what I'm using, minus the before_restart and after_restart blocks: https://gist.github.com/1440050

Show
Cameron Johnston added a comment - - edited Here's what I'm using, minus the before_restart and after_restart blocks: https://gist.github.com/1440050
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

I'm terribly with the deploy resource.

This recipe: 

deploy_revision 'chef' do
  revision "HEAD"
  action :deploy
  repository 'git://github.com/opscode/chef.git'
  user "root"
  group "root"
  deploy_to "/srv/chef"
  migrate false
  symlinks "notes" => "notes"
  symlink_before_migrate Hash.new
end

Setup and run like this:

sudo mkdir /srv ; cd /srv
sudo rm -rf chef /var/chef/cache/revision-deploys/chef ; sudo mkdir -p chef/shared ;  echo "something" | sudo tee chef/shared/notes
sudo chef-client _0.10.4_

Produces:

btm@btm-mbp-dev:/srv $ sudo chef-client _0.10.4_
[Tue, 06 Dec 2011 16:19:09 -0800] INFO: *** Chef 0.10.4 ***
[Tue, 06 Dec 2011 16:19:11 -0800] INFO: Run List is [recipe[deploy-symlinks]]
[Tue, 06 Dec 2011 16:19:11 -0800] INFO: Run List expands to [deploy-symlinks]
[Tue, 06 Dec 2011 16:19:11 -0800] INFO: Starting Chef Run for btm.foo.org
[Tue, 06 Dec 2011 16:19:12 -0800] INFO: Loading cookbooks [deploy-symlinks]
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: Storing updated cookbooks/deploy-symlinks/recipes/default.rb in the cache.
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: Processing deploy_revision[chef] action deploy (deploy-symlinks::default line 9)
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] cloning repo git://github.com/opscode/chef.git to /srv/chef/shared/cached-copy
[Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] sh(git clone  git://github.com/opscode/chef.git /srv/chef/shared/cached-copy)
Initialized empty Git repository in /srv/chef/shared/cached-copy/.git/
[Tue, 06 Dec 2011 16:21:10 -0800] INFO: deploy_revision[chef] checked out branch: HEAD reference: df38da887a542d1452dba5e09eba38ffc1bb2edc
[Tue, 06 Dec 2011 16:21:10 -0800] INFO: deploy_revision[chef] copied the cached checkout to /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] made pre-migration symlinks
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] purged directories in checkout log, tmp/pids, public/system
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] created directories before symlinking tmp,public,config
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] linked shared paths into current release: notes => notes
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] made pre-migration symlinks
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] linked release /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc into production at /srv/chef/current
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] updated symlinks
[Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] deployed to /srv/chef
[Tue, 06 Dec 2011 16:21:13 -0800] INFO: Chef Run complete in 121.396072 seconds
[Tue, 06 Dec 2011 16:21:13 -0800] INFO: Running report handlers
[Tue, 06 Dec 2011 16:21:13 -0800] INFO: Report handlers complete
btm@btm-mbp-dev:/srv $ find . -name notes -exec ls -l '{}' \;
lrwxrwxrwx 1 root root 22 2011-12-06 16:21 ./chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc/notes -> /srv/chef/shared/notes
total 4
-rw-r--r-- 1 root root 1212 2011-12-06 16:21 topic-queue-benchmarks.md
-rw-r--r-- 1 root root 10 2011-12-06 16:18 ./chef/shared/notes
total 4
-rw-r--r-- 1 root root 1212 2011-12-06 16:21 topic-queue-benchmarks.md
btm@btm-mbp-dev:/srv $ cat chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc/notes
something

On 0.10.6.rc.4:

btm@btm-mbp-dev:/srv $ sudo chef-client _0.10.6.rc.4_
[Tue, 06 Dec 2011 16:28:54 -0800] INFO: *** Chef 0.10.6.rc.4 ***
[Tue, 06 Dec 2011 16:28:56 -0800] INFO: Run List is [recipe[deploy-symlinks]]
[Tue, 06 Dec 2011 16:28:56 -0800] INFO: Run List expands to [deploy-symlinks]
[Tue, 06 Dec 2011 16:28:56 -0800] INFO: Starting Chef Run for btm.foo.org
[Tue, 06 Dec 2011 16:28:56 -0800] INFO: Running start handlers
[Tue, 06 Dec 2011 16:28:56 -0800] INFO: Start handlers complete.
[Tue, 06 Dec 2011 16:28:57 -0800] INFO: Loading cookbooks [deploy-symlinks]
[Tue, 06 Dec 2011 16:28:57 -0800] INFO: Processing deploy_revision[chef] action deploy (deploy-symlinks::default line 9)
[Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] cloning repo git://github.com/opscode/chef.git to /srv/chef/shared/cached-copy
[Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] sh(git clone  git://github.com/opscode/chef.git /srv/chef/shared/cached-copy)
Initialized empty Git repository in /srv/chef/shared/cached-copy/.git/
[Tue, 06 Dec 2011 16:31:08 -0800] INFO: deploy_revision[chef] checked out branch: HEAD reference: df38da887a542d1452dba5e09eba38ffc1bb2edc
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] copied the cached checkout to /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] set user to root
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] set group to root
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] made pre-migration symlinks
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] purged directories in checkout log, tmp/pids, public/system
[Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] created directories before symlinking tmp,public,config
[Tue, 06 Dec 2011 16:31:09 -0800] ERROR: deploy_revision[chef] (deploy-symlinks::default line 9) has had an error
[Tue, 06 Dec 2011 16:31:09 -0800] ERROR: deploy_revision[chef] (/var/chef/cache/cookbooks/deploy-symlinks/recipes/default.rb:9:in `from_file') had an error:
deploy_revision[chef] (deploy-symlinks::default line 9) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /srv/chef/shared/notes: File exists - /srv/chef/shared/notes
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:379:in `create_dir_unless_exists'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:276:in `link_tempfiles_to_current_release'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:275:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:275:in `link_tempfiles_to_current_release'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:179:in `symlink'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:127:in `deploy'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:73:in `action_deploy'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:384:in `with_rollback_on_error'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:72:in `action_deploy'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource.rb:440:in `send'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource.rb:440:in `run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:45:in `run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `each'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:76:in `converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/client.rb:312:in `converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/client.rb:160:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:239:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:229:in `loop'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:229:in `run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application.rb:67:in `run'
/usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/chef-client:26
/usr/bin/chef-client:19:in `load'
/usr/bin/chef-client:19
[Tue, 06 Dec 2011 16:31:09 -0800] ERROR: Running exception handlers
[Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Saving node information to /var/chef/cache/failed-run-data.json
[Tue, 06 Dec 2011 16:31:09 -0800] ERROR: Exception handlers complete
[Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Chef::Exceptions::FileNotFound: deploy_revision[chef] (deploy-symlinks::default line 9) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /srv/chef/shared/notes: File exists - /srv/chef/shared/notes
Show
Bryan McLellan [Opscode] added a comment - I'm terribly with the deploy resource. This recipe: deploy_revision 'chef' do revision "HEAD" action :deploy repository 'git: //github.com/opscode/chef.git' user "root" group "root" deploy_to "/srv/chef" migrate false symlinks "notes" => "notes" symlink_before_migrate Hash. new end Setup and run like this: sudo mkdir /srv ; cd /srv sudo rm -rf chef / var /chef/cache/revision-deploys/chef ; sudo mkdir -p chef/shared ; echo "something" | sudo tee chef/shared/notes sudo chef-client _0.10.4_ Produces: btm@btm-mbp-dev:/srv $ sudo chef-client _0.10.4_ [Tue, 06 Dec 2011 16:19:09 -0800] INFO: *** Chef 0.10.4 *** [Tue, 06 Dec 2011 16:19:11 -0800] INFO: Run List is [recipe[deploy-symlinks]] [Tue, 06 Dec 2011 16:19:11 -0800] INFO: Run List expands to [deploy-symlinks] [Tue, 06 Dec 2011 16:19:11 -0800] INFO: Starting Chef Run for btm.foo.org [Tue, 06 Dec 2011 16:19:12 -0800] INFO: Loading cookbooks [deploy-symlinks] [Tue, 06 Dec 2011 16:19:13 -0800] INFO: Storing updated cookbooks/deploy-symlinks/recipes/ default .rb in the cache. [Tue, 06 Dec 2011 16:19:13 -0800] INFO: Processing deploy_revision[chef] action deploy (deploy-symlinks:: default line 9) [Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] cloning repo git: //github.com/opscode/chef.git to /srv/chef/shared/cached-copy [Tue, 06 Dec 2011 16:19:13 -0800] INFO: deploy_revision[chef] sh(git clone git: //github.com/opscode/chef.git /srv/chef/shared/cached-copy) Initialized empty Git repository in /srv/chef/shared/cached-copy/.git/ [Tue, 06 Dec 2011 16:21:10 -0800] INFO: deploy_revision[chef] checked out branch: HEAD reference: df38da887a542d1452dba5e09eba38ffc1bb2edc [Tue, 06 Dec 2011 16:21:10 -0800] INFO: deploy_revision[chef] copied the cached checkout to /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] made pre-migration symlinks [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] purged directories in checkout log, tmp/pids, public /system [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] created directories before symlinking tmp, public ,config [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] linked shared paths into current release: notes => notes [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] made pre-migration symlinks [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] linked release /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc into production at /srv/chef/current [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] updated symlinks [Tue, 06 Dec 2011 16:21:11 -0800] INFO: deploy_revision[chef] deployed to /srv/chef [Tue, 06 Dec 2011 16:21:13 -0800] INFO: Chef Run complete in 121.396072 seconds [Tue, 06 Dec 2011 16:21:13 -0800] INFO: Running report handlers [Tue, 06 Dec 2011 16:21:13 -0800] INFO: Report handlers complete btm@btm-mbp-dev:/srv $ find . -name notes -exec ls -l '{}' \; lrwxrwxrwx 1 root root 22 2011-12-06 16:21 ./chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc/notes -> /srv/chef/shared/notes total 4 -rw-r--r-- 1 root root 1212 2011-12-06 16:21 topic-queue-benchmarks.md -rw-r--r-- 1 root root 10 2011-12-06 16:18 ./chef/shared/notes total 4 -rw-r--r-- 1 root root 1212 2011-12-06 16:21 topic-queue-benchmarks.md btm@btm-mbp-dev:/srv $ cat chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc/notes something On 0.10.6.rc.4: btm@btm-mbp-dev:/srv $ sudo chef-client _0.10.6.rc.4_ [Tue, 06 Dec 2011 16:28:54 -0800] INFO: *** Chef 0.10.6.rc.4 *** [Tue, 06 Dec 2011 16:28:56 -0800] INFO: Run List is [recipe[deploy-symlinks]] [Tue, 06 Dec 2011 16:28:56 -0800] INFO: Run List expands to [deploy-symlinks] [Tue, 06 Dec 2011 16:28:56 -0800] INFO: Starting Chef Run for btm.foo.org [Tue, 06 Dec 2011 16:28:56 -0800] INFO: Running start handlers [Tue, 06 Dec 2011 16:28:56 -0800] INFO: Start handlers complete. [Tue, 06 Dec 2011 16:28:57 -0800] INFO: Loading cookbooks [deploy-symlinks] [Tue, 06 Dec 2011 16:28:57 -0800] INFO: Processing deploy_revision[chef] action deploy (deploy-symlinks:: default line 9) [Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] cloning repo git: //github.com/opscode/chef.git to /srv/chef/shared/cached-copy [Tue, 06 Dec 2011 16:28:58 -0800] INFO: deploy_revision[chef] sh(git clone git: //github.com/opscode/chef.git /srv/chef/shared/cached-copy) Initialized empty Git repository in /srv/chef/shared/cached-copy/.git/ [Tue, 06 Dec 2011 16:31:08 -0800] INFO: deploy_revision[chef] checked out branch: HEAD reference: df38da887a542d1452dba5e09eba38ffc1bb2edc [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] copied the cached checkout to /srv/chef/releases/df38da887a542d1452dba5e09eba38ffc1bb2edc [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] set user to root [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] set group to root [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] made pre-migration symlinks [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] purged directories in checkout log, tmp/pids, public /system [Tue, 06 Dec 2011 16:31:09 -0800] INFO: deploy_revision[chef] created directories before symlinking tmp, public ,config [Tue, 06 Dec 2011 16:31:09 -0800] ERROR: deploy_revision[chef] (deploy-symlinks:: default line 9) has had an error [Tue, 06 Dec 2011 16:31:09 -0800] ERROR: deploy_revision[chef] (/ var /chef/cache/cookbooks/deploy-symlinks/recipes/ default .rb:9:in `from_file') had an error: deploy_revision[chef] (deploy-symlinks:: default line 9) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /srv/chef/shared/notes: File exists - /srv/chef/shared/notes /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:379:in `create_dir_unless_exists' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:276:in `link_tempfiles_to_current_release' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:275:in `each' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:275:in `link_tempfiles_to_current_release' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:179:in `symlink' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:127:in `deploy' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:73:in `action_deploy' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:384:in `with_rollback_on_error' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/provider/deploy.rb:72:in `action_deploy' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource.rb:440:in `send' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource.rb:440:in `run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:45:in `run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `each' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:81:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/runner.rb:76:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/client.rb:312:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/client.rb:160:in `run' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:239:in `run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:229:in `loop' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application/client.rb:229:in `run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/../lib/chef/application.rb:67:in `run' /usr/lib/ruby/gems/1.8/gems/chef-0.10.6.rc.4/bin/chef-client:26 /usr/bin/chef-client:19:in `load' /usr/bin/chef-client:19 [Tue, 06 Dec 2011 16:31:09 -0800] ERROR: Running exception handlers [Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Saving node information to / var /chef/cache/failed-run-data.json [Tue, 06 Dec 2011 16:31:09 -0800] ERROR: Exception handlers complete [Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Stacktrace dumped to / var /chef/cache/chef-stacktrace.out [Tue, 06 Dec 2011 16:31:09 -0800] FATAL: Chef::Exceptions::FileNotFound: deploy_revision[chef] (deploy-symlinks:: default line 9) had an error: Chef::Exceptions::FileNotFound: Cannot create directory /srv/chef/shared/notes: File exists - /srv/chef/shared/notes
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

This fixes for me. Cameron, can you test as well please, I'm uploading 0.10.6.rc.5 right now.

commit da39fa1fa3c1fe323900b5653d8d1d2d4aad3ffa
Author: Bryan McLellan <btm@opscode.com>
Date:   Tue Dec 6 16:48:31 2011 -0800

    CHEF-630: do not create directories for links

diff --git a/chef/lib/chef/provider/deploy.rb b/chef/lib/chef/provider/deploy.rb
index 8ede7d3..7e7ba5a 100644
--- a/chef/lib/chef/provider/deploy.rb
+++ b/chef/lib/chef/provider/deploy.rb
@@ -273,7 +273,6 @@ class Chef
 
         links_info = @new_resource.symlinks.map { |src, dst| "#{src} => #{dst}" }.join(", ")
         @new_resource.symlinks.each do |src, dest|
-          create_dir_unless_exists(::File.join(@new_resource.shared_path, src))
           begin
             FileUtils.ln_sf(::File.join(@new_resource.shared_path, src), ::File.join(release_path, dest))
           rescue => e
Show
Bryan McLellan [Opscode] added a comment - This fixes for me. Cameron, can you test as well please, I'm uploading 0.10.6.rc.5 right now. commit da39fa1fa3c1fe323900b5653d8d1d2d4aad3ffa Author: Bryan McLellan <btm@opscode.com> Date: Tue Dec 6 16:48:31 2011 -0800 CHEF-630: do not create directories for links diff --git a/chef/lib/chef/provider/deploy.rb b/chef/lib/chef/provider/deploy.rb index 8ede7d3..7e7ba5a 100644 --- a/chef/lib/chef/provider/deploy.rb +++ b/chef/lib/chef/provider/deploy.rb @@ -273,7 +273,6 @@ class Chef links_info = @new_resource.symlinks.map { |src, dst| "#{src} => #{dst}" }.join( ", " ) @new_resource.symlinks.each do |src, dest| - create_dir_unless_exists(::File.join(@new_resource.shared_path, src)) begin FileUtils.ln_sf(::File.join(@new_resource.shared_path, src), ::File.join(release_path, dest)) rescue => e
Hide
Permalink
Cameron Johnston added a comment -

Fix is working for me. Thanks!

Show
Cameron Johnston added a comment - Fix is working for me. Thanks!
Hide
Permalink
Juanje Ojeda added a comment -

I've been trying to use this resource for awhile and I still need to crreate the :deploy_to directory before the deploy's definition.

I'm testing on version Chef 0.10.8.

The error I get is because deploy tries to enforce_ownership before the directory be created, so it fails when the directory does not exist:

...
[Fri, 04 May 2012 02:28:01 -0700] ERROR: Running exception handlers
[Fri, 04 May 2012 02:28:01 -0700] ERROR: Exception handlers complete
[Fri, 04 May 2012 02:28:01 -0700] DEBUG: Re-raising exception: Errno::ENOENT - deploy_revision[/opt/redmine] (redmine::default line 90) had an error: Errno::ENOENT: No such file or directory - /opt/redmine
/opt/ruby/lib/ruby/1.8/fileutils.rb:1231:in `chown'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:1231:in `chown'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:967:in `chown_R'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:1331:in `traverse'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:965:in `chown_R'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:964:in `each'
  /opt/ruby/lib/ruby/1.8/fileutils.rb:964:in `chown_R'
  /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:234:in `enforce_ownership'
  /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:118:in `deploy'
  /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:73:in `action_deploy'
  /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:383:in `with_rollback_on_error'
  /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:72:in `action_deploy'
...

It can be easily fixed by moving up the verify_directories_exist at the deploy method:

--- a/chef/lib/chef/provider/deploy.rb
+++ b/chef/lib/chef/provider/deploy.rb
@@ -115,8 +115,8 @@ class Chef
       end
 
       def deploy
-        enforce_ownership
         verify_directories_exist
+        enforce_ownership
         update_cached_repo
         copy_cached_repo
         install_gems

BTW, the test pass because of the mocks. I think it probably should use real files/directories for this test. It could slow down a bit the test, but it will me more real and it avoid this kind of false positives.

Show
Juanje Ojeda added a comment - I've been trying to use this resource for awhile and I still need to crreate the :deploy_to directory before the deploy's definition. I'm testing on version Chef 0.10.8 . The error I get is because deploy tries to enforce_ownership before the directory be created, so it fails when the directory does not exist: ... [Fri, 04 May 2012 02:28:01 -0700] ERROR: Running exception handlers [Fri, 04 May 2012 02:28:01 -0700] ERROR: Exception handlers complete [Fri, 04 May 2012 02:28:01 -0700] DEBUG: Re-raising exception: Errno::ENOENT - deploy_revision[/opt/redmine] (redmine:: default line 90) had an error: Errno::ENOENT: No such file or directory - /opt/redmine /opt/ruby/lib/ruby/1.8/fileutils.rb:1231:in `chown' /opt/ruby/lib/ruby/1.8/fileutils.rb:1231:in `chown' /opt/ruby/lib/ruby/1.8/fileutils.rb:967:in `chown_R' /opt/ruby/lib/ruby/1.8/fileutils.rb:1331:in `traverse' /opt/ruby/lib/ruby/1.8/fileutils.rb:965:in `chown_R' /opt/ruby/lib/ruby/1.8/fileutils.rb:964:in `each' /opt/ruby/lib/ruby/1.8/fileutils.rb:964:in `chown_R' /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:234:in `enforce_ownership' /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:118:in `deploy' /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:73:in `action_deploy' /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:383:in `with_rollback_on_error' /opt/ruby/lib/ruby/gems/1.8/gems/chef-0.10.8/bin/../lib/chef/provider/deploy.rb:72:in `action_deploy' ... It can be easily fixed by moving up the verify_directories_exist at the deploy method: --- a/chef/lib/chef/provider/deploy.rb +++ b/chef/lib/chef/provider/deploy.rb @@ -115,8 +115,8 @@ class Chef end def deploy - enforce_ownership verify_directories_exist + enforce_ownership update_cached_repo copy_cached_repo install_gems BTW, the test pass because of the mocks. I think it probably should use real files/directories for this test. It could slow down a bit the test, but it will me more real and it avoid this kind of false positives.
Hide
Permalink
Daniel DeLeo added a comment -

Do you think you could add a regression test for this? You should be able to tell rspec to expect method calls in a certain order using @obj.should_receive(:method).ordered.

I agree with your comment about having a test for this that does not use mocks or stubs. We've started putting these kinds of tests under spec/functional in the tree.

Show
Daniel DeLeo added a comment - Do you think you could add a regression test for this? You should be able to tell rspec to expect method calls in a certain order using @obj.should_receive(:method).ordered . I agree with your comment about having a test for this that does not use mocks or stubs. We've started putting these kinds of tests under spec/functional in the tree.
Hide
Permalink
Juanje Ojeda added a comment -

Hi Daniel,

I did use the .ordered method for the test and it works fine, thanks.

But I was wondering if it's really necesary to call enforce_ownership twice? It seems that with the second one is enough.

def deploy
    verify_directories_exist                                                                            
->  enforce_ownership                                                                                   
    update_cached_repo                                                                                  
    copy_cached_repo                                                                                    
    install_gems                                                                                        
->  enforce_ownership                                                                                   
    callback(:before_migrate, @new_resource.before_migrate)                                             
...

Is there any reason I don't see? Can the first call be removed?

Thanks

Show
Juanje Ojeda added a comment - Hi Daniel, I did use the .ordered method for the test and it works fine, thanks. But I was wondering if it's really necesary to call enforce_ownership twice? It seems that with the second one is enough. def deploy verify_directories_exist -> enforce_ownership update_cached_repo copy_cached_repo install_gems -> enforce_ownership callback(:before_migrate, @new_resource.before_migrate) ... Is there any reason I don't see? Can the first call be removed? Thanks
Hide
Permalink
Juanje Ojeda added a comment -

The proposed fix and the regresion test are at the pull-request.

Show
Juanje Ojeda added a comment - The proposed fix and the regresion test are at the pull-request .
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

Removing triage bit so it'll get looked at again at code review.

Show
Bryan McLellan [Opscode] added a comment - Removing triage bit so it'll get looked at again at code review.
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

Looks good, marked for merge.

Show
Bryan McLellan [Opscode] added a comment - Looks good, marked for merge.
Hide
Permalink
Juanje Ojeda added a comment -

Hummm... something wrong with this bug? Any plans with it to be merge?

Show
Juanje Ojeda added a comment - Hummm... something wrong with this bug? Any plans with it to be merge?
Hide
Permalink
Bryan McLellan [Opscode] added a comment -

Merged to master. Thanks for the added work Juanje.

Show
Bryan McLellan [Opscode] added a comment - Merged to master. Thanks for the added work Juanje.

People

  • Assignee:
    Unassigned
    Reporter:
    Daniel DeLeo
Vote (0)
Watch (2)

Dates

  • Created:
    Updated:
    Resolved: