Details
-
Type:
Bug
-
Status:
Closed
-
Priority:
Blocker
-
Resolution: Fixed
-
Affects Version/s: 0.9.6
-
Fix Version/s: 0.10.4
-
Component/s: Chef Client
-
Labels:None
Description
The package "libmysqlclient-dev" is a virtual package that doesn't show up in apt-cache policy, but specifying it for installation with apt-get install will auto-resolve dependencies:
lenny1test:~ $ apt-cache policy libmysqlclient-dev libmysqlclient-dev: Installed: (none) Candidate: (none) Version table: lenny1test:~ $ apt-cache show libmysqlclient-dev lenny1test:~ $ echo $? 0 lenny1test:~ $ sudo apt-get install libmysqlclient-dev Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting libmysqlclient15-dev instead of libmysqlclient-dev The following extra packages will be installed: libmysqlclient15-dev libmysqlclient15off mysql-common zlib1g-dev The following NEW packages will be installed: libmysqlclient15-dev libmysqlclient15off mysql-common zlib1g-dev 0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded. Need to get 9716kB of archives. After this operation, 31.3MB of additional disk space will be used. Do you want to continue [Y/n]?
Activity
- All
- Comments
- History
- Activity
- Transitions Summary
Please merge with: http://github.com/toomasp/chef/tree/CHEF-1439
Chef::Provider::Package::Apt when loading current resource - should create a current resource with the name of the new_resource - should set the current resources package name to the new resources package name - should run aptitude show with the package name - should read stdout on aptitude show - should set the installed version to nil on the current resource if package state is not installed - should set the installed version if package has one - should set the candidate version if aptitude show has one - should raise an exception if aptitude show fails - should not raise an exception if aptitude show succeeds - should raise an exception if aptitude show does not return a candidate version - should return the current resouce - should set candidate version to new package name if virtual package Chef::Provider::Package::Apt install_package - should run apt-get install with the package name and version - should run apt-get install with the package name and version and options if specified Chef::Provider::Package::Apt Chef::Provider::Package::Apt upgrade_package - should run install_package with the name and version Chef::Provider::Package::Apt Chef::Provider::Package::Apt remove_package - should run apt-get remove with the package name - should run apt-get remove with the package name and options if specified Chef::Provider::Package::Apt when purging a package - should run apt-get purge with the package name - should run apt-get purge with the package name and options if specified Chef::Provider::Package::Apt when preseeding a package - should get the full path to the preseed response file - should run debconf-set-selections on the preseed file if it has changed - should not run debconf-set-selections if the preseed file has not changed Finished in 0.053094 seconds 22 examples, 0 failures
This is a great patch, it will make man people's lives easier. A few notes:
- it's best not to use predicate method names (i.e., ending with a '?') for methods that mutate state, as in most cases these methods don't modify state in ruby so it's counter to expectations to do that.
- I know a lot of our existing tests are really bad offenders here, but you don't need to mock out strings, you can just use strings. For example, since ShellOut#stdout returns a string, you don't need to stub #each_line on it, you can just return a string that matches the expected output of that command and not worry about how the underlying code processes the string.
Those are just nitpicks though, this patch is solid and a really sweet improvement to the provider. Thanks!
This patch fixes what it supposed to fix, but broke normal apt functionalities, so I will reopen this ticket. The code has been reverted in the master branch, for now.
I found the following problems with the patch, please take a look and fix so that we can merge the patch back to master.
Problem 1:
The code tries to read the "Depends:" line and call check_package_state recursively for the whole "Depends:" line, which causes syntax error. For example:
aptitude show sudo
has the following line:
Depends: libc6 (>= 2.8), libpam0g (>= 0.99.7.1), libpam-modules
And then it tries to do:
aptitude show libc6 (>= 2.8), libpam0g (>= 0.99.7.1), libpam-module
which gets syntax error near unexpected token `('
Problem 2:
After a rough look, the logic may not be completely right as well. Still use the sudo package as example, on a box sudo has been installed, running aptitude show sudo returns the following:
Package: sudo
State: installed
Automatically installed: no
Version: 1.7.2p1-1ubuntu5
Priority: important
Section: admin
Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Uncompressed Size: 602k
Depends: libc6 (>= 2.8), libpam0g (>= 0.99.7.1), libpam-modules
Conflicts: sudo-ldap
Replaces: sudo-ldap
Provided by: sudo-ldap
Description: Provide limited super user privileges to specific users
Sudo is a program designed to allow a sysadmin to give limited root privileges
to users and log root activity. The basic philosophy is to give as few
privileges as possible but still allow people to get their work done.
This version is built with minimal shared library dependencies, use the
sudo-ldap package instead if you need LDAP support.
Say we have a recipe like this:
package "sudo" do
action :install
end
And then the current code tries to call check_package_state recursively for "sudo-ldap" after doing so for "sudo", which is not necessary/correct in this case since sudo is installed and it should really not trying to see if sudo-ldap is installed or not.
Tests for the 2 problems Nou mentioned are passing now - available at http://github.com/toomasp/chef/tree/CHEF-1439 again. BUT - as for the logic that chooses which virtual pacakge to install - any better ideas?
This appears to be still happening with Chef 0.10 on Debian 5.0 (Lenny)
Chef run output:
[Thu, 02 Jun 2011 16:32:38 +0000] ERROR: package[mysql-devel] (mysql::client line 20) has had an error [Thu, 02 Jun 2011 16:32:38 +0000] ERROR: Running exception handlers [Thu, 02 Jun 2011 16:32:39 +0000] FATAL: Saving node information to /var/chef/cache/failed-run-data.json [Thu, 02 Jun 2011 16:32:39 +0000] ERROR: Exception handlers complete [Thu, 02 Jun 2011 16:32:39 +0000] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out [Thu, 02 Jun 2011 16:32:39 +0000] FATAL: Chef::Exceptions::Package: package[mysql-devel] (mysql::client line 20) had an error: apt does not have a version of package libmysqlclient-dev ...... slice20013067:~# cat /var/chef/cache/chef-stacktrace.out Generated at Thu Jun 02 16:32:39 +0000 2011 Chef::Exceptions::Package: package[mysql-devel] (mysql::client line 20) had an error: apt does not have a version of package libmysqlclient-dev /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/provider/package/apt.rb:57:in `load_current_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource.rb:416:in `run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/runner.rb:45:in `run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/runner.rb:78:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/runner.rb:78:in `each' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/runner.rb:78:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection.rb:94:in `execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in `step' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/resource_collection.rb:92:in `execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/runner.rb:76:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/client.rb:312:in `converge' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/client.rb:160:in `run' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/application/client.rb:239:in `run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/application/client.rb:229:in `loop' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/application/client.rb:229:in `run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/../lib/chef/application.rb:66:in `run' /usr/lib/ruby/gems/1.8/gems/chef-0.10.0/bin/chef-client:26 /usr/bin/chef-client:19:in `load'
Assorted debug info:
slice20013067:~# apt-cache policy libmysqlclient-dev libmysqlclient-dev: Installed: (none) Candidate: (none) Version table: slice20013067:~# apt-cache show libmysqlclient-dev slice20013067:~# echo $? 0 slice20013067:~# sudo apt-get install libmysqlclient-dev Reading package lists... Done Building dependency tree Reading state information... Done Note, selecting libmysqlclient15-dev instead of libmysqlclient-dev The following extra packages will be installed: libmysqlclient15-dev zlib1g-dev The following NEW packages will be installed: libmysqlclient15-dev zlib1g-dev 0 upgraded, 2 newly installed, 0 to remove and 38 not upgraded. Need to get 7753kB of archives. After this operation, 26.7MB of additional disk space will be used. Do you want to continue [Y/n]? ^C
changing back to fixed/resolved. just realized this was never merged in.
The new fix was verified working and was merged to master.
Thanks Toomas for the great work.
https://github.com/opscode/chef/commit/16dd475cd3470de9f1f3818c22916ba6670b806f
0.10.2 was a security release, 0.10.4 is the next maintenance release.
Bulk edit: this ticket should have been previously merged for 0.10.2, but will be released in 0.10.4
Testing prerelease Gem Chef 0.10.4-rc.2, I ran into issues installing the imagemagick package.
I think it's just some bad string parsing, note the comma:
[Thu, 28 Jul 2011 13:17:22 -0500] DEBUG: sh(aptitude show libbz2-1.0,)
Note the issue in the last comment, aptitude show is called on packages with a comma appended in some circumstances.
Bumping this to a blocker, we'll get it fixed and ship a new RC.
https://github.com/opscode/chef/commit/33085fdf2e1b481ba6e87c82004702d21e5fd6dd
Regex fix + test merged. 0.10.4.rc.3 released to rubygems.org
list of virtual packages for future reference: http://www.debian.org/doc/packaging-manuals/virtual-package-names-list.txt
I tried the latest Gem with imagemagick on a clean Ubuntu 10.04. Now it thinks it's installed, when Aptitude reports it isn't. Not sure why imagemagick is being so difficult. ![]()
[Fri, 29 Jul 2011 23:48:16 -0500] INFO: *** Chef 0.10.4.rc.3 *** <snip> [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Loading Recipe imagemagick via include_recipe [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Found recipe default in cookbook imagemagick [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Loading from cookbook_path: /tmp/vagrant-chef/cookbooks-0 [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Converging node vagrantup.com [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Processing package[imagemagick] on vagrantup.com [Fri, 29 Jul 2011 23:48:18 -0500] INFO: Processing package[imagemagick] action install (imagemagick::default line 24) [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Checking package status for imagemagick [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: sh(aptitude show imagemagick) Package: imagemagick New: yes State: not installed Version: 7:6.5.7.8-1ubuntu1.1 Priority: optional Section: graphics Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com> Uncompressed Size: 319k Depends: libbz2-1.0, libc6 (>= 2.2.5), libfreetype6 (>= 2.2.1), libgomp1 (>= 4.2.1), libice6 (>= 1:1.0.0), libjpeg62, liblcms1 (>= 1.15-1), libltdl7 (>= 2.2.6b), libmagickcore2 (>= 7:6.5.7.8), libmagickwand2 (>= 7:6.5.7.8), libsm6, libtiff4, libx11-6, libxext6, libxt6, zlib1g (>= 1:1.1.4) Recommends: libmagickcore2-extra Suggests: imagemagick-doc, transfig Provided by: graphicsmagick-imagemagick-compat Description: image manipulation programs ImageMagick is a software suite to create, edit, and compose bitmap images. It can read, convert and write images in a variety of formats (over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD, PNG, Postscript, SVG, and TIFF. Use ImageMagick to translate, flip, mirror, rotate, scale, shear and transform images, adjust image colors, apply various special effects, or draw text, lines, polygons, ellipses and B?zier curves. All manipulations can be achieved through shell commands as well as through an X11 graphical interface (display). Homepage: http://www.imagemagick.org/ [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Virtual package provided by libbz2-1.0 [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Checking package status for libbz2-1.0 [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: sh(aptitude show libbz2-1.0) Package: libbz2-1.0 State: installed Automatically installed: no Version: 1.0.5-4ubuntu0.1 Priority: important Section: libs Maintainer: Ubuntu Core Developers <ubuntu-devel-discuss@lists.ubuntu.com> Uncompressed Size: 131k Depends: libc6 (>= 2.4) Description: high-quality block-sorting file compressor library - runtime This package contains libbzip2 which is used by the bzip2 compressor. bzip2 is a freely available, patent free, high-quality data compressor. It typically compresses files to within 10% to 15% of the best available techniques, whilst being around twice as fast at compression and six times faster at decompression. bzip2 compresses files using the Burrows-Wheeler block-sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by more conventional LZ77/LZ78-based compressors, and approaches the performance of the PPM family of statistical compressors. The archive file format of bzip2 (.bz2) is incompatible with that of its predecessor, bzip (.bz). Homepage: http://www.bzip.org/ [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: package[imagemagick] is already installed - nothing to do [Fri, 29 Jul 2011 23:48:18 -0500] INFO: Chef Run complete in 1.692067 seconds [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Cleaning the checksum cache [Fri, 29 Jul 2011 23:48:18 -0500] INFO: Running report handlers [Fri, 29 Jul 2011 23:48:18 -0500] INFO: Report handlers complete [Fri, 29 Jul 2011 23:48:18 -0500] DEBUG: Exiting
We shouldn't even be touching imagemagick with this code. That isn't a virtual package. I'm working on some refactoring.
commit 4c9b9e60c93491f68abe15f49996ab6db4b88b60
If a virtual package is provided by a single real package, we ask apt to install the virtual package and it in turn installs the real package. The same goes for removal.
If a virtual package is provided by multiple real package, we throw an exception asking the user to be more explicit, which is what apt does.
aptitude is slow.
apt-cache policy doesn't differentiate between virtual packages with one provider and those with more than one.
https://github.com/opscode/chef/commit/47ade45014af680a1e16a6b9c0f021bd277509f5
refactored to use 'apt-cache policy' again. now we parse the output of 'apt-cache showpkg' if we think we're a virtual package to verify that there is a provider via 'Reverse Provides:'
aptitude will reveal that the package is virtual and list the package name that provides the actual package