Tuesday, August 12, 2014

ChefDK is easier to install Berkshelf than via gem on Amazon Linux AMI

I ended up installing the latest berkshelf, 3.x via gem on Amazon Linux AMI because it is quite difficult to deal with the dependency issues. Instead, I decided to use Chef Development Kit because it contains Berkshelf and is easy to install without worrying about such issues.

Installing ChefDK followed by the official instruction
$ sudo yum install https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chefdk-0.2.0-2.el6.x86_64.rpm

Confirming the installation pathpath
$ ll `which chef`
lrwxrwxrwx 1 root root 20 Aug 11 10:38 /usr/bin/chef -> /opt/chefdk/bin/chef

Confirming Berkshelf installed
$ /opt/chefdk/embedded/bin/gem list | grep 'berkshelf'
berkshelf (3.1.3)
berkshelf-api (1.4.0)
berkshelf-api-client (1.2.0)

Verifying the main components of ChefDK
$ chef verify
Running verification for component 'berkshelf'
Running verification for component 'test-kitchen'
Running verification for component 'chef-client'
Running verification for component 'chef-dk'
..............
---------------------------------------------
Verification of component 'chef-dk' succeeded.
Verification of component 'berkshelf' succeeded.
Verification of component 'chef-client' succeeded.
Verification of component 'test-kitchen' succeeded.

Installing knife solo
Just make sure that you install knife solo because ChedDK does not contain it.
$ chef gem install knife-solo

The following is the output that I failed to install chef via gem.

Installing berkshelf without specifying the version
Failed to install because of the lack of the dependent libraries.
$ gem i berkshelf
Fetching: addressable-2.3.6.gem (100%)
Successfully installed addressable-2.3.6
Fetching: multipart-post-2.0.0.gem (100%)
Successfully installed multipart-post-2.0.0
Fetching: faraday-0.9.0.gem (100%)
Successfully installed faraday-0.9.0
Fetching: berkshelf-api-client-1.2.0.gem (100%)
Successfully installed berkshelf-api-client-1.2.0
Fetching: hashie-2.1.2.gem (100%)
Successfully installed hashie-2.1.2
Fetching: buff-extensions-1.0.0.gem (100%)
Successfully installed buff-extensions-1.0.0
Fetching: varia_model-0.4.0.gem (100%)
Successfully installed varia_model-0.4.0
Fetching: buff-config-1.0.1.gem (100%)
Successfully installed buff-config-1.0.1
Fetching: buff-ruby_engine-0.1.0.gem (100%)
Successfully installed buff-ruby_engine-0.1.0
Fetching: buff-shell_out-0.1.1.gem (100%)
Successfully installed buff-shell_out-0.1.1
Fetching: minitar-0.5.4.gem (100%)
Successfully installed minitar-0.5.4
Fetching: retryable-1.3.5.gem (100%)
Successfully installed retryable-1.3.5
Fetching: buff-ignore-1.1.1.gem (100%)
Successfully installed buff-ignore-1.1.1
Fetching: hitimes-1.2.2.gem (100%)
Building native extensions.  This could take a while...
ERROR:  Error installing berkshelf:
        ERROR: Failed to build gem native extension.

    /usr/bin/ruby2.0 extconf.rb
checking for clock_gettime() in -lrt... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib64
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/ruby2.0
        --with-rtlib
        --without-rtlib
/usr/share/ruby/2.0/mkmf.rb:434:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
        from /usr/share/ruby/2.0/mkmf.rb:519:in `try_link0'
        from /usr/share/ruby/2.0/mkmf.rb:534:in `try_link'
        from /usr/share/ruby/2.0/mkmf.rb:720:in `try_func'
        from /usr/share/ruby/2.0/mkmf.rb:950:in `block in have_library'
        from /usr/share/ruby/2.0/mkmf.rb:895:in `block in checking_for'
        from /usr/share/ruby/2.0/mkmf.rb:340:in `block (2 levels) in postpone'
        from /usr/share/ruby/2.0/mkmf.rb:310:in `open'
        from /usr/share/ruby/2.0/mkmf.rb:340:in `block in postpone'
        from /usr/share/ruby/2.0/mkmf.rb:310:in `open'
        from /usr/share/ruby/2.0/mkmf.rb:336:in `postpone'
        from /usr/share/ruby/2.0/mkmf.rb:894:in `checking_for'
        from /usr/share/ruby/2.0/mkmf.rb:945:in `have_library'
        from extconf.rb:10:in `

'
Gem files will remain installed in /home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2 for inspection. Results logged to /home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/gem_make.out

Installing hitimes via gem
Failed because the development tools should be installed before that.
$ gem i hitimes --verbose
HEAD https://rubygems.org/latest_specs.4.8.gz
302 Moved Temporarily
HEAD https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
304 Not Modified
Installing gem hitimes-1.2.2
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/.travis.yml
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/CONTRIBUTING.md
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/HISTORY.md
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/LICENSE
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/Manifest.txt
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/README.md
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/Rakefile
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/examples/benchmarks.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/examples/stats.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/extconf.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_instant_clock_gettime.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_instant_osx.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_instant_windows.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_interval.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_interval.h
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_stats.c
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/hitimes_stats.h
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/java/src/hitimes/Hitimes.java
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/java/src/hitimes/HitimesInterval.java
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/java/src/hitimes/HitimesService.java
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/java/src/hitimes/HitimesStats.java
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/metric.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/mutexed_stats.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/paths.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/stats.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/timed_metric.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/timed_value_metric.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/value_metric.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/lib/hitimes/version.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/hitimes_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/interval_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/metric_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/mutex_stats_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/paths_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/spec_helper.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/stats_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/timed_metric_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/timed_value_metric_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/value_metric_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/spec/version_spec.rb
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/tasks/default.rake
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/tasks/extension.rake
/home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/tasks/this.rb
Building native extensions.  This could take a while...
/usr/bin/ruby2.0 extconf.rb
checking for clock_gettime() in -lrt... *** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
        --with-opt-dir
        --without-opt-dir
        --with-opt-include
        --without-opt-include=${opt-dir}/include
        --with-opt-lib
        --without-opt-lib=${opt-dir}/lib64
        --with-make-prog
        --without-make-prog
        --srcdir=.
        --curdir
        --ruby=/usr/bin/ruby2.0
        --with-rtlib
        --without-rtlib
/usr/share/ruby/2.0/mkmf.rb:434:in `try_do': The compiler failed to generate an executable file. (RuntimeError)
You have to install development tools first.
        from /usr/share/ruby/2.0/mkmf.rb:519:in `try_link0'
        from /usr/share/ruby/2.0/mkmf.rb:534:in `try_link'
        from /usr/share/ruby/2.0/mkmf.rb:720:in `try_func'
        from /usr/share/ruby/2.0/mkmf.rb:950:in `block in have_library'
        from /usr/share/ruby/2.0/mkmf.rb:895:in `block in checking_for'
        from /usr/share/ruby/2.0/mkmf.rb:340:in `block (2 levels) in postpone'
        from /usr/share/ruby/2.0/mkmf.rb:310:in `open'
        from /usr/share/ruby/2.0/mkmf.rb:340:in `block in postpone'
        from /usr/share/ruby/2.0/mkmf.rb:310:in `open'
        from /usr/share/ruby/2.0/mkmf.rb:336:in `postpone'
        from /usr/share/ruby/2.0/mkmf.rb:894:in `checking_for'
        from /usr/share/ruby/2.0/mkmf.rb:945:in `have_library'
        from extconf.rb:10:in `

'
ERROR:  Error installing hitimes:         ERROR: Failed to build gem native extension.     Building has failed. See above output for more information on the failure. Gem files will remain installed in /home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2 for inspection. Results logged to /home/ec2-user/.gem/ruby/2.0/gems/hitimes-1.2.2/ext/hitimes/c/gem_make.out

Installing "Development libraries" and "Development tools"
Succeeded without any fail.
$ sudo yum -y groupinstall "Development Libraries" "Development tools"
$ rpm -qa |grep ruby
ruby20-2.0.0.451-1.14.amzn1.x86_64
rubygem20-json-1.7.7-101.27.amzn1.x86_64
ruby20-libs-2.0.0.451-1.14.amzn1.x86_64
rubygem20-psych-2.0.0-1.14.amzn1.x86_64
rubygems20-2.0.14-1.14.amzn1.noarch
rubygem20-rdoc-4.0.1-2.18.amzn1.noarch
ruby20-devel-2.0.0.451-1.14.amzn1.x86_64
ruby20-irb-2.0.0.451-1.14.amzn1.noarch
ruby-2.0-0.3.amzn1.noarch
rubygem20-bigdecimal-1.2.0-1.14.amzn1.x86_64

Installing hitimes with gem again
Succeeded.
$ gem i hitimes --verbose
HEAD https://rubygems.org/latest_specs.4.8.gz
302 Moved Temporarily
HEAD https://s3.amazonaws.com/production.s3.rubygems.org/latest_specs.4.8.gz
304 Not Modified
Installing gem hitimes-1.2.2
...
Successfully installed hitimes-1.2.2
Parsing documentation for hitimes-1.2.2
Parsing sources...
100% [14/14]  lib/hitimes/version.rb                                           
Installing ri documentation for hitimes-1.2.2
Done installing documentation for hitimes after 0 seconds
1 gem installed

Installing berkshel via gem
Failed because libgecode failed to be compiled.
$ gem i berkshelf --verbose
...
virtual memory exhausted: Cannot allocate memory
make[1]: *** [gecode/int/extensional.o] Error 1
make[1]: *** Waiting for unfinished jobs....
fvirtual memory exhausted: Cannot allocate memory
make[1]: *** [gecode/int/rel.o] Error 1


make[1]: Leaving directory `/home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2/ext/libgecode3/vendor/gecode-3.7.3'
make: *** [compilelib] Error 2
extconf.rb:98:in `block in run': Failed to build gecode library. (GecodeBuild::BuildError)
        from extconf.rb:97:in `chdir'
        from extconf.rb:97:in `run'
        from extconf.rb:104:in `

'
ERROR:  Error installing berkshelf:         ERROR: Failed to build gem native extension.     Building has failed. See above output for more information on the failure. Gem files will remain installed in /home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2 for inspection. Results logged to /home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2/ext/libgecode3/gem_make.out

Installing libgecode with gem
Failed to complie libgecode.
$ gem i dep-selector-libgecode
...
virtual memory exhausted: Cannot allocate memory
make[1]: *** [gecode/int/element.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2/ext/libgecode3/vendor/gecode-3.7.3'
make: *** [compilelib] Error 2
extconf.rb:98:in `block in run': Failed to build gecode library. (GecodeBuild::BuildError)
        from extconf.rb:97:in `chdir'
        from extconf.rb:97:in `run'
        from extconf.rb:104:in `

Gem files will remain installed in /home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2 for inspection.
Results logged to /home/ec2-user/.gem/ruby/2.0/gems/dep-selector-libgecode-1.0.2/ext/libgecode3/gem_make.out


ChefDK is quite helpful to resolve dependency issues.

1 comment:

iJAWS@Doorkeeper