DevKit

Ruby gems can roughly be divided into two types. Gems that during installation do not require external tools and those that require them. First type of gems are either completely written in Ruby or have pre-built binaries for a system they are installed on. Second type of gems are written in C/C++ and require build tools capable of compiling C/C++ source and creating shared libraries. No matter if you use Linux, OS X or Windows operating system you must have build tools in order to install such gems.

One of the goals of RubyInstaller project was to make it possible to install gems written in C/C++ on Windows. Solution was named DevKit. It is a set of MSYS and MinGW based build tools that make it easy and simple to build native C/C++ Ruby extensions. For Ruby 2.2 DevKit is based on 32 bit MinGW 4.7.2 compiler suite for Windows.

Go ahead and download RubyInstaller's Ruby DevKit self extracting archive and unpack it. It is recommended to unpack DevKit to a path without spaces since MSYS tools do not handle such paths correctly. Throughout the book I'll assume DevKit is located at C:\Ruby\DevKit.

You have to take some precautions if you want to assure proper DevKit functioning. If you have Cygwin or other MSYS/MinGW build tools in the path you should remove them. Usually these tools keep their configuration files (.bashrc, .bash_profile) in user's home directory and these files can be a cause of DevKit malfunctioning. Before you go on with DevKit installation check your system for any of these.

RubyInstaller's DevKit comes with Ruby script, dk.rb, that is used to inject it to all Ruby versions you have installed on the system. You will find dk.rb file in C:\Ruby\DevKit directory. Let's execute it now and check the output:

c:\Ruby\DevKit>ruby dk.rb

Configures an MSYS/MinGW based Development Kit (DevKit) for
each of the Ruby installations on your Windows system. The
DevKit enables you to build many of the available native
RubyGems that don't yet have a binary gem.

Usage: ruby dk.rb COMMAND [options]

where COMMAND is one of:

  init     prepare DevKit for installation
  review   review DevKit install plan
  install  install required DevKit executables

and 'install' [options] are:

  -f, --force  overwrite existing helper scripts

As indicated by the message you have to use one of three commands: init, review and install when you execute script. Moreover commands should be executed in the same order as they are listed in the output.

First command collects data about installed Ruby versions on the system. Goto directory where you have unpacked DevKit (you'll see in a moment why) and execute script passing it init command.

c:\>cd c:\Ruby\DevKit

c:\Ruby\DevKit>ruby dk.rb init
[INFO] found RubyInstaller v2.2.1 at c:/Ruby/22

Initialization complete! Please review and modify the auto-generated
'config.yml' file to ensure it contains the root directories to all
of the installed Rubies you want enhanced by the DevKit.

Script has created config.yml file in the folder from which script was executed. That's why we switched to DevKit directory before we started it. As you can see only one Ruby version has been found, and that's the one for which we have used installer. As Pik, DevKit searches your system's registry for installed Rubies and for each version found, it displays a message and writes necessary data to the configuration file. Here is the code that lies behind this “magic”

def self.scan_for(key)
  ris = []
  [Win32::Registry::HKEY_LOCAL_MACHINE,
  Win32::Registry::HKEY_CURRENT_USER].each do |hive|
    begin
      hive.open(key) do |ri_key|
        ri_key.each_key do |skey, wtime|
          # read the install location if a version subkey
          if skey =~ /\d\.\d\.\d/
            ri_key.open(skey) do |ver_key|
              ri_root = ver_key['InstallLocation'].gsub('\\', '/')
              puts '[INFO] found RubyInstaller v%s at %s' % [ skey, ri_root ]
              ris << ri_root
            end
          end
        end
      end
    rescue Win32::Registry::Error
    end
  end
  ris
end

Script searches key within HKLM (HKEY_LOCAL_MACHINE) and HKCU (HKEY_CURRENT_USER) passed as an argument for sub-key that matches version pattern /\d\.\d\.\d/. This is regular expression that corresponds to three numbers (\d) separated by dots (\.). If it finds one, it reads InstallLocation string value and replaces Windows path separator, backslash ('\'), with path separator which is correctly handled by MinGW – forward slash ('/'). Finally it stores path in the array which is returned as a result of the method. Keys that are searched are Software\RubyInstaller\MRI and Software\RubyInstaller\Rubinius. These are only Rubies that are supported by DevKit at the moment. We should see what has been written to the configuration file.

# This configuration file contains the absolute path locations of all
# installed Rubies to be enhanced to work with the DevKit. This config
# file is generated by the 'ruby dk.rb init' step and may be modified
# before running the 'ruby dk.rb install' step. To include any installed
# Rubies that were not automagically discovered, simply add a line below
# the triple hyphens with the absolute path to the Ruby root directory.
#
# Example:
#
# ---
# - C:/ruby19trunk
# - C:/ruby192dev
#
---
- c:/Ruby/22

Besides detailed explanation in the comments, script has actually serialized Ruby array with only one element, path to the Ruby version installed with the installer. Does it mean we cannot use DevKit for other Ruby versions we built or just unpacked? Of course not. We may modify this file and manually add all versions we intend to use DevKit with, as long as we follow YAML specification for serializing arrays.

YAML uses particular sequence of characters for array elements. Each entry begins on its own line and sequences indicates each entry with a dash and space (-). Sequence of scalars, as it is stated in the YAML specification, or entries in the Ruby array is therefore written as:

- Ruby
- is a
- great programming language

After parsing it in Ruby this set will result in an Array object with three elements:

[“Ruby”, “is a”, “great programming language”]

With this background we are ready to add Ruby versions that we have on the system. Apart from Ruby 2.2.1p85 we have Ruby Dev. For each additional version we have on the system we must add one line that starts with a dash-space sequence followed by the full path where it is on the disk. Final version of config.yml file should be (comments omitted):

---
- c:/Ruby/22
- c:/Ruby/Dev

Next suggested dk.rb command is review. This command will check our configuration file and if it is valid, information will be printed out:

c:\Ruby\DevKit>ruby dk.rb review
Based upon the settings in the 'config.yml' file generated
from running 'ruby dk.rb init' and any of your customizations,
DevKit functionality will be injected into the following Rubies
when you run 'ruby dk.rb install'.

C:/Ruby/22
C:/Ruby/Dev

As expected dk.rb has found all Ruby versions we added, beside the one that was original added by init command. It is time to inject DevKit in all listed Ruby versions. Execute script with install command:

c:\Ruby\DevKit>ruby dk.rb install
[INFO] Updating convenience notice gem override for 'C:/Ruby/22'
[INFO] Installing 'C:/Ruby/22/lib/ruby/site_ruby/devkit.rb'
[INFO] Updating convenience notice gem override for 'C:/Ruby/Dev'
[INFO] Installing 'C:/Ruby/Dev/lib/ruby/site_ruby/devkit.rb'

Script has installed devkit.rb file in site_ruby directories in each Ruby version that we added to configuration file. Directory site_ruby has special meaning. This is the place where you want to put Ruby extensions other than those managed by Rubygems. Therefore if you want to extend Ruby without building new gem this is the place where you will put your scripts. You shouldn't use it too often. As a matter of act it should be used carefully and only if you really want to add some low-level extension. DevKit is exactly that – low level Ruby extension for Windows operating systems. Without it Ruby on Windows would not be capable to build native gems. So what is actually written in devkit.rb?

# enable RubyInstaller DevKit usage as a vendorable helper library
unless ENV['PATH'].include?('c:\\Ruby\\DevKit\\mingw\\bin') then
  puts 'Temporarily enhancing PATH to include DevKit...'
  ENV['PATH'] = 'c:\\Ruby\\DevKit\\bin;c:\\Ruby\\DevKit\\mingw\\bin;' + ENV['PATH']
end
ENV['RI_DEVKIT'] = 'c:\\Ruby\\DevKit'
ENV['CC'] = 'gcc'
ENV['CPP'] = 'cpp'
ENV['CXX'] = 'g++'

Script first checks whether our DevKit is in the path. If it is not, it prints out informational message and adds necessary directories at the beginning of the path so they are very first directories searched for the executable files after directory from which executable is invoked. In other words if some Ruby, script tries to execute for example gcc, g++ or some other MSYS/MinGW executable and if it is not found in the current executing directory, first paths where they will be looked for are our DevKit directories. Besides, it sets few environment variables needed for building native gems. Very clever solution, isn't it? We do not have to have these directories in the path all the time. When needed they will be added.

DevKit's dk.rb script has printed out one more type of message though. This was [INFO] Updating convenience notice gem override for <ruby_version>. Script has obviously updated something so let's check what. The relevant part of code in dk.rb is:

target_ruby.each do |folder|
  target = File.join(folder, 'defaults', 'operating_system.rb')
  FileUtils.mkdir_p File.dirname(target)

  if File.exist?(target)
    content = File.read(target)
    case
    when content !~ /^#.*DevKit/o
      # handle original and new token-based comments
      puts "[INFO] Updating existing gem override for '#{path}'"
      File.open(target, 'a') { |f| f.write(gem_override) }
    when content =~ /^# #{DEVKIT_START} missing DevKit/o
      # replace missing DevKit/build tool convenience notice
      puts "[INFO] Updating convenience notice gem override for '#{path}'"
      update_gem_override(target)
    else
      puts "[INFO] Skipping existing gem override for '#{path}'" unless $options[:force]

      if $options[:force]
        puts "[WARN] Updating (with backup) existing gem override for '#{path}'"
        update_gem_override(target)
      end
    end

  else
    puts "[INFO] Installing '#{target}'"
    File.open(target, 'w') { |f| f.write(gem_override) }
  end
end

Without going in too much details, what can be seen from the code is that script is looking for operating_system.rb file in some directories. Depending whether it has found old versions of the file, it updates them and if operating_system.rb file is not found at all it will create new one if it is forced by passing --force argument to install command. The operating_system.rb file is bundled with all RubyInstaller versions and they alter a way Rubygems install gems. Content of newly created, or updated, operating_system.rb file is:

Gem.pre_install do |gem_installer|
  unless gem_installer.spec.extensions.empty?
    unless ENV['PATH'].include?('c:\\Ruby\\DevKit\\mingw\\bin') then
      Gem.ui.say 'Temporarily enhancing PATH to include DevKit...'
         if Gem.configuration.verbose

      ENV['PATH'] = 'c:\\Ruby\\DevKit\\bin;c:\\Ruby\\DevKit\\mingw\\bin;' + ENV['PATH']
    end
    ENV['RI_DEVKIT'] = 'c:\\Ruby\\DevKit'
    ENV['CC'] = 'gcc'
    ENV['CPP'] = 'cpp'
    ENV['CXX'] = 'g++'
  end
end

The code calls singleton method pre_install of Gem module and passes it a block. This method is actually a hook that will be called prior to any gem installation. It accepts a block of code, converts it to Proc object and adds it to the list of procedures that will be called before any gem is installed.

Block passed to the pre-install hook checks whether gem has native extensions. If it has, it further checks for tools essential for building them. These tools are compiler (gcc), shell script (sh) and make. For each of them script executes simple shell command and if it was successfully finished it does nothing. Otherwise it informs us that we need build tools and where to get them. It is strongly recommended to run so called smoke test and check if DevKit works correctly:

c:\>gem install json --platform=ruby
Fetching: json-1.8.2.gem (100%)
Temporarily enhancing PATH to include DevKit...
Building native extensions.  This could take a while...
Successfully installed json-1.8.2
Parsing documentation for json-1.8.2
Installing ri documentation for json-1.8.2
Done installing documentation for json after 1 seconds
1 gem installed

First command installs json gem. Argument –-platform=ruby tells Rubygems not to look for pre-built binaries for our system, but to install gem from sources. Since json gem is implemented in C we will need build tools for its installation.

DevKit comes with one more utility script. Batch file devkitvars.bat is used to add patch to DevKit folders at the beginning of the path so we can use build tools from any Command Prompt. Open up new Command Prompt window and execute this batch file.

C:\>c:\Ruby\DevKit\devkitvars.bat
Adding the DevKit to PATH...

C:\>gcc --version
gcc (rubenvb-4.7.2-release) 4.7.2
Copyright (C) 2012 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

This batch file is useful if you are developing your own native gems and want to build them without installing. After you execute it you will have complete build tool chain available in the Command Prompt.