Building Ruby

Rubyinstaller Ruby Version Managers

Building Ruby

RubyInstaller project doesn't provide Ruby installers only. It creates development environment, based on free MinGW tool-chain, that can be used to build Ruby and its components. Project consists of number of Rake tasks aim to build, test and package various Ruby versions.

Before we start to explore these tasks we have to clone RubyInstaller project. Open new Command Prompt or Git bash shell or, if you have followed instructions from the book, open corresponding task in ConEmu. Go to our projects directory created in the previous chapter and clone project from GitHub (git://github.com/oneclick/rubyinstaller.git). The whole process in Git's bash shell should look something like this:

$ cd /c/projects/ruby
$ git clone git://github.com/oneclick/rubyinstaller.git
Cloning into rubyinstaller...
More Git clone messages....

Now when you have complete RubyInstaller's code it is time to see what Rake tasks it defines. Open your Ruby Console, go to the directory where you have cloned RubyInstaller's repository and type rake -T.

cd c:\projects\rubyinstaller
C:\projects\rubyinstaller>rake -T
You need rdoc 3.12 and rdoc_chm 3.1.0 gems installed
in order to build the docs tasks.
Try `gem install rdoc -v 3.12` and later `gem install rdoc_chm -v 3.1.0`

rake book                      # Download and extract The Book of Ruby
rake clean                     # Remove any temporary products
rake clobber                   # Remove any generated file
rake default                   # Build Ruby 1.8
rake devkit                    # Build DevKit installer and/or archives
rake devkit:ls                 # List available DevKit flavors
rake release:upload[version]   # Upload release files
rake ruby18                    # compile Ruby 1.8
rake ruby19                    # compile Ruby 1.9
rake ruby20                    # compile Ruby 2.0.0
rake ruby21                    # compile Ruby 2.1
rake ruby22                    # compile Ruby 2.2
rake ruby23:install            # install rubyinstaller-2.3.0.exe
rake ruby23:package            # generate packages for ruby 2.3.0
rake ruby23:package:archive    # generate ruby-2.3.0-i386-mingw32.7z
rake ruby23:package:docs       # generate ruby-2.3.0-doc-chm.7z
rake ruby23:package:installer  # generate rubyinstaller-2.3.0.exe
rake ruby23:repackage          # rebuild rubyinstaller-2.3.0.exe

The most important tasks in the list are rake rubyXX. As their descriptions denote, they build various Ruby versions. Ruby versions 1.8.x are obsolete and we will not use them. Expect these tasks to be removed from the project.

RubyInstaller automates the whole process of downloading build tools, configuring build environment, downloading sources and additional libraries needed for building Ruby and finally compiling it.

A list of all packages RubyInstaller uses is kept in config\ruby_installer.rb file. Here is excerpt of the file related to Ruby 1.8:

Ruby18 = OpenStruct.new(
      :version => '1.8.7-p374',
      :short_version => 'ruby18',
      :url => "http://ftp.ruby-lang.org/pub/ruby/1.8",
      :checkout => 'http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7',
      :checkout_target => 'downloads/ruby_1_8',
      :target => 'sandbox/ruby_1_8',
      :doc_target => 'sandbox/doc/ruby18',
      :build_target => 'sandbox/ruby18_build',
      :install_target => 'sandbox/ruby18_mingw',
      :patches => 'resources/patches/ruby187',
      :configure_options => [
        '--enable-shared',
        '--with-winsock2',
        '--disable-install-doc',
        "CFLAGS='-g -O2 -DFD_SETSIZE=256'"
      ],
      :files => [
        'ruby-1.8.7-p374.tar.bz2'
      ],
      :dependencies => [
        :gdbm, :iconv, :openssl, :pdcurses, :zlib, :tcl, :tk
      ],
      :excludes => [
        'libcharset-1.dll'
      ],
      :installer_guid => '{F6377277-9DF1-4a1f-A487-CB5D34DCD793}'
    )

In the first line of OpenStruct member definition Ruby version is set. Checkout options are used if we want to build Ruby not from archives, but from the Subversion repository. After that, directory where sources will be unpacked is defined and it is followed by build and install directories. The install directory here doesn't mean directory where Ruby will be installed by installer. It is directory within RubyInstaller output directory where all Ruby files needed for installer to be packed will be copied. Next option is important for build process since it defines configuration flags passed to configure script.

You might wonder what configure script is? This script is used to query the system on which it is run for platform architecture, build and run-time dependencies, system environment, etc. Based on gathered information configure script generates Makefile which is then used to build application. Script accepts arguments and these arguments are defined in the configure options part. Even though you can change configure options directly in the ruby_installer.rb file there is a much better way which we will explain soon.

Next three options in RubyInstaller's configuration file define full name of archive with Ruby sources, libraries that Ruby depends on and excluded library. Last option is not important at the moment so we will skip it. Similar set of options is defined for Ruby 2.2 and subset of these options is used for other packages used during build process.

Now I will briefly walk you through other Rake tasks which are important if you decide to build Ruby from sources. RubyInstaller unpacks sources and builds libraries and Ruby in the sandbox folder. Tasks clean and clobber are used to remove temporary and all files created during build process respectively.

You already saw how to choose which version of Ruby will be built but you can alter build procedure by passing “arguments” to Rake task. RubyInstaller does not use standard Rake way of passing parameters. Namely, if Rake task is defined in such a way that it accepts parameters it is invoked with parameter value enclosed in brackets:

rake namespace:task_name[param]

RubyInstaller on the other hand uses environment variables as input parameters. There are several environment variables that RubyInstaller's Rake tasks understand but most important for us at the moment are: LOCAL, NODEPS and DKVER. First environment variable, LOCAL, should hold the path where we keep sources from which Ruby will be built. In other words if we do not want to leave to RubyInstaller to download Ruby sources for particular version we can do that manually and then tell RubyInstaller where they are. As a matter of act this is the way we will build Ruby later. The second environment variable tells RubyInstaller to skip building dependencies and the third one which compiler tool-chan it should use.

Up to now we didn't say anything about compiler that is used to build Ruby except that MinGW based tool-chain is used. It is now time to take a closer look which compilers are available. Files in the config\compilers directory within RubyInstaller project keep a list of compilers that can be used. You can also get a list of supported compilers with

C:\Projects\rubyinstaller>rake devkit:ls

=== Available DevKit's ===
    mingw-32-3.4.5
    mingw-32-4.6.2
    mingw64-32-4.7.2
    mingw64-64-4.7.2
    mingwbuilds-32-4.7.3
    mingwbuilds-64-4.7.3
 => tdm-32-4.5.2          [default]
    tdm-32-4.6.1
    tdm-32-4.7.1
    tdm-64-4.6.1
    tdm-64-4.7.1

As you can see there are lot of defined compilers. Some of them are kept only for building Ruby 1.8.7 and will probably be removed in the future. Others are kept for testing purposes. Even though default version is set to tdm-32-4.5.2 the one that will use is mingw64-32-4.7.2 because at the moment of writing this book default version cannot compile Ruby sources (see comment on GitHub issue).

Before we build latest Ruby development version I owe you one more explanation. I mentioned a way that you can alter default project setting without a need to change RubyInstaller's code. During build, RubyInstaller checks for existence of override directory within your RubyInstaller project directory. If it finds it all Ruby files from that folder are loaded and you can use this to change project settings. Code that loads these ruby files looks like:

# scan all override definitions and load them
Dir.glob('override/*.rb').sort.each do |f|
  begin
    puts "Loading override #{File.basename(f)}" if Rake.application.options.trace
    require f
  rescue StandardError => e
    warn "WARN: Problem loading #{f}: #{e.message}"
  end
end

Therefore if we want to change, for example, which level of optimization compiler uses when it builds application we can do it by creating file optimization.rb within override directory with just one line of code:

RubyInstaller::Ruby22.configure_options << “optflags='-O0'”

If you remember configure_options is an array in the Ruby22 OpenStruct object. This array holds arguments that will be passed to configure script when it is invoked. Line given above just adds one more item to the configure_options array. But wouldn't it be nice that we can conditionally use new arguments? Let's improve code a little bit:

if ENV[“OPTIM”] then
  puts '[INFO] Changing compile optimization flags'
  RubyInstaller::Ruby22.configure_options << “optflags='-O0'”
end

Now our script checks whether environment variable OPTIM is defined and only if it is configure_options are changed. Starting Rake task in a standard way rake ruby22 will use default RubyInstaller settings but with rake ruby22 OPTIM=1 changes from our script will be applied. Due to the way how Ruby treats expressions in if statement you can put anything after equals sign except nil and false. Same way is used to change LOCAL, NODEPS and DKVER environment variables during Rake tasks invocation.

As stated above we will build latest Ruby development version. Since RubyInstaller uses latest official releases we have to download sources by ourself. We can do that by telling RubyInstaller to checkout sources from Ruby Subversion repository (by defining TRUNK environment variable) or telling it where sources are located on our disk. We will use second option - sources from our local disk. First we will clone Git mirror of Ruby Subversion repository which can be found on GitHub (git://github.com/ruby/ruby.git). Open Git bash shell in ConEmu and clone Ruby sources in C:\projects\ruby-mirror directory.

$ cd projects/
$ git clone git://github.com/ruby/ruby.git ruby-mirror
Cloning into ruby-mirror...
More Git clone messages...

Finally start ruby22 task with LOCAL variable set to the full path where you just cloned Ruby sources and DKVER set to mingw64-32-4.7.2.

C:\projects\rubyinstaller>rake ruby22 LOCAL=”c:\projects\ruby-mirror” DKVER=mingw64-32-4.7.2

Now you just have to be patient while RubyInstaller downloads compiler tool-chain, dependencies, builds them and finally configures and builds Ruby. Time needed for all this depends on speed of your network connection and a speed of your computer. When build is finished you will find latest version of Ruby 2.3.0dev in the sandbox\ruby22_mingw folder. Let's copy this folder to C:\Ruby and rename it to Dev. If you remember first Ruby version was installed in the C:\Ruby\22 folder. This way we are putting all Rubies in the same root directory.

Adding this version to ConEmu will let us easily switch between different Ruby versions. However we haven't installed it through installer so we need to copy resources\installer\setrbvars.bat file to C:\Ruby\Dev\bin directory. Now open ConEmu Setup tasks dialog box and add new task and for Shell insert:

C:\Windows\System32\cmd.exe /E:ON /K c:\Ruby\Dev\bin\setrbvars.bat

Set Title to Ruby Dev console.

Rubyinstaller Ruby Version Managers

Ruby on Windows Guides

by Boško Ivanišević

Building Ruby