I love Ruby blocks

While implementing a change in the AWS CPI, I once again got to use a block in Ruby - a feature I really like.

This is a simplified version of the first shot at the change looked:

def user_data
    ...

    if has_dns?(spec)
      data["dns"] = get_dns_servers(spec)
    end

    ...
  end

  def has_dns?(spec)
    spec.has_key?("dns")
  end

  def get_dns_servers(spec)
    spec["dns"]["nameservers"]
  end

This works, but it doesn't follow the tell don't ask principle, i.e. it first asks about the state and then takes action on it if it is in the expected state, instead of just asking for things to be done.

By refactoring it to use a method that takes a block instead, this is now shorter and easier to read.

def user_data
    ...

    with_dns(spec) do |servers|
      data["dns"] = servers
    end

    ...
  end

  def with_dns(spec)
    if spec.has_key?("dns")
      yield spec["dns"]["nameservers"]
    end
  end

The only downside is that it isn't clear what value(s) are yielded when you just see with_dns(spec). If you name the variable(s) properly, you can remedy that for someone else who reads the code, but you still need know what is yielded. I do that by documenting the method using yard and the @yield tag.

DTrace in Ruby 2.0

Being a long-time DTrace user while working on Solaris, I really miss using it on our production systems. Luckily I use OS X, so I've got DTrace while developing, but there hasn't been "real" support for it until now!

Ruby 2.0 just got built in DTrace support as of commit 4c740bae, and to build your own Ruby (dev) with DTrace support in rbenv, follow these steps:

Start by installing OpenSSL, as Ruby 2.0 isn't compatible with the OpenSSL in OS X. The easiest way is to use homebrew:

brew install openssl

Next grab the Ruby 2.0 source from github (git is available in homebrew too in case you don't already have it installed).

git clone git@github.com:ruby/ruby.git

Now configure, make & install it:

$ ./configure --prefix=~/.rbenv/versions/2.0.0 \
  --with-opt-dir=`brew --prefix openssl`
$ make
$ make install

Now you can try it out in the current shell:

$ rbenv shell 2.0.0
$ ruby --version
ruby 2.0.0dev (2012-12-11 trunk 38301) [x86_64-darwin12.2.0]

Running DTrace

I'll cover more advanced DTrace scripts in a follow-up post, but here is something to get you started...

Note that DTrace require elevated privileges, and I don't know of any other way in OS X to do that, than to run it through sudo.

Counting number of created objects

This script will count the number of objects created in this very naïve Ruby program

count.d

ruby*:::object-create
{
 @objects[copyinstr(arg0)] = count();
}

hello.rb

puts "hello world from Ruby #{RUBY_VERSION}!"

You have to pass sudo the full path of the Ruby 2.0 binary (/Users/martin/.rbenv/versions/2.0.0/bin/ruby) as dtrace and rbenv doesn't play nice together.

$ rbenv which ruby
/Users/martin/.rbenv/versions/2.0.0/bin/ruby
$ sudo dtrace -s count.d \
  -c '/Users/martin/.rbenv/versions/2.0.0/bin/ruby hello.rb'
dtrace: script 'y.d' matched 2 probes
hello world from Ruby 2.0.0!
dtrace: pid 43289 has exited

  #<Class:0x007ff1728e8710>                                    1
  ARGF.class                                                   1
  IOError                                                      1
  Mutex                                                        1
  NoMemoryError                                                1
  SystemStackError                                             1
  ThreadGroup                                                  1
  Time                                                         1
  LoadError                                                    2
  Object                                                       2
  Gem::Specification                                           8
  Gem::Version                                                11
  Hash                                                        11
  Gem::Requirement                                            24
  Array                                                       96
  String                                                     250

Developer mentoring

I'm mentoring Brett and Brian who attended DevBootCamp spring 2012, which is an interesting experience as it forces me to think about how I work and try to pass on any insights I've had over the years, sort of passing on my software craftsmanship. I'm not going to go into the "is programming science, art or a craft" debate, but I think the comparison to a carpenter isn't that far fetched.

Developers live and die by their tools and so does a carpenter. If you ever looked inside a carpentry you have seen an abundance of tools for many different purposes, and a good carpenter knows how to use them all.

One thing I've been trying to get across to my adepts is the importance of time and with that the importance of tools.

You will always be short on time as a programmer! When was the last time you worked on a project where you had all the time in the world to finish it? Thus, you need to master your tools, have a wide variety of them and also know when to use them - to maximize the time you can spend coding (or thinking about your code).

Tool mastery

Tool mastery is important as it will speed things up a lot. If you are editing a text file in your favorite editor and need to comment out a couple of lines, you can either use the mouse or arrow keys to move to the beginning of each line and insert a comment marker, or you can use the editor's keyboard shortcut to comment out the whole block of lines. The difference in time might not be much, say 5 seconds, but when you sum that up for a number of similar tasks over a week, it quickly adds up to an hour a week. This is about 5% of your effective coding time!

It is worth taking time learning your tools! The return on investment is great.

Even the keyboard is a tool you should take time to learn how to use properly. I wish someone had told me to learn how to touch type when I started with computers. I dread to calculate how many hours I could have saved if I could do 120 wpm instead of 80!

Expand your toolbox

Having a rich toolbox is also important, as for example if you don't know how to use vi you'll have no way to edit files on a remote server over ssh (yes, I know that there are ways to do it, but I'm sure you get my drift) other than to copy files back and edit them locally. Lacking a tool in your toolbox will make you loose time.

I'm sure you have heard of Maslov's hammer "if all you have is a hammer, everything looks like a nail". This very much applies to programming too.

Pick the right tool

There isn't one language which will be the best choose in all situations. You need to know when to use what language, which benefits and drawbacks it has. I'm sure you have your favorite language which you'd prefer to work with, but don't try to force the square peg in the round hole. If you are writing a process monitoring system in a memory constrained environment, Ruby is probably not the best answer - C is, even if it is harder to write good code in it.

You can cut down a tree with a hack-saw but it will not be efficient!

Setting up a Ruby development laptop on Mountain Lion

I've just reinstalled my MacBook Pro with Mountain Lion, and I always start from scratch instead of doing an update, as I don't want to bring along old cruft - and it is a good way to make sure I don't bring infected/trojaned files (if there were any).

Here are the the (very terse) steps I took to get my Ruby development environment back up, more a reminder to myself than a installation cookbook, but I'm sure it will be helpful for someone...

Install Mountain Lion

Duh!

Install XCode

Available in the App Store, also make sure to install the command line tools.

Install rbenv

There is a chicken and egg problem here: to install rbenv you need git to clone the repo, and to install git you need homebrew, and to install homebrew you need ruby. So work around that, pull down a tar-ball of the rbenv first.

cd ~
mkdir tmp
cd tmp
wget https://github.com/sstephenson/rbenv/tarball/master
tar xf master
mv sstephenson-rbenv-* ~/.rbenv

Install Ruby

Add the following to ~/.gemrc to avoid generating ri & rdoc.

gem: --no-ri --no-rdoc

I use yard instead, which is much nicer!

Get the latest version of Ruby from http://www.ruby-lang.org/en/downloads/ (1.9.3-p194 at the time of writing)

cd ~/tmp
tar xzf ruby-1.9.3-p194.tar.gz
cd ruby-1.9.3-p194
./configure --prefix $HOME/.rbenv/versions/1.9.3-p194
make
make install

Also do the normal rbenv in your .bash_profile

Install homebrew

Now that ruby is available, homebrew can be installed

ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)

Install git

brew install git

Install libyaml

brew install libyaml

Reinstall rbenv & Ruby

Now we have all pieces to install rbenv for real

cd ~
rm -rf .rbenv
git clone git://github.com/sstephenson/rbenv.git .rbenv
cd ~/tmp/ruby-1.9.3-p194
make distclean
./configure --prefix $HOME/.rbenv/versions/1.9.3-p194
make
make install

Install X11

Mountain Lion doesn't come with X11 headers any more, and if you try to compile Ruby 1.8 it requires X11 for tcl/tk, so it will fail with:

/usr/include/tk.h:78:23: error: X11/Xlib.h: No such file or directory

You can get X11 from http://xquartz.macosforge.org/landing/
After that you need to set

export CPPFLAGS=-I/opt/X11/include

Install Ruby 1.8

If you don't need tcl/tk & X11 in Ruby 1.8 you can just supply the flags

--disable-tcl --disable-tk

to your configure command, but if you do, you are now ready to compile 1.8

./configure --prefix $HOME/.rbenv/versions/1.8.7-p370

Done!

After this you are done!

martin@mbp[master]$ rbenv versions
  1.8.7-p370
  1.9.2-p320
* 1.9.3-p194 (set by /Users/martin/.rbenv/version)

Happy Ruby hacking :)

Custom RSpec matcher for Kwalify

I've been helping a friend learn how to program Ruby and TDD for one of his pet-projects, and we were writing some test cases for a class that loads and parses a yaml file with Kwalify, and the result didn't read very well:

it "should have 2 errors" do
  validator.validate(document).size.should == 2
end

it "should have validation error" do
  validator.validate(load_yaml("empty.yml")).each do |error|
    error.is_a?(Kwalify::ValidationError).should be_true
  end
end

So I decided to write a custom matcher for RSpec, and it now reads:

it "should have 2 errors" do
  validator.validate(asset("empty.yml")).should have_error(2)
end

it "should have validation error" do
  validator.validate(asset("empty.yml")).should have_validation_error
end

I've made the source available on github and also created a gem for it (rspec-kwalify), so all you have to do to use it is to add this to your Gemfile

group :development, :test do
 gem "rspec-kwalify"
end

I'm not sure if kwalify-rspec is a better name - perhaps I should change?

Moving a directory between two git repos with preserved history

When we open sourced Cloud Foundry  BOSH I was in charge of extracting some directories from an existing private git repo into a new public repo. This was done by cloning the original repository and then using git filter-branch to remove the directories which should not be made public, and then push the remaining directories to the new (empty) repo.

git filter-branch --prune-empty --tree-filter 'rm -rf dir1 dir2 dir3' HEAD
git remote add origin https://github.com/cloudfoundry/vcap-tools.git
git push origin master

Note that if you leave out --prune-empty you'll be left with a bunch of commit messages about the removed directories, which isn't what you want!

Today Vadim told me that I forgot the dashboard_v2 directory, so I had to figure out how to extract just that directory with history and add it to the vcap-tools repo!

This required a little more research to do then the initial move. The first step is to go into the old private repo (tools) and remove everything but dashboard_v2, then recreate the directory as git filter-branch --subdirectory-filter will leave the contents of the directory in the repository root.

cd ~/cf/tools
git filter-branch --prune-empty --subdirectory-filter dashboard_v2 -- --all
mkdir dashboard_v2
git mv BUILD-NOTES pom.xml src dashboard_v2
git commit -m 'dashboard_v2 extraction'

Now I can switch to bringing in the commits from tools into the now existing vcap-tools repo.

cd ~/cf/vcap-tools
git remote add tools ~/cf/tools
git fetch tools
git branch tools remotes/tools/master
git merge tools
git remote rm tools

I now have the dashboard_v2 directory with full history in the vcap-tools repository! The only thing that remain now is to push the changes to the origin. If we weren't using gerrit, it would just have been git push, but now it is gerrit push instead (using our helper gem). However, I need wait for my gerrit permissions to be upgraded so I have forge author and forge committer on that repository as I'm pushing other people's changes.

TextMate 2 and rmate

I just downloaded TextMate 2 alpha and am very pleased with what I see!

One feature in particular I love is the rmate script. It lets you edit files on a remote server on your local system in TextMate using ssh tunneling.

To set it up you just need to do the following steps:

First open the preferences and select the Terminal, make sure "Accept rmate connections" is checked, then make a note of the port number.
Next click on the blue rmate link and TextMate will open the rmate script in a window. Now save the script somewhere, e.g. in ~/Downloads.

Once rmate is saved, you need to start Terminal and run scp to copy rmate to the remote system (called micro in this case), then log on and move the script into place

scp ~/Downloads/rmate micro:
ssh micro
sudo mv rmate /usr/local/bin

Finally you need to edit your local ~/.ssh/config file and add automatic remote forwarding from the system to your local TextMate port (52698)

Host micro
User vcap
Hostname 192.168.100.202
RemoteForward 52698 127.0.0.1:52698

Now you can log on to the remote system and edit files with rmate

ssh micro
rmate /src/foo.rb

And a new TextMate window labeled "micro:/src/foo.rb" will open up on your local system!

With the change in ~/.ssh/config you don't need to specify the remote port forwarding on the command line each time. If you want the tunnel to be present for all ssh connections, you just move the RemoteForward line to the top of the file.

Spending a lot of my time messing around with files on remote systems this is a very handy feature, and a cool side note is that rmate is written in Ruby - which I really enjoy working with :-)