%% Default fonts
%deffont "standard" xfont "helvetica-medium-r", tfont "arial.ttf"
%deffont "thick" xfont "helvetica-bold-r", tfont "arialb.ttf"
%deffont "typewriter" xfont "fixed-medium-r", tfont "cour.ttf"
%%
%% Default settings per each line number
%default 1 area 90 90, leftfill, size 2, fore "yellow", back "black", font "thick", bgrad
%default 2 size 6, vgap 10, prefix " ", ccolor "black"
%default 3 size 2, bar "gray70", vgap 10
%default 4 size 5, fore "white", vgap 30, prefix " ", font "standard"
%%
%% Default settings for tab-indented lines
%tab 1 size 5, vgap 40, prefix "  ", icon box "green" 50
%tab 2 size 4, vgap 40, prefix "      ", icon arc "yellow" 50
%tab 3 size 3, vgap 40, prefix "            ", icon delta3 "white" 40
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%center

Hacking Ruby

Ruby Conference 2004

Paul Brannan
paul@atdesk.com

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

What is "hacking ruby"?

	From foldoc:
%font "typewriter", size 3
  hack
    6. To interact with a computer in a playful and exploratory
    rather than goal-directed way.  "Whatcha up to?"  "Oh, just
    hacking."
%font "standard"
	This interaction often results in code that lets ruby do/express things it was never specifically designed to

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Why hack?

	Ruby is lacking a feature from <insert your favorite language here> that you can't bear to live without
	To do a task that would otherwise be difficult
	Proof-of-concept for an RCR
	To explore/debug extensions and/or ruby itself
	For fun!

%% This last reason is, of course, the most _____ reason

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Example of a "hack"
#ruby <<END
def numberize(str, startnum=1)
  result = ''
  n = startnum
  str.each_line do |line|
    result << "#{'%2d' % n} #{line}"
    n += 1
  end
  $lastnum = n
  return result
end

def code_block(str, startnum=1)
  return <<-ENDCODEBLOCK
%font "typewriter", size 3
#{numberize(str, startnum)}
%font "standard"
  ENDCODEBLOCK
end
END

#ruby <<END
code_block <<ENDCODEBLOCK
  class Array
    alias_method :concat, :old_concat
    def concat(*arrays)
      arrays.each { |array| old_concat(array) }
    end
  end

  a = [1, 2, 3]
  a.concat([1], [2], [3, 4])
  p a #=> [1, 2, 3, 1, 2, 3, 4]
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Ruby, how do I hack thee?  Let me count the ways...

	Pure Ruby (using set_trace_func, callcc, etc.)
	Already-written extensions (enumerator, stringio, etc.)
	Using libdl (e.g. evil.rb)
	Writing your own extensions
	Hacking the interpreter itself

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

The Ruby Hacker's Guide

%% A talk on hacking Ruby could not possibly be complete without
%% mentioning this book!

	In Japanese: http://i.loveruby.net/ja/rhg/
	Can't read Japanese?  Use babelfish!
	But babelfish doesn't work with iso-2022-jp?
	Convert it to euc-jp using iconv.
	But babelfish doesn't like large documents?
	Split the html into multiple files!
	Maybe someone has already done this work: http://rubystuff.org/rhg/convertrhg.rb

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Tools for hacking in pure Ruby

	set_trace_func
	method_missing
	append_features/extend_object
	binding
	callcc
	Thread.new

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

set_trace_func

	Provides a callback whenever the interpreter calls a method, evaluates a statement, etc.
	Used by the debugger for breakpoints, watchpoints, etc.

#ruby <<END
code_block <<ENDCODEBLOCK
  def foo; puts "foo!"; end
  set_trace_func proc { |*x| p x }
  foo()
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

set_trace_func (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK
["line", "test.rb", 3, nil, #<Binding:0x401b7b74>, false]
["call", "test.rb", 1, :foo, #<Binding:0x401b7a48>, Object]
["line", "test.rb", 1, :foo, #<Binding:0x401b7a0c>, Object]
["c-call", "test.rb", 1, :puts, #<Binding:0x401b7840>, Kernel]
["c-call", "test.rb", 1, :write, #<Binding:0x401b7804>, IO]
foo!["c-return", "test.rb", 1, :write, #<Binding:0x401b7700>, IO]
["c-call", "test.rb", 1, :write, #<Binding:0x401b75fc>, IO]

["c-return", "test.rb", 1, :write, #<Binding:0x401b750c>, IO]
["c-return", "test.rb", 1, :puts, #<Binding:0x401b741c>, Kernel]
["return", "test.rb", 1, :foo, #<Binding:0x401b732c>, Object]
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

callcc

	Used to save the current point of execution
	Execution can be resumed from the saved point
	Similar to creating a stopped thread
	Can be used to access data you can't normally access

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Put it together...

#ruby <<END
code_block <<ENDCODEBLOCK
def Binding.of_caller(&block)
  old_critical = Thread.critical; Thread.critical = true
  count = 0
  cc = nil; result = callcc {|c| cc = c }
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Binding.of_caller (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK, 5
  tracer = lambda do |*args|
    type, context = args[0], args[4]
    if type == "return"
      count += 1
      if count == 2
        set_trace_func(nil)
        cc.call(eval("binding", context), nil)
      end
    elsif type != "line"
      set_trace_func(nil)
      cc.call(nil, lambda { raise(ArgumentError) })
    end
  end
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Binding.of_caller (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK, 19
  unless result
    set_trace_func(tracer)
    return nil
  else
    Thread.critical = old_critical
    yield result
  end
end
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies

	Derived classes inherit from base clases
	No multiple inheritance
	Mixins inserted in the hierarchy above of the class they inherit from

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK
class Base; end
module M; end
class Derived < Base
  include M
  class << self; @foo = 1; end
end

require 'classtree'
Derived.classtree($stderr)
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK
Derived
|-class = #<Class:Derived>
| |-class = #<Class:Class>
<snip>
| +-super = #<Class:Base>
|   |-class = #<Class:Class> (*)
|   +-super = #<Class:Object> (*)
+-super = #<M:0x401b7958>
  +-class = M
    |-class = Module (*)
    +-super = #<Kernel:0x401afbb8>
      +-class = Kernel (*)
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Put the class hierarchy to work

Overriding methods cleanly (not so useful on Ruby 2.0...):

#ruby <<END
code_block <<ENDCODEBLOCK
  def override_method(*symbols, &block)
    new_methods = Module.new
    old_methods = Module.new
    new_methods.instance_eval { include old_methods }
    include new_methods

    symbols.each do |symbol|
      old_methods.__send__(:define_method, symbol, instance_method(symbol))
      new_methods.module_eval(&block)
      define_method(symbol, new_methods.instance_method(symbol))
    end
  end
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overriding methods (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK
class Array
  override_method(:concat) do
    def concat(*arrays)
      arrays.each { |array| super(array) }
    end
  end
end

a = [1, 2, 3]
a.concat([1], [2], [3, 4])
p a #=> [1, 2, 3, 1, 2, 3, 4]
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overriding methods (cont'd)

#ruby <<END
code_block <<ENDCODEBLOCK
Array
|-class = #<Class:Array>
| |-class = #<Class:Class>
<snip>
| +-super = #<Class:Object> (*)
+-super = #<#<Module:0x401af474>:0x401af438>
  +-class = #<Module:0x401af474>
    |-class = #<Class:#<Module:0x401af474>>
    | |-class = #<Class:Module> (*)
    | +-super = Module (*)
    +-super = #<Kernel:0x401af2d0>
      +-class = Kernel (*)
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

evil.rb

%% evil.rb uses a technique for accessing the ruby interpreter's internals
%% using libdl.  Why is this desirable?  Why not just use an extension?  Well,
%% because extensions have to be compiled; a library that uses libdl can be run
%% as-is.
%%
	http://evil.rubyforge.org
	Reproduces Ruby's C data structures using DL
	Why not just use an extension?
	Extensions have to be compiled; DL-based libraries run as-is.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Object#become

	Swaps one object for another
	One stupid implementation:
#ruby <<END
code_block <<ENDCODEBLOCK
static VALUE ruby_become(VALUE self, VALUE newobj) {
  /* Not implemented here: type check, frozen check, $SAFE check */

  struct RObject * robj1 = ROBJECT(obj1);
  struct RObject * robj2 = ROBJECT(obj2);
  struct RObject tmp_obj;

  tmp_obj = *robj1;
  *robj1  = *robj2;
  *robj2  = tmp_obj;

  return self;
}
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

The power of Object#become: a lazy-evaluator

#ruby <<END
code_block <<ENDCODEBLOCK
class LazyEvaluator < KernellessObject
  def initialize(&block)
    @p = block
  end

  def method_missing(*args, &block)
    obj = @p.call
    obj.become(self) # self.become doesn't exist
    self.__send__(*args, &block)
  end
end
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Accessing interpreter internals

	C functions/data in ruby.h and intern.h
	Non-static data not defined in a header file
	Accessing some data is a bit more tricky

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

How to access interpreter internals without modifying the interpreter

%% This is a potentially very powerful technique.  It involves getting a
%% pointer to a data structure (the hard part), then using that pointer.
%% But how can a pointer be used if it is pointing to a data structure
%% that is defined in a .c file?  Simple: use a ruby script to extract
%% the data structure from the .c file and put it into a header file.
%%
%% Is this technique safe?  I'd classify it as "mostly harmless."  It's
%% not 100% portable, but will work with many C platforms.  And even if
%% it doesn't work for your target platform, you can still use this
%% technique in the "proof-of-concept" stage before you write a
%% full-blown patch for the interpreter.
%%
	Get a pointer to the data structure
	Dereference the pointer
	Need to define a data type that mirrors the type defined in the C file
	Create this data type using a script
	Can't use this trick to access static data

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Inspecting Ruby's internals: Nodedump

	A C extension that dumps Ruby's nodes to the screen
	Works on Ruby 1.6.x
	Can't download it anymore

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Inspecting Ruby's internals: Nodewrap

	A C extension for directly inspecting/manipulating the code in a running program
	Example:
#ruby <<END
code_block <<ENDCODEBLOCK
  irb(main):001:0> require 'nodewrap'
  => true
  irb(main):002:0> l = lambda { 1 + 1 }
  => #<Proc:0x401fbd78@(irb):2>
  irb(main):003:0> l.body
  => #<Node::CALL:0x401fa680>
  irb(main):004:0> l.body.members.each { |m| p [m, l[m]] }
  ["recv", 2]
  ["args", 2]
  ["mid", 2]
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Marshalling methods with Nodewrap

#ruby <<END
code_block <<ENDCODEBLOCK
irb(main):001:0> require 'nodewrap'
=> true
irb(main):002:0> class Foo; def foo; puts 'foo!'; end; end
=> nil
irb(main):003:0> s = Marshal.dump(Foo.instance_method(:foo).body)
=> "\004\010u:\020Node::SCOPE..."
irb(main):004:0> n = Marshal.load(s)
=> #<Node::SCOPE:0x401f1918>
irb(main):005:0> class Bar; end; Bar.instance_eval { add_method(:foo, n, 0) }
=> nil
irb(main):006:0> Bar.new.foo
foo!
=> nil
ENDCODEBLOCK
END

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Dumping nodes to stdout with Nodewrap

%font "typewriter", size 2
  $ cat test.rb
  def foo
    1+1
  end

  $ ruby -rnwdump test.rb
  NODE_NEWLINE at test.rb:1
  |-nth = 1
  +-next = NODE_DEFN at test.rb:1
    |-defn = NODE_SCOPE at test.rb:3
    | |-rval = false
    | |-tbl = nil
    | +-next = NODE_BLOCK at test.rb:1
    |   |-next = NODE_BLOCK at test.rb:3
    |   | |-next = false
    |   | +-head = NODE_NEWLINE at test.rb:2
    |   |   |-nth = 2
    |   |   +-next = NODE_CALL at test.rb:2
    |   |     |-recv = NODE_LIT at test.rb:2
    |   |     | +-lit = 1
    |   |     |-args = NODE_ARRAY at test.rb:2
    |   |     | |-alen = 1
    |   |     | |-head = NODE_LIT at test.rb:2
    |   |     | | +-lit = 1
    |   |     | +-next = false
    |   |     +-mid = :+
    |   +-head = NODE_ARGS at test.rb:1
    |     |-cnt = 0
    |     |-rest = -1
    |     +-opt = false
    |-mid = :foo
    +-noex = PRIVATE
%font "standard"

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% Inspecting Ruby's internals: bruby
%% 
%% TODO

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% TODO: interpreter/VM hacks

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% TODO: lazy-evaluation of expresions?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Using Ripper

	Could be used to implement macros in Ruby?
	Interface documented in ripper.y
	Examples can be found on Rubyforge

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Where to learn more

	The Pickaxe book
	README.EXT (came with ruby)
	The ruby source (especially object.c and eval.c -- grep through them
	by searching for define_method)
	The Ruby Hacker's Guide
	ruby-talk mailing list
%%http://84.128.196.153/exotic/