%% Default fonts
%deffont "standard" xfont "helvetica-medium-r", tfont "arial.ttf"
%deffont "thick" xfont "helvetica-bold-r", tfont "arialb.ttf"
%deffont "typewriter" xfont "fixed-medium-r", tfont "cour.ttf"
%%
%% Default settings per each line number
%default 1 area 90 90, leftfill, size 2, fore "yellow", back "black", font "thick", bgrad
%default 2 size 6, vgap 10, prefix " ", ccolor "black"
%default 3 size 2, bar "gray70", vgap 10
%default 4 size 5, fore "white", vgap 30, prefix " ", font "standard"
%%
%% Default settings for tab-indented lines
%tab 1 size 5, vgap 40, prefix "  ", icon box "green" 50
%tab 2 size 4, vgap 40, prefix "      ", icon arc "yellow" 50
%tab 3 size 3, vgap 40, prefix "            ", icon delta3 "white" 40
%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page
%center

Hacking Ruby

Ruby Conference 2004

Paul Brannan
paul@atdesk.com

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

What is "hacking ruby"?

	From foldoc:
%font "typewriter", size 3
  hack
    6. To interact with a computer in a playful and exploratory
    rather than goal-directed way.  "Whatcha up to?"  "Oh, just
    hacking."
%font "standard"
	This interaction often results in code that lets ruby do/express things it was never specifically designed to

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Why hack?

	Ruby is lacking a feature from <insert your favorite language here> that you can't bear to live without
	To do a task that would otherwise be difficult
	Proof-of-concept for an RCR
	To explore/debug extensions and/or ruby itself
	For fun!

%% This last reason is, of course, the most _____ reason

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Example of a "hack"

%font "typewriter", size 3
 1   class Array
 2     alias_method :concat, :old_concat
 3     def concat(*arrays)
 4       arrays.each { |array| old_concat(array) }
 5     end
 6   end
 7 
 8   a = [1, 2, 3]
 9   a.concat([1], [2], [3, 4])
10   p a #=> [1, 2, 3, 1, 2, 3, 4]

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Ruby, how do I hack thee?  Let me count the ways...

	Pure Ruby (using set_trace_func, callcc, etc.)
	Already-written extensions (enumerator, stringio, etc.)
	Using libdl (e.g. evil.rb)
	Writing your own extensions
	Hacking the interpreter itself

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

The Ruby Hacker's Guide

%% A talk on hacking Ruby could not possibly be complete without
%% mentioning this book!

	In Japanese: http://i.loveruby.net/ja/rhg/
	Can't read Japanese?  Use babelfish!
	But babelfish doesn't work with iso-2022-jp?
	Convert it to euc-jp using iconv.
	But babelfish doesn't like large documents?
	Split the html into multiple files!
	Maybe someone has already done this work: http://rubystuff.org/rhg/convertrhg.rb

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Tools for hacking in pure Ruby

	set_trace_func
	method_missing
	append_features/extend_object
	binding
	callcc
	Thread.new

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

set_trace_func

	Provides a callback whenever the interpreter calls a method, evaluates a statement, etc.
	Used by the debugger for breakpoints, watchpoints, etc.

%font "typewriter", size 3
 1   def foo; puts "foo!"; end
 2   set_trace_func proc { |*x| p x }
 3   foo()

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

set_trace_func (cont'd)

%font "typewriter", size 3
 1 ["line", "test.rb", 3, nil, #<Binding:0x401b7b74>, false]
 2 ["call", "test.rb", 1, :foo, #<Binding:0x401b7a48>, Object]
 3 ["line", "test.rb", 1, :foo, #<Binding:0x401b7a0c>, Object]
 4 ["c-call", "test.rb", 1, :puts, #<Binding:0x401b7840>, Kernel]
 5 ["c-call", "test.rb", 1, :write, #<Binding:0x401b7804>, IO]
 6 foo!["c-return", "test.rb", 1, :write, #<Binding:0x401b7700>, IO]
 7 ["c-call", "test.rb", 1, :write, #<Binding:0x401b75fc>, IO]
 8 
 9 ["c-return", "test.rb", 1, :write, #<Binding:0x401b750c>, IO]
10 ["c-return", "test.rb", 1, :puts, #<Binding:0x401b741c>, Kernel]
11 ["return", "test.rb", 1, :foo, #<Binding:0x401b732c>, Object]

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

callcc

	Used to save the current point of execution
	Execution can be resumed from the saved point
	Similar to creating a stopped thread
	Can be used to access data you can't normally access

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Put it together...

%font "typewriter", size 3
 1 def Binding.of_caller(&block)
 2   old_critical = Thread.critical; Thread.critical = true
 3   count = 0
 4   cc = nil; result = callcc {|c| cc = c }

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Binding.of_caller (cont'd)

%font "typewriter", size 3
 5   tracer = lambda do |*args|
 6     type, context = args[0], args[4]
 7     if type == "return"
 8       count += 1
 9       if count == 2
10         set_trace_func(nil)
11         cc.call(eval("binding", context), nil)
12       end
13     elsif type != "line"
14       set_trace_func(nil)
15       cc.call(nil, lambda { raise(ArgumentError) })
16     end
17   end

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Binding.of_caller (cont'd)

%font "typewriter", size 3
19   unless result
20     set_trace_func(tracer)
21     return nil
22   else
23     Thread.critical = old_critical
24     yield result
25   end
26 end

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies

	Derived classes inherit from base clases
	No multiple inheritance
	Mixins inserted in the hierarchy above of the class they inherit from

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies (cont'd)

%font "typewriter", size 3
 1 class Base; end
 2 module M; end
 3 class Derived < Base
 4   include M
 5   class << self; @foo = 1; end
 6 end
 7 
 8 require 'classtree'
 9 Derived.classtree($stderr)

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Class hierarchies (cont'd)

%font "typewriter", size 3
 1 Derived
 2 |-class = #<Class:Derived>
 3 | |-class = #<Class:Class>
 4 <snip>
 5 | +-super = #<Class:Base>
 6 |   |-class = #<Class:Class> (*)
 7 |   +-super = #<Class:Object> (*)
 8 +-super = #<M:0x401b7958>
 9   +-class = M
10     |-class = Module (*)
11     +-super = #<Kernel:0x401afbb8>
12       +-class = Kernel (*)

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Put the class hierarchy to work

Overriding methods cleanly (not so useful on Ruby 2.0...):

%font "typewriter", size 3
 1   def override_method(*symbols, &block)
 2     new_methods = Module.new
 3     old_methods = Module.new
 4     new_methods.instance_eval { include old_methods }
 5     include new_methods
 6 
 7     symbols.each do |symbol|
 8       old_methods.__send__(:define_method, symbol, instance_method(symbol))
 9       new_methods.module_eval(&block)
10       define_method(symbol, new_methods.instance_method(symbol))
11     end
12   end

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overriding methods (cont'd)

%font "typewriter", size 3
 1 class Array
 2   override_method(:concat) do
 3     def concat(*arrays)
 4       arrays.each { |array| super(array) }
 5     end
 6   end
 7 end
 8 
 9 a = [1, 2, 3]
10 a.concat([1], [2], [3, 4])
11 p a #=> [1, 2, 3, 1, 2, 3, 4]

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Overriding methods (cont'd)

%font "typewriter", size 3
 1 Array
 2 |-class = #<Class:Array>
 3 | |-class = #<Class:Class>
 4 <snip>
 5 | +-super = #<Class:Object> (*)
 6 +-super = #<#<Module:0x401af474>:0x401af438>
 7   +-class = #<Module:0x401af474>
 8     |-class = #<Class:#<Module:0x401af474>>
 9     | |-class = #<Class:Module> (*)
10     | +-super = Module (*)
11     +-super = #<Kernel:0x401af2d0>
12       +-class = Kernel (*)

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

evil.rb

%% evil.rb uses a technique for accessing the ruby interpreter's internals
%% using libdl.  Why is this desirable?  Why not just use an extension?  Well,
%% because extensions have to be compiled; a library that uses libdl can be run
%% as-is.
%%
	http://evil.rubyforge.org
	Reproduces Ruby's C data structures using DL
	Why not just use an extension?
	Extensions have to be compiled; DL-based libraries run as-is.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Object#become

	Swaps one object for another
	One stupid implementation:
%font "typewriter", size 3
 1 static VALUE ruby_become(VALUE self, VALUE newobj) {
 2   /* Not implemented here: type check, frozen check, $SAFE check */
 3 
 4   struct RObject * robj1 = ROBJECT(obj1);
 5   struct RObject * robj2 = ROBJECT(obj2);
 6   struct RObject tmp_obj;
 7 
 8   tmp_obj = *robj1;
 9   *robj1  = *robj2;
10   *robj2  = tmp_obj;
11 
12   return self;
13 }

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

The power of Object#become: a lazy-evaluator

%font "typewriter", size 3
 1 class LazyEvaluator < KernellessObject
 2   def initialize(&block)
 3     @p = block
 4   end
 5 
 6   def method_missing(*args, &block)
 7     obj = @p.call
 8     obj.become(self) # self.become doesn't exist
 9     self.__send__(*args, &block)
10   end
11 end

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Accessing interpreter internals

	C functions/data in ruby.h and intern.h
	Non-static data not defined in a header file
	Accessing some data is a bit more tricky

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

How to access interpreter internals without modifying the interpreter

%% This is a potentially very powerful technique.  It involves getting a
%% pointer to a data structure (the hard part), then using that pointer.
%% But how can a pointer be used if it is pointing to a data structure
%% that is defined in a .c file?  Simple: use a ruby script to extract
%% the data structure from the .c file and put it into a header file.
%%
%% Is this technique safe?  I'd classify it as "mostly harmless."  It's
%% not 100% portable, but will work with many C platforms.  And even if
%% it doesn't work for your target platform, you can still use this
%% technique in the "proof-of-concept" stage before you write a
%% full-blown patch for the interpreter.
%%
	Get a pointer to the data structure
	Dereference the pointer
	Need to define a data type that mirrors the type defined in the C file
	Create this data type using a script
	Can't use this trick to access static data

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Inspecting Ruby's internals: Nodedump

	A C extension that dumps Ruby's nodes to the screen
	Works on Ruby 1.6.x
	Can't download it anymore

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Inspecting Ruby's internals: Nodewrap

	A C extension for directly inspecting/manipulating the code in a running program
	Example:
%font "typewriter", size 3
 1   irb(main):001:0> require 'nodewrap'
 2   => true
 3   irb(main):002:0> l = lambda { 1 + 1 }
 4   => #<Proc:0x401fbd78@(irb):2>
 5   irb(main):003:0> l.body
 6   => #<Node::CALL:0x401fa680>
 7   irb(main):004:0> l.body.members.each { |m| p [m, l[m]] }
 8   ["recv", 2]
 9   ["args", 2]
10   ["mid", 2]

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Marshalling methods with Nodewrap

%font "typewriter", size 3
 1 irb(main):001:0> require 'nodewrap'
 2 => true
 3 irb(main):002:0> class Foo; def foo; puts 'foo!'; end; end
 4 => nil
 5 irb(main):003:0> s = Marshal.dump(Foo.instance_method(:foo).body)
 6 => "u:Node::SCOPE..."
 7 irb(main):004:0> n = Marshal.load(s)
 8 => #<Node::SCOPE:0x401f1918>
 9 irb(main):005:0> class Bar; end; Bar.instance_eval { add_method(:foo, n, 0) }
10 => nil
11 irb(main):006:0> Bar.new.foo
12 foo!
13 => nil

%font "standard"

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Dumping nodes to stdout with Nodewrap

%font "typewriter", size 2
  $ cat test.rb
  def foo
    1+1
  end

  $ ruby -rnwdump test.rb
  NODE_NEWLINE at test.rb:1
  |-nth = 1
  +-next = NODE_DEFN at test.rb:1
    |-defn = NODE_SCOPE at test.rb:3
    | |-rval = false
    | |-tbl = nil
    | +-next = NODE_BLOCK at test.rb:1
    |   |-next = NODE_BLOCK at test.rb:3
    |   | |-next = false
    |   | +-head = NODE_NEWLINE at test.rb:2
    |   |   |-nth = 2
    |   |   +-next = NODE_CALL at test.rb:2
    |   |     |-recv = NODE_LIT at test.rb:2
    |   |     | +-lit = 1
    |   |     |-args = NODE_ARRAY at test.rb:2
    |   |     | |-alen = 1
    |   |     | |-head = NODE_LIT at test.rb:2
    |   |     | | +-lit = 1
    |   |     | +-next = false
    |   |     +-mid = :+
    |   +-head = NODE_ARGS at test.rb:1
    |     |-cnt = 0
    |     |-rest = -1
    |     +-opt = false
    |-mid = :foo
    +-noex = PRIVATE
%font "standard"

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% Inspecting Ruby's internals: bruby
%% 
%% TODO

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% TODO: interpreter/VM hacks

%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% %page
%% 
%% TODO: lazy-evaluation of expresions?

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Using Ripper

	Could be used to implement macros in Ruby?
	Interface documented in ripper.y
	Examples can be found on Rubyforge

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%page

Where to learn more

	The Pickaxe book
	README.EXT (came with ruby)
	The ruby source (especially object.c and eval.c -- grep through them
	by searching for define_method)
	The Ruby Hacker's Guide
	ruby-talk mailing list
%%http://84.128.196.153/exotic/