Avoiding Pitfalls in C Extensions
RubyConf.new(2007)
Paul Brannan
Avoiding Pitfalls in C Extensions
Observation
Werewolf
Pair programming
Pay attention to detail
Two reasons for writing extensions
Speed
Interface with other languages
Concepts that apply to one usually apply to the other
A simple example - adding two numbers
#include <ruby.h>
/* Add integers v1 and v2 and return the result */
static VALUE add(VALUE self, VALUE v1, VALUE v2)
{
/* Convert our arguments */
int i1 = NUM2INT(v1);
int i2 = NUM2INT(v2);
/* Do the math */
int result = i1 + i2;
Adding two numbers (cont'd)
/* Convert and return the result */
return INT2NUM(result);
}
/* Initialize the library */
void Init_add()
{
rb_define_global_function("add", add, 2);
}
Building the extension
extconf.rb:
require 'mkmf'
create_makefile('add')
$ ruby extconf.rb
creating Makefile
Building the extension
test.rb:
require 'add'
puts add(2, 3)
$ ruby test.rb
5
A real-world example
Let's build something fun
hello world in C!
nah
Let's build...
A JIT compiler!
Tools we'll need:
Ruby
C compiler
libjit
Adding numbers with libjit
Create a function signature
Create and lock a context
Specify the instructions in the function body
Compile the function
Unlock the context
So that works...
But it's not a JIT compiler yet
We still need:
A parser
A script to compile and run our code
A wrapper for libjit - littlejit
Littlejit - pieces we need
Functions
Create
Append instructions
Compile
Call/apply
Values (variables)
Result of an arithmetic instruction
Constants
Literals, e.g. 1, 2, 3
Littlejit - pieces we need
Functions
Create
Append instructions
Compile
Call/apply
Values (variables)
Result of an arithmetic instruction
Constants
Literals, e.g. 1, 2, 3
Naming conventions
Two variables with the same name?
foo_v vs. foo
v vs. pv (bignum.c)
method vs. mdata (eval.c)
dir vs. dp (dir.c)
emitter vs. emitterPtr (syck/rubyext.c)
obj vs. data (curses.c)
Personal preference
Be consistent!
Data conversion
unsigned long num_args = NUM2ULONG(num_args_v);
Lots of integer conversions
NUM2ULONG
NUM2UINT
FIX2ULONG
FIX2INT
...
Consider:
Valid ranges for your data types
Valid ranges for your data
Prefer NUM2 macros since they do validation
Littlejit - pieces we need
Functions
Create
Append instructions
Compile
Call/apply
Values (variables)
Result of an arithmetic instruction
Constants
Literals, e.g. 1, 2, 3
Exception safety
Exception-safe code is better code
Use rb_ensure or rb_protect to clean up resources in C extensions
Check return values
C APIs rarely use exceptions
Check return values at every point and convert error codes to exceptions
Unit testing
How do we unit test this code?
Most C libraries aren't unit testable
Inject a layer so we can use mock objects
Don't bother - we have to write an integration test anyway
Rewrite the library so it is unit testable
Littlejit - pieces we need
Functions
Create
Append instructions
Compile
Call/apply
Values (variables)
Result of an arithmetic instruction
Constants
Literals, e.g. 1, 2, 3
Ownership
Critical to consider who owns a wrapped object
If Ruby does not own the object, Ruby should not destroy it
If a parent object owns the object and the parent obejct is owned by Ruby, the parent
object should be marked
Littlejit - pieces we need
Functions
Create
Append instructions
Compile
Call/apply
Values (variables)
Result of an arithmetic instruction
Constants
Literals, e.g. 1, 2, 3
So that works...
But it's not a JIT compiler yet
We still need:
A parser
A script to compile and run our code
A wrapper for libjit
Is it safe?
Security concerns:
Arbitrary code execution in high $SAFE levels
Use of potentially harmful tainted data
What have we learned?
Consistency in naming
Consistency in data types and conversions
Check return values
Exception safety
Unit testing
Object ownership
Security
Another example
Stock symbol lookup application
Using boost::shared_ptr
typedef boost::shared_ptr<Symbol_Info>
Symbol_Info_shptr;
Consider ownership
boost::shared_ptr owns the object
Ruby owns the shared pointer
Exception conversion
Simple rules:
Convert C++ exceptions to Ruby exceptions at C++/Ruby
boundaries
Convert Ruby exceptions to C++ exceptions at Ruby/C++
boundaries
A database
Use Pstore - it's simple
Need marshal_dump/marshal_load
Oft-forgotten methods in C extensions
marshal_dump/marshal_load
dup/clone
to_s/inspect
Extensions - things that can go wrong
Conversions
Exceptions
Incorrect object initialization
Inheritance
Pointer to base vs. pointer to derived
Incorrect use of Ruby API
Tools
SWIG
Simplified Wrapper Interface Generator
Rice
Ruby Interface for C++ Extensions
like boost++python
Rice and integer conversions
VALUE x_v = ULONG2NUM(x);
Object y_v = to_ruby(y);
unsigned long x = NUM2ULONG(x_v);
unsigned long y = from_ruby<unsigned long>(y_v);
Arrays in Rice
Array a;
a.push(42);
a[0] = 10;
std::cout << a[0] << std::end; // => 10
Defining classes in Rice
class Animal : public Organism
{
public:
virtual ~Animal() = 0;
virtual char const * speak() = 0;
};
Defining classes in Rice (cont'd)
class Dog : public Animal
{
public:
virtual ~Dog() { }
virtual char const * speak()
{
return "Woof woof";
}
};
Defining classes in Rice (cont'd)
define_class<Animal>("Animal")
.define_method("speak", &Animal::speak);
define_class<Dog, Animal>("Dog")
.define_constructor(Constructor<Dog>());
Final thoughts
Be observant
Pay attention to detail
Think about:
Exceptions and edge cases
Object ownership
Testing
Use a tool
Questions?>