Going Native Frederick Cheung CTO, dressipi.com
@fglc2 / spacevatican.org
Monday, 10 December 12 Why?
• Performance (bottleneck) • Platform specific functionality • Access to best in class libraries
Monday, 10 December 12 Downsides
• Generally harder • Memory management • ruby c api poorly documented • Crashes harder when things go wrong • Slower to write - ruby is very concise and expressive
Monday, 10 December 12 Options • Native java code • “Classic” C-extension • SWIG • RICE • RubyInline • FFI
Monday, 10 December 12 Calling java from JRuby • Just do it - stupidly easy • JRuby handles most of the type conversion, method name conversion require 'java' java_import java.lang.System version = System.getProperties.get("java.runtime.version") version = System.properties["java.runtime.version"]
Monday, 10 December 12 JRuby continued • Can subclass java classes in ruby • Can implement a java interface in ruby • Rescue/raise java exceptions from ruby • Blocks handled nicely java.lang.Thread.new do puts 'hi' end
Monday, 10 December 12 C extensions
• C code written using the MRI api - like the internals of ruby itself • Compatible with MRI, Rubinius, partly with jruby • Intricacies of the API not well documented (README.EXT, headers) • C api can change between versions (but then so can the ruby API)
Monday, 10 December 12 What does a C extension do?
Monday, 10 December 12 Write methods in C and attach them to ruby objects
static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ ... } void Init_keychain(){ rb_cKeychain = rb_const_get(rb_cObject, rb_intern("Keychain")); rb_define_singleton_method(rb_cKeychain, "find", RUBY_METHOD_FUNC(rb_keychain_find), -1); }
Monday, 10 December 12 Write methods in C and attach them to ruby objects
static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ ... } void Init_keychain(){ rb_cKeychain = rb_const_get(rb_cObject, rb_intern("Keychain")); rb_define_singleton_method(rb_cKeychain, "find", RUBY_METHOD_FUNC(rb_keychain_find), -1); }
Monday, 10 December 12 Write methods in C and attach them to ruby objects
static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ ... } void Init_keychain(){ rb_cKeychain = rb_const_get(rb_cObject, rb_intern("Keychain")); rb_define_singleton_method(rb_cKeychain, "find", RUBY_METHOD_FUNC(rb_keychain_find), -1); }
Monday, 10 December 12 Wrap native data structures
SecKeychainRef keychainRef = ... VALUE result = Data_Wrap_Struct(rb_cKeychain, NULL, CFRelease, keychainRef);
SecKeychainRef keychain=NULL; Data_Get_Struct(ruby_object, struct OpaqueSecKeychainRef, keychain);
Monday, 10 December 12 Wrap native data structures
SecKeychainRef keychainRef = ... VALUE result = Data_Wrap_Struct(rb_cKeychain, NULL, CFRelease, keychainRef);
SecKeychainRef keychain=NULL; Data_Get_Struct(ruby_object, struct OpaqueSecKeychainRef, keychain);
Monday, 10 December 12 Wrap native data structures
SecKeychainRef keychainRef = ... VALUE result = Data_Wrap_Struct(rb_cKeychain, NULL, CFRelease, keychainRef);
SecKeychainRef keychain=NULL; Data_Get_Struct(ruby_object, struct OpaqueSecKeychainRef, keychain);
Monday, 10 December 12 Convert/check types
• rb_float_new / RFLOAT_VALUE • CheckType(foo, T_STRING) • StringValueCStr(foo) • FIX2INT, INT2FIX, LL2NUM, ... • NIL_P • RTEST
Monday, 10 December 12 Gets verbose quickly
def find(first_or_all, kind, options={}) if options[:some_option] ... end end
Monday, 10 December 12 static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ VALUE kind; VALUE options; VALUE first_or_all;
rb_scan_args(argc, argv, "2:", &first_or_all, &kind, &options);
Check_Type(first_or_all, T_SYMBOL); Check_Type(kind, T_STRING); if(options){ if(RTEST(rb_hash_aref(options, SYM2ID(rb_intern("some_option"))))) { ... } } }
Monday, 10 December 12 static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ VALUE kind; VALUE options; VALUE first_or_all;
rb_scan_args(argc, argv, "2:", &first_or_all, &kind, &options);
Check_Type(first_or_all, T_SYMBOL); Check_Type(kind, T_STRING); if(options){ if(RTEST(rb_hash_aref(options, SYM2ID(rb_intern("some_option"))))) { ... } } }
Monday, 10 December 12 static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ VALUE kind; VALUE options; VALUE first_or_all;
rb_scan_args(argc, argv, "2:", &first_or_all, &kind, &options);
Check_Type(first_or_all, T_SYMBOL); Check_Type(kind, T_STRING); if(options){ if(RTEST(rb_hash_aref(options, SYM2ID(rb_intern("some_option"))))) { ... } } }
Monday, 10 December 12 static VALUE rb_keychain_find(int argc, VALUE *argv, VALUE self){ VALUE kind; VALUE options; VALUE first_or_all;
rb_scan_args(argc, argv, "2:", &first_or_all, &kind, &options);
Check_Type(first_or_all, T_SYMBOL); Check_Type(kind, T_STRING); if(options){ if(RTEST(rb_hash_aref(options, SYM2ID(rb_intern("some_option"))))) { ... } } }
Monday, 10 December 12 Call a method with a block
static VALUE doSomething(VALUE yielded_object, VALUE rb_context,int argc, VALUE *argv){ Context *context = NULL; Data_Get_Struct( rb_context, Context, context); ... return Qnil; }
Context *context = ... VALUE wrapped_struct = Data_Wrap_Struct(rb_cFindContext, NULL,NULL, context); rb_block_call(rb_cUser, rb_intern("find_each"), 0, NULL, RUBY_METHOD_FUNC(doSomething), wrapped_struct);
Monday, 10 December 12 Call a method with a block
static VALUE doSomething(VALUE yielded_object, VALUE rb_context,int argc, VALUE *argv){ Context *context = NULL; Data_Get_Struct( rb_context, Context, context); ... return Qnil; }
Context *context = ... VALUE wrapped_struct = Data_Wrap_Struct(rb_cFindContext, NULL,NULL, context); rb_block_call(rb_cUser, rb_intern("find_each"), 0, NULL, RUBY_METHOD_FUNC(doSomething), wrapped_struct);
Monday, 10 December 12 Call a method with a block
static VALUE doSomething(VALUE yielded_object, VALUE rb_context,int argc, VALUE *argv){ Context *context = NULL; Data_Get_Struct( rb_context, Context, context); ... return Qnil; }
Context *context = ... VALUE wrapped_struct = Data_Wrap_Struct(rb_cFindContext, NULL,NULL, context); rb_block_call(rb_cUser, rb_intern("find_each"), 0, NULL, RUBY_METHOD_FUNC(doSomething), wrapped_struct);
Monday, 10 December 12 • Anything is possible but it can be quite laborious
• Not much typesafety - nearly everything is a VALUE
• Sometimes unhelpful api naming: rb_str_new, rb_str_new2, rb_str_new3, rb_str_new4, rb_str_new5
Monday, 10 December 12 SWIG
• Generates code for C extensions automatically from a marked up header file • can target many languages (ruby, python, php, ocaml, perl, ...) • generated code is pretty illegible • interfaces often feel unnatural to me
Monday, 10 December 12 RICE
• C++ library wrapping the ruby c api, smoothing inconsistencies • Makes wrapping C++ classes easy • type conversions handled via template functions • implements STL iterators for Array, Hash
Monday, 10 December 12 • tries to leverage type safety • converts between C++ and ruby exceptions • Doesn’t seem very active - 6 commits in past year
Monday, 10 December 12 RubyInline
• Write C extensions without some of the hassle • generates, compiles and loads extension at runtime • doesn’t work in irb
Monday, 10 December 12 require 'inline' class Factorial inline :C do |builder| builder.c_singleton <<-SRC
long calculate(long n){ long result = 1; for(long i = 2; i<=n; i++){ result *= i; } return result; }
SRC end end
Monday, 10 December 12 require 'inline' class Factorial inline :C do |builder| builder.c_singleton <<-SRC
long calculate(long n){ long result = 1; for(long i = 2; i<=n; i++){ result *= i; } return result; }
SRC end end
Monday, 10 December 12 static VALUE calculate(VALUE self, VALUE _n) { long n = NUM2LONG(_n); long result = 1; for(long i = 2; i<=n; i++){ result *= i; } return LONG2NUM(result); }
#ifdef __cplusplus extern "C" { #endif void Init_Inline_Factorial_7b051bd7f8f35cf0422f74703bc98c19() { VALUE c = rb_cObject; c = rb_const_get(c, rb_intern("Factorial")); rb_define_singleton_method(c, "calculate", (VALUE(*)(ANYARGS))calculate, 1); } #ifdef __cplusplus } #endif
Monday, 10 December 12 static VALUE calculate(VALUE self, VALUE _n) { long n = NUM2LONG(_n); long result = 1; for(long i = 2; i<=n; i++){ result *= i; } return LONG2NUM(result); }
#ifdef __cplusplus extern "C" { #endif void Init_Inline_Factorial_7b051bd7f8f35cf0422f74703bc98c19() { VALUE c = rb_cObject; c = rb_const_get(c, rb_intern("Factorial")); rb_define_singleton_method(c, "calculate", (VALUE(*)(ANYARGS))calculate, 1); } #ifdef __cplusplus } #endif
Monday, 10 December 12 static VALUE calculate(VALUE self, VALUE _n) { long n = NUM2LONG(_n); long result = 1; for(long i = 2; i<=n; i++){ result *= i; } return LONG2NUM(result); }
#ifdef __cplusplus extern "C" { #endif void Init_Inline_Factorial_7b051bd7f8f35cf0422f74703bc98c19() { VALUE c = rb_cObject; c = rb_const_get(c, rb_intern("Factorial")); rb_define_singleton_method(c, "calculate", (VALUE(*)(ANYARGS))calculate, 1); } #ifdef __cplusplus } #endif
Monday, 10 December 12 module Foo inline do |builder| builder.add_compile_flags '-x c++', '-lstdc++', '-Wall', '-Werror' builder.prefix <<-SRC # line #{__LINE__ + 1} "#{__FILE__}" class foo { ... } SRC builder.c_raw <<-SRC VALUE attr_1(VALUE self){ foo *data = NULL; Data_Get_Struct( self, foo, data); return INT2FIX(data->get_attr_1()); } SRC end end
Monday, 10 December 12 module Foo inline do |builder| builder.add_compile_flags '-x c++', '-lstdc++', '-Wall', '-Werror' builder.prefix <<-SRC # line #{__LINE__ + 1} "#{__FILE__}" class foo { ... } SRC builder.c_raw <<-SRC VALUE attr_1(VALUE self){ foo *data = NULL; Data_Get_Struct( self, foo, data); return INT2FIX(data->get_attr_1()); } SRC end end
Monday, 10 December 12 module Foo inline do |builder| builder.add_compile_flags '-x c++', '-lstdc++', '-Wall', '-Werror' builder.prefix <<-SRC # line #{__LINE__ + 1} "#{__FILE__}" class foo { ... } SRC builder.c_raw <<-SRC VALUE attr_1(VALUE self){ foo *data = NULL; Data_Get_Struct( self, foo, data); return INT2FIX(data->get_attr_1()); } SRC end end
Monday, 10 December 12 module Foo inline do |builder| builder.add_compile_flags '-x c++', '-lstdc++', '-Wall', '-Werror' builder.prefix <<-SRC # line #{__LINE__ + 1} "#{__FILE__}" class foo { ... } SRC builder.c_raw <<-SRC VALUE attr_1(VALUE self){ foo *data = NULL; Data_Get_Struct( self, foo, data); return INT2FIX(data->get_attr_1()); } SRC end end
Monday, 10 December 12 Rubyinline - conclusion
• Easy to deploy - no extra steps required • great for small hotspots / simple interfaces • In general no syntax highlighting, code completion etc. • On jruby uses java instead
Monday, 10 December 12 And now, for something completely different
Monday, 10 December 12 FFI
• based on libffi • Ruby DSL for binding C functions / global variables • No compiler toolchain required, no C code to write • Supported by jruby, mri, rubinius
Monday, 10 December 12 require 'ffi'
module C extend FFI::Library
ffi_lib 'c'
attach_function 'puts', [:string], :int
end
C.puts "Hello world"
Monday, 10 December 12 require 'ffi'
module C extend FFI::Library
ffi_lib 'c'
attach_function 'puts', [:string], :int
end
C.puts "Hello world"
Monday, 10 December 12 require 'ffi'
module C extend FFI::Library
ffi_lib 'c'
attach_function 'puts', [:string], :int
end
C.puts "Hello world"
Monday, 10 December 12 require 'ffi'
module C extend FFI::Library
ffi_lib 'c'
attach_function 'puts', [:string], :int
end
C.puts "Hello world"
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC callback :qsort_cmp, [:pointer, :pointer ], :int attach_function :qsort, [:pointer, :ulong, :ulong, :qsort_cmp], :int end
def sort(array_of_ints) p = FFI::MemoryPointer.new(:int32, array_of_ints.size) p.put_array_of_int32(0, array_of_ints) LibC.qsort(p, array_of_ints.size, 4) do |p1, p2| i1 = p1.get_int32(0) i2 = p2.get_int32(0) i1 <=> i2 end p.get_array_of_int32(0, array_of_ints.size) end
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC callback :qsort_cmp, [:pointer, :pointer ], :int attach_function :qsort, [:pointer, :ulong, :ulong, :qsort_cmp], :int end
def sort(array_of_ints) p = FFI::MemoryPointer.new(:int32, array_of_ints.size) p.put_array_of_int32(0, array_of_ints) LibC.qsort(p, array_of_ints.size, 4) do |p1, p2| i1 = p1.get_int32(0) i2 = p2.get_int32(0) i1 <=> i2 end p.get_array_of_int32(0, array_of_ints.size) end
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC callback :qsort_cmp, [:pointer, :pointer], :int attach_function :qsort, [:pointer, :ulong, :ulong, :qsort_cmp], :int end
def sort(array_of_ints) p = FFI::MemoryPointer.new(:int32, array_of_ints.size) p.put_array_of_int32(0, array_of_ints) LibC.qsort(p, array_of_ints.size, 4) do |p1, p2| i1 = p1.get_int32(0) i2 = p2.get_int32(0) i1 <=> i2 end p.get_array_of_int32(0, array_of_ints.size) end
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC callback :qsort_cmp, [:pointer, :pointer], :int attach_function :qsort, [:pointer, :ulong, :ulong, :qsort_cmp], :int end
def sort(array_of_ints) p = FFI::MemoryPointer.new(:int32, array_of_ints.size) p.put_array_of_int32(0, array_of_ints) LibC.qsort(p, array_of_ints.size, 4) do |p1, p2| i1 = p1.get_int32(0) i2 = p2.get_int32(0) i1 <=> i2 end p.get_array_of_int32(0, array_of_ints.size) end
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC
class Timezone < FFI::Struct layout :tz_minuteswest, :int, :tz_dsttime, :int end class Timeval < FFI::Struct layout :tv_sec, :time_t, :tv_usec, :suseconds_t end attach_function :gettimeofday, [Timeval, Timezone], :int end
out = LibC::Timeval.new LibC.gettimeofday(out, nil) puts "the time according to libc is #{out[:tv_sec]}"
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC
class Timezone < FFI::Struct layout :tz_minuteswest, :int, :tz_dsttime, :int end class Timeval < FFI::Struct layout :tv_sec, :time_t, :tv_usec, :suseconds_t end attach_function :gettimeofday, [Timeval, Timezone], :int end
out = LibC::Timeval.new LibC.gettimeofday(out, nil) puts "the time according to libc is #{out[:tv_sec]}"
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC
class Timezone < FFI::Struct layout :tz_minuteswest, :int, :tz_dsttime, :int end class Timeval < FFI::Struct layout :tv_sec, :time_t, :tv_usec, :suseconds_t end attach_function :gettimeofday, [Timeval, Timezone], :int end
out = LibC::Timeval.new LibC.gettimeofday(out, nil) puts "the time according to libc is #{out[:tv_sec]}"
Monday, 10 December 12 module LibC extend FFI::Library ffi_lib FFI::Library::LIBC
class Timezone < FFI::Struct layout :tz_minuteswest, :int, :tz_dsttime, :int end class Timeval < FFI::Struct layout :tv_sec, :time_t, :tv_usec, :suseconds_t end attach_function :gettimeofday, [Timeval, Timezone], :int end
out = LibC::Timeval.new LibC.gettimeofday(out, nil) puts "the time according to libc is #{out[:tv_sec]}"
Monday, 10 December 12 Beyond toy examples
• OS X Keychain wrapper written using C api and using FFI • https://github.com/fcheung/keychain ~ 1000 lines of ruby, ~ 600 lines of real code • https://github.com/fcheung/keychain_c/ ~ 700 lines of C • Both versions pass the same set of specs • FFI version behaves nicer and has a reusable set of corefoundation wrappers
Monday, 10 December 12 FFI summary
• Great when you have a C library to wrap • Small performance penalty over a C extension (often irrelevant) • Much easier/nicer (in my experience) than writing C extensions
Monday, 10 December 12 The good stuff
• No knowledge of MRI C api required • Better compatibility across implementations • Glue code is now ruby instead of C - easier to write and easier to be ruby-like • less typechecking/wrapping/unwrapping boilerplate • FFI yielded more reusable components (core_foundation gem)
Monday, 10 December 12 But...
• Takes a little getting used to • Constants / enums are a pain • you still need to understand C, memory management • Some incompatibilities between FFI implementations • Still crashes just as hard if you mess it up
Monday, 10 December 12 Objective-C
• Not handled by ffi out of the box • Can write wrapper library exposing C entry points • Is just C, can use ffi to call objc_msgSend, objc_getClass, ...
Monday, 10 December 12 Questions?
• [email protected] • @fglc2 • github.com/fcheung • http://spacevatican.org
Monday, 10 December 12