JRuby: Not Just Another Ruby Impl Charles Nutter Thomas Enebo

1 Introductions: Who am I? • Charles Oliver Nutter – [email protected] • Thomas Enebo - [email protected] • Engineers at Sun, JRuby core developer • 10+ years Java experience each > Also C/C++, Win32, C#, .NET, JavaME/CLDC, Perl, LiteStep, JINI, JavaEE • Working to make Ruby a first-class JVM language • Working to make JVM a better home for dynlangs

2 Agenda • What Is JRuby • Ruby Design Issues • The JRuby Way • JRBuilder/Cheri • NetBeans Ruby Support • Compatibility Metrics • JRuby Compiler and Performance Metrics • Conclusion and Q/A

3 What Is JRuby • Started in 2002 • Java implementation of Ruby language • Open source, many active contributors • Aiming for compatibility with current Ruby version • Easy integration with Java libraries, infrastructure > Call to Ruby from Java via JSR223, BSF > Use Java classes from Ruby (e.g. Script Java) • Growing set of external projects based on JRuby > JRuby-extras (ActiveRecord-JDBC, rails-integration, ...)

4 Why Ruby on Java? • Ruby only supports green threads • Unicode story is rather weak • Performance often considered “slow” • Too great a change for many organizations • Still relatively new, perceived as “untried” • Existing investments in Java infrastructure • Vast, proven collection of Java libraries • Complete platform independence

5 Ruby Design Issues • Ruby 1.8: Green threading > Can't scale across processors/cores > C libraries won't or can't yield to Ruby threads > One-size-fits-all scheduler doesn't fit all • Ruby 1.9: Native, non-parallel threading > Core classes, extensions not ready for parallel exec > Lots of work to ensure thread-safe internals > Move to native threads, but not running in parallel – **May change before 2.0

6 Ruby Design Issues • Ruby 1.8: Partial Unicode support > Internet-connected apps must have solid Unicode > Ruby provides partial support, but not consistent > App developers must roll their own: Rails MultiByte • Ruby 1.9: Full Unicode (M17N) but drastic change > String interface changes to per-char, not per-byte – Breaks existing consumers of those methods – Duplicate methods to allow per-byte access > Each string can have its own encoding – Mixed encoding app behavior unclear – Much more complicated than “just support Unicode”

7 Ruby Design Issues • Ruby 1.8: Slower than most languages > 1.8 is usually called “fast enough” > ...but routinely finishes last on benchmarks > ...and no plans to improve in 1.8 • Ruby 1.9: Improvement, but still more to do > 3-4x faster on some targeted tests > ...but no change on others > AOT compilation > More to do: no JIT; GC, threading still slow; > Not likely to be “done” soon (1.9.1 in December)

8 Ruby Design Issues • Ruby 1.8: Memory management > Simple design > Good for many apps, but (probably?) doesn't scale > Stop-the-world GC, no parallel or generational collection • Ruby 1.9: No change > Improved performance means more garbage > Multiple native threads, potentially more contention > GC problems could multiply, cutting into profits

9 Ruby Design Issues • Ruby 1.8: C language extensions > C is difficult to write well > Badly-behaved extensions can crash Ruby runtime > Threading, GC issues > Portable, but often must be recompiled > No security restrictions • Ruby 1.9: No change > Can't afford to break extensions > Ruby C API exposure limits changes to Ruby core > Common performance advice: “Write it in C”

10 Other Ruby Issues • Politics > You want me to switch to what? > ...and it needs new servers/software/training? > Potentially better accepted by 1.9 • Legacy > Lots of Java apps and code in the world – Extensive library of Java frameworks – Many experienced users, developers, administrators > Large existing investment in Java infrastructure – Server software, developer and admin training – Existing services, frameworks

11 The JRuby Way • Native threading > Scale across all processors/cores in system > Concurrent execution, even in extensions > Allow system to schedule threads > Ensure reasonable safety in core classes

12 The JRuby Way • World-class, native Unicode support > Provide Ruby's byte[]-based String – ...but also provide native Rails multibyte “Chars” class – ...and you can also use Java's UTF-16 String > Java has complete, reliable Unicode support – ...and all libraries are Unicode-ready – ...and all IO channels support many encodings

13 The JRuby Way • Scalable Performance > Make interpreter as fast as possible – No reason we can't interpret as fast as Ruby 1.8 > Support Ruby 1.9/2.0 bytecode engine – Same resulting performance gains in JRuby as in Ruby 1.9 – Future-proof > Compile to Java bytecode – AOT and JIT compilation – Let HotSpot take over – JIT compilation – Code inlining – Dynamic optimization

14 The JRuby Way • Let Java manage memory > Best memory management and GC in the world > Wide variety of GC options – Concurrent – Generational – Real-time > Scales up to enormous applications and loads > Battle-tested: millions of deployments worldwide

15 The JRuby Way • Java-based extensions > Easier to write than C > Nearly impossible to crash runtime or VM > Truly portable: write once, run anywhere > No GC, threading, or security issues: it's all Java > Clean separation between core and extension API > Easier to expose Java libraries than C libraries > “Everybody” knows Java

16 The JRuby Way • Politics don't get in the way > JRuby is “just another Java library” > Easier to change web framework than app architecture > Minimal impact to dev, admin processes > Ten years of mainstream Java • Legacy integrates just fine > Call existing services, libraries directly from Ruby > Implement services in Ruby > Deploy to existing servers > Same proven scalability, reliability...just a new language

17 What Can It Do? • Most “pure” Ruby code runs just fine now • Rake runs well, though no official testing effort • RubyGems, ditto, but it's hit more often • Projects using JRuby + RSpec now • Rails looking better and better > Some small-scale production apps in the wild > Rails unit tests above 98% passing in JRuby > More and more of Rails in our regression suites • Other combinations of JRuby + X popping up daily

18 Yeah, But What Else? • Ruby + Java = Awesome > Java provides libraries and VM Ruby needs – Extensive, flexible libraries – Often too complicated to use...why? – Perhaps Java is good for frameworks, but not for calling them? – Perhaps there's a better option for “gluing” libraries together? > Ruby provides agile, flexible, fun language Java needs – Unix metaphor: libraries use C, apps often use dynlangs – Applied to Java: JVM is our kernel, Java is our C – Ruby and its ilk finally complete the platform > More powerful than either alone

19 JRBuilder/Cheri • JRBuilder/Cheri project by Bill Dortch > A Ruby “SwingBuilder” > Groovy-like builder syntax for Swing – But written entirely in Ruby – ...and only a couple kloc > Also inspired by F3 – Declarative syntax for Swing/Java2D UIs > Data binding support similar to JSR 295 “Beans Binding” > Additional Java integration enhancements – Java array and primitive manipulation – Some merged into JRuby proper, more to come

20 Demo: JRBuilder/Cheri - A Ruby SwingBuilder

21 NetBeans Ruby Support • Tor Norbye started work in September • Uses JRuby's parser • Available in NetBeans 6 Milestone 7 > Ruby support in “Update Center” > Still early, but very promising • Editor features: syntax coloring, completion (methods, string escapes, regex, variables...), variable renaming, hyperlinking (go to definition) • IDE features: running tests and scripts, Ruby projects, Rails projects, in-IDE WEBrick server

22 Demo: Netbeans Ruby Support

23 Measuring Compatibility • How do you implement a language with no spec? • ...and no complete suite of functional tests? • Make do with what's available > Partial suites (MRI's tests, BFTS, Rubicon, ruby_test) > Applications' own suites (RSpec, Rails so far) > Our own suite of tests, expanded over time > Other implementations' tests (Rubinius, others soon?) • Document as we go > Community spec: www.headius.com/rubyspec > JRuby mailing list archives, www.headius.com/jrubywiki

24 Compatibility: Ruby Tests • Ruby's sample test (ruby_src/sample/test.rb) > 95% or better passing > Remaining is mostly POSIXy stuff we can't support > Popular measure of compatibility (XRuby, Ruby.NET) > ...but very limited in scope • Ruby's language unit tests (ruby_src/test/ruby) > 90% or better passing > “second step” toward compatibility? • Rubicon (rubyforge.org/projects/rubytests) > Many tests 100%; perhaps 80% passing overall > Not the most reliable suite, but oldest, largest 25 Compatibility: Rails 1.2.1

> ActiveSupport – 498 tests, 1849 assertions, 4 failures, 0 errors (99.19%) > ActionPack – 1157 tests, 4868 assertions, 6 failures, 2 errors (99.30%) > ActionMailer – 64 tests, 133 assertions, 4 failures, 2 errors (90.63%) > ActionWebService – 96 tests, 531 assertions, 0 failures, 0 errors (100%) > ActiveRecord – 1012 tests, 3658 assertions, 41 failures, 6 errors (95.35%) > 2827 tests, 62 failures or errors (97.70%)

26 Compatibility: Others • BFTS (rubyforge.org/projects/bfts) > Maybe 90% passing > Fairly complete, but for very few core classes • ruby_test (rubyforge.org/projects/ruby_test) > Unknown; just started looking at it recently • RSpec's specs (rspec.rubyforge.org) > 99% passing • Our own regression suite: 100%!!! > Coverage exceeds 80%

27 Measuring Performance • No standard set of benchmarks for Ruby • Alioth “Shootout” tests > Wide range of algorithms tested > Most are memory-intensive > Much-maligned among Rubyists • Ruby 1.9 benchmarks > Narrow but solid range of tests > Primarily testing areas YARV was designed to improve > Subject of recent “Ruby shootout” • Rails requests-per-second

28 Performance Metrics • Rails requests-per-second (WEBrick) • Still 100% interpreted, but some playing with JIT • Ruby 1.8.4 > static - 301.09 req/s > dynamic - 121.74 req/s • JRuby 0.9.8 (Java 6 server VM) > static – 180.03 req/s > Dynamic – 50.53 req/s

29 Performance: JRuby RRuunn Timmee 40 60.000 55.000 35 Ruby 1.8.4 50.000 Ruby 1.8.4 45.000 JRuby 30 JRuby 40.000 25 35.000 30.000 20 25.000 20.000 15 15.000 10.000 10 5.000 0.000 i t 5 t r s r k r t y e t k y s e d 1 h e p ib d n t ix a p s c o e p 2 s f n m d t e f v e a e re a a v c r t i o a r e n n x u r o o t is o e t r i o p n is r r o o g i c p _ o p j u in e o a l n a rd r e w c e t r u i h _ l a a o t m o d h n t s u n p t lo _ u b jo lo i r n _ s u lb s o s a g t p s a b p m t c _ n s e s n o p e d o _ _ lo m _ e o _ _ n e e _ l e e _ e _ le w _ p re o _ _ t p i r z _ c a e r a o _ t e s d _ p 2 2 d a o _ c o 2 r r m le _ 1 a e t p 1 a p h _ _ 2 m i l _ _ t h t o n e p o s x n _

0 s 1 _ _ t a c t o m m a 1 2 s _ h m s 1 p 2 n m s e w m a e a p

t m o

r s 2 e e n o k s y n p e e d v c v d 1 y m v _ v s f lo u v

e p _ l r _

a w p a v o n e o o u o v n i m m s m m i t r i e n r o v w m e c n t im p r o h e m

o i m _ _ c m p l o p o t _ o a v v

s m i p v s v s u p t a c _n s c _ s e _ p n v o c e e _ _ _ p

_ _ s o _ p a t p _ p d _ r 2 o _ c _ o m 2 2 p a p e 1 _ _

n a _ x _ y p t d s _ m o s e o 1 l m m o o p lo e s m v p 1 2

p o o v v a l a p _ e v m 2 o _ s s lo m a a o n v s

p v e

s m _ p p o m r a v s _ v h t 2 _ m 3 v m v 30 Performance: JRuby TTiimemess Fasstteerr 2.020.00 X faster 1.010.00 X faster 0.00 0.00 -1.00 -1.00 -2.00 -2.-030.00 -3.-040.00 -4.-050.00 -6.00 -5.00 -7.00 -6.00 t t i t v t r r n r s d -8.00 n 2 e p o p s k n y d e p i y h k s d 1 e e o t ix r a s a c m t ib t f o e re e e d o c p o n x o e a r o o i r v u o a a f n r i c r i is a j u n t e g o t r _ u p r p h o t is o w j c e t n h _ e a r r l l lo t lo n o lb n a ie _ m o a t s n

-7.00 _ d d t u u m s s i o p b g s u r p a a _ b l _ e s o t c e e e e n s e re n p s o _ w le n o e e t o e t l _ _ _ m p i _ _ c _ o d t _ _ _ d r z e r e p a _ h p le _ c a m r a _ a _ l m 2 2 p _ a i t h r 2 t s o 1 e n o t p 1 n _ _ _ _ 2 a p x c t _ _ o p e 1 p o _ h e s _ n s s e o a 1 2 w m m 1 2 r a m e s y m n s p m s a 2 m l

-8.00 o u _ p v v w _ c m v m v

r s _ t e r

o lo v

p o p im d m e m c m x v k m m s i o o m s e p d e _ n o v _ _ p u o i r m o a r _ o c n n s a t o v r p v _ t w o v c h v c l o p p

p o t

p a d v m _ o p u d s s y s _ _ e c _ e p o p l w n l _ s n _ e i _ t a m p o a p _ a _ r 2 _ a o o h t 1 m

lo _

np r 1 _ o o _ a 2 2 a n _ o o e w _ 1 m e s s

p l a u s m 2 p m s v p _ o r p _ o s v m m v p m m

a v _

c h p o v v p o _ v t a l o 2

s _ m 3 v m v

31 JRuby Compiler • Ahead-of-time > Compile .rb file to .class file > Directly executable, or require/loadable > Package compiled Ruby like compiled Java • Just-in-time > Heavily hit methods compiled at runtime > Run existing apps, scripts without modification > Optimize methods generated at runtime • Incremental design > Fall back on interpreter for unimplemented bits

32 Demo: JRuby Compiler

33 Performance: JRuby Compiler RRuunn Timmee 202.000.0000

181.080.0000 Ruby 1.8.4 Ruby 1.8.4 JRuby -C JRuby -C 161.060.0000 JRuby -C no sJRtaurtbupy -C Svr 141.040.0000

121.020.0000

101.000.0000

8.080.0000 6.000 6.000 4.000 4.000 2.000 2.000

0.000 i p 2 p b k a o p o th y d f1 fi a r o o g ra o i 0.000 _ t a lo l n r th n p _ t le lo _ a e u p p _ i le d le _ _ a ib p k p ai h op i p2 te op _ th 2 ay m od 2 _f a _ta p tar w lo h o s lo 1 ng rr _ th m pp pp a _ _ ile w lelo e d_ m le m _a 2 e v a a pp p wh _ hi n te v 1_ v 2 m _m a o _ p w _ es m vm v 2 oop op_ o_n v vm lol lolo sos

34 Performance: JRuby Compiler TiTmimeess FFaasstteerr 4.540.00

4.030.50 -C X fas-Cte rX faster -C no startup X faster 3.530.00 -C Svr X faster 3.00 2.50 2.50 2.00 2.00 1.50 1.50 1.010.00 0.500.50 0.00.00 -0.-050.50 -1.00 -1.00 -1.50 i p 2 p h y d ib k a o p o t o f1 f a r o o o g ra i -1.50 _ t a l o l n r th n p _ t e l _ a e u p _ il e d le _ _ p p p i h il e _ m 2 a fib a ak p ra op h p2 t o1p th 2 ay _ od _ _t a ta w lo o s lo ng m rr thm pp p _ _ ile w elo e d_ m le v _a 2 e v a ap pp p h _ hil n te v 1_ 2 m _m a o _w p w _ es m vm v 2 lo op o p_ o _n v vm lo lo loo s so

35 What Next for JRuby? • Viable for development today > Not perfect, but what is? • Rapidly approaching MRI on many fronts • Many capabilities beyond MRI today • More and more apps running > Does yours? > Shouldn't you ensure that it does? • Bright future for Ruby on the Java platform

36 JRuby Roadmap • Early March, 0.9.8 release > Official “Rails Support” > First release of experimental compiler > Hundreds of bug fixes, some complete lib rewrites • April timeframe, 0.9.9 > JIT compiler enabled, AOT handles “most” scripts > Continuing app compatibility work > Expanding Rails with WAR, enterprise API support • May timeframe, 1.0RC? > Wrapping up, smoothing the edges

37 More Information • JRuby: www.jruby.org • JRuby Wiki: headius.com/jrubywiki • RubySpec Wiki: headius.com/rubyspec • JRBuilder: www2.webng.com/bdortch/jrbuilder • Cheri: www2.webng.com/bdortch/cheri • NetBeans Ruby: wiki.netbeans.org/wiki/view/Ruby • Charlie's Blog: headius.blogspot.com • Tom's Blog: bloglines.com/blog/ThomasEEnebo

38 Become Super Powerful with JRuby

Q/A Thank You! [email protected]

39