Ruby’s C Extension Problem and How We’re Solving It RubyConf 2016
Chris Seaton Research Manager Oracle Labs November 2016
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to provide some insight into a line of research in Oracle Labs. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Oracle reserves the right to alter its development plans and practices at any time, and the development, release, and timing of any features or functionality described in connection with any Oracle product or service remains at the sole discretion of Oracle. Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 3 Lots of people want to make Ruby faster
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 4 JRuby logo copyright (c) Tony Price 2011, licensed under the terms of the Creative Commons Attribution-NoDerivs 3.0 Unported (CC BY-ND 3.0) Ruby logo copyright (c) 2006, Yukihiro Matsumoto, licensed under the terms of the Creative Commons Attribution-ShareAlike 2.5 agreement Rubinius logo licensed under the terms of the Creative Commons Attribution-NoDerivs 3.0 Unported Appfolio logo © AppFolio, Inc. 2016 Maglev logo Copyright © 2008-2010 GemStone Systems OMR logo copyright Eclipse Foundation
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 5 C extensions – the original solution for performance
ruby.rb
Ruby Interpreter
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 6 C extensions – the original solution for performance
extension.c ruby.rb
C Compiler
Ruby Interpreter
extension.so
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 7 C extensions – the original solution for performance
extension.c ruby.rb
C Compiler
Ruby Interpreter Extension
extension.so
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 8 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 9 Why C extensions hold us back
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 10 C Extension
C Extension API
Ruby Interpreter
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 11 C Extension
C Extension API
JRuby MRI Rubinius
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 12 Bad news – this isn’t really a thing in practice
C Extension
C Extension API
JRuby MRI Rubinius
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 13 C Extension
MRI
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 14 C Extension
? ? ? ? ? ? JRuby
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 15 C Extension
? ?
MRI 3.0
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 16 String pointers
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 17 Array pointers
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 18 Data fields
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 19 Lack of caching when you are in C
Last time we called to_s this is the method we used
?
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 20 The black box
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 21 The black box
= 16
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 22 The black box
= ?
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 23 Previous solutions to the C extension problem
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 24 Denial • Everyone should use the FFI or Fiddle – FFI and Fiddle are two ways to call C functions directly from Ruby – 2.1 billion lines of code in RubyGems, 0.5 billion of it is C extension code – It might be nice if people used FFI instead of C extensions… but they don’t… so little point in continuing to argue about it
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 25 Bargaining • Attempt to implement the C extension API as best as possible, alongside optimisations • Generally involves a lot of copying • JRuby used this approach in the past, Rubinius still uses it – JRuby only ran 60% of C extensions I tried – Rubinius ran 90% – Worse: when they didn’t work they just ground to a halt, no clear failure point
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 26 Bargaining • Try to improve the C extension API over time – The JavaScript (V8) and Java C extension APIs don’t have these problems because they have better designed APIs that don’t expose internals
– Steady progress in this direction, has helped
– But even OpenSSL doesn’t use these new methods! “ “
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 27 Depression • JRuby unfortunately had to give up on their C extension work – They didn’t have the resources to maintain it after the original developer moved on – Limited compatibility and limited performance – In the end, in was removed entirely – Maybe it’ll return in the future (they could use the same approach as us)
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 28 Acceptance • JRuby encourage Java extensions instead of C extensions • Try to optimise Ruby while keeping most of the internals the same – IBM’s OMR adds a new GC and JIT to Ruby while keeping support for C extensions – The techniques they can use are therefore limited – And so performance increases expected from OMR are more modest
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 29 Interlude: JRuby+Truffle
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 30 JVM
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 31 JVM
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 32 JVM
JIT
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 33 JVM
JIT
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 34 JVM Graal
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 35 Truffle
JVM Graal
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 36 Truffle
Graal VM
JVM
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 37 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 38 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 39 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 40 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 41 Our radical new solution for C extensions…
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 42 ruby.rb extension.c
C Compiler
Ruby Interpreter extension
extension.so
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 43 ruby.rb extension.c
Ruby Interpreter C Interpreter
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 44 extension.c
LLVM C Compiler
ruby.rb extension.ll
Ruby Interpreter LLVM Interpreter
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 45 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 46 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 47 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 48 Optimise Ruby and C together
ruby.rb extension.c
Optimisations
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 49 Optimise Ruby and C together
Optimisations
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 50 Interesting problems and their solutions
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 51 Defining the C extension API in Ruby
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 52 Imaginary strings
H e l l o , R u b y C o n f !
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 53 Imaginary strings
A Tale of Two String Representations Kevin Menard - RubyKaigi 2016
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 54 Imaginary strings
H e l l o , R u b y C o n f !
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 55 Imaginary strings
H e l l o , R u b y C o n f !
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 56 Results
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 57 35.0
30.0 MRI (s/s) 25.0
20.0
15.0 Running Pure Ruby
Speedup Relative to 10.0
5.0
0.0 MRI With C Extension Rubinius With C JRuby With C JRuby+Truffle With C JRuby+Truffle With C Extension Extension Extension Extension (No Inline)
Matthias Grimmer, Chris Seaton, Thomas Wuerthinger, Hanspeter Moessenboeck: Dynamically Composing Languages in a Modular Way: Supporting C Extensions for Dynamic Languages Modularity '14 Proceedings of the 14th International Conference on Modularity
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 58 Some limitations
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 59 You do need the source code of the C extension • Means no closed source C extensions – Is this a problem in reality for anyone? – I’m not aware of any closed source C extensions – C extensions in turn using closed source libraries like database drivers is fine
extension.c libpropietarydatabase.so ✓
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 60 You can’t store pointers to Ruby objects in native code • If your C extension uses a compiled library, such as libssl.so – You can’t give that compiled library a reference to a Ruby object – The Ruby object may not really exist – The GC may want to move the object
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 61 By the way… • It is probably still best to use the FFI if you are writing new extensions – Wide support across Ruby implementations – Although we don’t actually implement the FFI in JRuby+Truffle yet – Implementing the FFI in JRuby+Truffle would be a great internship project! • If you do write a C extension for performance – Write a pure Ruby baseline version as well • Or if you just needed better performance: – Write pure Ruby code – Run with JRuby+Truffle
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 62 Java extensions
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 63 Java Extension
JRuby
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 64 Java Extension
? ? ? ? ? ? JRuby+Truffle
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 65 Extension.java
Java Compiler
ruby.rb Extension.class
Ruby Interpreter Java Bytecode Interpreter
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 66 This could be a direction for MRI as well
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 67 Evan Phoenix: store the LLVM IR of the MRI implementation code and JIT it Ruby: 2020 - RubyKaigi 2015 Keynote
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 68 A quick status update on JRuby+Truffle
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 69 Classic research benchmarks – 10-20x faster than MRI
45 40 MRI 35 30 25 20 15 10 5 0 Speedup Compared to
Truffle JRuby+invokedynamic MRI
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 70 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 71 ‘optcarrot’ NES emulator benchmark – 9x faster than MRI
10 9 8
MRI 7 6 5 4 3
Speedup Relative to 2 1 0 Truffle JRuby+invokedynamic MRI
Image copyright NY - http://nyphotographic.com/ - Creative Commons 3 - CC BY-SA 3.0
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 72 Ruby specs 99% Language specs 96% Core library specs Standard library specs* 78% coverage is very limited here; probably a bit misleading
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 73 Rails tests 100% Active Support Active Model Active Record 100% Action View Action Pack Basic functionality works Action Mailer 98% Railties Sprockets-Rails Active Job 37% Spring
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 74 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 75 So then why can’t we run real applications yet? • C extensions are still a work in progress – Almost no database drivers – No openssl – No nokogiri – Prevents us running almost everything unfortunately • The specs don’t have perfect coverage • Our sophisticated optimisations mean the program state space is huge – Lots more to test – Lots more to tune for performance
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 76 Search for ’graal otn’
www.oracle.com/technetwork/oracle-labs/program-languages
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 77 Search for ‘github graalvm’
github.com/graalvm
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 78 chrisseaton.com/rubytruffle Freenode #jruby gitter.im/jruby/jruby @ChrisGSeaton
‘jruby truffle’
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 79 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 80 Acknowledgements
Oracle Oracle (continued) Oracle Interns JKU Linz University of California, Irvine Danilo Ansaloni Roland Schatz Brian Belleville Prof. Hanspeter Mössenböck Prof. Michael Franz Stefan Anzinger Chris Seaton Miguel Garcia Benoit Daloze Gulfem Savrun Yeniceri Cosmin Basca Doug Simon Shams Imam Josef Eisl Wei Zhang Daniele Bonetta Štěpán Šindelář Alexey Karyakin Thomas Feichtinger Matthias Brantner Zbyněk Šlajchrt Stephen Kell Matthias Grimmer Purdue University Petr Chalupa Lukas Stadler Andreas Kunft Christian Häubl Prof. Jan Vitek Jürgen Christ Codrut Stancu Volker Lanting Josef Haider Tomas Kalibera Laurent Daynès Jan Štola Gero Leinemann Christian Huber Petr Maj Gilles Duboscq Jaroslav Tulach Julian Lettner Stefan Marr Lei Zhao Martin Entlicher Michael Van De Vanter Joe Nash Manuel Rigger Brandon Fish Adam Welc David Piorkowski Stefan Rumzucker T. U. Dortmund Bastian Hossbach Christian Wimmer Gregor Richards Bernhard Urban Prof. Peter Marwedel Christian Humer Christian Wirth Robert Seilbeck Helena Kotthaus Mick Jordan Paul Wögerer Rifat Shariyar University of Edinburgh Ingo Korb Vojin Jovanovic Mario Wolczko Christophe Dubach Peter Kessler Andreas Wöß Alumni Juan José Fumero Alfonso University of California, Davis David Leopoldseder Thomas Würthinger Erik Eckstein Ranjeet Singh Prof. Duncan Temple Lang Kevin Menard Michael Haupt Toomas Remmelg Nicholas Ulle Jakub Podlešák Christos Kotselidis Aleksandar Prokopec Hyunjin Lee LaBRI University of Lugano, Switzerland Tom Rodriguez David Leibs Floréal Morandat Prof. Walter Binder Chris Thalinger Sun Haiyang Till Westmann Yudi Zheng
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 81 Safe Harbor Statement The preceding is intended to provide some insight into a line of research in Oracle Labs. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. Oracle reserves the right to alter its development plans and practices at any time, and the development, release, and timing of any features or functionality described in connection with any Oracle product or service remains at the sole discretion of Oracle. Any views expressed in this presentation are my own and do not necessarily reflect the views of Oracle.
Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 82 Copyright © 2016, Oracle and/or its affiliates. All rights reserved. | 83