Jitter for version 0.0.0.1200-47b1-dirty, updated in August 2017 Luca Saiu This is the manual for Jitter (version 0.0.0.1200-47b1-dirty, last updated on 30 August 2017), an efficient virtual machine generator. Copyright c 2017 Luca Saiu. Written by Luca Saiu. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Front-Cover texts and with the Back-Cover text being \You have freedom to copy and modify this manual, like GNU software.". A copy of the GNU Free Documentation License is available along with the software source code as doc/COPYING.DOC, and also on the web at https:// www.gnu.org/licenses/fdl.html. i Table of Contents Introduction ::::::::::::::::::::::::::::::::::::::::: 1 Audience :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 History :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 1 About language virtual machines::::::::::::::::::::::::::::::::::::: 3 When not to use VMs :::::::::::::::::::::::::::::::::::::::::::: 3 When to use VMs :::::::::::::::::::::::::::::::::::::::::::::::: 4 Software and hardware requirements ::::::::::::::::::::::::::::::::: 6 Design goals ::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 6 Performance :::::::::::::::::::::::::::::::::::::::::::::::::::::: 6 Generality :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 6 Minimizing and factoring machine dependencies ::::::::::::::::::: 6 Expressivity and convenience ::::::::::::::::::::::::::::::::::::: 7 Non-goals :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 7 Comparison with other systems :::::::::::::::::::::::::::::::::::::: 8 Comparison with ahead-of-time compilers ::::::::::::::::::::::::: 8 Comparison with JITs :::::::::::::::::::::::::::::::::::::::::::: 8 C is only an implementation tool :::::::::::::::::::::::::::::: 8 Comparison with Vmgen and GForth ::::::::::::::::::::::::::::: 8 License :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 9 Contacting the author ::::::::::::::::::::::::::::::::::::::::::::::: 9 1 Working with the Jitter sources :::::::::::::: 11 1.1 Building conventions and tips :::::::::::::::::::::::::::::::::: 11 1.1.1 Cross-compiling [probably not: better to merge this into the previous sections] ::::::::::::::::::::::::::::::::::::::::::: 11 1.2 Configuration ::::::::::::::::::::::::::::::::::::::::::::::::: 11 1.3 Generating the documentation ::::::::::::::::::::::::::::::::: 12 1.4 Building::::::::::::::::::::::::::::::::::::::::::::::::::::::: 12 1.5 Running the test suite ::::::::::::::::::::::::::::::::::::::::: 12 1.5.1 A test suite is not a correctness proof ::::::::::::::::::::: 14 1.6 Installing :::::::::::::::::::::::::::::::::::::::::::::::::::::: 14 2 Tutorial ::::::::::::::::::::::::::::::::::::::::: 15 2.1 Using a VM ::::::::::::::::::::::::::::::::::::::::::::::::::: 15 2.1.1 Predefined frontend::::::::::::::::::::::::::::::::::::::: 15 2.1.2 Disassembly :::::::::::::::::::::::::::::::::::::::::::::: 15 2.1.3 C API :::::::::::::::::::::::::::::::::::::::::::::::::::: 15 2.1.4 Building a Jittery program:::::::::::::::::::::::::::::::: 15 2.1.4.1 jitterc-config ::::::::::::::::::::::::::::::::::::: 15 2.1.4.2 Make :::::::::::::::::::::::::::::::::::::::::::::::: 15 2.1.4.3 Autoconf and Automake ::::::::::::::::::::::::::::: 15 2.2 A VM definition ::::::::::::::::::::::::::::::::::::::::::::::: 15 ii 3 Invoking :::::::::::::::::::::::::::::::::::::::: 16 4 Writing VM specifications::::::::::::::::::::: 17 4.1 Syntax :::::::::::::::::::::::::::::::::::::::::::::::::::::::: 17 4.1.1 Header ::::::::::::::::::::::::::::::::::::::::::::::::::: 17 4.1.2 Embedding C code ::::::::::::::::::::::::::::::::::::::: 17 4.1.3 Meta-instructions ::::::::::::::::::::::::::::::::::::::::: 17 4.1.3.1 Meta-instruction headers::::::::::::::::::::::::::::: 17 4.1.3.2 Meta-instruction properties :::::::::::::::::::::::::: 17 4.1.3.3 Embedding C code in meta-instructions :::::::::::::: 17 4.2 Instruction rewriting :::::::::::::::::::::::::::::::::::::::::: 17 4.3 Emacs major mode for Jitter :::::::::::::::::::::::::::::::::: 17 4.4 Performance tips :::::::::::::::::::::::::::::::::::::::::::::: 17 4.4.1 Performance tips for VM programs:::::::::::::::::::::::: 18 5 Standalone VMs:::::::::::::::::::::::::::::::: 19 6 C API ::::::::::::::::::::::::::::::::::::::::::: 20 6.1 Initialization :::::::::::::::::::::::::::::::::::::::::::::::::: 20 6.2 Finalization ::::::::::::::::::::::::::::::::::::::::::::::::::: 20 6.3 VM state:::::::::::::::::::::::::::::::::::::::::::::::::::::: 20 6.4 VM programs ::::::::::::::::::::::::::::::::::::::::::::::::: 20 6.4.1 Program structure :::::::::::::::::::::::::::::::::::::::: 20 6.4.2 Adding labels::::::::::::::::::::::::::::::::::::::::::::: 20 6.4.3 Adding instructions::::::::::::::::::::::::::::::::::::::: 20 6.4.4 Specialization::::::::::::::::::::::::::::::::::::::::::::: 20 6.4.5 Multiple programs :::::::::::::::::::::::::::::::::::::::: 20 6.5 Running VM programs :::::::::::::::::::::::::::::::::::::::: 20 6.6 Concurrency::::::::::::::::::::::::::::::::::::::::::::::::::: 20 7 Compiling VMs :::::::::::::::::::::::::::::::: 21 7.1 Two-stage build ::::::::::::::::::::::::::::::::::::::::::::::: 21 7.2 Cross-compiling VMs :::::::::::::::::::::::::::::::::::::::::: 21 8 Internals :::::::::::::::::::::::::::::::::::::::: 22 8.1 Specialization internals :::::::::::::::::::::::::::::::::::::::: 22 8.2 Patch-ins :::::::::::::::::::::::::::::::::::::::::::::::::::::: 22 8.3 Rewriting internals :::::::::::::::::::::::::::::::::::::::::::: 22 8.4 Dispatch models ::::::::::::::::::::::::::::::::::::::::::::::: 22 8.4.1 switch dispatching ::::::::::::::::::::::::::::::::::::::: 22 8.4.2 Direct threading :::::::::::::::::::::::::::::::::::::::::: 22 8.4.3 Minimal threading :::::::::::::::::::::::::::::::::::::::: 22 8.4.4 No threading ::::::::::::::::::::::::::::::::::::::::::::: 22 8.4.4.1 Fast labels ::::::::::::::::::::::::::::::::::::::::::: 22 8.5 Architecture-specific notes ::::::::::::::::::::::::::::::::::::: 22 8.5.1 Aarch64 :::::::::::::::::::::::::::::::::::::::::::::::::: 24 iii 8.5.2 Alpha :::::::::::::::::::::::::::::::::::::::::::::::::::: 24 8.5.3 ARM ::::::::::::::::::::::::::::::::::::::::::::::::::::: 24 8.5.4 AVR ::::::::::::::::::::::::::::::::::::::::::::::::::::: 24 8.5.5 i386 :::::::::::::::::::::::::::::::::::::::::::::::::::::: 24 8.5.6 m68k ::::::::::::::::::::::::::::::::::::::::::::::::::::: 24 8.5.7 MIPS :::::::::::::::::::::::::::::::::::::::::::::::::::: 25 8.5.8 OpenRISC ::::::::::::::::::::::::::::::::::::::::::::::: 25 8.5.9 PowerPC ::::::::::::::::::::::::::::::::::::::::::::::::: 26 8.5.10 RISC-V ::::::::::::::::::::::::::::::::::::::::::::::::: 26 8.5.11 s390 ::::::::::::::::::::::::::::::::::::::::::::::::::::: 26 8.5.12 SH :::::::::::::::::::::::::::::::::::::::::::::::::::::: 26 8.5.13 SPARC ::::::::::::::::::::::::::::::::::::::::::::::::: 26 8.5.14 x86 64 :::::::::::::::::::::::::::::::::::::::::::::::::: 27 8.6 Porting:::::::::::::::::::::::::::::::::::::::::::::::::::::::: 27 8.6.1 Required files ::::::::::::::::::::::::::::::::::::::::::::: 27 9 This is free software ::::::::::::::::::::::::::: 28 Bibliography:::::::::::::::::::::::::::::::::::::::: 29 Index:::::::::::::::::::::::::::::::::::::::::::::::: 30 Appendix A Stuff still to be written :::::::::::: 35 A.1 (Meta-)language :::::::::::::::::::::::::::::::::::::::::::::: 36 A.2 Frontend performance::::::::::::::::::::::::::::::::::::::::: 37 A.3 Markup samples for the reference part::::::::::::::::::::::::: 37 A.4 GCC options ::::::::::::::::::::::::::::::::::::::::::::::::: 38 A.5 Among design goals ::::::::::::::::::::::::::::::::::::::::::: 38 A.6 About tracing JITs ::::::::::::::::::::::::::::::::::::::::::: 38 A.7 VM instruction coarseness, or granularity ::::::::::::::::::::: 39 A.8 What a \basic block" is ::::::::::::::::::::::::::::::::::::::: 39 Appendix B Stuff still to be implemented:::::: 41 B.1 Emacs mode :::::::::::::::::::::::::::::::::::::::::::::::::: 41 B.2 mmap :::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 41 B.3 Word size on the target machine :::::::::::::::::::::::::::::: 41 B.4 Non-relocatable instructions::::::::::::::::::::::::::::::::::: 41 B.5 Machine register use: single base [almost done now]:::::::::::: 42 B.5.1 Base pointer not at offset zero :::::::::::::::::::::::::::: 42 B.6 Alias argument class :::::::::::::::::::::::::::::::::::::::::: 42 B.6.1 Scratch register not reserved:::::::::::::::::::::::::::::: 42 B.7 Frontend optimization::::::::::::::::::::::::::::::::::::::::: 43 B.8 Frontend cleanup ::::::::::::::::::::::::::::::::::::::::::::: 43 B.9 Memory directives :::::::::::::::::::::::::::::::::::::::::::: 43 B.10 Stack slot access ::::::::::::::::::::::::::::::::::::::::::::: 43 B.11 Convenience register spilling ::::::::::::::::::::::::::::::::: 43 iv B.12 A ridiculous i386/x86 64 optimization :::::::::::::::::::::::: 43 B.13 No-threading: generate one C compilation unit per VM instruction ::::::::::::::::::::::::::::::::::::::::::::: 44 1 Introduction Jitter is a software automatically building a portable, very efficient language virtual machine with close to native-code performance, starting from a relatively high-level specification provided by the user. The VM state may use registers, stacks, a combination of both or any runtime structure defined by user code; whenever possible the generator attempts to map