LLVMLinux: Embracing the Dragon Presented by: Behan Webster (LLVMLinux project lead) LLVMLinux project Presentation Date: 2014.08.22 Clang/LLVM ● LLVM is a Toolchain Toolkit (libraries from which compilers and related technologies can be built) ● Clang is a C/C++ toolchain LLVMLinux project New Clang Benchmarks (compilation) Timed ImageMagick Compilation v6.8.1-10 Time To Compile Seconds, Less Is Better OpenBenchmarking.org GCC 4.8.2 60.35 SE +/- 0.36 GCC 4.9.0 RC1 60.59 SE +/- 0.04 LLVM Clang 3.5 20140413 29.61 SE +/- 0.14 14 28 42 56 70 Powered By Phoronix Test Suite 5.0.1 http://www.phoronix.com/scan.php?page=article&item=gcc49_compiler_llvm35&num=1 LLVMLinux project New Clang Benchmarks (execution) ebizzy v0.3 Records/s Seconds, More Is Better OpenBenchmarking.org GCC 4.8.2 42849 SE +/- 109.85 GCC 4.9.0 RC1 42950 SE +/- 44.08 LLVM Clang 3.5 20140413 43202 SE +/- 53.03 9000 18000 27000 36000 45000 Powered By Phoronix Test Suite 5.0.1 1. (CC) gcc options: -pthread -lpthread -O3 -march=native http://www.phoronix.com/scan.php?page=article&item=gcc49_compiler_llvm35&num=1 LLVMLinux project Other Interesting LLVM Related Projects ● Clang Static Analyzer ● Energy consumption analysis of programs using LLVM ● CUDA ● llvmpipe (Galium3D) ● OpenCL (most implementations are based on LLVM) ● Clang is one of the Android NDK compilers ● Renderscript in Android is based on LLVM ● Code transformation tools LLVMLinux project The LLVMLinux Project Goals ● Fully build the Linux kernel for multiple architectures, using the Clang/LLVM toolchain ● Discover LLVM/Kernel issues early and find fixes quickly across both communities ● Upstream patches to the Linux Kernel and LLVM projects ● Bring together like-minded developers ● Enable the kernel community to do more in depth analysis of the kernel code LLVMLinux project LLVMLinux Build/Test System ● Fetches, patches, builds, tests: clang, kernel, qemu, etc – git clone http://git.linuxfoundation.org/llvmlinux.git – cd llvmlinux/target/vexpress (or x86_64) – make LLVMLinux project Patched Mainline Kernel Tree ● A mainline kernel tree with all LLVMLinux patches applied on top is now available: – git://git.linuxfoundation.org/llvmlinux/kernel.git ● Dated llvmlinux branches – remotes/origin/llvmlinux-2014.07.30 ● The master branch is rebased regularly LLVMLinux project Supported Architectures ● X86 (i586 and x86_64) ● ARM ● AARCH64 ● And now MIPS (Work in progress) LLVMLinux project Linux-Next ● LLVMLinux project now has a branch which is pulled into Linux-Next – git://git.linuxfoundation.org/llvmlinux/kernel.git – remotes/origin/for-next ● Broader testing of our patches by more people ● Last step before being submitted to mainline LLVMLinux project Improved Buildbot Farm ● New better buildbot master ● 4 slave build servers (2 more on the way) ● More CI builds for targets are in the works ● Configs for known good compiler versions and kernels LLVMLinux project Other Avenues of Interest ● Clang Static Analysis of the Linux kernel ● Kernel specific Checkers (GSoC) ● Compiling the Android kernel and AOSP with clang ● Supporting Linaro LLVM team ● Adding clang support to yocto ● Building better tools based on LLVM for the kernel community LLVMLinux project LLVMLinux Project Status ● LLVM/clang: – All LLVMLinux patches for LLVM are Upstream – Newer LLVM patches to support the Linux kernel are mostly being added by upstream maintainers ● Linux Kernel: – Roughly 47 kernel patches for various arches – LLVMLinux branch in linux-next LLVMLinux project Kernel Patches ● Patches still working their way upstream Architecture Number of patches Submitted all 13 1 arm 10 8 aarch64 16 4 x86_64 8 0 TOTALS 47 13 LLVMLinux project Kbuild ● Basic Kbuild support for clang and x86 has been upstreamed ● Specific support for ARM and aarch64 ready to be upstreamed LLVMLinux project Integrated Assembly Status ● David Woodhouse added .code16 support for X86 ASM ● Renato Golin, Vicinius Tinti, Saleem Abdulrasool and Stepan Dyatkovskiy are working on fixing IA issues in clang to support the Linux ARM kernel code (and ultimately AARCH64) ● For now we disable the IA and use GNU as instead LLVMLinux project Different option passing ● gcc passes -march to GNU as ● clang doesn't... (Bug submitted PR) -CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto +CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto -Wa,-march=armv8-a+crypto LLVMLinux project extern inline: Different for gnu89 and gnu99 ● GNU89/GNU90 (used by gcc) – Function will be inlined where it is used – No function definition is emitted – A non-inlined function may also be provided ● GNU99/C99 (used by clang) – Function will be inlined where it is used – An external function is emitted – No other function of the same name may be provided. ● Solution? Use “static inline” instead. ● Only still an issue for ARM support for ftrace (submitted) LLVMLinux project Attribute Order ● gcc is less picky about placement of __attribute__(()) ● clang requires it at the end of the type or variable -struct __read_mostly va_alignment va_align = { +struct va_alignment __read_mostly va_align = { LLVMLinux project Named Registers ● Named registers for X86 kernel have been fixed in mainline kernel by Andi Kleen ● clang now supports using a globally named register for the stack pointer ● For ARM/AARCH64 move to using a global in asm/thread_info.h register unsigned long current_stack_pointer asm ("sp"); LLVMLinux project ARM percpu patch ● One of the uses of Named Registers in the ARM code is due to a deficiency in gcc. ● The new code which works with gcc fails in clang. ● Solution, provide routines for both, and choose at compile time ● Gcc: asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : "Q" (*sp)); ● Clang: asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : : "memory"); LLVMLinux project Section Mismatch Issues (MergedGlobals) ● By default clang merges globals with internal linkage into one: MergedGlobals ● Allows globals to be addressed using offsets from a base pointer ● Can reduce the number of registers used ● Modpost uses symbol names to look for section mismatches ● MergedGlobals breaks modpost (false positive section mismatches) ● Current solution: use -mno-global-merge to stop global merging ● Updates to modpost may allow this optimization to be enabled again LLVMLinux project Deprecated clang options ● clang options have been deprecated -KBUILD_CFLAGS += $(call cc-option, -fcatch-undefined-behavior) +KBUILD_CFLAGS += $(call cc-option, -fsanitize=undefined-trap) +KBUILD_CFLAGS += $(call cc-option, -fsanitize-undefined-trap-on-error) LLVMLinux project ARM eabi support ● Clang emits code which uses the “aeabi” ARM calls which are implemented in compiler-rt (equivalient to libgcc) ● Compiler-rt doesn't easily cross compile yet... void __aeabi_memcpy(void *dest, const void *src, size_t n) void __aeabi_memmove(void *dest, const void *src, size_t n) void __aeabi_memset(void *s, size_t n, int c) LLVMLinux project Variable Length Arrays In Structs ● VLAIS isn't supported by Clang (undocumented gcc extension) char vla[n]; /* Supported, C99/C11 */ struct { char flexible_member[]; /* Supported, C99/C11 */ } struct_with_flexible_member; struct { char vlais[n]; /* Explicitly not allowed by C99/C11 */ } variable_length_array_in_struct; ● VLAIS is used in the Linux kernel in a number of places, spreading mostly through reusing patterns from data structures found in crypto LLVMLinux project VLAIS Removal Example - struct { - struct shash_desc shash; - char ctx[crypto_shash_descsize(hash)]; - } desc; + char desc[sizeof(struct shash_desc) + + crypto_shash_descsize(hash)] CRYPTO_MINALIGN_ATTR; + struct shash_desc *shash = (struct shash_desc *)desc; unsigned int i; - desc.shash.tfm = hash; + shash->tfm = hash; (from crypto/hmac.c) LLVMLinux project VLAIS Removal Example (cont.) (the missing pieces) #define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) #define CRYPTO_MINALIGN ARCH_KMALLOC_MINALIGN #define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN))) struct shash_desc { struct crypto_shash *tfm; u32 flags; void *__ctx[] CRYPTO_MINALIGN_ATTR; }; LLVMLinux project Status of VLAIS in the Linux Kernel ● USB Gadget patch is in mainline ● Mac80211 patch is in mainline ● Netfilter patch is in mainline ● Only the VLAIS patches for crypto are left: (apparmor, btrfs, dm-crypt, hmac, libcrc32c, testmgr) LLVMLinux project How Can You Help? ● Make it known you want to be able to use Clang to compile the kernel ● Test LLVMLinux patches ● Report bugs to the mailing list ● Help get LLVMLinux patches upstream ● Work on unsupported features and Bugs ● Submit new targets and arch support ● Patches welcome LLVMLinux project Embrace the Dragon. He's cuddly. Thank you http://llvm.linuxfoundation.org LLVMLinux project Contribute to the LLVMLinux Project ● Project wiki page – http://llvm.linuxfoundation.org ● Project Mailing List – http://lists.linuxfoundation.org/mailman/listinfo/llvmlinux – http://lists.linuxfoundation.org/pipermail/llvmlinux/ ● IRC Channel – #llvmlinux on OFTC – http://buildbot.llvm.linuxfoundation.org/irclogs/OFTC/%23llvmlinux/ ● LLVMLinux Community on Google Plus LLVMLinux project.