LLVMLinux: Embracing the Dragon

Presented by: Behan Webster (LLVMLinux project lead)

LLVMLinux project Presentation Date: 2014.08.22 Clang/LLVM

● LLVM is a Toolchain Toolkit (libraries from which and related technologies can be built) ● Clang is a C/C++ toolchain

LLVMLinux project New Clang Benchmarks (compilation)

Timed ImageMagick Compilation v6.8.1-10 Time To Compile

Seconds, Less Is Better OpenBenchmarking.org

GCC 4.8.2 60.35 SE +/- 0.36

GCC 4.9.0 RC1 60.59 SE +/- 0.04

LLVM Clang 3.5 20140413 29.61 SE +/- 0.14

14 28 42 56 70

Powered By Phoronix Test Suite 5.0.1 http://www.phoronix.com/scan.php?page=article&item=gcc49_compiler_llvm35&num=1 LLVMLinux project New Clang Benchmarks (execution)

ebizzy v0.3 Records/s

Seconds, More Is Better OpenBenchmarking.org

GCC 4.8.2 42849 SE +/- 109.85

GCC 4.9.0 RC1 42950 SE +/- 44.08

LLVM Clang 3.5 20140413 43202 SE +/- 53.03

9000 18000 27000 36000 45000

Powered By Phoronix Test Suite 5.0.1 1. (CC) gcc options: -pthread -lpthread -O3 -march=native http://www.phoronix.com/scan.php?page=article&item=gcc49_compiler_llvm35&num=1 LLVMLinux project Other Interesting LLVM Related Projects

● Clang Static Analyzer ● Energy consumption analysis of programs using LLVM ● CUDA

● llvmpipe (Galium3D)

● OpenCL (most implementations are based on LLVM)

● Clang is one of the Android NDK compilers

● Renderscript in Android is based on LLVM

● Code transformation tools

LLVMLinux project The LLVMLinux Project Goals

● Fully build the kernel for multiple architectures, using the Clang/LLVM toolchain ● Discover LLVM/Kernel issues early and find fixes quickly across both communities ● Upstream patches to the and LLVM projects ● Bring together like-minded developers ● Enable the kernel community to do more in depth analysis of the kernel code LLVMLinux project LLVMLinux Build/Test System

● Fetches, patches, builds, tests: clang, kernel, , etc – git clone http://git.linuxfoundation.org/llvmlinux.git – cd llvmlinux/target/vexpress (or x86_64) – make

LLVMLinux project Patched Mainline Kernel Tree

● A mainline kernel tree with all LLVMLinux patches applied on top is now available: – git://git.linuxfoundation.org/llvmlinux/kernel.git ● Dated llvmlinux branches – remotes/origin/llvmlinux-2014.07.30 ● The master branch is rebased regularly

LLVMLinux project Supported Architectures

● X86 (i586 and x86_64)

● ARM

● AARCH64

● And now MIPS (Work in progress)

LLVMLinux project Linux-Next

● LLVMLinux project now has a branch which is pulled into Linux-Next – git://git.linuxfoundation.org/llvmlinux/kernel.git – remotes/origin/for-next ● Broader testing of our patches by more people ● Last step before being submitted to mainline

LLVMLinux project Improved Buildbot Farm

● New better buildbot master ● 4 slave build servers (2 more on the way) ● More CI builds for targets are in the works ● Configs for known good versions and kernels

LLVMLinux project Other Avenues of Interest

● Clang Static Analysis of the Linux kernel ● Kernel specific Checkers (GSoC) ● Compiling the Android kernel and AOSP with clang ● Supporting LLVM team ● Adding clang support to yocto ● Building better tools based on LLVM for the kernel community

LLVMLinux project LLVMLinux Project Status

● LLVM/clang: – All LLVMLinux patches for LLVM are Upstream – Newer LLVM patches to support the Linux kernel are mostly being added by upstream maintainers ● Linux Kernel: – Roughly 47 kernel patches for various arches – LLVMLinux branch in linux-next

LLVMLinux project Kernel Patches

● Patches still working their way upstream

Architecture Number of patches Submitted all 13 1 arm 10 8 aarch64 16 4 x86_64 8 0 TOTALS 47 13

LLVMLinux project Kbuild

● Basic Kbuild support for clang and x86 has been upstreamed ● Specific support for ARM and aarch64 ready to be upstreamed

LLVMLinux project Integrated Assembly Status

● David Woodhouse added .code16 support for X86 ASM ● Renato Golin, Vicinius Tinti, Saleem Abdulrasool and Stepan Dyatkovskiy are working on fixing IA issues in clang to support the Linux ARM kernel code (and ultimately AARCH64) ● For now we disable the IA and use GNU as instead

LLVMLinux project Different option passing

● gcc passes -march to GNU as ● clang doesn't... (Bug submitted PR)

-CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto +CFLAGS_aes-ce-cipher.o += -march=armv8-a+crypto -Wa,-march=armv8-a+crypto

LLVMLinux project extern inline: Different for gnu89 and gnu99

● GNU89/GNU90 (used by gcc) – Function will be inlined where it is used – No function definition is emitted – A non-inlined function may also be provided ● GNU99/C99 (used by clang) – Function will be inlined where it is used – An external function is emitted – No other function of the same name may be provided. ● Solution? Use “static inline” instead.

● Only still an issue for ARM support for (submitted) LLVMLinux project Attribute Order

● gcc is less picky about placement of __attribute__(()) ● clang requires it at the end of the type or variable

-struct __read_mostly va_alignment va_align = { +struct va_alignment __read_mostly va_align = {

LLVMLinux project Named Registers

● Named registers for X86 kernel have been fixed in mainline kernel by Andi Kleen ● clang now supports using a globally named register for the stack pointer

● For ARM/AARCH64 move to using a global in asm/thread_info.h register unsigned long current_stack_pointer asm ("sp");

LLVMLinux project ARM percpu patch

● One of the uses of Named Registers in the ARM code is due to a deficiency in gcc. ● The new code which works with gcc fails in clang.

● Solution, provide routines for both, and choose at compile time

● Gcc: asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : "Q" (*sp));

● Clang: asm("mrc p15, 0, %0, c13, c0, 4" : "=r" (off) : : "memory");

LLVMLinux project Section Mismatch Issues (MergedGlobals)

● By default clang merges globals with internal linkage into one: MergedGlobals

● Allows globals to be addressed using offsets from a base pointer

● Can reduce the number of registers used

● Modpost uses symbol names to look for section mismatches

● MergedGlobals breaks modpost (false positive section mismatches)

● Current solution: use -mno-global-merge to stop global merging

● Updates to modpost may allow this optimization to be enabled again

LLVMLinux project Deprecated clang options

● clang options have been deprecated

-KBUILD_CFLAGS += $(call cc-option, -fcatch-undefined-behavior) +KBUILD_CFLAGS += $(call cc-option, -fsanitize=undefined-trap) +KBUILD_CFLAGS += $(call cc-option, -fsanitize-undefined-trap-on-error)

LLVMLinux project ARM eabi support

● Clang emits code which uses the “aeabi” ARM calls which are implemented in compiler-rt (equivalient to libgcc) ● Compiler-rt doesn't easily cross compile yet... void __aeabi_memcpy(void *dest, const void *src, size_t n) void __aeabi_memmove(void *dest, const void *src, size_t n) void __aeabi_memset(void *s, size_t n, int c)

LLVMLinux project Variable Length Arrays In Structs

● VLAIS isn't supported by Clang (undocumented gcc extension) char vla[n]; /* Supported, C99/C11 */ struct { char flexible_member[]; /* Supported, C99/C11 */ } struct_with_flexible_member; struct { char vlais[n]; /* Explicitly not allowed by C99/C11 */ } variable_length_array_in_struct;

● VLAIS is used in the Linux kernel in a number of places, spreading mostly through reusing patterns from data structures

found in crypto LLVMLinux project VLAIS Removal Example

- struct { - struct shash_desc shash; - char ctx[crypto_shash_descsize(hash)]; - } desc; + char desc[sizeof(struct shash_desc) + + crypto_shash_descsize(hash)] CRYPTO_MINALIGN_ATTR; + struct shash_desc *shash = (struct shash_desc *)desc; unsigned int i;

- desc.shash.tfm = hash;

+ shash->tfm = hash; (from crypto/hmac.c) LLVMLinux project VLAIS Removal Example (cont.) (the missing pieces)

#define ARCH_KMALLOC_MINALIGN __alignof__(unsigned long long) #define CRYPTO_MINALIGN ARCH_KMALLOC_MINALIGN #define CRYPTO_MINALIGN_ATTR __attribute__ ((__aligned__(CRYPTO_MINALIGN))) struct shash_desc { struct crypto_shash *tfm; u32 flags; void *__ctx[] CRYPTO_MINALIGN_ATTR; };

LLVMLinux project Status of VLAIS in the Linux Kernel

● USB Gadget patch is in mainline ● Mac80211 patch is in mainline ● patch is in mainline ● Only the VLAIS patches for crypto are left: (, , dm-crypt, hmac, libcrc32c, testmgr)

LLVMLinux project How Can You Help?

● Make it known you want to be able to use Clang to compile the kernel ● Test LLVMLinux patches

● Report bugs to the mailing list

● Help get LLVMLinux patches upstream

● Work on unsupported features and Bugs

● Submit new targets and arch support

● Patches welcome

LLVMLinux project Embrace the Dragon. He's cuddly.

Thank you

http://llvm.linuxfoundation.org

LLVMLinux project Contribute to the LLVMLinux Project

● Project wiki page – http://llvm.linuxfoundation.org ● Project Mailing List – http://lists.linuxfoundation.org/mailman/listinfo/llvmlinux – http://lists.linuxfoundation.org/pipermail/llvmlinux/ ● IRC Channel – #llvmlinux on OFTC – http://buildbot.llvm.linuxfoundation.org/irclogs/OFTC/%23llvmlinux/

● LLVMLinux Community on Google Plus LLVMLinux project