Analyzing Code for KDE/Qt && Other Open Source Software
Total Page:16
File Type:pdf, Size:1020Kb
Analyzing code for KDE/Qt && other open source software http://leetcode.cn/2016/11/analyzing-code-for-kde-qt-open-source-components.html Leslie Zhai 普华——操作系统整合解决方案专家 Senior Software Engineer iSOFT Infrastructure Software co., Ltd; KDE developer KDE-China.org && Leetcode.cn 1 WHY 目录 HOWTO Contents 2 3 REFERENCE Part1 WHY? VULNERABILITIES OPTIMIZATION WHY -VULNERABILITIES 普华——操作系统整合解决方案专家 WHY - OPTIMIZATION 普华——操作系统整合解决方案专家 Identifying potential bottlenecks; Detecting problems in memory management; Detecting problems in C/C++ via static/dynamic analyzer source code analysis tool; Analyzing the CPU usage of embedded applications and Linux desktop applications; Part 2 HOWTO Static Analyzer Sanitizer libFuzzer CFI Safe Stack HOWTO – static analyer for k3b v2.10.0 普华——操作系统整合解决方案专家 HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 scan-build -k -v -V cmake .. -DCMAKE_INSTALL_PREFIX=/usr \ -DKDE_INSTALL_LIBDIR=lib \ -DKDE_INSTALL_LIBEXECDIR=lib \ -DKDE_INSTALL_USE_QT_SYS_PATHS=ON \ -DK3B_BUILD_API_DOCS=ON \ -DK3B_ENABLE_PERMISSION_HELPER=ON \ -DK3B_DEBUG=ON scan-build -k -v -V make HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 Q_UNUSED(dialogOpen); HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 http://clang-analyzer.llvm.org/annotations.html#custom_assertions HOWTO - static analyzer for k3b v2.10.0 普华——操作系统整合解决方案专家 LLVMBUG-28331 HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 echo ‘include(ECMEnableSanitizers)’ >> CMakeLists.txt cmake .. -DCMAKE_INSTALL_PREFIX=/usr \ -DCMAKE_CXX_COMPILER=clang++ \ -DECM_ENABLE_SANITIZERS='address;leak;undefined' \ -DECM_ENABLE_SANITIZERS='address;thread;leak;undefined'\ -DKDE_INSTALL_LIBDIR=lib \ -DKDE_INSTALL_LIBEXECDIR=lib \ -DKDE_INSTALL_USE_QT_SYS_PATHS=ON \ -DK3B_BUILD_API_DOCS=ON \ -DK3B_ENABLE_PERMISSION_HELPER=ON \ -DK3B_DEBUG=ON HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 in /data/project/kde/libkcddb/libkcddb/kcddbconfig.cpp:43:9: runtime error: member call on address 0x607000047280 which does not point to an object of type 'KConfigSkeletonGenericItem<QString>' 0x607000047280: note: object is of type 'KCoreConfigSkeleton::ItemString' 1c 00 00 60 78 40 69 ba be 7f 00 00 20 85 dc be be 7f 00 00 20 8d dc be be 7f 00 00 e0 8d dc be ^~~~~~~~~~~~~~~~~~~~~~~ vptr for 'KCoreConfigSkeleton::ItemString' SUMMARY: AddressSanitizer: undefined-behavior /data/project/kde/libkcddb/libkcddb/kcddbconfig.cpp:43:9 HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 /data/project/kde/k3b/libk3b/tools/libisofs/iso_fs.h:215:9: runtime error: load of misaligned address 0x61d0002fb32e for type 'unsigned int', which requires 4 byte alignment 0x61d0002fb32e: note: pointer points here 00 00 22 00 13 00 00 00 00 00 00 13 00 08 00 00 00 00 08 00 74 05 01 04 2f 07 00 02 00 00 01 00 ^ SUMMARY: AddressSanitizer: undefined-behavior /data/project/kde/k3b/libk3b/tools/libisofs/iso_fs.h:215: 9 HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 /data/project/kde/k3b/libk3b/core/k3bjob.cpp:118:39: runtime error: member call on address 0x7ffeb29f01 d0 which does not point to an object of type 'K3b::JobHandler' 0x7ffeb29f01a0: note: object is base class subobject at offset 48 within object of type 'K3b::BurnProgressDialog' 00 01 00 00 b0 44 95 01 00 00 00 00 80 6a 40 01 60 61 00 00 f8 46 95 01 00 00 00 00 00 00 00 00 ^ SUMMARY: AddressSanitizer: undefined-behavior /data/project/kde/k3b/libk3b/core/k3bjob.cpp:118:39 HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 ================================================================= ==14489==ERROR: LeakSanitizer: detected memory leaks Direct leak of 256 byte(s) in 1 object(s) allocated from: #0 0x5989c8 in __interceptor_malloc (/data/project/kde/k3b/build/src/k3b+0x5989c8) #1 0x7fdb06d38ef9 (/usr/lib/libfontconfig.so.1+0x1def9) HOWTO – sanitizer for k3b v2.10.0 普华——操作系统整合解决方案专家 Indirect leak of 4 byte(s) in 1 object(s) allocated from: #0 0x5d0780 in operator new[](unsigned long) (/data/project/kde/k3b/build/src/k3b+0x5d0780) #1 0x7fc8bd1a02a1 (/usr/lib/libQt5Core.so.5+0x2b02a1) #2 0x7fc8bd186ccf in QMetaMethod::parameterTypes() const (/usr/lib/libQt5Core.so.5+0x296ccf) SUMMARY: AddressSanitizer: 97381 byte(s) leaked in 921 allocation(s). https://github.com/google/sanitizers/wiki/AddressSanitizer http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer HOWTO – ThreadSanitizer 普华——操作系统整合解决方案专家 clang-3.9: error: invalid argument '-fsanitize=address' not allowed with '-fsanitize=thread' clang-3.9: error: invalid argument '-fsanitize=thread' not allowed with '-fsanitize=memory' clang-3.9: error: invalid argument '-fsanitize=leak' not allowed with '-fsanitize=thread' The sanitizers “address”, “memory” and “thread” are mutually exclusive, but why? LLVMBUG-31352 HOWTO – ThreadSanitizer 普华——操作系统整合解决方案专家 WARNING: ThreadSanitizer: data race (pid=17199) Read of size 4 at 0x0000006f4738 by thread T2: #0 MyThread2::run() /data/project/LeetcodeCN/leetcodecn.github.io/src/qtbase/test.cpp:41:15 (test+0x0000004bc0b6) #1 <null> <null> (libQt5Core.so.5+0x0000000acd77) Previous write of size 4 at 0x0000006f4738 by thread T1: #0 MyThread1::run() /data/project/LeetcodeCN/leetcodecn.github.io/src/qtbase/test.cpp:24:15 (test+0x0000004bb6c5) #1 <null> <null> (libQt5Core.so.5+0x0000000acd77) HOWTO – ThreadSanitizer 普华——操作系统整合解决方案专家 Location is global 'Global' of size 4 at 0x0000006f4738 (test+0x0000006f4738) Thread T2 'MyThread2' (tid=17202, running) created by main thread at: #0 pthread_create <null> (test+0x000000427ea6) #1 QThread::start(QThread::Priority) <null> (libQt5Core.so.5+0x0000000ac278) #2 __libc_start_main <null> (libc.so.6+0x000000020290) Thread T1 'MyThread1' (tid=17201, finished) created by main thread at: #0 pthread_create <null> (test+0x000000427ea6) #1 QThread::start(QThread::Priority) <null> (libQt5Core.so.5+0x0000000ac278) #2 __libc_start_main <null> (libc.so.6+0x000000020290) https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual HOWTO – libFuzzer 普华——操作系统整合解决方案专家 llvm/lib/Fuzzer/build.sh → libFuzzer.a #include <QString> #include <QDebug> extern "C" int LLVMFuzzerInitialize(int *argc, char ***argv) { qDebug() << "DEBUG:" << __PRETTY_FUNCTION__ << *argc; qDebug() << "DEBUG:" << __PRETTY_FUNCTION__ << *argv[0]; return 0; } extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size) { qDebug() << "DEBUG:" << __PRETTY_FUNCTION__ << QString::fromRawData((const QChar *)Data, Size); return 0; } HOWTO – libFuzzer 普华——操作系统整合解决方案专家 llvm/lib/Fuzzer/FuzzerInterface.h LLVMFuzzerCustomMutator Given the same Seed produces the same mutation LLVMFuzzerCustomCrossOver Combines pieces of Data1 & Data2 together into Out LLVMFuzzerMutate Experimental, may go away in future HOWTO – libFuzzer 普华——操作系统整合解决方案专家 cmake .. -DCMAKE_CXX_COMPILER=clang++ \ -DCMAKE_CXX_FLAGS="-Wall -fsanitize-coverage=edge,indirect-calls,8bit- counters -fsanitize=address -fsanitize=undefined -fno-omit-frame-pointer -fPIE -g" -fsanitize-coverage=func for function-level coverage -fsanitize-coverage=bb for basic-block-level coverage -fsanitize-coverage=edge for edge-level coverage -fsanitize-coverage=indirect-calls for additional caller-callee coverage -fsanitize-coverage=8bit-counters for coverage counters HOWTO – libFuzzer 普华——操作系统整合解决方案专家 ./test_fuzzer DEBUG: int LLVMFuzzerInitialize(int *, char ***) 1 DEBUG: int LLVMFuzzerInitialize(int *, char ***) ./test_fuzzer INFO: Seed: 3156976563 INFO: Loaded 0 modules (0 guards): INFO: -max_len is not provided, using 64 INFO: A corpus is not provided, starting from an empty corpus #0 READ units: 1 HOWTO – libFuzzer 普华——操作系统整合解决方案专家 #1 INITED cov: 107 bits: 104 corp: 1/1b exec/s: 0 rss: 20Mb DEBUG: int LLVMFuzzerTestOneInput(const uint8_t *, size_t) "ਊ \u0000" DEBUG: int LLVMFuzzerTestOneInput(const uint8_t *, size_t) "\u0001\u0000\u0000\u0000ਊ \u0000\u0000\u0000\u0002\u0000" DEBUG: int LLVMFuzzerTestOneInput(const uint8_t *, size_t) "\u0001\u0000\uFFFF\uFFFF\uFFFF\uFFFF\u0000\u0000ਊ \u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0003\u0000" #4 NEW cov: 110 bits: 107 corp: 2/19b exec/s: 0 rss: 21Mb L: 18 MS: 3 CopyPart-CMP-CMP- DE: "\x01\x00\x00\x00\x00\x00\x00\x00"- "\xff\xff\xff\xff\xff\xff\xff\xff"- DEBUG: int LLVMFuzzerTestOneInput(const uint8_t *, size_t) "\uFFFF\uFFFF\uFFFF\uFFFF\uFFFF\uFFFF\u0000\u0000ਊ \u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0003\u0000" HOWTO – libFuzzer 普华——操作系统整合解决方案专家 QTBUG-57553 ==23824==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60600000e933 at pc 0x000000468e12 bp 0x7ffc87217e20 sp 0x7ffc872175d0 READ of size 2 at 0x60600000e933 thread T0 #0 0x468e11 in __interceptor_memcpy.part.45 (/data/project/LeetcodeCN/leetcodecn.github.io/src/qtbase/build/test_fuzzer+0 x468e11) #1 0x7f49243498b1 in QString::append(QChar const*, int) (/usr/lib/libQt5Core.so.5+0x1338b1) #2 0x7f49243ae9c6 in QDebug::putString(QChar const*, unsigned long) (/usr/lib/libQt5Core.so.5+0x1989c6) #3 0x511df8 in QDebug::operator<<(QString const&) /usr/include/qt/QtCore/qdebug.h:154:52 https://github.com/google/fuzzer-test-suite HOWTO – CFI && Safe Stack 普华——操作系统整合解决方案专家 -fsanitize=cfi-{vcall,nvcall,derived-cast,unrelated-cast} -flto -fsanitize=safe-stack http://leetcode.cn/2016/12/cfi-safestack.html HOWTO – static analyer for cdrkit v1.1.11 普华——操作系统整合解决方案专家 clang -O4 sends your code to Jeff Dean for a complete rewrite.