DOCSLIB.ORG

Studying the Real World Today's Topics

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Studying the Real World Today's Topics Studying the real world Today's topics Free and open source software (FOSS) What is it, who uses it, history Making the most of other people's software Learning from, using, and contributing Learning about your own system Using tools to understand software without source Free and open source software Access to source code Free = freedom to use, modify, copy Some potential benefits Can build for different platforms and needs Development driven by community Different perspectives and ideas More people looking at the code for bugs/security issues Structure Volunteers, sponsored by companies Generally anyone can propose ideas and submit code Different structures in charge of what features/code gets in Free and open source software Tons of FOSS out there Nearly everything on myth Desktop applications (Firefox, Chromium, LibreOffice) Programming tools (compilers, libraries, IDEs) Servers (Apache web server, MySQL) Many companies contribute to FOSS Android core Apple Darwin Microsoft .NET A brief history of FOSS 1960s: Software distributed with hardware Source included, users could fix bugs 1970s: Start of software licensing 1974: Software is copyrightable 1975: First license for UNIX sold 1980s: Popularity of closed-source software Software valued independent of hardware Richard Stallman Started the free software movement (1983) The GNU project GNU = GNU's Not Unix An operating system with unix-like interface GNU General Public License Free software: users have access to source, can modify and redistribute Must share modifications under same license Many of the tools we use are GNU bash, emacs, gcc, gdb, make coreutils (ls, head, tail, sort, which, ...) Linus Torvalds Invented the Linux kernel Fast Forward GNU/Linux has become very popular Feedback loop: use Linux because it supports lots of hardware, contribute support for your hardware back Companies open source tools and libraries More users, more apps for their platform Where to get it Generally through version control glibc: https://sourceware.org/git/?p=glibc.git musl: https://git.musl-libc.org/cgit/musl busybox: https://git.busybox.net/busybox/ Many projects on GitHub gcc: https://github.com/gcc-mirror/gcc linux: https://github.com/torvalds/linux Of course, not limited to C/systems code Learning from real-world software New functions, syntax, concepts E.g. strtok, memmove Learning from real-world software Code quality Style standards and enforcement GNU Coding Standards Learning from real-world software Code quality Style standards and enforcement GNU Coding Standards Linux kernel coding style: First off, I’d suggest printing out a copy of the GNU coding standards, and NOT read it. Burn them, it’s a great symbolic gesture. -- https://www.kernel.org/doc/html/v4.10/process/coding-style.html Learning from real-world software Testing strategies and frameworks Busybox testsuite: https://git.busybox.net/busybox/tree/testsuite Learning from real-world software Development workflows Many small changes vs. large patches Code review Release cycles Best (or at least common...) practices "You can either hang out in the Android Loop or the HURD loop." Link Getting involved Install and use it Experiment with changes, new releases User communities Forums, mailing lists Ask questions, request features Bug/issue trackers See what kinds of issues come up Discussions about design, process, portability, ... Example: GRUB stack exploit static int grub_username_get (char buf[], unsigned buf_size) { unsigned cur_len = 0; int key; while (1) { key = grub_getkey (); [...] if (key == '\b') { cur_len--; grub_printf ("\b"); continue; } [...] } grub_memset( buf + cur_len, 0, buf_size - cur_len); [...] } -- http://hmarco.org/bugs/CVE-2015-8370-Grub2-authentication-bypass.html Example: Flash memcpy Description of problem: Strange sound when playing mp3 on website using flash (using Shockwave Flash 10.2 d161). ----- The trigger of the problem is the glibc version. [...] ----- valgrind? ----- Looking at the changelog for glibc-2.12.90-4 shows: * Fri Jul 02 2010 Andreas Schwab <[email protected]> - 2.12.90-4 - Improve 64bit memcpy/memmove for Atom, Core 2 and Core i7 I used chromium to run valgrind on the flash plugin [...] ==2100== Thread 9: ==2100== Source and destination overlap in memcpy(0x256d7170, 0x256d7570, 1280) ==2100== at 0x4A06A3A: memcpy (mc_replace_strmem.c:497) https://bugzilla.redhat.com/show_bug.cgi?id=638477 Example: Flash memcpy So in the kernel we have a pretty strict "no regressions" rule, and that if people depend on interfaces we exported having side effects that weren't intentional, we try to fix things so that they still work unless there is a maJor reason not to. So I'm disappointed glibc just closes this as NOTABUG. There's no real reason to do the copy backwards that I can see, so doing it that way is Just stupid. -- Linus Torvalds https://bugzilla.redhat.com/show_bug.cgi?id=638477 Example: Flash memcpy (In reply to comment #39) > The only stupidity is crap software violating well known rules that have > existed forever. Umm. Bugs happen. That's a fact. You can call it "crap software" all you like, but the thing is, if memcpy doesn't warn about overlaps, there's no test coverage, and in that case even well-designed software will have bugs. Then the question becomes one of "Why break it?" -- Linus Torvalds https://bugzilla.redhat.com/show_bug.cgi?id=638477 Aside: Licenses Myth: If code is published online, I can use it. WRONG! There are lots of issues with licensing and permitted use Permissive licenses (BSD, MIT, Apache) Few restrictions on use, modification, distribution Can use in proprietary software "Copyleft" licenses (GNU GPL) Must make source code of changes available Can only be integrated into projects with compatible licenses Many companies have "open source disclosures" with lots of interesting code Aside: Licenses No license = no permission "If you find software that doesn’t have a license, that generally means you have no permission from the creators of the software to use, modify, or share the software. Although a code host such as GitHub may allow you to view and fork the code, this does not imply that you are permitted to use, modify, or share the software for any purpose." -- https://choosealicense.com/ Learning from binaries You don't need the source code to learn how things work Can use the same techniques you've learned in 107 on real programs strings, valgrind, objdump/gdb Some new tools: ltrace, strace, file Takeaways You've learned lots of super practical skills Command line/unix, reading C code, inspecting programs through assembly Where to go from here Learn from the process and designs of others Use open source projects to run with your own ideas Get involved with a project that interests you.
Load more

Recommended publications
  • The Kernel Report
    The kernel report (ELC 2012 edition) Jonathan Corbet LWN.net [email protected] The Plan Look at a year's worth of kernel work ...with an eye toward the future Starting off 2011 2.6.37 released - January 4, 2011 11,446 changes, 1,276 developers VFS scalability work (inode_lock removal) Block I/O bandwidth controller PPTP support Basic pNFS support Wakeup sources What have we done since then? Since 2.6.37: Five kernel releases have been made 59,000 changes have been merged 3069 developers have contributed to the kernel 416 companies have supported kernel development February As you can see in these posts, Ralink is sending patches for the upstream rt2x00 driver for their new chipsets, and not just dumping a huge, stand-alone tarball driver on the community, as they have done in the past. This shows a huge willingness to learn how to deal with the kernel community, and they should be strongly encouraged and praised for this major change in attitude. – Greg Kroah-Hartman, February 9 Employer contributions 2.6.38-3.2 Volunteers 13.9% Wolfson Micro 1.7% Red Hat 10.9% Samsung 1.6% Intel 7.3% Google 1.6% unknown 6.9% Oracle 1.5% Novell 4.0% Microsoft 1.4% IBM 3.6% AMD 1.3% TI 3.4% Freescale 1.3% Broadcom 3.1% Fujitsu 1.1% consultants 2.2% Atheros 1.1% Nokia 1.8% Wind River 1.0% Also in February Red Hat stops releasing individual kernel patches March 2.6.38 released – March 14, 2011 (9,577 changes from 1198 developers) Per-session group scheduling dcache scalability patch set Transmit packet steering Transparent huge pages Hierarchical block I/O bandwidth controller Somebody needs to get a grip in the ARM community.
    [Show full text]
  • The Linux Kernel Module Programming Guide
    The Linux Kernel Module Programming Guide Peter Jay Salzman Michael Burian Ori Pomerantz Copyright © 2001 Peter Jay Salzman 2007−05−18 ver 2.6.4 The Linux Kernel Module Programming Guide is a free book; you may reproduce and/or modify it under the terms of the Open Software License, version 1.1. You can obtain a copy of this license at http://opensource.org/licenses/osl.php. This book is distributed in the hope it will be useful, but without any warranty, without even the implied warranty of merchantability or fitness for a particular purpose. The author encourages wide distribution of this book for personal or commercial use, provided the above copyright notice remains intact and the method adheres to the provisions of the Open Software License. In summary, you may copy and distribute this book free of charge or for a profit. No explicit permission is required from the author for reproduction of this book in any medium, physical or electronic. Derivative works and translations of this document must be placed under the Open Software License, and the original copyright notice must remain intact. If you have contributed new material to this book, you must make the material and source code available for your revisions. Please make revisions and updates available directly to the document maintainer, Peter Jay Salzman <[email protected]>. This will allow for the merging of updates and provide consistent revisions to the Linux community. If you publish or distribute this book commercially, donations, royalties, and/or printed copies are greatly appreciated by the author and the Linux Documentation Project (LDP).
    [Show full text]
  • Executable Code Is Not the Proper Subject of Copyright Law a Retrospective Criticism of Technical and Legal Naivete in the Apple V
    Executable Code is Not the Proper Subject of Copyright Law A retrospective criticism of technical and legal naivete in the Apple V. Franklin case Matthew M. Swann, Clark S. Turner, Ph.D., Department of Computer Science Cal Poly State University November 18, 2004 Abstract: Copyright was created by government for a purpose. Its purpose was to be an incentive to produce and disseminate new and useful knowledge to society. Source code is written to express its underlying ideas and is clearly included as a copyrightable artifact. However, since Apple v. Franklin, copyright has been extended to protect an opaque software executable that does not express its underlying ideas. Common commercial practice involves keeping the source code secret, hiding any innovative ideas expressed there, while copyrighting the executable, where the underlying ideas are not exposed. By examining copyright’s historical heritage we can determine whether software copyright for an opaque artifact upholds the bargain between authors and society as intended by our Founding Fathers. This paper first describes the origins of copyright, the nature of software, and the unique problems involved. It then determines whether current copyright protection for the opaque executable realizes the economic model underpinning copyright law. Having found the current legal interpretation insufficient to protect software without compromising its principles, we suggest new legislation which would respect the philosophy on which copyright in this nation was founded. Table of Contents INTRODUCTION................................................................................................. 1 THE ORIGIN OF COPYRIGHT ........................................................................... 1 The Idea is Born 1 A New Beginning 2 The Social Bargain 3 Copyright and the Constitution 4 THE BASICS OF SOFTWARE ..........................................................................
    [Show full text]
  • Chapter 1 Introduction to Computers, Programs, and Java
    Chapter 1 Introduction to Computers, Programs, and Java 1.1 Introduction • The central theme of this book is to learn how to solve problems by writing a program . • This book teaches you how to create programs by using the Java programming languages . • Java is the Internet program language • Why Java? The answer is that Java enables user to deploy applications on the Internet for servers , desktop computers , and small hand-held devices . 1.2 What is a Computer? • A computer is an electronic device that stores and processes data. • A computer includes both hardware and software. o Hardware is the physical aspect of the computer that can be seen. o Software is the invisible instructions that control the hardware and make it work. • Computer programming consists of writing instructions for computers to perform. • A computer consists of the following hardware components o CPU (Central Processing Unit) o Memory (Main memory) o Storage Devices (hard disk, floppy disk, CDs) o Input/Output devices (monitor, printer, keyboard, mouse) o Communication devices (Modem, NIC (Network Interface Card)). Bus Storage Communication Input Output Memory CPU Devices Devices Devices Devices e.g., Disk, CD, e.g., Modem, e.g., Keyboard, e.g., Monitor, and Tape and NIC Mouse Printer FIGURE 1.1 A computer consists of a CPU, memory, Hard disk, floppy disk, monitor, printer, and communication devices. CMPS161 Class Notes (Chap 01) Page 1 / 15 Kuo-pao Yang 1.2.1 Central Processing Unit (CPU) • The central processing unit (CPU) is the brain of a computer. • It retrieves instructions from memory and executes them.
    [Show full text]
  • Some Preliminary Implications of WTO Source Code Proposala INTRODUCTION
    Some preliminary implications of WTO source code proposala INTRODUCTION ............................................................................................................................................... 1 HOW THIS IS TRIMS+ ....................................................................................................................................... 3 HOW THIS IS TRIPS+ ......................................................................................................................................... 3 WHY GOVERNMENTS MAY REQUIRE TRANSFER OF SOURCE CODE .................................................................. 4 TECHNOLOGY TRANSFER ........................................................................................................................................... 4 AS A REMEDY FOR ANTICOMPETITIVE CONDUCT ............................................................................................................. 4 TAX LAW ............................................................................................................................................................... 5 IN GOVERNMENT PROCUREMENT ................................................................................................................................ 5 WHY GOVERNMENTS MAY REQUIRE ACCESS TO SOURCE CODE ...................................................................... 5 COMPETITION LAW .................................................................................................................................................
    [Show full text]
  • Android (Operating System) 1 Android (Operating System)
    Android (operating system) 1 Android (operating system) Android Home screen displayed by Samsung Nexus S with Google running Android 2.3 "Gingerbread" Company / developer Google Inc., Open Handset Alliance [1] Programmed in C (core), C++ (some third-party libraries), Java (UI) Working state Current [2] Source model Free and open source software (3.0 is currently in closed development) Initial release 21 October 2008 Latest stable release Tablets: [3] 3.0.1 (Honeycomb) Phones: [3] 2.3.3 (Gingerbread) / 24 February 2011 [4] Supported platforms ARM, MIPS, Power, x86 Kernel type Monolithic, modified Linux kernel Default user interface Graphical [5] License Apache 2.0, Linux kernel patches are under GPL v2 Official website [www.android.com www.android.com] Android is a software stack for mobile devices that includes an operating system, middleware and key applications.[6] [7] Google Inc. purchased the initial developer of the software, Android Inc., in 2005.[8] Android's mobile operating system is based on a modified version of the Linux kernel. Google and other members of the Open Handset Alliance collaborated on Android's development and release.[9] [10] The Android Open Source Project (AOSP) is tasked with the maintenance and further development of Android.[11] The Android operating system is the world's best-selling Smartphone platform.[12] [13] Android has a large community of developers writing applications ("apps") that extend the functionality of the devices. There are currently over 150,000 apps available for Android.[14] [15] Android Market is the online app store run by Google, though apps can also be downloaded from third-party sites.
    [Show full text]
  • Android Operating System
    Software Engineering ISSN: 2229-4007 & ISSN: 2229-4015, Volume 3, Issue 1, 2012, pp.-10-13. Available online at http://www.bioinfo.in/contents.php?id=76 ANDROID OPERATING SYSTEM NIMODIA C. AND DESHMUKH H.R. Babasaheb Naik College of Engineering, Pusad, MS, India. *Corresponding Author: Email- [email protected], [email protected] Received: February 21, 2012; Accepted: March 15, 2012 Abstract- Android is a software stack for mobile devices that includes an operating system, middleware and key applications. Android, an open source mobile device platform based on the Linux operating system. It has application Framework,enhanced graphics, integrated web browser, relational database, media support, LibWebCore web browser, wide variety of connectivity and much more applications. Android relies on Linux version 2.6 for core system services such as security, memory management, process management, network stack, and driver model. Architecture of Android consist of Applications. Linux kernel, libraries, application framework, Android Runtime. All applications are written using the Java programming language. Android mobile phone platform is going to be more secure than Apple’s iPhone or any other device in the long run. Keywords- 3G, Dalvik Virtual Machine, EGPRS, LiMo, Open Handset Alliance, SQLite, WCDMA/HSUPA Citation: Nimodia C. and Deshmukh H.R. (2012) Android Operating System. Software Engineering, ISSN: 2229-4007 & ISSN: 2229-4015, Volume 3, Issue 1, pp.-10-13. Copyright: Copyright©2012 Nimodia C. and Deshmukh H.R. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
    [Show full text]
  • Version 7.8-Systemd
    Linux From Scratch Version 7.8-systemd Created by Gerard Beekmans Edited by Douglas R. Reno Linux From Scratch: Version 7.8-systemd by Created by Gerard Beekmans and Edited by Douglas R. Reno Copyright © 1999-2015 Gerard Beekmans Copyright © 1999-2015, Gerard Beekmans All rights reserved. This book is licensed under a Creative Commons License. Computer instructions may be extracted from the book under the MIT License. Linux® is a registered trademark of Linus Torvalds. Linux From Scratch - Version 7.8-systemd Table of Contents Preface .......................................................................................................................................................................... vii i. Foreword ............................................................................................................................................................. vii ii. Audience ............................................................................................................................................................ vii iii. LFS Target Architectures ................................................................................................................................ viii iv. LFS and Standards ............................................................................................................................................ ix v. Rationale for Packages in the Book .................................................................................................................... x vi. Prerequisites
    [Show full text]
  • Introduction to Linux Kernel Driver Programming
    IntroductionIntroduction toto LinuxLinux kernelkernel driverdriver programmingprogramming Introduction to Linux kernel driver programming The Linux kernel device model Authors and license ● Authors – Michael Opdenacker ([email protected]) Founder of Bootlin, kernel and embedded Linux engineering company https://bootlin.com/company/staff/michael-opdenacker ● License – Creative Commons Attribution – Share Alike 4.0 https://creativecommons.org/licenses/by-sa/4.0/ – Document sources: https://github.com/e-ale/Slides Need for a device model ● For the same device, need to use the same device driver on multiple CPU architectures (x86, ARM…), even though the hardware controllers are different. ● Need for a single driver to support multiple devices of the same kind. ● This requires a clean organization of the code, with the device drivers separated from the controller drivers, the hardware description separated from the drivers themselves, etc. Driver: between bus infrastructure and framework In Linux, a driver is always interfacing with: ● a framework that allows the driver to expose the hardware features in a generic way. ● a bus infrastructure, part of the device model, to detect/communicate with the hardware. Let’s focus on the bus infrastructure for now Device model data structures The device model is organized around three main data structures: ● The struct bus_type structure, which represent one type of bus (USB, PCI, I2C, etc.) ● The struct device_driver structure, which represents one driver capable of handling certain devices on a certain bus. ● The struct device structure, which represents one device connected to a bus The kernel uses inheritance to create more specialized versions of struct device_driver and struct device for each bus subsystem.
    [Show full text]
  • GNU Octave Beginner's Guide
    GNU Octave Beginner's Guide Become a profcient Octave user by learning this high-level scientfc numerical tool from the ground up Jesper Schmidt Hansen BIRMINGHAM - MUMBAI GNU Octave Beginner's Guide Copyright © 2011 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmited in any form or by any means, without the prior writen permission of the publisher, except in the case of brief quotatons embedded in critcal artcles or reviews. Every efort has been made in the preparaton of this book to ensure the accuracy of the informaton presented. However, the informaton contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, its dealers, and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark informaton about all of the companies and products mentoned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this informaton. First published: June 2011 Producton Reference: 2150611 Published by Packt Publishing Ltd. 32 Lincoln Road Olton Birmingham, B27 6PA, UK. ISBN 978-1-849513-32-6 www.packtpub.com Cover Image by John Quick ([email protected]) Credits Author Project Coordinator Jesper Schmidt Hansen Joel Goveya Reviewers Proofreaders Piotr Gawron Lesley Harrison Kenneth Geisshirt Clyde Jenkins Jordi Gutérrez Hermoso Lynda Sliwoski Acquisiton Editor Indexers Usha Iyer Hemangini Bari Tejal Daruwale Development Editor Monica Ajmera Mehta Roger D'souza Graphics Technical Editor Nilesh R.
    [Show full text]
  • Université De Montréal Context-Aware
    UNIVERSITE´ DE MONTREAL´ CONTEXT-AWARE SOURCE CODE IDENTIFIER SPLITTING AND EXPANSION FOR SOFTWARE MAINTENANCE LATIFA GUERROUJ DEPARTEMENT´ DE GENIE´ INFORMATIQUE ET GENIE´ LOGICIEL ECOLE´ POLYTECHNIQUE DE MONTREAL´ THESE` PRESENT´ EE´ EN VUE DE L'OBTENTION DU DIPLOME^ DE PHILOSOPHIÆ DOCTOR (GENIE´ INFORMATIQUE) JUILLET 2013 ⃝c Latifa Guerrouj, 2013. UNIVERSITE´ DE MONTREAL´ ECOLE´ POLYTECHNIQUE DE MONTREAL´ Cette th`ese intitul´ee: CONTEXT-AWARE SOURCE CODE IDENTIFIER SPLITTING AND EXPANSION FOR SOFTWARE MAINTENANCE pr´esent´eepar: GUERROUJ Latifa en vue de l'obtention du dipl^ome de: Philosophiæ Doctor a ´et´ed^ument accept´eepar le jury d'examen constitu´ede: Mme BOUCHENEB Hanifa, Doctorat, pr´esidente M. ANTONIOL Giuliano, Ph.D., membre et directeur de recherche M. GUEH´ ENEUC´ Yann-Ga¨el, Ph.D., membre et codirecteur de recherche M. DESMARAIS Michel, Ph.D., membre Mme LAWRIE Dawn, Ph.D., membre iii This dissertation is dedicated to my parents. For their endless love, support and encouragement. iv ACKNOWLEDGMENTS I am very grateful to both Giulio and Yann for their support, encouragement, and intel- lectual input. I worked with you for four years or even less, but what I learned from you will last forever. Giulio, your passion about research was a source of inspiration and motivation for me. Also, your mentoring and support have been instrumental in achieving my goals. Yann, your enthusiasm and guidance have always been a strength for me to keep moving forward. Research would not be as much fun without students and researchers to collaborate with. It has been a real pleasure and great privilege working with Massimiliano Di Penta (University of Sannio), Denys Poshyvanyk (College of William and Mary), and their teams.
    [Show full text]
  • The GNU C Programming Tutorial
    Edition 4.1 The GNU C Programming Tutorial Mark Burgess Faculty of Engineering, Oslo College Ron Hale-Evans Copyright c 2002 Free Software Foundation, Inc. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.1 or any later version published by the Free Software Foundation; there being no Invariant Section, with the Front-Cover Texts being \A GNU Manual", and with the Back-Cover Texts as in (a) below. A copy of the license is included in the section entitled \GNU Free Documentation License". (a) The FSF's Back-Cover Text is: \You have freedom to copy and modify this GNU Manual, like GNU software. Copies published by the Free Software Foundation raise funds for GNU development." Function pointers i Table of Contents Preface ...................................... xi 1 Introduction............................... 1 1.1 The advantages of C..................................... 1 1.2 Questions for Chapter 1 ................................. 2 2 Using a compiler........................... 3 2.1 Basic ideas about C ..................................... 3 2.2 The compiler ........................................... 4 2.3 File names .............................................. 4 2.4 Errors .................................................. 5 2.4.1 Typographical errors ............................ 6 2.4.2 Type errors .................................... 6 2.5 Questions for Chapter 2 ................................. 6 3 The form of a C program................... 9
    [Show full text]