ECE 2400 / ENGRD 2140 Systems Programming Course Overview

Christopher Batten School of Electrical and Computer Engineering Cornell University http://www.csl.cornell.edu/courses/ece2400 What is Computer Systems Programming? Activity Trends in Computer Systems Programming Course Logistics

ECE 2400 / ENGRD 2140 Computer Systems Programming

What is Computer Systems Programming?

Application-Level Software Activity: Comparing Algorithms

System-Level Trends in Computer Systems Programming Software Course Logistics

ECE 2400 Course Overview 2 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Applications vs. Technology

Application

Gap too large to bridge in one step (but there are exceptions, e.g., a magnetic compass)

Technology

ECE 2400 Course Overview 3 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Applications vs. Technology

Application

Technology

ECE 2400 Course Overview 3 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics The Computer Systems Stack

Application Algorithm Traditional Programming Language Computer Science Compiler Computer Engineering is at the Instruction Set Architecture interface between hardware and software Microarchitecture and considers the entire system Register-Transfer Level Gate Level Traditional Computer Engineering Circuits Electrical Engineering Devices Technology In its broadest definition, computer engineering is the development of the abstraction/implementation layers that allow us to execute information processing applications efficiently using available manufacturing technologies

ECE 2400 Course Overview 4 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Python for Application-Level Programming

Application I High-level, user- Algorithm facing software Programming Language Enable productively Operating System I developing Compiler applications that Instruction Set Architecture provide new Microarchitecture Application-Level Software functionality to Register-Transfer Level Gate Level users

Computer Engineering Circuits ? I Enable productively Devices collecting, Technology analyzing, visualizing data I Sometimes called a productivity-level language

ECE 2400 Course Overview 5 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics /C++ for System-Level Programming

Application I Connects Algorithm application software Programming Language to the low-level Operating System computer hardware Compiler Enables carefully Instruction Set Architecture I managing Microarchitecture Application-Level Software performance and Register-Transfer Level Gate Level resource constraints System-Level Computer Engineering Circuits Software? I Sometimes called Devices an efficiency-level Technology language

ECE 2400 Course Overview 6 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Dynamically Interpreted vs. Statically Compiled

def min(a,b): int min( int a, if a < b: Python Python C/C++ C/C++ int b ) c = a Source Code Library Code Source Code Library Code { else int c; c = b if ( a < b ) return c c = a; else "Runtime" "Runtime" Ahead-of-Time Ahead-of-Time c = b; Compiler Compiler Compiler Compiler return c; }

LOAD_FAST pushq %rbp LOAD_FAST Python Python Machine Machine movq %rsp,%rbp COMPARE_OP cmpl %esi,%edi POP_JUMP_IF_F Bytecode Bytecode Object File Object File cmovl %edi,%esi LOAD_FAST movl %esi,%eax STORE_FAST popq %rbp JUMP_FORWARD retq LOAD_FAST Execute STORE_FAST w/ Interpreter Linker 1010101 LOAD_FAST 100100010001001 RETURN_VALUE 11100111110111 111101001110111 100010011111000 Machine 1011101 Binary 11000011 The standard Python Execute interpreter is called CPython Binary and it is written in C!

ECE 2400 Course Overview 7 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Computer Systems Programming is Diverse

Application Algorithm I Python, MATLAB Programming Language Ruby, Javascript Operating System I Compiler I SQL, LINQ Instruction Set Architecture I NumPy Microarchitecture Application-Level I GUI frameworks Software Register-Transfer Level Gate Level System-Level Computer Engineering Circuits Software? Devices I Interpreters Technology I Compilers I Databases I Numerical libraries I Operating systems I Embedded control

ECE 2400 Course Overview 8 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Aside: C/C++ for Application-Level Software

Application Algorithm Programming Language Operating System Compiler Instruction Set Architecture Microarchitecture Application-Level Application-LevelSoftware Register-Transfer Level & Gate Level System-Level Software Computer Engineering Circuits ? Devices Technology

ECE 2400 Course Overview 9 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics A Tale of Two Programming Languages

Python Programming Language C/C++ Programming Language

I Introduced: 1991 I Introduced: 1972(C), 1979(C++) I Most of the machine details are I Most of the machine details are hidden from exposed to the programmer

I Programmer gives up some I Programmer is in complete control for improved productivity control for improved efficiency

I Easily supports multiple I Easily supports multiple programming paradigms programming paradigms

I Extensive standard library is I More limited standard library is included included

I Slow and memory inefficient I Fast and memory efficient

ECE 2400 Course Overview 10 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Comparing the Popularity of Python vs. C/C++

The 2021 Top Programming Languages, IEEE Spectrum

ECE 2400 Course Overview 11 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Comparing the Performance of Python vs. C/C++

300

100 50 30

10 5 3

1 C Go Perl Lisp C++ Rust PHP Java Swift Ruby Program Time / Fastest Program Time Pascal Python Fortran The Computer Language Benchmarks Game

ECE 2400 Course Overview 12 / 33 • What is Computer Systems Programming? • Activity Trends in Computer Systems Programming Course Logistics Program = Algorithm + Data Structure

While this course covers C/C++ and system-level programming, this course also builds off of your prior programming experience to further develop your understanding of algorithms and data structures

Algorithm Data Structure

I Algorithm: Clear set of steps to solve any problem instance in a particular class of problems

I Data Structure: Way of efficiently organizing and storing data along with operations for accessing and manipulating this data

ECE 2400 Course Overview 13 / 33 What is Computer Systems Programming? • Activity • Trends in Computer Systems Programming Course Logistics

ECE 2400 / ENGRD 2140 Computer Systems Programming

What is Computer Systems Programming?

Application-Level Software Activity: Comparing Algorithms

System-Level Trends in Computer Systems Programming Software Course Logistics

ECE 2400 Course Overview 14 / 33 What is Computer Systems Programming? • Activity • Trends in Computer Systems Programming Course Logistics Activity: Comparing Algorithms

I Application: Sort 16 numbers I Activity Steps . 1. Half the class will use Algorithm A, half uses Algorithm B . 2. When instructor starts timer, flip over worksheet . 3. Sort 16 numbers using assigned algorithm . 4. Lookup when completed and write time on worksheet . 5. Raise hand . 6. When everyone is finished, then analyze data

I Algorithm A repeat 16 times find smallest number not crossed off in input list copy smallest number to next open entry in output list cross smallest number off input list

ECE 2400 Course Overview 15 / 33 What is Computer Systems Programming? • Activity • Trends in Computer Systems Programming Course Logistics Activity: Comparing Algorithms

I Algorithm B

repeat 8 times, once for each pair in column 1 copy smallest from input pair into next entry in column 1 copy largest from input pair into next entry in column 1

repeat 4 times, once for group of 4 in column 2 repeat 4 times compare top two numbers not crossed off in both groups copy smallest number to next open entry in column 2 cross smallest number off input list

... and so on ...

ECE 2400 Course Overview 16 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics

ECE 2400 / ENGRD 2140 Computer Systems Programming

What is Computer Systems Programming?

Application-Level Software Activity: Comparing Algorithms

System-Level Trends in Computer Systems Programming Software Course Logistics

ECE 2400 Course Overview 17 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics Trend towards IoT and Cloud w/ Novel Hardware

Roughly every decade a new, smaller, lower priced computer class forms based on a new programming platform resulting in entire new industries

8 10 8 Supercomputers 10 Minicomputers 7 7 10 10 Mainframes Personal 6 6 10 10 Scalable 5 Mainframes Clusters 3 10 Handhelds 5 Minicomputers 10 Cloud 4 Computing 10 3 4 10 Workstations 10 2 Volume in cm Price in Dollars Laptops 10 3 Workstations 10 Personal 1 10 Computers Laptops 2 Handhelds 0 10 G. Bell. "Bell's Law for the Birth and 10 Y. Lee et al. "Modular 1mm3 Die- Internet Internet Death of Computer Classes." of Things Stacked Sensing Platform ..." CACM, Jan 2008. JSSC, Jan 2013. of Things 1950 1960 1970 19801990 2000 2010 2020 1950 1960 1970 1980 2000 201019902020

ECE 2400 Course Overview 18 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics Trend towards IoT and Cloud w/ Novel Hardware

Roughly every decade a new, smaller, lower priced computer class forms based on a new programming platform resulting in entire new industries

8 Cloud Computing 10 8 Supercomputers 10 Minicomputers 7 7 I Often requires low-latency, 10 10 Mainframes high-throughput to meet overallPersonal 6 Computers 6 10 10 application requirements Scalable 5 Mainframes Clusters 3 10 Handhelds 5 Minicomputers I Increasingly w/ specialized HW 10 Cloud 4 Computing 10 3 4 Internet-of-Things 10 Workstations 10 2 Volume in cm Price in Dollars Laptops 10 Very limited resource constraints 3 I Workstations 10 Personal 1 10 Computers (e.g., energy,Laptops memory) 2 Handhelds 0 10 G. Bell. "Bell's Law for the Birth and 10 IY. LeeRequires et al. "Modular 1mm3 carefully Die- managing Internet Internet Death of Computer Classes." of Things Stacked Sensing Platform ..." CACM, Jan 2008. JSSC,these Jan 2013. resourcesof Things to meet overall 1950 1960 1970 19801990 2000 2010 2020 1950application 1960 1970 1980 requirements 2000 201019902020 I Increasingly w/ specialized HW

ECE 2400 Course Overview 18 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics Example Application: Image Recognition

Starfish

Dog

ECE 2400 Course Overview 19 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics Machine Learning (ML): Training vs. Inference

Training Model forward "starfish" labels

=? "dog"

error backward many images

forward Inference "dog" few images

ECE 2400 Course Overview 20 / 33 What is Computer Systems Programming? Activity • Trends in Computer Systems Programming • Course Logistics Computer Systems Programming in ML

Cloud Google TPU Computing I Training is done using the TensorFlow C++ framework I Training can take weeks I Google TPU is custom chip I High-level ML frameworks use C++ under the hood

Movidius Myriad 2 I Custom chip for ML on embedded IoT devices I Carefully crafted C/C++ ML libraries for inference Internet I Embedded control also of Things in C/C++

ECE 2400 Course Overview 21 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics •

ECE 2400 / ENGRD 2140 Computer Systems Programming

What is Computer Systems Programming?

Application-Level Software Activity: Comparing Algorithms

System-Level Trends in Computer Systems Programming Software Course Logistics

ECE 2400 Course Overview 22 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • ECE 2400 Within the Engineering Curriculum

Application Algorithm CS 1110 / CS 1112 Intro to Computing Programming Language ECE 2400 Computer Systems Programming Operating System ECE 3140 Embedded Systems Compiler Instruction Set Architecture ECE 4760 Design with Microcontrollers Microarchitecture ECE 4750 Computer Architecture Register-Transfer Level Gate Level ECE 2300 Digital Logic & Computer Org

Computer Engineering Circuits Devices Technology ECE 2400 is also an ENGRD and thus satisfies the engineering distribution requirement ECE 2400 can be an excellent way to generally incorporate programming into your non-ECE engineering curriculum

ECE 2400 Course Overview 23 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • Course Objectives

I describe a variety of algorithms and data structures and how to analyze these algorithms and data structures in terms of time and space complexity

I apply the C/C++ programming languages to implement algorithms and data structures using different programming paradigms

I evaluate algorithm and data structure alternatives and make a compelling qualitative and/or quantitative argument for one approach

I create non-trivial C/C++ programs (roughly 1,000 lines of code) and the associated testing strategy from an English language specification

I write concise yet comprehensive technical reports that describe a program implemented in C/C++, explain the testing strategy used to verify functionality, and evaluate the program to characterize its performance and memory usage

ECE 2400 Course Overview 24 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • Course Structure

I Part 1: Procedural Programming . introduction to C; variables; expressions; functions; conditional & iteration statements; recursion; static types; pointers; arrays; dynamic allocation I Part 2: Basic Algorithms and Data Structures . lists; vectors; complexity analysis; sorting algorithms: insertion, selection, merge, quick, radix; ADTs: stacks, queues, priority queues, sets, maps I Part 3: Multi-Paradigm Programming . transition to C++; namespaces; flexible function prototypes; references; exceptions; new/delete; object oriented programming: C++ classes and inheritance for dynamic polymorphism; generic programming: C++ templates for static polymorphism; functional programming: C++ functors and lambdas; concurrent programming: C++ threads and atomics I Part 4: More Algorithms and Data Structures . trees (binary search trees; binary heaps); tables (lookup tables; hash tables); graphs (DFS, BFS, shortest path, minimum spanning trees)

ECE 2400 Course Overview 25 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • zyBook: New Interactive Online Textbook

ECE 2400 Course Overview 26 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • Programming Assignments

I PA1–3: Fundamentals . PA1: Math functions . PA2: List and Vector Data Structures . PA3: Sorting Algorithms I PA4–5: Handwriting Recognition System . PA5: Linear vs. Binary Searching . PA5: Trees vs. Tables I Every programming assignment involves . C/C++ “agile” programming . State-of-the-art tools for build systems, version control, continuous integration, code coverage . Performance measurement . Short technical report

ECE 2400 Course Overview 27 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics •

Application-Level Software

System-Level Software

ECE 2400 Course Overview 28 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics •

Nick Cebry Ryan McMahon Guadalupe Bernal Michael Egbueze ECE Phd ECE MEng ECE Junior CS Senior

Eric Hall Sonal Parab Anya Prabowo Chidera Wokonko ECE Senior CS Senior ECE Junior ECE Junior

ECE 2400 Course Overview 29 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics •

ECE 2400 Course Overview 30 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • Frequently Asked Questions

I I have not taken CS 1110 nor CS 1112, can I take this class? . We assume some basic programming experience, discuss with instructor I ECE Majors – How does ECE 2400 satisfy degree requirements? . ECE 2400 can count as your second ENGRD course . ECE 2400 can count as an outside-ECE technical elective . ECE 2400 satisfies the ECE advanced programming requirement I CS Majors – Can I use ECE 2400 in place of CS 2110? . Yes but you should probably take CS 2110 I ECE/CS Dual Majors – Can I use ECE 2400 in place of CS 2110? . Absolutely! (NEW) I CS Minors – Can I use ECE 2400 in place of CS 2110? . Absolutely! (NEW)

ECE 2400 Course Overview 31 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming • Course Logistics • Frequently Asked Questions

I Other Majors – How does ECE 2400 satisfy degree requirements? . ECE 2400 can count as one of your two required ENGRD courses . CS 2110 and ECE 2400 are in the same ENGRD category, so you cannot use both of them as your two ENGRD courses I Can I take both ECE 2400 and CS 2110? . Sure! (recall popularity and performance data)

ECE 2400 Course Overview 32 / 33 What is Computer Systems Programming? Activity Trends in Computer Systems Programming Course Logistics

Take-Away Points

I Computer systems programming involves developing software to connect the low-level computer hardware to high-level, user-facing Application-Level application software and usually requires careful Software consideration of performance and resource constraints System-Level Software I We are entering an exciting era where computer systems programming will play a critical role in enabling both cloud computing and the internet-of-things

ECE 2400 Course Overview 33 / 33