Project 1: Boot Process of an Embedded System, Its BSP and Development Environment

LECTURE: MP-6171 SISTEMAS EMPOTRADOSDE ALTO DESEMPEÑO

Project 1: Boot Process of an Embedded System, its BSP and Development Environment

Lecturers: MSc. José Araya Martínez MSc. Sergio Arriola-Valverde

First term, 2020 0

Contents

1 Introduction3 1.1 Administrative Matters...... 3 1.1.1 Team Formation...... 3 1.1.2 Forum and Communication...... 3 1.1.3 Plagiarism...... 4 1.2 Development Environment Overview...... 4 1.2.1 Nomenclature...... 5

2 Setting up the Host5 2.1 Downloading the VM and Configuring VirtualBox...... 6 2.2 Version Control: GIT...... 7 2.3 Configure Local Area Network with Static IP...... 8 2.4 Configure a TFTP Server for File Sharing...... 9 2.5 Set NFS for Mounting a Remote File System...... 10 2.6 Installing the Eclipse IDE for C/C++ Developers...... 10

3 Setting up our Embedded System and its Board Support Package 11 3.1 Using a Pre-compiled Image: Raspbian...... 12 3.2 Establishing UART Connection with the RPI...... 13 3.3 How Does an Embedded Device Boot?...... 16 3.3.1 The Specific Case of the Raspberry Pi 4...... 18 3.4 First Stage Bootloader: BootROM...... 19 3.5 Second Stage Bootloader: Das U-Boot...... 19 3.5.1 Toolchain for Cross-Compilation...... 20 3.5.2 Cross-compiling U-Boot...... 21 3.5.3 Configuring our Target to Load U-Boot...... 22 3.5.4 Configuring Static IP in U-Boot...... 24 3.6 The Linux Kernel...... 25 3.6.1 Cross-compiling the Linux Kernel...... 26 3.6.2 Loading Kernel from TFTP...... 29 3.7 YOCTO, File System and the Toolchain...... 32 3.7.1 Generating a File System with Yocto...... 32 3.7.2 Deploying a Remote Network File System (NFS)...... 36 3.7.3 Configure the Embedded System to mount a NFS...... 37 3.8 Testing the Target’s SSH Server...... 40 3.9 Cross-compiler and Toolchain...... 40 3.9.1 Install the Generated ARM Toolchain...... 40 3.9.2 Yocto Plugin for the Eclipse IDE...... 42

1 0

3.9.3 Conﬁguring the Yocto Plugin and Create Example Project.... 42

4 Create a Custom Meta-Layer in Yocto 44

5 The Application: Composite Trapezoidal Numerical Integration 45 5.1 Motivation and Description...... 46 5.2 Prototyping and Programming Process...... 46 5.2.1 Prototyping...... 47 5.2.2 Programming...... 47

6 Deliverables and Grading 48 6.1 Work ﬂow requirements...... 48 6.2 Folder Structure of your Deliverables...... 49 6.3 Grading...... 50 6.4 Deliverables Submission...... 50

2 1.1.2

1 Introduction

In this project, each team will be guided through the process of bringing up an embedded system. During the project we will learn what is the common boot process of an industrial embedded device and the specific case of the Raspberry Pi 4 (RPI4), additionally the students will need to make use of the widely-used Linaro cross-compilation toolchain and throw some light on the powerful but complex Yocto Project. All this without forgetting to keep an eye on the code’s version control as is usual in every serious development team nowadays. The students will be guided through the setup of a cross development environment (host and target). For this, we will go through the compilation and configuration of the multiple software components needed to orchestrate a cross development environment with the ultimate purpose of developing and optimizing our own application. Towards the end of the project, each team will implement an application using the Trape- zoidal Numerical Integration concept. To do so, we will first prototype our application in a high-level programming language, then we will implement using the more efficient C language and at the end we will include it into our custom File System as a standard application of our Linux distribution.

1.1 Administrative Matters Before we get down to business, some administrative affairs are to be discussed:

1.1.1 Team Formation Work teams of 2 students should be made up. Please send as soon as possible a list with the team members and their respective Email (the one associated with GIT) to the following Email Addresses: [email protected] and [email protected].

1.1.2 Forum and Communication This project will be evaluated remotely, for this reason having a suitable online platform turns out to be very important to facilitate the communication between students and lecturers. In order to do so, we will adopt a "community" approach by means of a forum in the TecDigital platform. In the forum, all students can create new topics as questions arise. All discussions are public to all the course members so that any fellow student can answer and/or add more information to the proposed question. Please avoid sending project- related queries directly to the lecturers Email, as this prevents other students to beneﬁt from the answer. In the forum we all are a team! The only restriction is to not share

3 1.2

working source code in the forum, instead, we can create discussion and propose ideas and concepts that lead to the solution.

1.1.3 Plagiarism Any evidence of plagiarism will be thoroughly investigated. If it is corroborated, the team will receive a zero grade in the project and the incident will be communicated to the corresponding institutional authorities for further processing.

1.2 Development Environment Overview Embedded systems are ubiquitous; they are present everywhere in our daily-life activities. This project will introduce students in both: the setup of the development environment and some best design and implementation practices concerning embedded systems. A Raspberry Pi 4 will be used as target platform. As Figure1 shows, our development environment consists of two main components: • Host: Serves as main development platform. As it typically has more computing capacity than the target, all the design and implementation will be done here. • Target: This is the test and debug environment. Once a design phase has been concluded, the target platform is used as a test system. Here we validate correct functioning, proﬁle and benchmark our application. It is even possible to measure the energy and efﬁciency of our algorithms for energy-aware applications.

Figure 1: General view of the development environment

It is worth noting that there are 2 connections between the host and target: • UART: Thanks to its simplicity, this is usually the ﬁrst communication established with the target even at early stages of the board bring-up. It will serve to send commands to the bootloader and get logging information of the boot process.

4 2.1

• Ethernet: Because UART does not a allow a high transfer rate (normally up to a range of hundreds of kBps), we need a faster communication method to share large ﬁles in a reasonable amount of time. For this reason the TFTP protocol will be used over Ethernet to share the device tree and Linux kernel at boot time. In addition the Target will mount a File System present in the Host over the NTFS protocol.

1.2.1 Nomenclature As we will work with different command-line consoles during the setup of the development environment, it is necessary to specify where the commands are to be executed. Table1 summarizes the prompt symbols for the different command-line consoles.

Table 1: Prompt symbols for the multiple command-line consoles during the project.

Prompt symbol Description $ Host Linux PC => Target: U-Boot @ Target: Linux

2 Setting up the Host

Once we have reviewed the main hardware components of our set up, it is time to check that all required software is installed in the host to support our development. In order to carry out the installation process quickly, a virtual machine (VM) with some pre-installed software will be provided. Besides making the installation easier, a virtual machine isolates many potential problems from our native system and allows us all to have the same base conﬁguration to avoid version mismatch of software tools. Only this Virtual Machine will be supported during the project, due to time and resource limitations we cannot attend questions if you decide to go for a native installation. Please note the following concerning the virtual machine: • Any reference to host in this document will refer to this virtual machine. • Using the provided VM is highly recommended. However, if for any reason you prefer to use a native Ubuntu installation please make sure your system match: – Ubuntu 18.04.4 LTS – GIT 2.17.1 – Eclipse for C/C++ Developers

5 2.1

2.1 Downloading the VM and Configuring VirtualBox 1. Downloading the .zip: Please go to the following link and download the .zip file. 2. Extracting the files: As the download is available to everyone, the .zip file is protected with a password. Unpack all contents of the .zip file in a hard drive with at least 180GB of free space (it can also be an external HD). The provided virtual hard drive (.vdi file) allocates memory dynamically as needed up to 256GB. Password: MaesTria_TeC 3. Install VirtualBox: Download the VirtualBox 5.2 or greater (for instance 6.0 version) binaries matching your current operating system from here and install the application. 4. Create a new VM in your VirtualBox: Go to Machine → New (a) Name: Ubuntu18.04 → Type: Linux → Version Ubuntu (64-bit). (b) Select a RAM memory of at least 4GB. Warning: Never go beyond the green limit of VirtualBox. If you cannot assign 4GB of RAM to your virtual machine, you can try with less but it will be much slower and the compilation process might fail in Sections 3.6.1 and/or 3.7. The Virtual Machine is equipped with a 8GB swap memory in case your RAM gets full, but as it is physically located in the hard disk it will be much slower. (c) Select: Use an existing virtual hard disk file and browse the downloaded .vdi file (d) Click in the "settings" button to adjust the configuration of your new VM by taking into account the available physical resources of your PC. Give to your VM as much resources as you can without going into an invalid VirtualBox setting. (e) Select 4 CPUs or more if your PC allows it and enable PAE/NX if you have the option to do so. Note: In order to configure a shared folder between your VM and native system follow this link. 5. Start the Virtual Machine and Configure Credentials: Start the VM by clicking in the "Start" button and change the host password so that you can remember it. The current temporary password is: programIT.

6 2.2

If you want to change the password of your user, please open a terminal in the host (Ctrl + Alt + t) and type the following command followed by enter:

1 $ passwd 2

Note: If you are having problems at any time with your VM’s clipboard or shared folders consider reinserting VirtualBox Guest Additions as explained here.

2.2 Version Control: GIT Version controlling is a powerful concept to keep track of your project progress, keep your information safe and work in parallel along with your team fellows. For this reason, version control is widely used in every serious software project, including ours. We will be using GIT to keep track of our project. In order to meet the current industrial standards and to ease the parallel work of the students, all projects must be kept under the GIT version control. All documents associated with the project (source files, Eclipse files, documentation, etc) must be kept updated in one single repository. Access to the working repository must be granted to the lecturer in order to be able to clone its contents and evaluate the work done by the team. Refer to Section6 to know what deliverables you have to push to your GitHub repository. Some basic GIT commands will be discussed in a tutorial during the lecture for those students who have not yet experience with version controlling in software projects. Addi- tional information can be found in the provided links during the tutorial session. In the host (Virtual Machine), GIT 2.17.1 is already installed. Please install it if you are not using the provided VM. 1. Step 1: Create a GitHub Account. In order to use GIT we need a server to host our project. GitHub provides a server with free accounts for teams. Please go to GitHub and create a free account with your TEC-Email. During the first lectures of the course, a GIT tutorial with some best practices will be offered to get up to speed all those students who haven’t yet worked with this development tool. The correct utilization of GIT will be evaluated, see Section6 for more details. 2. Step 2: Configure your GIT Credentials. Prior to work with GIT it is necessary to enter the credentials of the above-created

7 2.3

account in the host. In order to do so, type the following commands in the host terminal substituting your information:

1 $ git conﬁg --global user.name"YOUR_FIRST_NAME YOUR_LAST_NAME" 2 $ git conﬁg --global user.email"YOUR_GIT_ASSOCIATED_EMAIL" 3

2.3 Configure Local Area Network with Static IP In this section we will carry out three main steps: 1. Physically connect the Ethernet cable between the RPI and the PC. 2. Create an Ethernet bridge adapter in the VM’s configuration. 3. Configure static IP addresses to all devices connected to this local area network. Table2 shows the static IP assignation of all three connected devices.

Device Static IP Netmask Gateway PC (laptop) 192.168.8.1 255.255.255.0 - Virtual Machine (Ubuntu host) 192.168.8.2 255.255.255.0 - Target (RPI 4) 192.168.8.3 255.255.255.0 -

Table 2: Static IP assignation of Virtual Machine (host), PC and Target

Connect the Ethernet cable between your RPI and your computer and perform the following steps: 1. Turn off your virtual machine. And in the VirtualBox application go to Machine → Settings → Network → Adapter 2. 2. Setup your bridge adapter as shown in Figure2. The adapter type, MAC Address and Network Name correspond to the parameters of your local computer. 3. According to your physical operating system, investigate how to set up the wired (Ethernet) connection of your PC (laptop) with the static IP settings shown in Table 2. 4. Setup a static IP in your Virtual Machine (Ubuntu host) as shown in Table2. To do so, you can follow the Ubuntu Desktop section of the following post. Note: leave the Gateway ﬁeld empty and automatic DNS.

8 2.4

5. Test the connection between your virtual machine and PC (laptop). You can just open a terminal in your virtual machine and type:

1 $ ping 192.168.8.1 2 # You should get something like this: 3 PING 192.168.8.1 (192.168.8.1) 56(84) bytes of data. 4 64 bytes from 192.168.8.1: icmp_seq=1 ttl=63 time=0.734 ms 5 64 bytes from 192.168.8.1: icmp_seq=2 ttl=63 time=0.845 ms 6 64 bytes from 192.168.8.1: icmp_seq=3 ttl=63 time=0.872 ms 7 ... 8

This proofs that there is an actual network connection between your physical PC and your virtual machine. We will use this connection as bridge only so that we can have a local area network between the host (virtual machine) and target (RPI). 6. When we get to Section 3.5.4 we will ﬁnish the Ethernet conﬁguration within the target itself.

Figure 2: Bridge adapter conﬁguration in VirtualBox

Note: For a larger setup on which you are simultaneously developing with multiple tar- gets, you would usually connect all your boards to a network switch and assign IPs dynamically. For our purposes a static IP assignation is enough.

2.4 Conﬁgure a TFTP Server for File Sharing The Trivial File Transfer Protocol (TFTP) is based on the User Datagram Protocol (on port 69) to implement a simple way to transfer ﬁles between devices connected within a

9 2.6

local area network. We will use TFTP in Section 3.5.4 to transfer the kernel image into the RPI’s RAM. Figure1 show the physical connection between the host and target. In the provided virtual machine there is a running TFTP server in the /tftpboot path. In order to check if the TFTP server is running correctly, type:

1 $ sudo service tftpd-hpa status 2 3 # You should see: 4 5 • tftpd-hpa.service - LSB: HPAs tftp server 6 Loaded: loaded (/etc/init.d/tftpd-hpa; generated) 7 Active: active (running) since Sun 2020-03-22 07:07:50 CST; 3s ago 8 . 9 . 10 .

Note: If you are experiencing problems or want to install the TFTP server again, please refer to [2,9]. Make sure you get your server running in /tftpboot.

2.5 Set NFS for Mounting a Remote File System Did you know you can boot a computer without a hard drive? The Network File System (NFS) allows the target to mount its file system remotely on the host’s hard drive. We will use this protocol to have a better access from the host to all files accessible to the target. This will allow us to modify files on the host and have immediate access to the updated files in the target. Figure1 shows the physical Ethernet connection to implement the NFS protocol on it. Just as for TFTP, there is a NFS server already running in the provided virtual machine. The address of the NFS server is /var/nfs/rootfs. To check that the server is working properly, type:

1 $ sudo service nfs-kernel-server status

Note: If you are experiencing problems or want to install the NFS server again, please refer to [3,6]. Just make sure you get your NFS working on the /var/nfs/rootfs path.

2.6 Installing the Eclipse IDE for C/C++ Developers We can use an IDE to aid the process of creating and cross-compiling our application. For that purpose we will use the Eclipse IDE.

10 3

Note: If you are using the provided Virtual Machine you can jump to the next section as this is already installed. 1. Install the Java Development Kit:

1 $ sudo apt-get install openjdk-11-jdk 2

2. Download the Eclipse IDE for C/C++ Developers from here, extract it and copy the folder called eclipse into the pat /opt/, for instance:

1 $ cd /folder_where_you_extracted_the_.zip 2 $ cd eclipse-cpp-2020-03-R-incubation-linux-gtk-x86_64/ 3 $ sudo cp -r eclipse/ /opt/ 4

3. Execute it:

1 $ /opt/eclipse/eclipse 2

Once it starts, choose the default path for your Eclipse Workspace: /home/project2/eclipse- workspace 4. Close it for now. We will conﬁgure it for cross compilation in Section 3.9.

3 Setting up our Embedded System and its Board Sup- port Package

In this section we will learn: • The boot process of an embedded device. • How to perform a board bring-up (configuration of the boot process from bootloader level) and creation of a Board Support Package (BSP): compile and configure the U- Boot bootloader, the Linux Kernel and a Yocto-based file system for the Raspberry Pi 4. • How to create a development environment to program, cross compile and test your application in a embedded system.

11 3.1

• The right way to use a Yocto layer to include your application into the compiled ﬁle system.

3.1 Using a Pre-compiled Image: Raspbian Raspbian is the Raspberry Pi Foundation’s official supported operating system (OS). It is a general purpose OS that allows to quickly boot your board and test the functionalities offered by its hardware. We will use Raspbian to: • Quickly check that the multiple RPI hardware is working fine before we start tailor- ing our OS and development environment. • Use the microSD partitions created by the Raspbian image and its proprietary first- stage bootloader to boot our custom Linux-based operating system on top of it. Getting Raspbian up and running is fairly simple, to do so, go through the following steps: 1. Download Raspbian Buster Lite in your VM from its official site. 2. Unpack your .zip file and remember the location of your unpacked .img file 3. Insert the microSD card into the USB Card Reader and connect it to your PC 4. Start your VM and click on Devices → USB → Card Reader in order to bridge your microSD with your VM.

Note: If you don’t see your USB card reader or you have problems connecting to it is it possible that you are trying to use an USB 3.0. To solve the problem follow the next steps: • Check the version of your VirtualBox by clicking on Help → About VirtualBox (it should be version 5.2.34) • Download the matching version of the Virtual Box Extension Pack from here: https://www.virtualbox.org/wiki/Download_Old_Builds_5_2 • Make sure the downloaded ﬁle is in your PC (not inside the VM but where you installed VirtualBox) • In VirtualBox go to: File → Preferences → Extensions → Adds New Package (icon on right side) and add your just downloaded extension pack • Turn off your VM and in VirtualBox go to Machine → Settings → USB and activate the USB 3.0 • Start your VM and try it again with your USB card reader

12 3.2

5. Now in order to ﬁnd your microSD type:

1 # In order to discover yourSD card look at the"NAME" column after typing: 2 $ lsblk -p 3 # It should be shown as/dev/sdX, whereX isa lower-case letter indicating the device 4 # The"SIZE" column must showa number that approximates the size of your microSD 5 # If you are not sure which device is your microSD, disconnect it and check with 6 #"lsblk-p" which device disappears from the list 7 8 # The"MOUNTPOINT" column shows where the partitions have been mounted(if they 9 # haven't been, it will be blank). Please unmount ALL partitions with: 10 $ umount /dev/sdX1 11 # Replace sdX1 with yourSD card's device name, and mounted partitions 12

6. What is left is to ﬂash your /dev/sdX microSD with the downloaded Raspbian image. To do so, type:

1 $ dd bs=4M if=raspbian.img of=/dev/sdX conv=fsync status=progress 2 # Where: 3 # raspbian.img= Path to your.img 4 #/dev/sdX= Your WHOLE microSD, not justa partition 5 # BE CAREFUL, IF YOU SELECT THE WRONG DEVICE YOU WILL BREAK YOUR VM 6

7. Insert your microSD with your brand-new Raspbian into your RPI4, connect the provided power supply and a HDMI monitor to the HDMI0. 8. The Raspbian Buster Lite is a minimal image without Desktop, you can login plug- ging a mouse and keyboard into your RPI and entering the following credentials: Username: pi Password: raspberry

Note: Don’t worry if you don’t have an extra mouse and keyboard, we will ﬁnd a workaround to command the RPI from our host during development. Just make sure Raspbian booted until the point you have to login.

3.2 Establishing UART Connection with the RPI Even tough the Universal Asynchronous Receiver-Transmitter (UART) protocol is slow in comparison to modern USB, QSPI, SATA, etc. protocols, its simplicity allows every

13 3.2

small processor to provide valuable debug information over UART even at early boot stages after power on. We need to establish communication with the RPI even when Linux (and all its drivers) hasn’t yet started, how to do that? This is where the benefits of simple communication protocols pay off, we will use UART at this stage to configure U-Boot and get kernel messages while loading the kernel and file system. Please do the following to get UART up and running: 1. Make sure your RPI is turned off and your UART to USB cable is not connected into the USB. Follow the pinout depicted in Figure3 and connect GND, TXD and RXD pins. Note: VCC MUST not be connected. 2. Attention: Make sure the GND pin is connected to the GPIO 6 according to Figure 3a and 3b. If you connect it one pin to the left you will cause a SHORT CIRCUIT!!!. 3. Connect the UART to USB cable into an USB 2.0 of your PC. If you need to use a USB 3.0 you must follow the steps described in the note of Step4 if you have not done it yet. 4. Start your VM and go to Devices → USB and select your USB to UART converter. 5. Open a Terminal in your VM and type:

1 $ dmesg | grep tty 2 # You should geta kernel message like this one: 3 [ 308.129746] usb 1-2: pl2303 converter now attached to ttyUSB0 4 # That means our USB to UART converter is connected to the ttyUSB0 port 5

6. We will use minicom to communicate over UART with the RPI. To do so start minicom with the following options (switches):

1 $ sudo minicom -w -b 115200 -c on -s 2 # Where: 3 #-w: Turns linewrap off 4 #-b: Set the baudrate of your RPI(default 115200) 5 #-D: Enter the device you found in the last step as/dev/your_port 6 #-c on: ANSI style color usage on 7 #-s: Enters setup mode 8 # Once you enter the setup mode, go to"Serial port setup" and: Press"A" to conﬁgure your 9 # port to match the one you found in the last step. 10 11 # Then, exit the conﬁguration menu. You should see the following: 12 13 Welcome to minicom 2.7.1

14 3.2

14 15 OPTIONS: I18n 16 Compiled on Aug 13 2017, 15:25:34. 17 Port /dev/ttyUSB0, 06:15:12 18 19 Press CTRL-A Z for help on special keys 20 21 # Put especial attention to the the text in orange 22 23 # To finish the configuration: press"CTRL-A"+"T" to configure the terminal. There press press"F" and entera Character tx delay of1 ms. 24

Note: to leave minicom type: Ctrl + a and then x. 7. Plug in the microSD with Raspbian into the card reader and make it visible to your VM (Devices → USB → Card Reader). In the boot partition you will find a config.txt file. Open it with a text editor and add the following lines at the end:

1 enable_uart=1 2

This will let the bootloader know that we want to use the UART port in the GPIO header. 8. Additionally make sure that the keyword "quiet" is not in the cmdline.txt, otherwise all debug messages will be omitted during the boot process. 9. Plug your microSD back into the RPI and turn it on. After multiple kernel messages you should get the following (Remember, user: pi and password raspberry):

1 Raspbian GNU/Linux 10 raspberrypi ttyS0 2 3 raspberrypi login: pi 4 Password: raspberry 5 . 6 . 7 . 8 pi@raspberrypi:~$ 9

After this process you are able to command your RPI over the terminal and perform any non-GUI tasks on your system.

15 3.3

(a) Pinout of the RPI’s 40-pin header [10] (b) UART physical connection

Figure 3: Establishing an UART connection with our RPI 4

After following the process described above we are sure that we can communicate with the target over UART, now we will understand the general booting process of an embedded system and analyse the speciﬁc case of the Raspberry Pi 4. After that we will craft our own system.

3.3 How Does an Embedded Device Boot? Have you ever asked yourself what happens right after pressing the start button of your smartphone?. Or how do embedded developers boot the first time their brand-new hardware? In the following sections we will attempt to address these questions. It turns out that if you understand the booting process of your Raspberry Pi, you can port this process from a high level perspective to pretty much any modern embedded system you can find out there. So make sure you understand the sequence and function of hardware and software components involved during the process. Figure4 shows an overview of the generic boot process of an industrial embedded system. In the following sections a short description of every step and software component will be drafted and a detailed description to compile and configure each of them for the RPI will be offered.

16 3.3

Figure 4: General boot process of an embedded system along with involved hardware and software components.

17 3.3.1

3.3.1 The Specific Case of the Raspberry Pi 4 As commented above, Figure4 shows the general case for an industrial embedded system. The specific case of the Raspberry Pi 4 has a main difference in comparison to the typical industrial case: There is no available information such as datasheet or reference manual describing the internal structure of the Broadcom BCM2711 SoC. This makes it impossible to create a Bootloader that interacts with the processor at early boot stages when its internal units (MMU, L1 cache, DMA, etc) are not yet initialized. The Broadcom BCM2711 SoC is designed for mass production in the consumer sector, which means it is not intended for engineers outside Broadcom to tailor their bootloader according to their needs, but to buy a custom-made bootloader from Broadcom itself. This usually speeds time-to-market at a cost of higher initial investment and technical dependency to the SoC manufacturer. The Broadcom BCM2711 has therefore originally the following boot process [8,4]: • First Stage Bootloader: – A first stage BootROM resides in the on-chip ROM of the processor. It is executed in the GPU (while the ARM cores are in reset). – The BootROM loads the main bootloader from EEPROM. • Second Stage Bootloader: – Bootloader checks it’s inbuilt BOOT_ORDER configuration item to determine which interface it should use to boot (microSD, Network or USB). – It looks in the selected interface for a start.elf binary and loads it. – The start.elf binary looks into a config.txt file for information to continue with the boot process, for instance how to find the kernel. For our purposes this start up scheme is not the most suitable for the following reasons: • The BootROM, bootloader and start.elf binaries are all proprietary software. Which means that there is no access to its source code, we only have the binaries. This makes it impossible to modify them in order to control the boot process according to our requirements. • So far the official network boot of the Raspberry Pi 4 is still under test. Only a beta (initial) version has been released (see RPI-EEPROM). If we use it at this stage we can expect bugs in the boot process and therefore lost of our time.

18 3.5

But, if we do not have access to the source code of the original RPI bootloader, how can we get a stable network boot to design a flexible development environment? To do so we are going to do as follows: • We are going to use the first and second stage bootloader of the RPI as our first stage bootloader. • To do so, we are going to describe in the Section 3.5 how to load U-Boot from the RPI’s second stage bootloader by modifying the config.txt file. • As the LPDDR4 RAM of the RPI will be initialized by the proprietary first stage bootloader, there is no need to use the SPL, so we will not use it. • Once U-Boot is running, we have all the flexibility of a mature and widely-used open source bootloader to perform a network boot. In the following chapters we describe the configuration and compilation of the elements present in Figure4.

3.4 First Stage Bootloader: BootROM As shown in Figure4, the ﬁrst instructions to be executed in the SoC after power up are the BootROM instructions. The BootROM is usually programmed into the on-chip ROM of the processor during the manufacturing process. It either reads the processor fuses or external bootstraps resistors attached to dedicated processor pins to determine the storage location of the second stage bootloader. Normally it is proprietary software produced by the processor manufacturer itself. For this reason we cannot modify it. In our case we will just use the provided BootROM by Broadcom. Please explain in your report what are the processor fuses and bootstrap resistors and what exactly of the boot process can be controlled with them. Mention a processor (architecture and manufacturer) for which you can control the boot process using both methods.

3.5 Second Stage Bootloader: Das U-Boot Das U-Boot or just U-Boot is an open source universal bootloader for multiple embedded system architectures. It was initially created by Wolfgang Denk (now managing director of DENX Software Engineering) by taking some Linux Kernel code as basis for the ﬁrst drivers and device initialization. As any open-source project, contributions are made in U-Boot by any developer able to follow the coding style and best practices established

19 3.5.1

by the project maintainers. U-Boot support for the RPI4 has been contributed by Andrei Gherzan [5]. As shown in Figure4, U-Boot is the main component of the second stage bootloader. Moreover, as discussed in 3.3.1, we do not need to use a SPL as the low-power DDR4 RAM of the target will be already initialized by the manufacturer’s first stage bootloader. We are going to compile U-Boot, thus, the first step is to configure our cross-compilation toolchain.

3.5.1 Toolchain for Cross-Compilation We are going to run our U-Boot bootloader in the Raspberry Pi, remember? This might be obvious to most of us, however we shall not forget what it implies: we need a compiler for a 64-bit ARM architecture instead of the default one installed in our machines (probably a x86 64-bit compiler). Compiling code to target a different architecture than the one on which the code is compiled is called cross-compilation. In our case, one option is to use the open source 64-bit ARM compiler provided by Linaro. We have already installed the latest Linaro compiler at /opt/linaro/ in the virtual machine for you. If for any reason you want to install it again, please go to https://www.linaro.org/downloads/ and download the gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu.tar.xz from the aarch64-linux-gnu binaries. Then: 1. Unpack into /opt/linaro/ with sudo privileges as follows:

1 # If the/opt/linaro folder does not exist: 2 $ mkdir -p /opt/linaro 3 # Unpack your downloaded ﬁle: 4 $ sudo tar -xf your-ﬁle.tar.xz -C /opt/linaro

2. Open the ﬁle ~/.bashrc with your favorite text editor and add the following alias at the end of the document (it is also already added in the provided virtual machine). For instance, type in the terminal:

1 # vim isa wide-used editor in the embedded world and is supported by most systems, 2 #for more information see https://www.vim.org/ 3 $ sudo vim ~/.bashrc

3. Add the line at the end:

1 alias crosscompiler='export KERNEL=kernel8;export ARCH=arm64;export CROSS_COMPILE=/ opt/linaro/gcc-linaro-7.5.0-2019.12-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-'

20 3.5.2

4. And ﬁnally source the ~/.bashrc:

1 $ source ~/.bashrc

3.5.2 Cross-compiling U-Boot Have we already mentioned that U-Boot is an open source project? It means we can have free access to the bleeding-edge version of the code, which is important in our case: support for the Raspberry Pi has been added during the last months. Let us download the mainline repository:

1 # Go to the folder where you want to clone the sources(note that the path/home is equivalent to~ 2 $ cd /home/project2/git 3 # If the folder does not exist, create it with:$ mkdir-p/your/folder 4 # Now use git to clone the mainline repository 5 $ git clone https://github.com/u-boot/u-boot.git 6 # Enter the folder containing the sources you just cloned 7 $ cd u-boot 8 # And checkouta stable version of the code supporting our target 9 $ git checkout v2020.04-rc3

We will need a couple of packages installed in the host to support the U-Boot compilation (already installed in the VM):

1 # Answer with"Y" during the installation process 2 $ sudo apt install u-boot-tools bison bc make ﬂex libssl-dev ncurses-*

As mentioned before, U-Boot has recently added support for the Raspberry Pi 4, it means that there is code supporting the boot process of the Broadcom BCM2711 processor and drivers that allow initialization and usage of the peripherals present in the Raspberry Pi 4 (RAM, micro-SD, Ethernet, etc). As U-Boot is an universal bootloader, it has drivers to support a wide range of processors and boards. We have to select those specific drivers that are compatible with our target. Fortunately U-Boot supports the Raspberry Pi 4 out-of-the-box, which means that there is a defconf file which stands for default configuration and selects specifically the drivers necessary for the RPI from the whole pool of drivers present in the U-Boot sources. Most

21 3.5.3

widely-used platforms have such a file in U-Boot. In order to configure U-Boot with its default RPI4 configuration type:

1 # You can see the Raspberry Pi4's default configuration by typing: 2 $ cat /your_u-boot_location/configs/rpi_4_defconfig 3 # We have to use the alias we just created to let the host know that we want to use 4 # the Linaro cross compiler instead of the x86 local gcc compiler 5 $ crosscompiler 6 # Now we will configureU-Boot with the default Raspberry Pi4 configuration 7 $ make rpi_4_defconfig 8 # By doing so the/configs/rpi_4_defconfig will be copied to.config 9 10 #U-Boot hasa kconfig menu to further configure it"graphically" 11 $ make menuconfig 12 # This is typically used to add or remove drivers fora specific device present in your 13 # custom boarde.g. booting from SPI or QSPI FLASH, SATA, etc. 14 # Exit the menu without doing any changes. We will come back to it in the Kernel configuration.

Now that we have conﬁgured U-Boot with the default conﬁguration for our platform, the next step is to cross compile it.

1 # Before compiling, make sure you are using the Linaro compiler by checking 2 $ echo ${CROSS_COMPILE} 3 # If the last command prompts the right location, issue the next command to compileU-Boot 4 # with all processor cores available in your machine, otherwise ﬁrst type:$ crosscompiler 5 $ make -j $(nproc) 6 # Note that"nproc" provides the number of cores in your computer

Once the compilation process has succeed, type the following command to identify your just-created U-Boot bootloader:

1 $ ls -lh 2 # You should havea binary calledu-boot.bin witha size of about 496K as follows: 3 -rwxr-xr-x 1 project2 project2 496K mar 1 09:44 u-boot.bin 4 # That is our binary!

Note: For more general information regarding U-Boot please visit the ofﬁcial Wiki at https://www.denx.de/wiki/U-Boot.

3.5.3 Conﬁguring our Target to Load U-Boot In this section we will use the boot mechanism of the Broadcom SoC to load U-Boot after executing the ﬁrst stage bootloader. To understand the process described here make sure

22 3.5.3

you read Sections 3.3 and 3.3.1. Additionally, make sure you followed steps described in Section 3.1 to have a working miscroSD with Raspbian and steps from Section 3.2 to have a working UART connection with your target. Once you have followed steps described above: 1. Plug the microSD card with Raspbian into your USB card reader and make it visible to your virtual machine as described in step 4 in Section 3.1. 2. Go to the boot partition of the microSD and copy your U-Boot binary (u-boot.bin) into that partition. 3. In the boot partition you should find a config.txt file. The first stage bootloader looks into the config.txt the keyword kernel= to determine the next binary to load (normally the Kernel if we use do not use U-Boot). 4. Add the following lines to the config.txt file [8]:

1 # As the Raspberry Pi4 hasa 64-bit architecture, we have to activate the 64-bit mode of the bootloader. 2 arm_control=0x200 3 4 # UseU-Boot 5 kernel=u-boot.bin 6 7 # Select the following Device Tree overlay to disable the Bluetooth in order to free the ttyAMA0 serial(needed to run the Kernel in the next chapter): 8 9 dtoverlay=disable-bt 10 11 # Additionally remove any other entry containing the keyword kernel= 12

5. Save the changes and plug back the microSD into the RPI. 6. Turn the RPI on and observe the UART communication in the VM (minicom). After some seconds you should see the following message:

1 U-Boot 2020.04-rc3 (Mar 01 2020 - 09:44:56 -0600) 2 3 DRAM: 3.9 GiB 4 RPI 4 Model B (0xc03112) 5 MMC: emmc2@7e340000: 0, mmcnr@7e300000: 1 6 Loading Environment from FAT... *** Warning - bad CRC, using default environment 7

23 3.5.4

8 In: serial 9 Out: serial 10 Err: serial 11 Net: eth0: genet@7d580000 12 Hit any key to stop autoboot: 0 13 U-Boot> 14

7. Hit any key to stop the auto boot process. 8. Congratulations! Now you are able to get U-Boot (one of the most used bootloaders in the world) working in your embedded devices! 9. Answer the next questions in your written Report: (a) What is a Device Tree? Why was it introduced and why is it so important? (b) What is a Device Tree Overlay? What can be controlled with it? Now we will take a look at Figure4 again. U-Boot is the third component in the execution process at start up. Its main task it to initialize the necessary peripherals and load the Linux Kernel. We will learn how to conﬁgure U-Boot to load the kernel from the host by using the TFTP connection we established in the Section 2.4.

3.5.4 Conﬁguring Static IP in U-Boot As you experienced in the last section, U-Boot provides a command line to allow con- ﬁguration of the boot process and bare-metal testing of the target’s hardware. To get an overview of all the commands available in the U-Boot version we just compiled, type in the minicom terminal:

1 => help 2 # You can print theU-Boot Environment with the"printenv" command. 3 # We will modify and set these environment variables and commands to meet our needs.

Now we will ﬁnish in U-Boot the local area network conﬁguration we started in Section 2.3. Make sure the Ethernet cable is connected and follow next steps: 1. According to Table2, set the IP address and Netmask of the Target as:

1 # Set the RPI staticIP: 2 => setenv ipaddr 192.168.8.3

24 3.6

3 4 # Set the static server(host) ip: 5 => setenv serverip 192.168.8.2 6 7 # Set the netmask: 8 => setenv netmask 255.255.255.0 9 10 # Make sure everything is set as desired(printenv= pri): 11 => pri ipaddr serverip netmask 12 13 # Save your changes into theU-Boot environment stored in your microSD: 14 => saveenv 15

2. Now we will check that the local area connection between host and target is up and running:

1 # First, you have to reset your board to make sure the new staticIP is used. 2 => reset 3 4 # OnceU-Boot has started again, check your connection with the host: 5 => ping 192.168.8.2 6 7 # If everything was set properly, you should get an equivalent message to this: 8 Using genet@7d580000 device 9 host 192.168.8.2 is alive 10

As Figure4 suggests, the next step after the Bootloader is to load the Linux Kernel. In the next section we will compile our kernel from sources and upload it to the target over the TFTP connectio set in Section 2.4.

3.6 The Linux Kernel The Linux Kernel origin dates back to 1991 when Linus Torvalds conceived it to run its personal computer. The Linux Kernel is a monolithic, Unix-like and most important: open-source OS kernel. Since its creation it has been ported to a wide variety of computing systems. Besides the desktop computing market, it dominates almost every other segment of computing, including: smartphones, smartwatches, tablets, servers and even supercomputers. As Figure5 shows, the Linux Kernel provides an abstraction layer between a wide range of hardware components (drivers) and the user space applications. Additionally, it pro-

25 3.6.1

vides a scheduler for both: processes and network tasks; it assigns time slots to all queued processes according to a selected scheduling criteria. Additionally, it provides security mechanisms to prevent user spacer applications (or viruses) to crash the system. You can ﬁnd the ofﬁcial (mainstream) Linux sources from the following repository: https: //github.com/torvalds/linux.git.

Figure 5: The Linux Kernel: an universal hardware abstraction layer [11]

In the next section we will learn how to compile the Linux kernel for a speciﬁc embedded system. As we will see, its compilation procedure is very similar to the U-Boot’s compilation process.

3.6.1 Cross-compiling the Linux Kernel Open a terminal in your Ubuntu Host and: 1. Go to the folder where you storage your sources:

1 # For instance: 2 $ cd ~/git 3

26 3.6.1

2. Clone the Linux Kernel 5.6 sources for the Raspberry Pi and check out the right branch (branch=rpi-5.6.y):

1 $ git clone --branch rpi-5.6.y https://github.com/raspberrypi/linux 2

3. Export the necessary cross-compiler variables before start the compilation of the Kernel:

1 # Source the ~/.bashrc if you have not done it yet 2 $ source ~/.bashrc 3 # And our alias again: 4 $ crosscompiler 5

4. Get into the folder you just cloned, check that the sources are present and create a directory to keep the result of the compilation:

1 # Enter the linux folder 2 $ cd linux/ 3 # List the ﬁles inside the folder and show human-readable information 4 $ ls -lh 5 # Createa new directory to storage the results of the compilation 6 $ mkdir our_binaries 7

5. Conﬁgure the Kernel with the default settings for the Broadcom BCM2711 processor and Raspberry Pi 4:

1 # Look for the right configuration for our target: 2 $ ls arch/arm64/configs/ 3 bcm2711_defconfig bcmrpi3_defconfig defconfig 4 5 # From this list, select the right RPI4 configuration and prepare folder we just created for compilation: 6 $ make O=our_binaries bcm2711_defconfig 7 8 # To geta quick overview of the selected Kernel configuration and packages: 9 $ cat our_binaries/.config 10 11 # We will makea change to our Kernel: Remove the. Type: 12 $ make O=our_binaries menuconfig 13

27 3.6.1

14 # Go to the Device Drivers section and deselect the"MMC/SD/SDIO card support"(by pressing the space key once). Then save and exit. 15

Why disabling the MMC/SD/SDIO driver?: (a) As we will be booting from the network, there is no need for SD card support in the kernel. (b) There are sporadic issues with the Raspberry Pi and some SD cards (see this), so we will avoid some problems in the future. (c) If you need SD support in the future, you can use the Kernel we will generate with Yocto in Section 3.7. 6. Compile the Linux Kernel:

1 # We will compile the Linux Kernel using several parallel threads, however this might takea couple of minutes. Meanwhile yo can geta cup of coffee:) 2 $ make O=our_binaries -j $(nproc) 3 4 # Once the compilation process has succeed, you should get: 5 . 6 . 7 . 8 LD [M] sound/usb/snd-usb-audio.ko 9 LD [M] sound/usb/snd-usbmidi-lib.ko 10 make[1]: Leaving directory '/home/project2/git/test/linux/our_binaries' 11

7. Identify the Kernel binary and copy it to the microSD card:

1 # The binaries we just created can be found under: 2 $ ls our_binaries/arch/arm64/boot -lh 3 drwxr-xr-x 30 project2 project2 4,0K mar 23 02:15 dts 4 -rwxr-xr-x 1 project2 project2 14M mar 23 02:30 Image 5 -rw-r--r-- 1 project2 project2 5,9M mar 23 02:30 Image.gz 6 -rw-r--r-- 1 project2 project2 14M mar 24 17:46 uImage 7 8 # The Image file contains the uncompressed (14MB) 64-bit Linux Kernel binaries! 9 10 # Copy the Image file into the TFTP directory. We will later useU-Boot to transfer this file ito the target's RAM and execute it: 11 12 $ sudo cp our_binaries/arch/arm64/boot/Image /tftpboot/

28 3.6.2

3.6.2 Loading Kernel from TFTP We just compiled our 64-bit Linux Kernel and copied the resulting binary into /tftpboot/, which is the folder we configured in Section 2.4 to share with our TFTP server. The following steps illustrate how to let U-Boot find this file, copy it into a specific RAM address, share some arguments with the kernel and pass the CPU control to Linux. 1. We want to copy the Kernel somewhere in the RAM, but, where would you put it? To answer this question please go to the minicom U-Boot console and type "bdinfo" to get general information about our board:

1 => bdinfo 2 arch_number = 0x0000000000000000 3 boot_params = 0x0000000000000100 4 DRAM bank = 0x0000000000000000 5 -> start = 0x0000000000000000 6 -> size = 0x000000003b400000 7 DRAM bank = 0x0000000000000001 8 -> start = 0x0000000040000000 9 -> size = 0x00000000bc000000 10 baudrate = 115200 bps 11 TLB addr = 0x000000003b3f0000 12 relocaddr = 0x000000003b359000 13 reloc off = 0x000000003b2d9000 14 irq_sp = 0x000000003af499c0 15 sp start = 0x000000003af499c0 16 FB base = 0x0000000000000000 17 Early malloc usage: 6e0 / 2000 18 fdt_blob = 0x000000003af499d0 19

Note that: (a) We have two banks of RAM, each with about 2GB of space. The ﬁrst starts at address 0x00000000 while the second at 0x40000000. (b) The load address and entry point of the Kernel is the RAM address 0x8000. This means that the bootloader will put the Kernel there while booting. As we noticed, the uncompressed Kernel has a size of about 14MB, therefore we have to leave enough space after the address 0x8000 so that the do not have a RAM overlap when the Kernel gets copied there.

29 3.6.2

(c) As we have 4GB of RAM we are not in a critical situation, however, you can typically ﬁnd Linux-based systems with as low as 64MB of RAM. There you will have to meditate about it a bit further. (d) In addition to the recommendations discussed above, investigate the default location in RAM of the device tree at boot time for the RPI4 and look for information about reserved RAM addresses that U-Boot uses to operate. Describe your ﬁndings in the written report and select your kernel_addr_r accordingly. 2. Considering the information from the previous point, set some U-Boot environment variables as follows:

1 # Set the kernel Address selected above: 2 => setenv kernel_addr_r 3 # LetU-Boot know the name of our Linux Image(the one we just compiled): 4 => setenv kernel Image 5

3. Then, create an U-Boot command to load the Kernel from TFTP and start it:

1 # Be specially careful to keep all spaces of the commands! 2 => setenv netboot 'tftp ${kernel_addr_r} ${kernel} && booti ${kernel_addr_r} - ${ fdtcontroladdr}' 3 4 # Additionally, set: 5 => setenv bootcmd 'run netboot' 6 # So that every timeU-Boot starts, out command gets executed 7

The netboot command has two parts, which are separated by &&: (a) The first one loads the ${kernel} = Image into the address ${kernel_addr_r} over TFTP. (b) The second one boots the Kernel located in ${kernel_addr_r} without an init RAM disk (that is why there is a dash -) and provides the Kernel with the hardware information from the device tree located at the address that you have to insvestigate. The device tree was placed there by the first-stage bootloader. Note: The && key executes the second part of our netboot command only if the first part succeeded. 4. The last step to let the kernel control the CPU is to pass the bootargs arguments to the kernel. These arguments contain further information about peripherals, where

30 3.6.2

to ﬁnd the File System, etc. For now we will only let the kernel know that we want to use the ttyAMA0 serial interface (the one connected to minicom).

1 => setenv bootargs 'console=ttyAMA0' 2 3 # Save all environment variables in the microSD so that you don't have to type it all again: 4 => saveenv 5

5. We are ready to start the Kernel!

1 # The"boot" command executes the"bootcmd" command we set before, which runs our" netboot" command: 2 => boot 3 . 4 . 5 [ 2.961187] Kernel Offset: disabled 6 [ 2.964672] CPU features: 0x10002,20006000 7 [ 2.968762] Memory Limit: none 8 [ 2.971825] ---[ end Kernel panic - not syncing: VFS: Unable to mou 9 nt root fs on unknown-block(0,0) ]--- 10

Ooops! Did you get a Kernel panic message as well? Well it turns out that the next step in the boot process is to look for a suitable File System. The Kernel tries to find a File System himself, but as we have not generated one yet, it is not able to find it. In the next chapter we will compile a File System ourselves and let the Kernel know where it is located in order to finish the boot process successfully. Note: If you are having problems to load the Kernel over TFTP try: (a) Restarting your TFTP service in the VM:

1 $ sudo service tftpd-hpa restart 2 3 # And then check it is running: 4 $ sudo service tftpd-hpa status 5

(b) If the problem persists check all static IPs of2 and make sure you can ping your VM from your Raspberry:

31 3.7.1

1 => ping 192.168.8.2 2

If the ping doesn’t work you have a deeper network problem or conﬁguration issue you might consider restarting one by one the devices involved in our setup and check again after each reset in this order: Raspberry Pi, VM and at last your physical PC. (c) Investigate in Internet how to get working a static local area network connection (and test it again with ping).

3.7 YOCTO, File System and the Toolchain We are going to use Yocto to generate our custom File System. Fortunately, unlike with U-Boot and the Kernel, there are both: ofﬁcial support and sufﬁcient updated information on the web regarding Yocto and the Raspberry Pi 4. For further information regarding the Yocto Project, the best place to start looking at is the Yocto Project Mega-Manual and documentation. You have to consider that the initial Yocto learn curve is steep, but it is worth if you want to dig deeper into the embedded world.

3.7.1 Generating a File System with Yocto In order to generate a custom ﬁle system, please consider these steps: 1. Make sure your virtual hard disk is in a physical storage with at least 180GB of free space. One characteristic of Yocto, is that it is intended to be independent from pre- compiled binaries. Nearly everything will be generated from sources, even the cross compiler, which will be paradoxically compiled locally and then used to compile all required modules for the target. This is a charming solution to make the ﬁle system generation self-contained, however it comes at a cost of requiring high resources in the local machine. 2. Open a Terminal in the virtual Host machine 3. If you are not using the provided virtual machine, install the following packages (otherwise jump to the next step):

1 $ sudo apt-get install gawk wget git-core diffstat unzip texinfo gcc-multilib build-essential chrpath socat libsdl1.2-dev xterm python 2

32 3.7.1

4. Cloning the Yocto sources:

1 $ cd~ 2 # You can also do:"$ cd ~/project2" just remember where is your Yocto folder 3 $ mkdir Yocto 4 $ cd Yocto 5 $ git clone -b zeus http://git.yoctoproject.org/git/poky 6

5. Source the following script to export all necessary environment variables required for compilation:

1 $ cd poky 2 $ source oe-init-build-env 3

6. Download the Raspberry Pi Yocto layer:

1 $ cd ~/Yocto 2 $ git clone -b zeus git://git.yoctoproject.org/meta-raspberrypi 3

7. Add the Raspberry Pi Yocto layer to the bblayers.conf ﬁle located at ~/Yocto/poky/build/- conf. It should look like this:

1 # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf 2 # changes incompatibly 3 POKY_BBLAYERS_CONF_VERSION ="2" 4 5 BBPATH ="${TOPDIR}" 6 BBFILES ?="" 7 8 BBLAYERS ?="\ 9 /home/project2/Yocto/poky/meta\ 10 /home/project2/Yocto/poky/meta-poky\ 11 /home/project2/Yocto/poky/meta-yocto-bsp\ 12 /home/project2/Yocto/meta-raspberrypi\ 13 " 14

33 3.7.1

8. Add/change the following lines in the ~/Yocto/poky/build/conf/local.conf:

1 # We want to builda 64-bit File System for the Raspberry Pi4 2 MACHINE ??="raspberrypi4-64" 3 4 # debug-tweaks allows us to login as superuser withouta password and the following packages are necessary to create our cross-compilatio toolchain: tools-debug eclipse-debug tools-sdk ssh-server-openssh 5 EXTRA_IMAGE_FEATURES ?=" debug-tweaks tools-debug eclipse-debug tools-sdk ssh- server-openssh" 6 7 # Allow compilation of proprietary software in our file system(nota problem for education purposes) 8 LICENSE_FLAGS_WHITELIST ="commercial" 9 10 # In order to make things compile much faster: 11 # Uncomment and set to allow bitbake to execute multiple tasks at once. 12 # Fora quadcore,BB_NUMBER_THREADS= "4", PARALLEL_MAKE="-j 4" would 13 # be appropriate. 14 BB_NUMBER_THREADS ="4" 15 # Also, make can be passed flags so it run parallel threadse.g.: 16 PARALLEL_MAKE ="-j4" 17 18 # Additionally, we have to configure our Serial console: 19 SERIAL_CONSOLES ="115200;ttyAMA0" 20

9. Once you have concluded the Yocto conﬁguration of the previous point, please type: Attention: Depending on the resources you gave to your virtual machine, this process will take several hours. During this time you might need more than just a coffee to keep you busy. Meanwhile, you can already start conﬁguring your IDE and your research to design your audio codec. But consider that your computer or virtual machine might not be responsive during the compilation process.

1 # Go into Poky 2 $ cd ~/Yocto/poky 3 4 # Make sure you are in the right environment: 5 $ source oe-init-build-env 6 7 # Clean the cache and remove output and downloaded ﬁles for the desired image(important for rebuilding aftera conﬁguration change)

34 3.7.1

8 $ bitbake -c cleanall core-image-base 9 10 # Start the Yocto compilation process of your File System: 11 $ bitbake core-image-base 12 13 # List results of Yocto compilation: 14 $ ls -lh tmp/deploy/images/raspberrypi4-64 | grep .tar 15 16 -rw-r--r-- 2 project2 project2 113M abr 16 12:50 core-image-base-raspberrypi4-64- 20200416184251.rootfs.tar.bz2 17 18 # Createa script to support the cross compilation of the application. 19 $ bitbake core-image-base -c populate_sdk 20 21 # Check the scrip we just created to install the cross-toolchain: 22 $ ls -lh tmp/deploy/sdk/ | grep .sh 23 24 -rwxr-xr-x 2 project2 project2 503M abr 16 13:28 poky-glibc-x86_64-core-image-base-aarch64 -raspberrypi4-64-toolchain-3.0.2.sh 25

Yeah! We have created a Linux file system of about 113MB (compressed) which fully supports the 64-bit Raspberry Pi 4 and the development of applications and debug tools. A file system of this size is not commonly found in general purpose computers, right? One of the reasons for that is that this file system does not contain yet a desktop (no GUI at all): it is a console-only image that fully supports our device. There you will find as well a Linux Kernel Image file, device tree blob, modules (drivers) and you can even generate an U-Boot binary. Note: If Yocto can create U-Boot and Kernel binaries for us, why did we compile them "manually" before? Some of the reasons are: (a) Modularity: If you are designing a Yocto Board Support Package (BSP) for a brand new custom board (with U-Boot, Kernel, File System, etc), you normally divide the BSP "problem" in smaller ones, just as we did. You first make sure your U-Boot and Kernel are working with the configuration you selected and then you integrate them all into your Yocto layer and File System. (b) Understanding: In order to understand how Yocto works, you first need to know what components are involved in the boot process and recognize their function. (c) Debugging: If you are facing problems to bring your system up, it normally

35 3.7.2

helps to isolate your problem. At the early design stages of a new embedded system you want to have every component isolated to each other so that you can ﬁnd and solve problems easily. (d) Once you have done all that, you can ﬁne-tune your system within Yocto and deliver a ready-to-use Yocto layer to the high-level “userland” guys of your team. Note: If you are interested in Yocto, you can have a deeper look at it in the lecture: "Sistemas Empotrados de Alto Desempeño".

3.7.2 Deploying a Remote Network File System (NFS) Do you remember the NFS server we created in Section 2.5? The moment to use it has come! 1. Go to your Yocto folder and extract the core-image-base-raspberrypi4-64-20200330223917.rootfs.tar.bz2 ﬁle into our /var/nfs/rootfs/

1 # Get into your main Yocto folder 2 cd ~/Yocto 3 4 # Extract the File System 5 sudo tar -xf poky/build/tmp/deploy/images/raspberrypi4-64/here_your.tar.bz2 -C /var/nfs/rootfs/ 6

Make sure to select the “here_your.tar.bz2” with the compressed core-image-base- raspberrypi4-64 ﬁle generated in the last section. 2. Verify that your new Linux File System has been extracted in the /var/nfs/rootfs/ directory:

1 $ ls /var/nfs/rootfs/ -lh 2 drwxr-xr-x 2 root root 4,0K mar 30 21:23 bin 3 drwxr-xr-x 2 root root 4,0K mar 30 17:10 boot 4 drwxr-xr-x 2 root root 4,0K mar 30 17:10 dev 5 drwxr-xr-x 25 root root 4,0K mar 30 21:24 etc 6 drwxr-xr-x 3 root root 4,0K mar 30 21:22 home 7 drwxr-xr-x 7 root root 4,0K mar 30 20:20 lib 8 drwxr-xr-x 2 root root 4,0K mar 30 17:10 media 9 drwxr-xr-x 2 root root 4,0K mar 30 17:10 mnt 10 dr-xr-xr-x 2 root root 4,0K mar 30 17:10 proc 11 drwxr-xr-x 2 root root 4,0K mar 30 17:10 run 12 drwxr-xr-x 2 root root 4,0K mar 30 21:23 sbin 13 dr-xr-xr-x 2 root root 4,0K mar 30 17:10 sys

36 3.7.3

14 drwxrwxrwt 2 root root 4,0K mar 30 17:10 tmp 15 drwxr-xr-x 10 root root 4,0K mar 30 20:08 usr 16 drwxr-xr-x 8 root root 4,0K mar 30 18:40 var 17 # That looks likea Linux root directory! 18

3. You can verify all compiled modules in your ﬁle system by looking into the manifest ﬁle:

1 # To check which version of the audio ﬁle we have, type: 2 $ cat poky/build/tmp/deploy/images/raspberrypi4-64/core-image-base-raspberrypi4-64.manifest | grep alsa 3 alsa-conf aarch64 1.1.9 4 alsa-state aarch64 0.2.0 5 alsa-states aarch64 0.2.0 6 alsa-utils-alsactl aarch64 1.1.9 7 alsa-utils-alsamixer aarch64 1.1.9 8 # Where: alsa is the package you are looking for. 9

3.7.3 Configure the Embedded System to mount a NFS The last step of this guide is to provide the Kernel with the required information to find and mount the network file system. To do so, please reboot your Raspberry and stop the boot process in U-Boot. Then: 1. First create some U-Boot variables that we are going to hand over to the kernel.

1 # Set the location of the NFS in the Host 2 => setenv nfsroot /var/nfs/rootfs 3

2. Then modify the bootargs to let the kernel know the location of the NFS:

1 => setenv bootargs"console=ttyAMA0,115200 root=/dev/nfs rw nfsroot=${serverip}:${nfsroot },v3,tcp ip=$ipaddr:$serverip::$netmask::eth0:off" 2 3 # Note the " " quotes: variables are replaced by their value now and not at boot time 4 # Let us see what we have set: 5 # Console: Lets the kernel know that we want to use the ttyAMA0 as our UART console, speciﬁcally ata 115200 baudrate. 6 # root: It is nota path, buta keyword to tell the kernel that we are booting froma remote location in the network(NFS)

37 3.7.3

7 # nfsroot: Information about the host and its NFS server 8 # ip: Network information of the target 9 10 # Before booting please save the entries we have done so far: 11 => saveenv 12

3. Boot your Raspberry Pi with custom Bootloader, Kernel and File System:

1 # You can either type inU-Boot: 2 => boot 3 # Or just reset manually your RPI4, you should get: 4 5 U-Boot> boot 6 Using genet@7d580000 device 7 TFTP from server 192.168.8.2; our IP address is 192.168.8.3 8 Filename 'Image'. 9 Load address: 0x40000000 10 Loading:################################################################# 11 . 12 . 13 . 14 1.2 MiB/s 15 done 16 Bytes transferred = 13806080 (d2aa00 hex) 17 ## Flattened Device Tree blob at3af499b0 18 Booting using the fdt blob at 0x3af499b0 19 Using Device Tree in place at 000000003af499b0, end 000000003af56dc1 20 21 Starting kernel ... 22 23 24 Poky (Yocto Project Reference Distro) 3.0.2 raspberrypi4-64 /dev/ttyAMA0 25 26 raspberrypi4-64 login: root 27 root@raspberrypi4-64:~# 28

CONGRATULATIONS! You just booted your custom embedded system from the local network. Note: (a) The user is root and there is no password.

38 3.8

(b) If you have problems to load the file system over NFS check the U-Boot environment variables, the NFS configuration in the virtual machine (/etc/exports) and that you have unpack your file system in the right location. You can reset the NFS server by typing:

1 # Reset the NFS Server 2 $ sudo service nfs-kernel-server reset 3 # Check the status 4 $ sudo service nfs-kernel-server status 5

(c) Investigate a bit more regarding NFS: i. Is the whole file system loaded into RAM when you mount a NFS? If not, which is the criteria followed by the Linux Kernel to load files from the file system into RAM? ii. What if we use a Initrd? Would it be completely copied into RAM at boot time? iii. What are the benefits and use cases of both of them? 4. The benefit to have a NFS:

1 # We are going to check that we have control from the host over the target's file system. For that, createa file in the host and enter some text: 2 3 # Creates/var/nfs/rootfs/home/root/test.txt and puts"Testing our NFS" into it: 4 $ echo"Testing our NFS" | sudo tee -a /var/nfs/rootfs/home/root/test.txt > /dev/null 5 # Prints the contents of/var/nfs/rootfs/home/root/test.txt 6 $ sudo cat /var/nfs/rootfs/home/root/test.txt 7 Testing our NFS 8 9 # Now go to the TARGET and type the following to go to the home directory: 10 @ cd~ 11 # Show files: 12 @ ls 13 test.txt 14 # And print its contents: 15 @ cat test.txt 16 Testing our NFS 17

This basic example illustrates how you can have control over the ﬁles of the Rasp- berry Pi 4. However, NFS is much more powerful, as we will experience.

39 3.9.1

3.8 Testing the Target’s SSH Server In Step8 of Section 3.7.1 we added into the local.conf a SSH server (ssh-server-openssh). Which means the target should have a SSH server running in the local area network for us to login. Let us test it: 1. Open a terminal in your host and type:

1 $ ssh [email protected] 2 Last login: Thu Apr 16 18:50:52 2020 3 root@raspberrypi4-64:~# 4 5 # Now your are logged into your Raspberry over SSH! 6

Good news: at this point we do not need the UART cable and minicom anymore. Now we can login into our running Raspberry over SSH (better than a simple serial connection).

3.9 Cross-compiler and Toolchain This is the last step before we can start developing C/C++ applications for our system. It basically consists of: 1. Use the Yocto auto-generated script located at ~/Yocto/poky/build/tmp/deploy/sdk/ to install our cross-toolchain in the host. 2. Add the Yocto Plugin to the Eclipse installation from Section 2.6. 3. Configure Eclipse to compile the project with the ARM cross-compiler instead of the native gcc compiler. 4. Test our cross-compiler with a classic "Hello World" application and run it in the target. Please consider that besides the basic cross-compilation setup illustrated here, you can extend this development environment for debugging, profiling, tracing and power estima- tions of your code. If you are interested on this investigate some tools such as oprofile, perf and powertop. All of them supported by the Yocto Plugin.

3.9.1 Install the Generated ARM Toolchain In step9 of Section 3.7 we have already generated a script to make the ARM Toolchain installation easy. This script is located at ~/Yocto/poky/build/tmp/deploy/sdk/ so to install it you just have to:

40 3.9.2

1. Execute the script:

1 $ ~/Yocto/poky/build/tmp/deploy/sdk/here_your_scrip.sh 2

2. Then press Enter to select the default installation directory: /opt/poky/3.0.2 3. Lets look what is inside the /opt/poky/3.0.2 folder and source the environment:

1 $ ls -lh /opt/poky/3.0.2 2 -rw-r--r-- 1 root root 3,8K abr 16 14:44 environment-setup-aarch64-poky-linux 3 -rw-r--r-- 1 root root 14K abr 16 14:44 site-conﬁg-aarch64-poky-linux 4 drwxr-xr-x 4 root root 4,0K abr 16 14:43 sysroots 5 -rw-r--r-- 1 root root 121 abr 16 14:44 version-aarch64-poky-linux 6

At least 2 elements of the toolchain folder should drag our attention: (a) The Environment Script: It sets the right environment variables, compiler and linker flags, etc so that we can cross compile an application in the current terminal session. (b) Sysroots: It is a file system where all the libraries and modules we included with Yocto are present. Those libraries are used to compile our application. 4. That being said, we must source the environment before any attempt of cross- compiling with a Makefile (but not with the Yocto Plugin for Eclipse) as follows:

1 # Let's check for example the default content of theCC variable before sourcing the environment: 2 $ echo $CC 3 # Surprise! TheCC variable has nothing yet! Let's then source the environment: 4 $ source /opt/poky/3.0.2/environment-setup-aarch64-poky-linux 5 # And check again theCC variable: 6 $ echo $CC 7 aarch64-poky-linux-gcc -mcpu=cortex-a72+crc+crypto -fstack-protector-strong -D_FORTIFY_ SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/opt/poky/3.0.2/ sysroots/aarch64-poky-linux 8 # Voilà! Now we see the right aarch64-poky-linux-gcc ARM compiler 9

41 3.9.3.0

3.9.2 Yocto Plugin for the Eclipse IDE Even though we can use the toolchain right away without any IDE, it is more comfortable if we don’t have to write the Makeﬁle ourselves to compile the application. For this reason we will install the Yocto Plugin in the IDE: 1. Launch from the same terminal the Eclipse IDE:

1 $ /opt/eclipse/eclipse 2

2. Go to Section "Sourcing the Environment and Setting up Eclipse" of this guide and follow steps 3 to 6 to install the Eclipse Plugin. Note: In step 5 of the guide select the version 2.6.1/oxygen/ of the plugin to match our Eclipse version.

3.9.3 Conﬁguring the Yocto Plugin and Create Example Project Once the Yocto Plugin has been installed, we just need to conﬁgure it to start developing C/C++ applications. Additionally we will create an example project to test our setup.

Yocto Plugin Follow the next few steps [7] to get an Eclipse C/C++ project up and running: 1. In Eclipse go to: Window → Preferences: (a) Select the Yocto Project SDK option on the left. (b) Under "Cross Compiler Options" select the radial button for "Standalone pre- built toolchain". (c) Set the "Toolchain Root Location" to the SDK you just installed: /opt/poky/3.0.2 (d) Set the "Sysroots Location": /opt/poky/3.0.2/sysroots/aarch64-poky-linux (e) The "Target Architecture" ﬁeld should automatically ﬁll with "aarch64-poky- linux" (f) Click "Apply and Close" to save settings.

42 3.9.3.0

Figure 6: Conﬁguring the Yocto Project SDK

Test Application Let us now create a test application to make sure everything is working before we start developing the actual algorithm. 1. Go to File → New → C/C++ Project 2. Select the "C++ Managed Build" and click Next 3. Expand "Yocto Project SDK CMake Project" and select "Hello World C++ CMake Project" 4. Name the Project "Toolchain_Test" and click Next 5. Enter your personal information (if you wish) and click Next 6. Click Finish 7. Click on the Build icon on the top left to build the project 8. Open a terminal in the host and copy the generated executable in the targt’s ﬁle system:

1 $ sudo cp ~/eclipse-workspace/Toolchain_Test/Debug/Toolchain_Test /var/nfs/rootfs/home/root/ 2

9. Login over SSH into the target(or use the minicom UART terminal) and execute the test application:

43 4

1 $ ssh [email protected] 2 Last login: Thu Apr 16 19:43:16 2020 from 192.168.8.2 3 # Now we are logged in the Raspberry Pi 4. Let's run the test by typing ./Toolchain_Test 4 root@raspberrypi4-64:~#./Toolchain_Test 5 Hello World! 6 root@raspberrypi4-64:~# 7

It seems that we are able to compile C/C++ applications for the Raspberry Pi!

Limitations of the Basic Development Environment So far we have only compiled a simple application and then manually copied the executable into the target’s file system. This method works for a "Hello World" example but is quite primitive when it comes to debugging a real program. From here on you should: 1. Investigate and configure remote debugging for your Eclipse project so that you have more convenient tools to find errors in your application. There is plenty information on the web on remote debugging with Eclipse, for instance, you can install Eclipse packages described in Section 3.2 of this tutorial and then follow this guide to try external debugging. 2. For those members of the team who do not have a Raspberry: you can use the hardware simulator QEMU to collaborate in the application development even without a Raspberry. (a) Investigate how you can generate a QEMU emulator with Yocto and use it with Eclipse to run your application. However: the last application test before the delivery should be made with a Raspberry and not with a simulator. (b) Hint: Check the "$ bitbake meta-ide-support" command. You can start from here.

4 Create a Custom Meta-Layer in Yocto

It is a common practice to include your application into the File System of your embedded device at the end of the application development process, that is, once it has been developed and debugged with a suitable IDE (e.g. Eclipse) and a cross-compilation environment like the one we just got working. The Yocto project allows custom packages generation which can be included into the compiled image through a deﬁned recipe scheme.

44 5.1

In this section a recipe example is shown which will inherit a custom package that prints "Hello world using Yocto" on prompt using C programming language and Autotools. It is highly recommended to read and follow the next steps to create a recipe for the above- mentioned purpose: 1. Go to Build an example package based on a local source archive, please read and pay special attention to how to build an example package (the main idea is to learn how is the recipe creation process using local source data). 2. In order to understand the file distribution, please check every file in this Github repository and analyze the file tree distribution. 3. After reading point 1 and 2, please download the folder example here, unzip files and check every file in terms of contents and source code. 4. Ok, here we go, let’s create a custom meta-layer! In this case we will create a meta- layer called meta-hello, this layer prints "Hello world using Yocto" in the command prompt. For better understanding of the recipe creation workflow, it is highly rec- ommend to watch and follow this video, please pay attention to every step to build your meta layer example in your Yocto distribution. (Hint: if you want to get rid of warnings in your bitbake process read and pay attention to this link).

5 The Application: Composite Trapezoidal Numerical In- tegration

This section aims to design and develop a high-level prototype in Octave and a custom meta-layer in Yocto of a composite trapezoidal rule from an established function f(x), which must be compiled into a Yocto image and executed on a Raspberry Pi 4. In relation to code development it is highly recommended to follow the Eclipse framework suggested in this project in terms of evaluating the application functionality using a cross-compile approach before you build your custom Yocto image with your custom meta-layer. In summary the expected development steps are: 1. Prototype your application in a high-level language (Octave). This will get you to understand the problem you are facing and have a reference implementation. 2. Port your application to C language using Eclipse, test it on the target system and debug it if necessary. 3. Include your faultless application into your Yocto image to hypothetically make the production of your devices easier and (hopefully) sell thousands of your just- designed embedded platforms around the world :)

45 5.2.1

5.1 Motivation and Description A numerical method is a procedure by which the numerical solution of certain problem is approximated by performing purely arithmetic and logical calculations. For resolution of deﬁnite integrals, there are various numerical methods that allow the result to be generated according to the mathematical function to be integrated. For this project, the composite trapezoidal rule will be used as application target. Figure7 depicts the composite trapezoidal rule.

Figure 7: Composite Trapezoidal Rule description, taken from [1].

Based on composite trapezoidal rule, the numerical method is described by (1):

" n−1 # Z b h X f(x)dx = f(a) + 2 f(x ) + f(b) (1) 2 j a j=1

b−a where h is deﬁned as n and n represents sub-intervals or equidistant intervals. If you want to read about Composite Trapezoidal Rule click here

5.2 Prototyping and Programming Process Based on the deﬁnite integral described by Equation2, you will estimate its numerical approximation using Octave prototyping and C programming process according to the upper value, lower value and sub-intervals deﬁned for your estimation.

Z b 1 2 dx (2) a 1 + x

Remember, a represents the upper value and b the lower value in this deﬁnite integral.

46 5.2.2

5.2.1 Prototyping For this part before typing your C program you will create a prototyping ﬁle named trapezoidal.m using Octave in order to evaluate the functionality. Please follow the next instructions during the prototyping stage: 1. If you don’t have the Octave software, please check your OS and install the latest version for your PC in this link 2. Include header information in your Octave ﬁle, this information is important for your code. Consider the following format in your code prototyping.

1 # Course:MP-6171 High Performance Embedded Systems 2 # Tecnologico de Costa Rica(www.tec.ac.cr) 3 # Developers Name:(email@...) 4 # This script is structured in Octave(https://www.gnu.org/software/octave/) 5 # General purpose: Composite Trapezoidal Rule Prototyping 6 # Input: Lower value, upper value and sub-intervals 7 # Output: numerical approximation 8

3. Program using an Octave script your Composite Trapezoidal numerical method and evaluate its functionality. 4. Finally remember to properly comment your script so that other people can easily understand your code. Useful information in source code comment practices here

5.2.2 Programming For this part it is recommended to use your code prototype as a base to write you source code using C and the Eclipse IDE cross-compiler framework which was set up in section 3.9.3. This in order to type and build your binary file before you compile it into your Yocto recipe. You will create a simple C program named trapezoidal.c which shall meet the following requirements: 1. You must create a new Yocto meta-layer called meta-tec. You can inherit from the image (i.e core-image-base) target for the standard configuration and your custom layers. Useful information in section Create a Custom Meta-Layer in Yocto 2. Please create a new Yocto recipe called: trapezoidal. You will find some useful information in Section4 and here. 3. The program should have the following options using the getop API and the order of the parameters should not affect the correct operation of your application. You will find some useful information for getop here

47 6.1

1 Usage 2 ./trapezoidal [ -u UpperValue ] [ -l LowerValue ] [ -n sub-intervals] 3 -u UpperValue speciﬁes upper value. 4 -l LowerValue speciﬁes lower value. 5 -a displays the information of the author of the program. 6 -h displays the usage message to let the user know how to execute the application. 7 8 ./trapezoidal -u 5 -l 0 -n 50 9

4. You are free to choose any build system as GNU make, Automake or CMake. Just make sure it builds correctly and it is totally integrated with the Yocto recipe. (It is recommended to use the Autotools process). 5. The numerical estimation process MUST be implemented in a C function (trapezoidal.c) with the following proposed prototype:

1 float Trapezoidal(float lower, float upper, int subInterval) 2 You can vary the arguments but NOT the function name. 3

6. Don’t forget, all binaries MUST be installed into the ﬁle system as part of the recipe compilation.

6 Deliverables and Grading

This section explains the work ﬂow requirements, deliverables and grading related to this project. Please read and make sure to organize your project execution and deliverables over work time and not just before the deadline.

6.1 Work ﬂow requirements In relation to the Git repository, several aspects are to be considered. It is important to follow them in order to organize your code development and prepare your repository: 1. Open a Git account on a free Git hosting (GitHub) 2. Create a Git repository named as follows: < group# > _HPCE_2020. Where the group# shall be the assigned group number. For example, for Group 1, the name of his Git repository shall be: group1_HPEC_2020

48 6.3

3. If the repository is private then provide access to users sercr0388 and jmarayam (GitHub) and make sure to provide write permissions. 4. The Git repository shall contain two main branches: master and develop 5. Initially the develop branch is created from the master. 6. When working on a project the student shall create a new working branch from develop and when the feature is ready the branch must be merged to develop. Any additional ﬁx or modiﬁcation after merge must require the process to be repeated (i.e. create the branch from develop and merge the changes later). Once the code in develop is ready it shall be merge to master and a tag must be created. The process is described in Figure8. Recommended link here

Figure 8: Repository structure for assignments or projects

6.2 Folder Structure of your Deliverables In relation to deliverables documentation and folder structure you MUST follow the folder organization in your user_id.tar.gz and Git repository as depicted below:

49 6.4

6.3 Grading The grading of this project will be based on Table3. Below we provide a summary of the proposed question along this document, please scroll up and find them to know in which context they where asked: 1. What is a Device Tree? Why was it introduced and why is it so important? 2. What is a Device Tree Overlay? What can be controlled with it? 3. Please explain in your report what are the processor fuses and bootstrap resistors and what exactly of the boot process can be controlled with them. Mention a processor (architecture and manufacturer) for which you can control the boot process using both methods. 4. In addition to the recommendations discussed above, investigate the default location in RAM of the device tree at boot time for the RPI4 and look for information about reserved RAM addresses that U-Boot uses to operate. Describe your findings in the written report and select your kernel_addr_r accordingly. 5. Investigate a bit more regarding NFS: (a) Is the whole file system loaded into RAM when you mount a NFS? If not, which is the criteria followed by the Linux Kernel to load files from the file system into RAM? (b) What if we use a Initrd? Would it be completely copied into RAM at boot time? (c) What are the benefits and use cases of both of them?

6.4 Deliverables Submission To submit your deliverables you have to: 1. Before sending your documentation and deliverables please follow the folder structure established on section 6.2. 2. Compress your deliverable folder in zip format and look in TEC-Digital for the delivery link for Project 1. 3. Delivery date is Sunday June 16th 2020, deadline 12:00 m.n afterwards, the link access in TEC-Digital will be closed.

50 6

Table 3: Project’s evaluation criteria.

Item Description Percentage Answer the questions stated along this document 15% Report Show your results 10% Analyze/discuss your results. Address problems you could not solve, 15% the way you tackled them and its probable cause A full (legible) boot process of your RPI4. You have to show the command line as your system 10% Video boot (U-Boot, TFTP, NFS and login as root) Show application compilation and operation 10% Yocto metal-layer, correct creation 5% Yocto Recipes, correct creation 5% Custom Autotools, GNU make or Cmake. Yocto 5% Usage, correct usage for the compilation program Application Git control versioning, delivery is correct. Using git with the layout suggestested and 5% following the required worf ﬂow Getopt implementation. 5% Correct usage of getopt for the command lines options Application meets with requeriments proposed and is totally functional providing 15% correct estimation result Total 100%

References

[1] R. L Burden and J.D Faires. Numerical differentiation & integration, 2011.

[2] Ubuntu Community. Tftp, 2017.

[3] Ubuntu Community. Setting up nfs: How to, 2018.

[4] Raspberry Pi Foundation. Pi4 bootﬂow, 2019.

[5] Andrei Gherzan. U-boot on raspberry pi (including rpi 4), 2019.

[6] Mark Drake Melissa Anderson. How to set up an nfs mount on ubuntu 18.04, 2018.

[7] Phytec. How-to: Set up yocto plugin for eclipse, 2020. [Online; accessed 19-April- 2020].

51 6

[8] Pranav. Porting of u-boot on raspberrypi-3, 2018.

[9] Sean Sosik-Hamor. Conﬁguring a tftp server on ubuntu for switch upgrades and maintenance, 2017.

[10] Raspberrypi Spy. Simple guide to the raspberry pi gpio header, 2012.

[11] Wikipedia contributors. Linux kernel — Wikipedia, the free encyclopedia, 2020. [Online; accessed 22-March-2020].