Boottorrent Documentation Release 0.1.1
Total Page:16
File Type:pdf, Size:1020Kb
boottorrent Documentation Release 0.1.1 Shreyansh Khajanchi May 06, 2020 Contents 1 Features 3 2 Contents 5 2.1 Installation................................................5 2.2 Quick Start Guide............................................8 2.3 Usage................................................... 10 2.4 Use cases................................................. 15 2.5 Architectural document......................................... 17 2.6 Internals................................................. 24 2.7 Frequently Asked Questions....................................... 32 2.8 Contributing............................................... 35 2.9 Credits.................................................. 37 2.10 History.................................................. 38 3 Indices and tables 39 i ii boottorrent Documentation, Release 0.1.1 BootTorrent allows distributed P2P BitTorrent based network booting of various Operating systems. The advantage of BootTorrent over standard network booting is that the bottleneck caused by a central server serving the image is reduced and BootTorrent reduces the linear scaling factor in total time to bring a cluster online fully functional by utilizing Peer-to-Peer protocols. The nodes share the image data among themselves, hence, improved boot times. +--------------------------------------------------------------------+ |+----------------------------+| | Fig1: A network of computers|+----+FE+----+|| | sharing data together after||Node|<---->|Node||| | receiving it from server. |+----++----+|| ||^^|| ||FE||FE|| |+------+1 Gbps+------+ 100 Mbps|v v|| ||Server|-------->|Switch|--------->|+----+FE+----+|| |+------+ Link+------+ Link(FE)||Node|<---->|Node||| ||+----++----+|| |+----------------------------+| +--------------------------------------------------------------------+ Cases where BootTorrent may be useful would be: • If the clients in your network are not getting enough bandwidth individually and have significant bandwidth being left unused, which causes increased boot times. With BootTorrent you can repurpose the remaining bandwidth to help clients mutually share it among themselves. • If your server (such as a laptop) can only connect to your cluster of computers via a comparatively slow link (such as WiFi or Fast-ethernet) then BootTorrent can help you mitigate the low bandwidth issues of network link. • If you have large number of computers at your disposal and you’re simply looking to deploy any given system image(s) (that may have been hand-crafted according to your needs) as painlessly as possible. BootTorrent can help you deploy it in three easy steps to the whole network. • If your current network boot server is unable to meet your requirements and deliver much needed performance, consider giving BootTorrent a try. Its distributed architecture will reduce the dependence on server, which means improved boot performance. For more details on use cases please refer to Use cases list and visit the documentation. We have data to back our claims. Check out the performance improvements here: [Images & Data courtesy of SL-Lab: http://sl-lab.it/dokuwiki/lib/exe/fetch.php/tesi:tesi_bruschi.pdf] The above images were created from tests done at the University of Milan (ITALY) during the development of the original “boottorrent” project. Contents 1 boottorrent Documentation, Release 0.1.1 WARNING: STATUS - ALPHA: DO NOT USE IN PRODUCTION! • Free software: GNU General Public License v3 • Documentation: https://boottorrent.readthedocs.io. 2 Contents CHAPTER 1 Features • Boot Linux Kernel with Kexec. • Includes Qemu to run any type of Operating system on client machine. • Ability to run a binary executable. • Client side user input to choose an OS. • Leverage BitTorrent to transfer files over the network. 3 boottorrent Documentation, Release 0.1.1 4 Chapter 1. Features CHAPTER 2 Contents 2.1 Installation 2.1.1 Dependencies Components this project depends on include: • Python 3.6 with Pip • Aria2 • mktorrent • Dnsmasq • bsdtar (provided by libarchive) • Opentracker (Optional) After installation, please make sure that binary files are available in your PATH variable. You can check it with this command (example for dnsmasq): $ which dnsmasq /usr/sbin/dnsmasq The package itself depends on a few Python libraries which are installed automatically by Pip package manager. These libraries are: • PyYAML • Jinja2 These dependencies are only for the host running BootTorrent. Please check your distribution specific guide for instructions on how to install them. For building BootTorrent and the list of build dependencies, please refer build dependencies. 5 boottorrent Documentation, Release 0.1.1 2.1.2 From sources Get the code The sources for BootTorrent can be downloaded from the Github repo. You can either clone the public repository: $ git clone git://github.com/shreyanshk/boottorrent Or download the tarball: $ curl -OL https://github.com/shreyanshk/boottorrent/tarball/dev Pre-Install: Build assets You can skip this section if you do not intent to customize BootTorrent as these resources are currently checked in with the repository Building assets requires additional software. This includes: • Golang • SliTaz LiveCD/installation with Tazlito Some Golang dependencies are also required to build, you can download them with these commands: $ cd phase1bootstrap/ $ make dldeps To build the SliTaz live image that is run on the client, copy the phase1bootstrap/slitaz/ directory to a Virtual Machine or computer running SliTaz with Tazlito installed. Then, open a console and execute: $ cd <path to copied directory> $ sudo tazlito gen-distro Hint: You can also customize the built image to include more packages, drivers or files etc. Read more here in the FAQ. It will ask if you want to ‘Repack packages from rootfs?’, please press ‘n’ and then enter. This will download SliTaz packages from the internet and make a custom live distribution usable with BootTorrent. We only need the final Kernel image and the initrd file which are generated in this process. These files are rootcd/boot/bzImage and rootcd/boot/rootfs.gz, please copy these files from this Virtual Machine or computer and place them inside boottorrent/assets/ph1 directory. To build the client TUI, execute: $ cd phase1bootstrap $ make initrd This will create the assets and place them at proper locations in the repository. Install First, check if you have a compatible version (>3.6) of Python. 6 Chapter 2. Contents boottorrent Documentation, Release 0.1.1 $ python --version Python 3.6.5 Otherwise, look at your distribution’s documentation to install it or use tools such as pyenv. You can install BootTorrent just for your account (this doesn’t require sudo) with pip: $ pip install --user <repository path> You can also do a global install with pip: $ sudo pip install <repository path> If BootTorrent conflicts with your previously installed packages. You can use virtualenv to setup a virtual environment and install inside it: $ virtualenv -p python3.6 venv $ source venv/bin/activate $ pip install <repository path> 2.1.3 From Pip First, check if you have a compatible version (>3.6) of Python. $ python --version Python 3.6.5 Otherwise, look at your distribution’s documentation to install it or use tools such as pyenv. After installing runtime dependencies, to install BootTorrent, you can install it just for your account (this doesn’t require sudo) with pip: $ pip install --user git+https://github.com/shreyanshk/boottorrent You can also do a global install with pip: $ sudo pip install git+https://github.com/shreyanshk/boottorrent If BootTorrent conflicts with your previously installed packages. You can use virtualenv to setup a virtual environment and install inside it: $ virtualenv -p python3.6 venv $ source venv/bin/activate $ pip install git+https://github.com/shreyanshk/boottorrent These are the preferred methods to install BootTorrent, as they will always install the most recent release. If you don’t have Pip installed, this Python installation guide can guide you through the process. 2.1.4 Updating/Reinstalling First, please uninstall the previous version with these commands: If you’ve installed BootTorrent locally only for the current user: 2.1. Installation 7 boottorrent Documentation, Release 0.1.1 $ pip uninstall boottorrent If you’ve installed BootTorrent globally (installed with sudo): $ sudo pip uninstall boottorrent Or, If you’ve used virtualenv: $ source <path to virtualenv>/bin/activate $ pip uninstall boottorrent Then, you can install BootTorrent back by any of the above methods. 2.2 Quick Start Guide Getting started with BootTorrent is very easy and this guide will show you the way towards booting a Linux based OS with BootTorrent. This guide assumes that you’ve a compiled kernel (bzImage) that you want to boot along with it’s corresponding initramfs (initrd.gz) and you have complete control over your network. This guide also assumes that you’ve successfully installed BootTorrent. If that is not the case please visit the Installation guide. 2.2.1 Make a BootTorrent env Change directory to where you want to create a new env and then execute: $ boottorrent init guidetest $ cd guidetest This initializes a new env with sane defaults and then changes to that env. 2.2.2 Configure the Linux kernel You need to place the kernel and initrd.gz file inside a sub-directory inside the oss/ directory. You can give any name to the sub-directory. Here, we are assuming the name is linuxos. The console command would look like this: Please substitute the brackets with appropriate