A distributed delay tolerant data transfer protocol for ISTSAT-1
Pedro Miguel Pereira Eugénio Andrea Gameiro
Thesis to obtain the Master of Science Degree in Bologna Master Degree in Telecommunications and Informatics Engineering
Supervisors: Prof. Rui Manuel Rodrigues Rocha Dr. Gonçalo Lopes
Examination Committee Chairperson: Prof. Ricardo Jorge Fernandes Chaves Supervisor: Prof. Rui Manuel Rodrigues Rocha Member of the Committee: Prof. Fernando Henrique Côrte-Real Mira da Silva
December 2018 Resumo
Um cubesat é um pequeno satélite artificial, de baixo custo, criado para efeitos de investigação cientí- fica espacial, que se tornou muito popular na comunidade científica. O ISTSAT-1 é o primeiro cubesat a ser desenvolvido no Instituto Superior Técnico (IST), com um tamanho de 10x10x10cm, respeitante a especificação cubesat 1U. As comunicações espaciais oferecem desafios diferentes daqueles encontrados em Terra. As ligações são tipicamente de baixo débito, com grandes atrasos e as disrupções podem ser frequentes. Tudo isto motivou a criação de protocolos Delay Tolerant (DT), uma arquitectura desenhada para lidar com os problemas característicos de ambientes disruptivos. No entanto, mesmo com protocolos Delay Tolerant (DT), a transmissão de ficheiros de elevada dimensão pode ser difícil. Isto é especialmente verdade no caso dos cubesats, que devido às baixas órbitas em que são colocados, sofrem de longos períodos de disrupção nas suas ligações com as estações de rádio em Terra. Com este projecto é proposta uma solução mais adequada. A criação de um protocolo Delay Toler- ant (DT) distribuído, capaz de realizar as tarefas já presentes nos restantes protocolos Delay Tolerant (DT), mas onde uma única transmissão pode ser distribuída por diferentes dispositivos (como satélites e estações de rádio em terra), aumentando assim o número de ligações, reduzindo os períodos de disrupção e melhorando assim o desempenho das transmissões espaciais.
Palavras chave: IST, nanosat, LEO, ISTSAT-1, distribuído, DT
ii Abstract
A cubesat is a very small, low cost, artificial satellite designed for space research purposes, very popu- lar in the academic community. The ISTSAT-1 is the first cubesat being developed in Instituto Superior Técnico (IST), with a size of a 10x10x10cm cube respecting the 1U cubesat standard. In space communications one finds different challenges from the ones on Earth. Links are typically un- stable, of low debit, of high delay and disruptions can be frequent. This motivated the creation of Delay Tolerant (DT) protocols, an architecture designed to deal with the characteristic problems of disruptive environments. However, even with DT, the transmission of big data files can be difficult. This is specially true in the case of cubesats. Due to their Low Earth Orbit (LEO) deployment, cubesats suffer from very long disruptions periods with the ground-stations on Earth. With this project we intend to create an enhanced solution. A distributed DT protocol capable of perform- ing the normal DT tasks, but in a manner where a single transmission can be distributed over different devices (like ground-stations or satellites), thus increasing the number of links, reducing the disruptions time periods and increasing the much needed performance of space transmissions.
Keywords: IST, nanosat, LEO, ISTSAT-1, distributed, DT
iii iv Contents
Resumo ...... ii Abstract ...... iii Contents ...... iv List of Tables ...... vii List of Figures ...... viii
Glossary xi
Acronyms xv
1 Introduction 1 1.1 Motivation ...... 2 1.2 Objectives and challenges ...... 3 1.3 Research contributions ...... 4 1.4 Document structure ...... 4
2 State of the art 5 2.1 DTN...... 5 2.2 Saratoga ...... 8 2.3 SCPS-TP ...... 10 2.4 T-CSP ...... 11 2.5 Comparison ...... 13
3 Architecture 15 3.1 Application scenarios ...... 15 3.1.1 One-to-one transmissions ...... 15 3.1.2 Multi-to-one transmissions ...... 16 3.1.3 One-to-multi transmissions ...... 17 3.1.4 Distributed transmissions ...... 18 3.2 Requirements ...... 19 3.3 Protocol ...... 20 3.3.1 Messages ...... 20 3.3.2 Algorithm ...... 22
v 3.4 Application Scenarios operations ...... 24 3.4.1 One-to-one transmissions ...... 24 3.4.2 Multi-to-one transmissions ...... 25 3.4.3 One-to-multi transmissions ...... 25 3.4.4 Distributed transmissions ...... 26 3.4.5 POCKET++ ...... 28
4 Implementation 29 4.1 Software Architecture ...... 29 4.1.1 Multi platform ...... 31 4.1.2 Linux ...... 32 4.1.3 OMNET ...... 33 4.1.4 Multi Network stack ...... 33 4.1.5 Data structures ...... 34 4.2 Programming language selection ...... 36 4.3 Build tools ...... 37 4.4 Code Analysis ...... 37 4.4.1 Static ...... 37 4.4.2 Dynamic ...... 37 4.5 Documentation ...... 37 4.5.1 Software reference ...... 38 4.6 Source Code Version Control ...... 38 4.7 Testing ...... 38 4.8 Development Environment ...... 39
5 Experimental evaluation 41 5.1 One to one ...... 42 5.1.1 One to one - Delay tolerant ...... 43 5.2 Multi to one ...... 45 5.3 One to multi ...... 45 5.4 Distributed ...... 47 5.5 Compression ...... 48
6 Conclusion and future work 51
Bibliography 53
vi List of Tables
2.1 Comparison of the existing solutions ...... 13
5.1 One to one transmission ...... 42
vii viii List of Figures
1.1 ADS-B System ...... 2 1.2 ISTSAT-1 Distributed transmission ...... 3
2.1 Store and Forward ...... 6 2.2 DTN stack ...... 7 2.3 DTN transmission comparison ...... 7 2.4 Mars network ...... 8 2.5 Saratoga Message Sequence Chart ...... 9 2.6 Saratoga stack ...... 10 2.7 Saratoga STATUS message ...... 10 2.8 T-CSP segment over CSP packet ...... 12 2.9 T-CSP transmission example ...... 12
3.1 One to one transmission ...... 16 3.2 Multi to one transmission ...... 17 3.3 Multicast transmission ...... 18 3.4 Distributed transmission ...... 19 3.5 DDTP Message types ...... 21 3.6 DDTP sender process state machine ...... 23 3.7 DDTP receiver process state machine ...... 23 3.8 MSC One-to-one transmission ...... 24 3.9 MSC Multi-to-one transmission ...... 25 3.10 One-to-multi transmission ...... 26 3.11 MCC Distributed Transmission ...... 27 3.12 Distributed transmission - cubesat cluster example ...... 28
4.1 DDTP modules ...... 30 4.2 DDTP software architecture ...... 31 4.3 DDTP software platform architecture ...... 32 4.4 DDTP - Network Abstraction Layer ...... 33 4.5 DDTP Network stack ...... 34 4.6 DDTP Multi stack transmission ...... 34
ix 5.1 Testing simulation topology ...... 42 5.2 DDTP vs Saratoga - One to one ...... 43 5.3 DDTP DT transmission ...... 44 5.4 Saratoga DT transmission ...... 44 5.5 Multi to one - DDTP vs Saratoga ...... 45 5.6 One to multi topology ...... 46 5.7 DDTP - One to multi transmission ...... 47 5.8 DDTP distributed transmission ...... 48 5.9 Pocket++ transmission ...... 49
x Glossary
ADS-B Automatic dependent surveillance-broadcast is a surveillance technology in which an aircraft de- termines its position via satellite navigation and periodically broadcasts it, enabling it to be tracked.
AX25 A data link layer protocol derived from the X.25 protocol suite and designed for use by amateur radio operators.
C A general-purpose, imperative computer programming language.
C++ introduces object-oriented programming (OOP) features the C language.
Cubesat A type of miniaturized, low cost satellite for space research of popular use in the academic community.
Doxygen Doxygen is a documentation generator, a tool for writing software reference documentation. The documentation is written within code, and is thus relatively easy to keep up to date.
ESA The European Space Agency is an intergovernmental organization of 22 member states dedicated to the exploration of space.
Ethernet A data link layer protocol, widely used in conjunction with the Transmission Control Protocol (TCP)/Internet Protocol (IP) stack.
FreeRTOS FreeRTOS is a real-time operating system kernel for embedded devices that has been ported to 35 microcontrollers.
FYS ESA’s Fly Your Satellite! (FYS) programme is a recurring, hands-on programme designed and managed by the ESA Education Office in close collaboration with European Universities, with the objective to complement academic education and inspire, engage, and better prepare university students for a more effective introduction to their future professions in the space sector.
ISTSAT-1 ISTSAT-1 is the first cubesat being developed in Portugal, by IST-UL, with a 1U size (103 cm cube) by students, professors and radio-amateurs. Its mission is first and foremost an educational one, joining together multiple students from different expertise and engineering programmes to foster their enthusiasm for space science and complementing their education with a challenging hands-on project.
xi Java Java is a general-purpose computer-programming language that is concurrent, class-based, object- oriented, and specifically designed to have as few implementation dependencies as possible.
Libcsp CubeSat Space Protocol (CSP) is a small network-layer delivery protocol designed for Cube- Sats. The idea was developed by a group of students from Aalborg University in 2008, and further developed for the AAUSAT3 CubeSat mission that was launched in 2013.
MCC A mission control center is a facility that manages a spacecraft mission and operations, including both personnel and equipment. It is part of the ground segment of spacecraft operations.
Netkit Netkit is an environment for setting up and performing networking experiments at low cost. It allows to "create" several virtual network devices (full-fledged routers, switches, computers, etc.) that can be easily interconnected in order to form a network. Netkit exploits open source software (mostly licensed under GPL) and is heavily based on the User Mode Linux (UML) variant of the Linux kernel.
OMNET OMNeT++ is a modular, component-based C++ simulation library and framework, primarily for building network simulators.
Peer2peer Peer-to-peer (P2P) computing or networking is a distributed application architecture that partitions tasks or workloads between peers. Peers are equally privileged, equipotent participants in the application. They are said to form a peer-to-peer network of nodes.
Pocket++ A simple, deterministic and very efficient compression algorithm developed by ESA for space- crafts with limited computacional resources. It can work on the real-time data stream and therefore one benefits from stored data compression for free, as well as the multiplication factor during play- back.
Python Python is an interpreted high-level programming language for general-purpose programming. Created by Guido van Rossum and first released in 1991, Python has a design philosophy that emphasizes code readability.
Ruby A dynamic, open source, reflective, object-oriented, general-purpose programming language with a focus on simplicity and productivity.
Saratoga Saratoga is a very scalable, peer-to-peer reliable transport protocol, capable of transferring very large files guaranteeing error-free data delivery.
Sattelite cluster Sattelite cluster is the concept that multiple satellites can work together in a group to accomplish the objective of one larger, usually more expensive, satellite.
SNACK Selective Negative Acknowledge - The receiver explicitly lists the segments in a stream that were not received (even if non-continuous).
xii UDP-lite Lightweight User Datagram Protocol (UDP-Lite) is a connectionless protocol similar to User Datagram Protocol (UDP) but that does not check the integrity of the payload.
Unicast Unicast is communication between a single sender and a single receiver over a network. The term exists in contradistinction to multicast, communication between a single sender and multiple receivers.
Valgrind Valgrind is a programming tool for memory debugging, memory leak detection, and profiling.
xiii xiv Acronyms
ACK Acknowledge.
CCSDS Consultative Committe for Space Data Systems.
CPU Center Processing Unit.
CSP Cubesat Space Protocol.
DDTP Distributed Delay Tolerant Protocol.
DICE Dynamic Ionosphere Cubesat Experiment.
DMC Disaster Monitoring Constellation.
DT Delay/Disruptive Tolerant.
DTN Delay/Disruptive Tolerant Networking.
EID Endpoint Identifier.
FTP File Transfer Protocol.
FYS Fly Your Satellite.
GDB GNU Debugger.
GS Ground Station.
IDE Integrated Development Environment.
IP Internet Protocol.
IPN Interplanetary Internet Project.
IPSec IP Security.
ISA Instruction Set Architecture.
ISL Inter Satellite Links.
xv IST Instituto Superior Técnico - University of Lisbon.
ISWCS International Symposium on Wireless Communication Systems.
ITU WRC-15 International Telecommunication Union’s World Radio communication Conference.
KISS Keep It Simple and Stupid.
LEO Low Earth Orbit.
LOS Line Of Sight.
MCC Mission Control Center.
MSC Message Sequence Chart.
MTU Maximum Transmission Unit.
NACK Negative Acknowledge.
NASA National Aeronautics and Space Administration.
NPAL Network Protocol Abstration Layer.
OS Operating System.
OSAL Operating System Abstration Layer.
OSI Open Systems Interconnection.
RDP Reliable Datagram Protocol.
SCPS Space Communications Protocol Specifications.
SCPS-FP SCPS File transfer Protocol.
SCPS-NP SCPS Network Protocol.
SCPS-SP SCPS Security Protocol.
SCPS-TP SCPS Transfer Protocol.
SNACK Selective Negative Acknowledge.
SNR Signal to Noise Ratio.
SSTL Surrey Satellite Technology Ltd.
T-CSP Tolerant CSP.
TCP Transmission Control Protocol.
xvi UDP User Datagram Protocol.
URI Uniform Resource Identifiers.
WSN Wireless Sensor Network.
xvii Chapter 1
Introduction
Space research has traditionally been limited to organizations who can afford the large investments as- sociated with it. This presents a major limitation in technological development. A popular new trend, specially in universities and the academic community, is the development of minia- turized satellites known as cubesats[1]. These satellites offer a major opportunity for organizations with limited resources to perform space research[2]. This is not only due to their small sizes, but since they can be built with common electronic components they are of low cost. Its design was proposed in 1999 by professors Jordi Puig-Suari of California Polytechnic State University and Bob Twiggs of Stanford University. Standard cubesats usually have a volume of 1 liter (103 cm cube), weight 1kg and are called 1U(one unit) cubesats. There are also some bigger models, scaled in the vertical axis, 2U(20x10x10cm) and 3U(30x10x10cm) and even as big as 6U and 12U. ISTSAT-1[3] is the first cubesat being developed in Portugal, by Instituto Superior Técnico - University of Lisbon (IST), with a 1U size (103 cm cube) by students, professors and radio-amateurs. The ISTSAT-1 mission is first and foremost an educational one, joining together multiple students from different exper- tise and engineering programmes to foster their enthusiasm for space science and complementing their education with a challenging hands-on project. In 2014, under unexplained circumstances, the Malaysia Airlines flight 370 plane disappeared in the mid- dle of the ocean. This event triggered a new discussion about safety in the airline industry and a growing interest in tracking airplanes in remote areas. In November 2015, the International Telecommunication Union’s World Radio communication Conference (ITU WRC-15) allocated a new frequency band to be used by the ADS-B[4] system in earth-space direction1. This enabled the possibility of tracking aircraft positions from space, at a global scale. Today, this signal is only monitored through ground-based sta- tions, with no coverage in oceanic routes and remote areas. The ADS-B system is a security feature that most planes today use, and all of them will be obliged to use by 2020. It periodically (each 0.5 seconds) emits a message with information about the plane, it’s location, identification and status. ISTSAT-1[5] will be used for carrying a feasibility study of the use of cubesats for receiving ADS-B signals from traveling
1https://www.itu.int/net/pressoffice/press_releases/2015/51.aspx
1 airplanes, as illustrated in figure 1.1. The ISTSAT-1 spacecraft is also being developed in the context of an ESA educational programme, the Fly Your Satellite (FYS), along with 5 other european teams devel- oping their own cubesat2. In this programme the ISTSAT-1 development will be assisted by experienced engineers and professionals from ESA that have developed, launched and operated several spacecrafts in their carer. This is a major opportunity that as drastically impacted this project. The FYS programme as also offered the ISTSAT-1 a launch opportunity, currently schedule for 2020. To address the problems associated with space communications (characterized by long distances, high delay environments, frequent disruptions or disconnections), a new communication paradigm emerged, the Delay/Disruptive Tolerant Networking (DTN). The DTN specification defines an hop-to-hop architec- ture where link disruption is acceptable and transmissions will pause and resume as necessary. This paradigm is already being used in the National Aeronautics and Space Administration (NASA) mars rover mission3.
Figure 1.1: ADS-B System
1.1 Motivation
Cubesats are normally deployed in Low Earth Orbit (LEO), which is defined as 300-1500km above the earth surface, with an orbital period of approximately 90 minutes at a speed of 7.8km/s. However, a big consequence of this, is that a particular ground-station only has a very small time-window (15-20 minutes) of line-of-sight to communicate with the satellite. Offering even more obstacles, space-link communications are very unstable, error prone, and of low debit. The challenges in space-link commu- nications are not traffic congestion (like on earth), but long propagation delays and high bit-error rates. The current approach to this problem is the use of DTN [6][7] protocols. These (in contrast to common, terrestrial protocols) are designed to tolerate large periods of delay (or disrupted links) and transmit at high bitrates when the link is available. This approach allows transmissions to be paused when the communication space link is unavailable, and to resume when a line-of-sight is once again available. The purpose of this project is to tackle this difficulties in a new, more powerful way. To create a distributed
2https://www.esa.int/Education/CubeSats_-_Fly_Your_Satellite/Six_new_CubeSat_missions_selected_for_next_ cycle_of_Fly_Your_Satellite 3http://mars.nasa.gov/mer/home/
2 protocol capable of expanding a single transmission session over several links and hosts (Ground Sta- tions (GSs) in this particular case). This is illustrated in figure 1.2, where the satellite, after starting a transmission with a given GS, leaving its line-of-sight and consequently disconnecting the link, does not need to perform a full orbit to resume the transmission. It can just continue the transmission with the next GS available in its orbit. This can re- sult in a very significant performance enhancement, both due to the very limited available time-windows and the typical characteristics of space communications, like high delays and low throughputs. Another interesting use case for such a solution is the transmission of data through, and within, cubesat clusters[8]. In this types of spacecraft constellations, such as the humsat project 4, transmissions can be tricky since every single node is moving on its own orbit, and also relative to earth’s GSs. Due to this, a distributed approach like the one proposed by this project can be very useful. With enough cubesats in a cluster, and taking advantage of a distributed solution, it could even be possible to create a real-time service for tracking ADS-B signals.
Figure 1.2: ISTSAT-1 Distributed transmission
1.2 Objectives and challenges
With this project we intend to create a Delay/Disruptive Tolerant (DT) protocol, prepared to deal with space communications related challenges that are already being dealt with in today existing DT proto- cols, but one that is capable of transmitting in a fully distributed manner. This means that, not only this protocol will be able to deal with disruptive prone environments, but it will also be capable of performing a single transmission through different nodes.
4https://www.humsat.org
3 In a typical cubesat communication, a DT transmission occurs between a ground station and a satellite (optional with other hops). This is not enough, specially when transmitting big data files. With the typi- cal DT protocols, this is handled by resuming the transmission (where it was left of) when the cubesat completes an orbit and the line-of-sight is restored. Note, however, the passes over the GS receiving (or transmitting) the big data are not always azimuth-favorable in consecutive orbits due to its quasi-polar characteristics[9]. Therefore, it might take a while to complete the entire transmission successfully for a given GS. With a distributed protocol the transmission could be performed over different ground-stations, located in different geographical locations. The proposed protocol should be bidirectional (i.e. capable of transmitting both in the uplink and down- link), capable of prioritization when dealing with several simultaneous transmissions, guarantee the cor- rect integrity of the received data and retransmit the broken segments. These are all common features found in transport protocols. This protocol is intended to be used over Open Systems Interconnection (OSI) layer 4 Reliable Datagram Protocol (RDP) protocol that in its turn will use Cubesat Space Protocol (CSP). CSP is a very popular, well tested, protocol for cubesat systems. It was designed for the purpose of space transmission and it is able to address different, possibly independent, modules of a given satellite. It works in both OSI layers 2 and 3 and its original implementation source code is freely available for download and changes.
1.3 Research contributions
During the development of this project, a scientific paper[10] was also written. It features the designed solution and some preliminary results. It was presented and published at the 15th International Sympo- sium on Wireless Communication Systems (ISWCS) conference in 2018.
1.4 Document structure
This report is organized in six chapters. The present one, Introduction, explains the proposed project, its objectives, motivations and challenges. The second one, State of the art, presents a comparison of the existing technologies, showing the need for this project and analyzing the good characteristics of the existing solutions, offering an useful insight for the design of this project. The third one, Architecture, focus one the logical design of developed solution. The fourth one, Implementation will explain in detail the protocol, taking as reference the design mentioned in the previous chapter. The fifth one, Experi- mental evaluation, will present practical results from experimental execution of the developed solution. And finally, the sixth one, Conclusion and future work, sums up the work of the whole document, the research involved, its conclusions and possible future work.
4 Chapter 2
State of the art
In this chapter we will compare some of the current solutions to the presented problem. These are used in the space communication link between the radio stations on earth (known as GS), and the orbiting satellites.
2.1 DTN
DTN [6] is a network architecture designed to address the problems of disruptive environments, like the ones found in this project related with space communications, (keeping in mind the inefficiency of terres- trial protocols to address this issues) like high delay environments, long distance communications and frequent disconnection of links. In this architecture a transmission is not terminated when its physical link is disconnected. Instead, link disruption is dealt with by pausing and resuming transmissions as needed, and by the use of multiple hops, in a store-and-forward[11] fashion. This strategy is illustrated in figure 2.1, where node A starts by transmitting to node B normally, but then it is forced to store the data, since a link is not available or was disrupted mid-transmission (as illustrated by the dotted line). When it does become available, the data is forwarded along the network using the same strategy at every node until it finally reaches its destination. To implement this architecture an end-to-end message-oriented overlay, called bundle layer [12], is de- fined over the OSI transport layer (layer 4). This compatibility layer ensures the independence of the DTN protocol from the rest of the stack, thus allowing a transmission to occur over different network stacks (like TCP/IP or AX25/CSP)[13]. In figure 2.2 this concept is illustrated, showing how a transmis- sion can occur over multiple regions, each with potentially different layers bellow the bundle layer. Devices implementing the bundle layer are called DTN nodes. Each node uses a persistent memory to store the transmissions being sent. This is done due to the disruption tolerant nature of the architecture. Since links are expected to disconnect, each node receives transmissions even if no output link is avail- able for the final receiver, stores the data, and then, when an output link becomes available, retransmits it towards its final destination. This means that a successful transmission can occur without, at any given moment, the existence of a complete path (with the use of hops) between the sender and the receiver.
5 Source
A
B Store
Carry
C Forward
Store D
Carry Normal link
E Forward
Disruptive link F
Destination
Figure 2.1: Store and Forward
Transmissions can be sent in a hop-by-hop manner, with each hop node storing persistently the data when its link is disconnected. Each DTN node is identified by an Endpoint Identifier (EID) which is based on Uniform Resource Identifiers (URI) [14]. In figure 2.3 a DTN transmission is illustrated and compared with a non DTN transmission over the in- ternet using the TCP/IP stack. In the top diagram (the non DTN transmission) a link disruption between any of hops means the termination of the transmission and the tear-down of the TCP connection. On the bottom diagram (the DTN transmission) a link disruption between any of the hops does not mean the termination of the transmission. Instead, all the nodes with valid links will continue to transmit. The ones without an output link will store the received data in their persistent memory, waiting for a chance to resume the transmission. This allows a transmission to be processed, never aborting or restarting when there is no connection between the sender and receiver. The concept of DTN began to be studied by the end of the 1990’s decade as part of the implementation of the Interplanetary Internet Project (IPN)1. A real life example of a DTN is the NASA Mars mission2. In this network the Mars Rover[15] (named curiosity), illustrated in figure 2.4, transmits data to one of the three orbiting satellites. They, in turn, store and retransmit this data to Earth when their orbit is favorable (when they have line-of-sight with Earth). While the Mars Rover could send the messages directly to Earth, this DTN approach enables it to upload data more efficiently and even when it has no line-of-sight with the Earth ground-stations. In summary, the DTN architecture addresses many of the problems of space networks that are charac- terized by an environment of long delays and discontinuous connectivity.
1http://ipnsig.org/ 2http://ipnsig.org/2013/07/09/curiosity-rover-the-store-forward-architecture/
6 Figure 2.2: DTN stack
Figure 2.3: DTN transmission comparison
7 Figure 2.4: Mars network
2.2 Saratoga
Saratoga[16][17], named after the USS Saratoga3, was first developed at Surrey Satellite Technology Ltd (SSTL) to download imagery from its remote-sensing Disaster Monitoring Constellation (DMC) satel- lites. Saratoga is a very scalable, peer-to-peer reliable transport protocol, capable of transferring very large files (exa-exabytes - 2128 B) guaranteeing error-free data delivery using a Selective Negative Acknowl- edge (SNACK) logic. Saratoga does not use any congestion control, it just sends the data at the highest possible rate in the link, with no regard for other flows. This is acceptable since space-links are typically dedicated. It is a protocol based in User Datagram Protocol (UDP) (can also be used with UDP-lite to minimize checksum computation overhead), designed to send data packets as fast as possible (since it is de- signed for private, low congestion links there is no need for congestion control) over several hops on privately owned links. It is also capable of transmitting over links with very long propagation delays since no forced timeouts are used. Saratoga uses a 32 B DATA message header, another 8B for the UDP header, 20 B for the IP header and another 18 B for the Ethernet header, for transmitting a payload of 1422 B (assuming a normal sized ethernet frame of 1500 B), offering a payload ratio of ∼95 %. The Saratoga protocol was developed with design goals of being very fast; to use cheap, reliable and robust internet technologies; provide more speed than TCP can offer; not caring about congestion, be- cause it is to be used as a single flow per link with no competition; support very big data file sizes; simultaneous delivery to multiple receivers. Two example implementations of the Saratoga specification are being developed, but are still incomplete. The first one is a Perl implementation4, that was written for interoperability testing, to validate the
3https://web.archive.org/web/20080328084822/http://www.chinfo.navy.mil/navpalib/ships/carriers/ histories/cv03-saratoga/cv03-saratoga.html 4http://saratoga.cvs.sourceforge.net/viewvc/saratoga/saratoga-v1-perl/
8 Figure 2.5: Saratoga Message Sequence Chart
Saratoga specification. It was not designed with performance in mind, which means that it is not suitable for a real scenario deployment, only for testing other implementations. The second one is a C++ implementation5, it is freely available for use and implements most of the Saratoga version 1 [18] (the latest one) specification. It is still incomplete but, unlike the perl implemen- tation, it is intended to be suitable for real world scenarios. Saratoga uses 5 types of messages:
• BEACON Sent periodically, the beacon message is used by each peer to declare its EID, its availability for receiving packets, and the max file descriptor (used to indicate the offset of the data segment) available (16, 32, 64 or 128 bits).
• REQUEST Used to request a file, directory listing, or a file deletion from a peer (in other word, a get transac- tion).
• METADATA Used to start the sending of a file, (in other words, a put transaction) it describes the file, sets the translation ID, and advertises the used file descriptor.
• DATA Used to send data. The file descriptor is used to indicate the offset of the data segment. An Acknowledge (ACK) message can be requested.
5http://sourceforge.net/p/saratoga/saratoga-vallona/ci/default/tree/
9 Figure 2.6: Saratoga stack
Figure 2.7: Saratoga STATUS message
• STATUS Used to report lists of missing data offsets (known has HOLESTOFILL) and errors. This imple- ments the SNACK logic of Saratoga. These are requested periodically by the sender (piggyback- ing a DATA message) asking the receiver to report the correct reception of the segments, and the missing HOLES as figure 2.7 illustrates. This means that the pace at which the ACK are sent is set by the sender. The receiver can also send STATUS messages when he "thinks" these are needed (e.g. start, restart, end of a transmission).
2.3 SCPS-TP
The Consultative Committe for Space Data Systems (CCSDS)[19][20] organization formulates recom- mendations to address problems found in space systems. One of this projects is the Space Communica- tions Protocol Specifications (SCPS), a set of extensions to existing protocols designed to address the typical problems encountered in space communication systems, with a great focus in promoting systems interoperability and cost reduction, these are divided in 4 categories:
10 • SCPS File transfer Protocol (SCPS-FP) - A set of extensions to File Transfer Protocol (FTP) to make it more bit efficient and to add advanced features such as record update within a file and integrity checking on file transfers. This is an optional protocol.
• SCPS Transfer Protocol (SCPS-TP) - SCPS-TP[21] uses a set of TCP options such as Negative Acknowledge (NACK), SNACK, modified slow start algorithms, modified congestion avoidance algorithms, and windows scaling. Additionally, SCPS-TP includes sender-side modifications to enhance the TCP performance in the stressed environments, such as long delays, high bit error rates, and significant asymmetries. As a result, the SCPS-TP stack is compatible with other recog- nized TCP implementations. SCPS-TP refers collectively to the protocols that provide full reliability (using TCP), best-effort reliability (using TCP with some modifications), and minimal reliability ser- vices (using UDP).
• SCPS Security Protocol (SCPS-SP) - SCPS-SP is an optional security protocol, and is compa- rable to IP Security (IPSec).
• SCPS Network Protocol (SCPS-NP) - A bit-efficient network protocol that is analogous to IP. However, it is not compatible with IP. The protocol is designed for use in space systems. It supports static or dynamic routing, precedence, and multiple routing options. This is an optional protocol.
In respect of efficiency, SCPS-TP uses a 20 B header, plus 20 B for the IP header, and 18 B for the Ethernet header, summing up to an overhead of 58 B for a 1442 B payload (assuming a normal Ethernet frame). This results in a payload ratio of ∼96 %. Some cubesat projects, such as the Dynamic Iono- sphere Cubesat Experiment (DICE) [22] are using the CCSDS standard. The SCPS-TP was designed to support a best-effort, minimal reliability strategy; to operate efficiently in a wide range of delay, band- width and errors conditions; operate efficiently in space-based processing environments; support priority connections; support connectionless multicasting; and support packet oriented applications.
2.4 T-CSP
CSP is a layer 4 protocol designed for space communication environments, specifically designed for cubesat systems. Tolerant CSP (T-CSP) is an extension of CSP, capable of performing in a DTN man- ner, and was designed and implemented by João Ferreira during his master’s degree dissertation in Communication Networks Engineering at Instituto Superior Técnico [3]. In his dissertation the goal was to send imagery data over an AX25 link, using CSP for both layer 2 and layer 3. Since AX25 has a very restrictive Maximum Transmission Unit (MTU) limit (256 B), a protocol capable of performing fragmenta- tion and packet recovery was necessary. T-CSP relies on a SNACK logic to ensure packet recovery. This approach was chosen due to the typical throughput asymmetry in space-links (The downlink tends to have more throughput). This means that a packet is only retransmitted if the receiver reports that it’s missing, otherwise its successful arrival is assumed.
11 In figure 2.8 a T-CSP segment is shown, where 1B is used for file identification, 2B for the segment number and another 2B for the total number of segments being transmitted. This structure allows for 255 possible different files and 65 535 segments per file, that results in a 15.44 MiB of max file size. Since space-links are characterized by being low debit this can be a reasonable value. In the case of overhead, since 29 B are used for transporting 247 B of payload, we are looking at a pay- load ratio of ∼90 %. T-CSP was designed to perform segment fragmentation, retransmit lost packets, and be compatible with the AX25 stack.
Figure 2.8: T-CSP segment over CSP packet
Figure 2.9 demonstrates a successful T-CSP file transmission. In the first step, the GS sends a command in a CSP packet (T-CSP is not a appropriate for sending commands), requesting the transmission of a file. In step two and three, the requested file is split into multiple segments that will be transmitted in T-CSP packets. These include their segment offset, since packets may be lost or not arrive in order. This process continues for until the n packet, at which point the GS software will be able to process the file as a whole.
Figure 2.9: T-CSP transmission example
12 2.5 Comparison
Table 2.1: Comparison of the existing solutions Features Saratoga SCPS-TP T-CSP Available implementation Yes No Yes Bidirectional Yes Yes Yes Auto-retransmission Yes Yes Yes Transmission error detection Yes Yes No Prioritization No Yes No Supported file sizes 256 EiB2 (2128 B) Not limited 15.44 MiB Overhead ∼5 % ∼4 % ∼10 % Congestion Control No Yes No Compatible with CSP No No Yes Distributed No No No
From these three protocols we can get a grasp of the type of problems encountered in space com- munications and the type of solutions chosen. Nevertheless, none of these are appropriate for resolving the presented problem. T-CSP was designed to work with the CSP protocol, uses a SNACK logic and retransmits lost segments but does not have error detection mechanisms. SCPS-TP covers a lot of the necessary requirements, but it is not compatible with the CSP protocol. Saratoga is also capable of a lot of what it is required, but is missing CSP compatibility. The major reason for the development of a new protocol, however, is that none of these are capable of transmitting in a distributed manner. This means that in all this cases, non of them are capable of taking usage of an available link to resume a transmission if this is not the original link through which the transmission was established.
13 14 Chapter 3
Architecture
With the distributed DT protocol, contrasting the current solutions, a transmission is not paused or can- celed when a link is disrupted, and instead, resumes over the next available node, only pausing indefi- nitely if absolutely necessary. This is specially important in LEO satellites like the cubesat systems that, due to their fast orbit, fly out of the transmitting GS line-of-sight very quickly. This solution removes the very time consuming need of waiting for a link re-establishment. Possibility requiring the passage of one or multiple full orbits, depending on the scenario in question. This is specially relevant when dealing with transmissions of big data files. Section 3.1.1, 3.1.2 and 3.1.4 will describe the solution for these scenarios. Another interesting example is the usage of a Cubesat to deliver a firmware update to a Wireless Sensor Network (WSN) with devices scattered over large distances. Here the simultaneous transmission of the data payload to all devices within the Cubesat Line Of Sight (LOS) would offer a major advantage over the typical Unicast transmission. Section 3.1.3 will describe this solution. The rest of this chapter will be dedicated to explaining the logical design and operations of the proposed solution.
3.1 Application scenarios
In this section we will describe the four major scenarios the proposed solution must solve, these are directly connected to the requirements described in section 3.2. The first two, One-to-one (section 3.1.1) and Multi-to-one (section 3.1.2), are already solved by current solutions but the last two(section 3.1.3 and 3.1.4), however, are not (with the exception of Saratoga protocol that can perform in a on-to-multi fashion).
3.1.1 One-to-one transmissions
The most simple scenario is a transmission with only one sender and one receiver. This is what most current solutions do, where the objective is to send or receive payload data from a system with a dis- ruptive network connection. Figure 3.1 demonstrates an example with a transmission between node A
15 and C through B with the dotted line representing a disruptive link. This can be achieved successfully by performing the transmission in a store-and-forward fashion, pausing and resuming the transmission accordingly to the link availability. In the example shown in figure 3.1 node B would store the data blocks received from node A, when the link with node C is unavailable, and forward them when it becomes available again. This is what current solutions do, as it will be further explained in section 3.4.1.
Source
A
B
C
Destination
Figure 3.1: One to one transmission
3.1.2 Multi-to-one transmissions
A slightly more complex scenario is one where several data payloads are transmitted at the same time, through the same intermediary nodes, and to possibly the same destination. This can be achieved by creating a protocol that allows multiple transmission to execute in parallel. In figure 3.2, an example is showed where three different connections are originated in nodes A, B and C and terminate in node F. An example of this would be a group of distant sensor networks placed in remote areas that need to deliver their data payloads to a collection server, and do so through a satellite link. This would be a multi- to-one example and would require a protocol capable of performing multiple parallel transmissions. The specific details of how this scenario is being addressed by the proposed solution are described in section 3.4.2.
16 Source 1 Source 2 Source 3
A B C
D
E
F Destination
Figure 3.2: Multi to one transmission
3.1.3 One-to-multi transmissions
A more complex scenario is the transmission of a data payload from one source to multiple destinations. This can obviously be done by simply transmitting the data N times for N destinations, but this is very inefficient. This is particularly true in the case of spacecrafts communicating with earth since, due to their altitude, they are gifted with a big line-of-sight radius that may cover more than one destination node. An example of this, that takes advantage of the large communication line of sight of a LEO spacecraft, is the transmission of a firmware update to N nodes on the terrain that are sufficiently close to each other to fit in the line-of-sight cone of the spacecraft. If one were to perform a multicast transmission, instead of executing N unicast transmissions, one could improve very significantly the overall transmission speed. This idea consist in bringing the multicast concept into DTNs. In the example shown in figure 3.3 the data blocks travel from node F to D in the same way they would in figure 3.1 and then are multicasted to node A, B and C. This offers the advantage of allowing three nodes to receive a payload while only performing one transmission. The major challenge in this scenario comes from the fact that each of the multicasted nodes (A, B and C) have links with different qualities due to their distances, meteorological conditions, Signal to Noise Ratio (SNR) and other characteristics. This means that packets that are successfully received by one node may be dropped by another. This challenge and the proposed solution will be explained in more detail in section 3.4.3.
17 Destination Destination Destination 1 2 3 A B C
D
E
F
Source
Figure 3.3: Multicast transmission
3.1.4 Distributed transmissions
The major feature of this project is its distributed nature. The intent is to allow a single transmission to be performed over several links, several nodes, with no fixed requirement over any single route. This works in a similar manner to a typical peer-to-peer network. In figure 3.4, an example is shown with a single transmission being performed from node C to A over several B nodes. Node C has several available links of disruptive nature and will use whichever is available at each moment to deliver the payload data blocks along the network. This results in not having to wait for the re-establishment of a specific link to resume a transmission, simply continuing sending data blocks over whatever links are available. The benefit of this feature is greater when there are multiple low quality links, but is however less relevant when a good quality link is available. A LEO cubesat with access to multiple GSs with low quality links is a case for which this approach could be very beneficial. Another example would be a Sattelite cluster where Inter Satellite Links (ISL)[23] are used to sent fragments of a big file to each satellite capable of transmitting the fragment to earth, thus increasing the overall transmission speed. This could be used, for example, for transmitting the ADS-B signal captured in space to earth in real-time.
18 Destination
A
B B B B
C
Source
Figure 3.4: Distributed transmission
3.2 Requirements
For the development of this project a list of requirements was defined. These are design decisions that will guide the development of an adequate solution to the presented problem:
• Distributed protocol The main goal of the project is to create a distributed protocol, capable of maintaining a single connection over different GSs or satellites, with different links, to transmit as continually as possible while the satellite line-of-sight time window passes over different geographical spots.
• Bidirectional communication Communication originated from both the ground-stations to the satellite (uplink), and from the satellite to the ground-stations (downlink), should be supported.
• Guarantee data integrity To guarantee data integrity means to be able to detecte if the received data suffered any alteration while being transmitted. Data corruption is a very common occurrence in radio communications, specially in such error prone environments has satellite communications. Data corruption can be detected with the use of a checksum algorithm. Generally this can be calculated, either at the transport layer level (calculated for each transmitted segment) or, once the transmission has ended, to the complete receive data, or both.
• Retransmission of broken segment To detect data corruption is one thing, to deal with it is another. The protocol should, by itself, detect and retransmit data that was incorrectly received.
19 • Designed to be used with layer 2/3 CSP Protocol CSP is a popular protocol designed for nanosats. It supports a connection oriented layer 4 protocol, network layer 3, data layer 2, and physical layer 1. For the purposes of this project’s distributed DTN protocol, CSP’s layer 2 and 3 implementation will be used.
• Use non-blocking mechanisms Since it is expected for the links to be disrupted and reconnected frequently, it is important that the implementation of the protocol does not leave the remaining program in an active wait state.
• The implementation should work in the same Operating Systems (OSs) where CSP works CSP implementation currently supports Windows, Linux, FreeRTOS, and MacOSX. The main fo- cus of this project will be on the Linux and FreeRTOS.
• The development should be performed according to ECSS-E-ST-40C[24] This standard developed by ESA covers all aspects of space software engineering including re- quirements definition, design, development, validation and maintenance. By following this stan- dard, one can produce better quality software, more prepared to be deployed in a real space environment. This is because this standard reflects the joint experience and knowledge gathered by ESA during many years of space activity and software development.
3.3 Protocol
The implemented Distributed Delay Tolerant Protocol (DDTP) protocol design was inspired on both the Saratoga protocol[18] (see section 2) and the peer-to-peer architecture[25]. In this section the term "transmission" will be used to describe a network communication between two directly connected nodes, and the term "session" to describe a DDTP, distributed, connection that spans several transmission over several nodes. The proposed solution must solve the scenarios described in section 3.1. It must outperform the current solutions, speeding up transmission times by the use of its distributed architecture. The major purpose of the implementation is the transmission of large data payloads from and to LEO cubesats, but it is also intended to solve the transmission of data in spacecraft clusters topologies.
3.3.1 Messages
A total of five message are used to communicate between network nodes, these are illustrated in figure 3.5:
• Session Info This is always the first message exchanged between two network nodes. It is used to describe the transmission session. Proceeding messages will be related to this session. It is part of the initial handshake that takes places in the beginning of any communication, that will be explained in more
20 detail in section 3.3.2. It’s contents include the session destination address, source address and priority.
• Metadata Describes a collection of data blocks to be transmitted. This includes the offset of this data blocks in the overall, distributed, transmission, the data blocks count and the checksums of these data blocks. This message is always sent after the session info message, during the initial handshake.
• Control This message is the most unique among the rest, since it is used to send session information between the two endpoint nodes. These are: the node who has all the data blocks and started the transmission, and the final destination node. This includes fin, snack, ack flags that inform the sending node of the termination of a session, a list of data blocks that were not received, and a sequence of data blocks that were received, respectively. This message represents the only direct communication that is transmitted between the two end nodes, although, it will most likely need to be relayed through intermediary nodes.
• Status Serves the purpose of delivering responses to a node. This is used in the final step of the initial handshake, in order to inform the node requesting a transmission if this is accepted or not. Multiple status codes are available for rejecting a transmission in order to inform the node of the reason for this rejection. This include lack of space, power saving mode, and already processing a priority transmission.
• ACK This is a simple acknowledge message. It serves the purpose of reporting to the seeding node that a sequence of blocks have been received. This information is important so that the sending node can mark these datablocks as successfully sent and reuse the storage space if necessary.
Describes a collection of data pieces, requesting the Metadata receiving node to accept the pieces and route them along to the final destination.
Error and status messages, normally sent as a response to Status other messages.
Describes a ongoing transmission between two nodes. This Session Info is used to iden�ty the session and consequently the des�na�on of the proceeding messages. Only processed by the end-to-end nodes. It is used to Control deliver connection related information to both ends. This includes retransmission requests, sacks, fin, ack.
ACK piece message. ACK
Figure 3.5: DDTP Message types
21 3.3.2 Algorithm
The protocol algorithm is defined by two state machines one for controlling the reception and another the transmission procedures. Moreover, one of the core rules of a session is that when a block is ex- changed, the receiving node takes in the responsibility for that block, meaning it should keep forwarding it along the route that will eventually reach its final session destination. For route determination each node looks at a table kept in memory. Each entry in the table contains met- ric, destination and route values. There are several possibilities for how the routing table values could be determined. The simplest one consists in static values, with a lot more complex examples being possible, like dynamically changing the table to reflect the best space-link route computed by analyzing the orbital parameters of a spacecraft orbit[26]. This is however outside the focus of this dissertation. Data blocks then flow trough the network, in no particular order, until they arrive at their destination. If a data block does not arrive after a long time (e.g. node crashed or a particular link has been offline for a long time) the receiving node sends a control message (figure 3.5) with a SNACK flag for the block in question. One of the protocol requirements is that it should be usable in low resource embedded systems, so some compromises need to be made. Instead of identifying each data block that is going to be sent through a cryptographic-safe hash, like in the case of a typical peer-to-peer protocol, a sequence number along with the transmission session ID is sent.
Each active node in the network starts in the standby state. This state is where the protocol remains when it has no work to do, more specifically when it doesn’t have anything to send or receive. There are two ways to leave this state, either a network communication is received from another node, moving therefore to the reception state machine (figure 3.7), or a request for a new transmission of data is received from the upper layers of software, moving the protocol to the sender state machine (figure 3.6). This process also moves the receiver into the pre recv session info state and the sender to the pre send session info state. In every exchange between two peers the Session info message (see figure 3.5) is always the first one sent. This message describes the session in question, its identification (the source address, session id pair) and priority. The transmission and reception of this message moves the receiving peer from the pre recv session info to the pre recv state, as illustrated in figure 3.7. If the sender intends to send data blocks, it will move to the pre sending state, but if a control message is to be sent, it will move to the send control state, as illustrated in figure 3.6. In the case of a control message, the receiver moves to the control state. At this point the receiver can send a message either accepting or rejecting the control message. If the message is rejected, it is up to the sender to find another route for delivering it to it’s destination. If however it is accepted, the receiver will store the message and forward to its destination. This moves both machine states to the end state. Control messages are only processed by the end-to-end transmission peers, every other peer will not look into its contents. The only exception to this rule is control messages with a retransmission flag. These may trigger a data block transmission in a intermediary peer if the peer contains the data block in
22 storage, either in alive or zombie state (see section 4.1.5). If a metadata is received, however, the receiver move to the pre recv 2 state, will parse it and retrieve the information about the data blocks that the sender intends to send. The sender on its hand will move to the pre sending 2 state. The receiver will respond with a status message reporting if the transmission is accepted or not. Reasons for rejecting include lack of storage space, no route available for the session and power saving modes. If the transmission is accept, however, the sender move to the sending state and starts sending the data blocks described in the metadata message until completion or the link drops, while the receiver move to the recv state and receives the mentioned data blocks.
Recv Status(reject) End
Recv last ACK Send Session_info Send Metadata Recv Status(accept)
Pre_send Pre sending Pre sending Sending Session_info p2 Standby Send next block Recv Status(reject)
Send Session_Info Send Control End
Recv Status(accept) and queue message
Figure 3.6: DDTP sender process state machine
Sent Status(Reject) Recv Session_info Recv Metadata Sent Status(accept) End
Pre_recv Pre_recv Pre_recv2 Recv Sent last ack session_info Standby
Send Status(reject) Recv next block
Recv Control Control End
Send Status(accept) and mark message has sent
Figure 3.7: DDTP receiver process state machine
23 3.4 Application Scenarios operations
Taking in consideration the scenarios described in section 3.1 and the solution proposed in section 3.3, the operational behavior of the developed protocol will now be explained in each scenario. This is a theoretical description that will focus on the more technical aspects, experimental results will presented in chapter 5.
3.4.1 One-to-one transmissions
Even though the first scenario is the most simple one, and it is already solved by the current solutions, it is still very relevant. This is because it allows one to test the protocol algorithm in a simple case first, before going to the more complex scenarios, and because it allow one to compare the proposed solution performance with the current solutions in the same context. In figure 3.8 a simplified Message Sequence Chart (MSC) of a transmission between two, directly con- nected, nodes is depicted. In this example, node A initiates the transmission by sending a session info and metadata messages. The session info describes the session itself. It contains the addresses, priority and session id (data structures will be explained in section 4.1.5). The metadata message de- scribes the data blocks that will be sent in this particular transmission. The receiver (node B) after receiving these messages will have all the information it needs to "decide" if it is able to accept and forward along the data blocks. Since in this particular example node B is the final destination, it will sent a status message informing node A that it accepts the transmission. After this node A starts sending data blocks. This process is very similar to the one described in section 2.2. This is to be expected since they are both solving the same scenario. The DDTP exchange however requires a higher network overhead since more metadata is sent. This may be decremental in this particular scenario, but will result in a much higher total throughput in the new scenarios.
A B
Session info
Metadata
Status
Data
Data ... Ack
Figure 3.8: MSC One-to-one transmission
24 3.4.2 Multi-to-one transmissions
A generalization of the previous scenario is the processing of multiple transmissions in parallel. This means that the protocol implementation should be able to handle the simultaneously reception of mes- sages from different sessions. In figure 3.9 an example of this is showed. Nodes A and C have data blocks to send along the network and have selected node C as the next hop, simultaneously. Just in the previous scenario, they sent a session info and metadata message to node B, describing the transmission itself and the data blocks that they intend to send. At this point, node B parses this data and responds with a status message, ether accepting or rejecting the requested transmission. In this example, node B has accepted both transmissions (this can be observed since node A and C proceeded by sending data blocks), but it could also had been the case that transmission A would had been rejected, or transmission B, or even both. Reasons for a rejection include not enough memory for storing the offered data, energy conserving modes and a priority transmission is already using the link. A B C
Session info Session info
Metadata Metadata
Status
Status
Data Data
Data Data
Ack ...Ack ...
Figure 3.9: MSC Multi-to-one transmission
3.4.3 One-to-multi transmissions
In figure 3.10 an example of this scenario is displayed. The major difference in this scenario is the usage of multicast messages. This means that one cannot rely on the receivers acquiring all the messages (there is not re-transmissions at the transport layer). This means that the data messages need to be sent together with the information about the DDTP session. This is displayed in the example figure, over the RDP layers, the necessary fields are inserted in the DATA message. There is offcourse no way to ensure that all the data blocks arrive at all the receivers correctly, but for this the retransmission message explained in section 3.3.1 will be use by the receiver to request a normal, unicast, retransmission of the mission data blocks.
25 DATA
Session Sequence Info
RDP Multicast
CSP Multicast
AX25
Figure 3.10: One-to-multi transmission
3.4.4 Distributed transmissions
In this section, while using the concepts introduced in the previous scenarios, a distributed transmission will be explained. In figure 3.11 an example is illustrated, where each black arrow corresponds to a one-to-one transmission (explained in section 3.4.1) and each red arrow represents a link drop. The example is divided in three steps, A, B, and C that correspond to three transmission opportunities in an orbit. The example corresponds to a spacecraft (ISTSAT-1 in the figure) downlinking to earth a data payload through multiple GSs. This is the major use case for which this protocol was developed. When the spacecraft (ISTSAT-1 in the figure) goes over the first GS (GS1 in the figure) it obtains its first transmission opportunity. During this time windows the ISTSAT-1 will transmit as much as possible of the data blocks until the link drops and it is forced stop. This pieces are then sent from the GS1 to the Mission Control Center (MCC) server through the Internet. An important thing to notice is that even though, in this example, multiple transmission will occur over different links, in a distributed manner, each individual transmission is performed in the same way as a one-to-one transmission from section 3.4.1. This process repeats two more time in step B and C until all data blocks are sent and reassembled in their final destination, the MCC server. Another interesting possible example is the transmission of data to earth from a cluster of cubesats. This is illustrated in figure 3.12, where the blue spacecraft, in order to transmit the data to earth, needs to forward the data through the remaining spacecrafts. The non-dotted lines represent good quality links, which are spacecraft-to-spacecraft and earth communications, and the dotted lines represent low quality disruptive links, which are spacecraft-GS communications. This example can also be solved by the distributed transmission procedure, where the first spacecraft (marked with an A in the figure) will first transmit the data to other spacecrafts that can reach earth,
26 who will in turn, according to their transmission opportunities, forward the data to earth. The spacecraft selection depends on the route algorithm used, as explained in section 4.1. If two or more GSs are at transmission range of the spacecraft, the selection of which will be used will also depend on the route algorithm. Transmission collisions are avoided since all packets are addressed to a single receiver.
ISTSAT-1 GS1 GS2 GS3 MCC Server
Transmission Transmission A)
Link Drop
Transmission
Transmission B)
Link Drop
Transmission
Transmission C)
Link Drop
Figure 3.11: MCC Distributed Transmission
27 A)
B)
Figure 3.12: Distributed transmission - cubesat cluster example
3.4.5 POCKET++
Pocket++[27] is a lossless compression algorithm developed by ESA, designed for spacecrafts with few resources. It is able to achieve a good compression ratio, specially on data payloads with period information, such as is usually the case with spacecrafts telemetries. For the ISTSAT-1 however, the POCKET++ compression will be used for compressing the captured ADS-B records, and thus it should work together with the DDTP protocol. Since DDTP does not look into the contents of the data payload it is transmitting, there is no need to adapt its architecure to support Pocket++. There are however some requirements for transmitting Pocket++ compressed data, of which the major one is that the data being compressed must be divided into periodic, fixed sized, frames. This means that the usage of Pocket++ over DDTP can impact the transmission quality, since each frame can only be decoded after it is completely received. The impact of such combination will be explained in chapter 5.
28 Chapter 4
Implementation
Any software project should enforce a clearly defined set of methodologies, tools and procedures to be used in its development. This very important for the success of any engineering project, but particularly in a project like this, that not only features a lot of members working together, from different engineering and professional backgrounds, but also where any error or mistake can be fatal, like it is typically the case in space related projects. In this section the complete methodology employed in the development of this project will be explained in detail, some originating from the ISTSAT-1 project, others from the ESA FYS programme and some from the DDTP project itself.The implementation will also be explained in detail. This includes all the design decisions that where made to accommodate the specified requirements in the best way possible.
4.1 Software Architecture
The software development was divided in three modular sections: protocol, route and storage, as pic- tured in figure 4.1. These were implemented in the form of independent software libraries, containing the core protocol implementation, the route calculation, and the payload data storage module, respec- tively. This approach was used be the DDTP implementation is designed to be used in different kinds of systems, with architectures and capabilities. The storage lib creates an abstraction level between the protocol lib and the selected storage medium, this results in an implementation that is not only designed for one storage method but can be more easily ported to multiple. In the illustrated example two mediums are available: RAM and flash storage. The route lib offers an interface for accessing the routing table and dynamically updating its entries if such is desired, allowing, due to its modular nature, the implementation of multiple route calculation algorithms. The protocol lib contains the actual software implementation of the DDTP protocol, including its algo- rithms and messages. The application level software can be any software that needs to perform a DDTP transmission. The only requirement it needs to fulfill is to interface with the protocol library in order to perform a DDTP
29 transmission.
Application
Route Lib Protocol Lib Storage Lib
Routes DDTP Storage
Algorithm A Algorithm B Flash Storage RAM Storage
Figure 4.1: DDTP modules
In figure 4.2 a simplified version of the software structure is illustrated. The project was divided into ten major files (more were used but are omitted here for the purpose of clarity and lack of relevance). These are:
• ddtp The main file that gives the upper layers of software access to the DDTP protocol. This includes functions as ddtp_recv and ddtp_send that serve the purpose of queuing data to be sent and received. In the example figure 4.2 these functions are being called by the main function.
• ddtp_sm The file that contains all the information related to the DDTP state machines explained in section 3.3.2.
• ddtp_conn This file defines the structures and functions related to each connection being processed. Together with ddtp_sm, this two files contain most of the internal state of the program when it is running.
• ddtn_messages Defines all the protocol messages, its construction, serialization and parsing.
• ddtp_rtable The protocol routes are stored and processed here. Can use external files as source of the routing algorithm.
• ddtp_storage Defines all the functions and structures related to the storage of data blocks. Can use external files to defines new storage mediums.
• ddtn_net All access to the lower network layers are defined in this file. The purpose of this file is simply to act as an interface to the actual files that defined each network layer. Depending on the compilation options and interface in question different network layers will be accessed.
30 • ddtn_net_rdp The file that implements the access to the actual RDP protocol. The real RDP protocol processing is done inside the libcsp C library.
• ddtn_net_inet The file that implements the access to the TCP/IP protocol.
• ddtn_net_omnet The file that implements the access to the OMNET simulated TCP/IP and UDP/IP protocol.
app
main()
ddtp.h
ddtp_close() ddtp_send() ddtp_create() ddtp_task() ddtp_init() ddtp_halt() ddtp_recv() ddtp_accept()
ddtp_sm.h ddtp_conn.h ddtp_messages.h ......
ddtp_net.h
ddtp_rtable.h ddtp_net_recv() ddtp_net_send() ddtp_net_accept()
... ddtp_net_create() ddtp_net_close() ddtp_net_bindandlisten()
ddtp_net_init() ddtp_net_halt()
ddtp_storage.h
... ddtp_rdp.h ddtp_inet.h ddtp_omnet.h ......
libcsp.h socket.h omnet_inet.h ......
Figure 4.2: DDTP software architecture
4.1.1 Multi platform
One of the DDTP protocol requirements is the support of multiple OSs. To achieve this, an architecture similar to the Libcsp was used. All system related functionality is isolated in separated modules, one
31 for each different OS. Every time a system related call is required by the software, instead of directly accessing a particular system interface (which would result in system lock-in), an internal DDTP inter- face module is called. This interface, called Operating System Abstration Layer (OSAL) in figure 4.3, will select the module of the particular system being called, make the necessary adjustments according to the system features and limitations, and execute the system depend functionality. This abstraction layer offer the possibility of easily porting the software to different platforms by only creating a new platform specific module that implements the specified interface. Without this technique it would be necessary to perform deep and unclear modifications to the source code, with a high probability of creating new problems and bugs. The compilation procedure was designed so that one is able to only include the modules corresponding to the necessary platforms, thus reducing the compiled executable size and memory footprint. Despite the OSAL interface, an effort was made to avoid using functionalities that were not available in all the targeted platforms, since doing so would require the development of this functionalities in the modules where it was unavailable. Particularity the FreeRTOS platform is much slimmer in terms of functionality than the remaining platforms. This design follows the common mentioned software development principle: The code should be de- signed to be extended, not modified. Modules for three different platforms were developed, which will be explained bellow, but more could be easily created.
Protocol Lib
DDTP
Operating System Abstraction Layer (OSAL)
FreeRTOS Linux OMNETPP
Figure 4.3: DDTP software platform architecture
4.1.2 Linux
The first platform for which the DDTP protocol was developed was Linux. This was the most obvious choice since all the development was done in Linux and it is also the most convenient system for execut- ing all the tests described in section 4.7, that were essential to maintaining the project quality. Moreover the ISTSAT-1 GS will running Linux, making it a very important platform for this project.
32 4.1.3 OMNET
For the purpose of testing and performance analysis the OMNET simulator was used. This simulator interfaces with the DDTP project twice, one for sending and receiving messages through the simulated network, and another for executing the protocol and the associated application itself. Since a simulator can create events, artificially accelerate the simulation time and extract data from the execution, such integration is necessary. For the protocol interface the OSAL level was used, like the other implemented platforms. The simulator itself was executed in Linux.
4.1.4 Multi Network stack
The DDTP protocol is being developed under three different network stacks: TCP/IP, CSP/RDP and OMNET’s simulated TCP/IP stack. This is illustrated in figure 4.4 where, in similarity to the approach used for the multi-platform problem described in section 4.1.1, an abstraction layer was used. This layer was named Network Protocol Abstration Layer (NPAL) and it abstracts every access between the DDTP protocol and its lower network stacks. This allows one to easily extend the project with new network stacks, add and remove stack modules without breaking the project (something very useful for memory constrained systems) and have multi stack compilations. This modular approach also allows one to test each module in isolation, something very useful for pinpointing problems early on.
Protocol Lib
DDTP
Network Protocol Abstration Layer (NPAL)
TCP/IPV4 RDP/CSP OMNETPP
Figure 4.4: DDTP - Network Abstraction Layer
Multiple network stacks are necessary for this project since communications will travel over two very distinct mediums, requiring different protocols. In figure 4.5 the multiple stacks are illustrated, where depending on the different medium, either the RDP/CSP or TCP/IP stacks will be used.
The CSP/RDP stack is useful for space-link communications, like the ones between GSs and space- crafts or spacecraft-spacecraft. This is due to its simplicity and low overhead. In this scenario the major concern is the quality of the link and not its congestion, in contrast with the typical earthly communication scenario like Internet connections.
33 Application App App App
Bundle Common over across DDTP DDTP DDTP all network regions Transport RDP TCP OMNETPP Network Specific for each CSP IP network region Simulator Data Link AX25 Ethernet
Figure 4.5: DDTP Network stack
App App
DDTP DDTP DDTP DDTP
RDP RDP RDP TCP TCP
CSP CSP CSP IP IP
AX25 AX25 AX25 Eth Ethernet
Figure 4.6: DDTP Multi stack transmission
The TCP/IP stack is, on the other hand, appropriate for Internet connections and is being used for the purpose of inter connecting GSs. Both TCP and RDP protocols offer guaranties of integrity, message order and delivery acknowledgment. These are the minimal requirements for a network stack running the DDTP protocol. The OMNET stack is only used for testing, debuging and performance evaluation purposes. It comprises of a simulated TCP/IP stack that runs inside the OMNET simulator. A single transmission can flow trough more than one stack as long there is a multi-stack node, like it is illustrated in figure 4.6. The nodes with multi stack capabilities are called gateways and are necessary for enabling communications between the different mediums.
4.1.5 Data structures
Each DDTP session is defined by four parameters:
• Source and destination address These addresses identify the nodes sending and receiving the data, respectively. These identify uniquely the nodes and are not related to their OSI Layer 3 addresses. Since the DDTP protocol was developed to work with big networks, something specially relevant in the case of cubesat clusters, this address has 32bits in size. Although, since the implementation was designed to be configurable, the size could be easily changed without requiring software adaptations.
• Session ID
34 This is a counter of how many sessions a node has initiated. Each node stores this value for its own sessions. Together with the source address the session id identifies uniquely a session. This pair of values is always the identification used for a session, and is sent in the metadata message in the beginning of a transmission, as explained in section 3.3.1.
• Priority Represents the priority value of a session related to other sessions that may be executed in parallel over a single link. This is an optional field.
In addition to these, a data block counter is used to identify each data block in a session. This is neces- sary since data blocks do not arrive in an orderly fashion, due to the distributed nature of the protocol, and to identify block in retransmission requests. In a regular Peer2peer protocol, a cryptography-safe computational digest would be used for both identifying the block and confirming their integrity. Although this solution is more elegant, it is not feasible in a cubesat like system where processing power is very limited, so the mentioned solution was chosen. Each data block stored in by a peer can be in three different states:
• Alive A datablock relative to a session is stored. And, as such, needs to be forwarded to the next peer in route to its destination.
• Dead The structure is free and can be safely used.
• Zombie The datablock is free and can be used, but contains a valid data block of a session. When a node transmits a data block to another one, it no longer needs to store it, but there is however no cost associated with keeping the data stored. Because of this, the data block state is simply changed to zombie and all software references to this block are maintained. Besides keeping this data available, even though it has already been sent, its real advantage comes from the fact that in the event of a retransmission request from the session destination node, the data block can be recovered and retransmitted at no added cost.
The project was developed with the intent of being highly customized. For the Linux implementation a key/value configuration file can be used to configure the software with all the possible parameters. This allows one to change the configurations without having to recompile the software. This can be very useful for GS configurations. For embedded systems however, like the ones using FreeRTOS, this solution is not possible. So the same parameters needs to be defined by hard-coding them in the software before uploading the resulting binary executable to the system. These parameters include the following values:
• interfaces
35 • interface costs
• routes
• gateway
• network stacks
• log level
• addresses
4.2 Programming language selection
One of the first decisions that it was necessary to make, was selecting the programming language to be used in the project development. A crucial decision that by itself establishes a lot of the project develop- ment foundations. There are two major requirements in the development of software to be deployed in a spacecraft: Reliability and low resource usage. Particularly in a Cubesat the resources available are very limited. In terms of resource usage, the C programming language as a clear advantage. By being a very low level language (sometimes even referred as high level assembly) its overhead is minimal, the available compilers produce very optimized machine code and it allows the programmer to access low level infor- mation that can be used to write more optimized code in certain instances. All this comes at the cost of programming complexity however. For this requirement all the modern, very popular, high level languages as Java, Python and Ruby are completely out of question. They would bring great benefits, not only by allowing the programmer to write less complex, more easily understandable and maintainable code, but also due to the security and robustness that their automatic memory management offers (the majority of errors in C written code originates during memory and pointers operations). However this advantages come at the cost of a very high level, resource demanding system, that is not acceptable for a Cubesat like the ISTSAT-1. In terms of reliability the C programming language is a very well known and tested solution. Its been constantly used by the industry since it was first developed in 1972, there are many well test, constantly in use, freely available compilers and has a lot of space heritage. Its lacks however a memory manager, thus requiring the programmer to perform all the memory accesses itself. This is typically a cause of many problems. The use of strict coding rules and guidelines (like the ones specified by NASA or ESA) can however greatly reduce this risk. The C programming language is also the most portable by far. Whichever combination of Center Pro- cessing Unit (CPU), Instruction Set Architecture (ISA), and hardware one selects, there will probably be a C compiler available. Since the ISTSAT-1 is still in development, this is an important factor. This also allows one to easily share code between different subsystems in the spacecraft and on earth. Considering the overall scenario, the C programming language was chosen for the development of this
36 project. Not only does it fulfill the needed requirements, but along with its space heritage and portability it makes a very solid choice.
4.3 Build tools
Since the project needs to be compiled and build in several different configurations, both due to its modular software approach explained in section 4.1 and supporting multiple platforms a flexible solution for compiling the project was required. The cmake tool was chosen due to its reliability from being a well tested and wildly used solution that is also multiplatform. The gitlab automatic test suite was also used, this not only allows the frequent build of different versions of the software during its development, but also running code analysis and several tests frequently.
4.4 Code Analysis
There are two types of code analysis that are of great benefit to any software project: Static and dynamic. Both were used in this project.
4.4.1 Static
For static analysis two tools were used, an Integrated Development Environment (IDE), named Inteljea Clion, and the clang compiler static analysis tool. These tools offer different advantages and are better used together. The clang compiler static analysis tool, however offers a slower but more deep insight into the state of the written code, checking for more complex bugs, coding styles inconsistencies, and other hard to track problems. These include checking for out of bounds, memory leaks, uninitialized variables and functions, cast problems, initialization problems, null pointers and some others.
4.4.2 Dynamic
For dynamic analysis two tools were used, GNU Debugger (GDB) and Valgrind. The GDB, a well known tool, allows one to monitor, pause, and edit the developed software during its execution. Valgrind on the other hand is a dynamic analysis tool capable of detecting memory leaks, uninitialized or un-allocated memory usage. This is extremely important for a project such as this, software bugs can be critical.
4.5 Documentation
A common problem that occurs in many software projects is the lack of documentation. Even a very well developed project can become unusable if there is no record of how it can be used. Particularly in complex or critical project (like space related ones) this requirement becomes even more serious. To
37 solve this problem two types of documentations were produced: Software reference documentation and Design documentation.
4.5.1 Software reference
Software reference documentation is a type of document that describes in detail the full API (ideally both internal and external to the project) of the developed software. This should detail all the relevant information about the project functions, variables, directories, files and anything else that may be rele- vant. This can be a daunting task, but if one uses the proper tools and consistently keeps documenting the developed software throughout the project, it becomes a simple task that will greatly improve the project’s quality. To achieve this purpose the Doxygen1 software was used, a tool that generates reference documenta- tion from analyzing annotated C source code. This results in both consistently documented code, in a standard format, and the production of a very detailed reference documentation.
4.6 Source Code Version Control
Using a source code version control tool is not necessary for any software project, but it was also highly recommended by the ESA FYS program. It allows one to keep a commented history of all the modifications done throughout the project, allows a hole team to work collaboratively while keeping a backup copy in a server. Initially subversion was used and latter the project was moved to git. Both tools are fine to use for this project, with the git tool being a more optimized, distributed solution and subversion working in a classical, centralized, server-client model. A git related tool called gitlab was used to complement the source version control with some other features like automatic testing capabilities that will explained in more detail in section 4.7.
4.7 Testing
Constant and thorough testing is a fundamental part of any software project, but it is even more important when it is space related. For the development of the DDTP protocol two types of tests were used Unit, and load tests.
• Unit Test Unit testing is a technique for checking and maintaining the developed code quality. For each unit of code, normally this can be functions, files or modules, one or more tests are developed. These will execute the unit with different input values, test its execution intermediary values through assert statements, and test the resulting output. This simple concept allows one to test the software with a very small granularity. This is very important in order to find problems early, precisely, and
1https://www.stack.nl/~dimitri/doxygen/
38 throughout the development of the project. New problems and bug can be discretely introduced each time one changes to a project. To perform this task an open-source C lib was used called munit 2.
• Load tests To ensure that the software is capable of functioning correctly when its operational limits are reached, load test are used. Theses tests run several executions of the software in limit push- ing situations (eg. over maximum connection requests, very high delays, very small delays) and assert whether the software is able to deal with this situations without deriving from following the specification (eg. crashing). To achieve this a simulator was used. Due to the exotic nature of this protocol (distributed radio communications) fully non simulated executions are challenging, and since an important factor of any kind of software test is to be able to execute it frequently, OMNET a network simulator was used. The usage of this simulator and its related environment will be explained in section 5.
4.8 Development Environment
All the software development was done in a Linux OS, with the remaining platforms and network stacks being implemented at a second stage. The first execution corresponded to simply running unit tests, which progressed to localhost transmissions until it was necessary to use a network emulator to execute more complex scenarios. For the purpose of executing the complete developed protocol (as opposed to the unit tests, that execute small unit individually), the OMNET3 was used. This software simulates a network typology, allowing one to prepare big and complex networks, both wired and wireless, with programmable event, while allowing the nodes to execute arbitrary C++ software. This means that it was possible to adapt the DDTP protocol to run in the simulated nodes of a OMNET network. Other possibilities were also considered, like the Netkit4 simulator, that even though it is a less powerfull solution compared to OMNET it does not require the implementation of a new network interface, as uses the TCP/IP Linux interface. This however proved not to be a sufficiently capable solution for the scenarios that needed to be emulated. In term of programming practices. a Keep It Simple and Stupid (KISS) approach was used, the clang-tidy tool was used for setting and keeping a code format and the ESA coding standard for space software was followed, in order to keep the software at a level of quality necessary for space deployment. One of the key requirements of this standard is that one should not use any type of dynamic memory allocation. All the memory resources should be allocated at the beginning of program execution. This is not only to avoid the typical memory allocation bugs but also to allow one to easily calculate how much memory the software will use, an important metric since spacecrafts tend to be very resource constrained.
2https://github.com/nemequ/munit 3https://www.omnetpp.org/ 4http://wiki.netkit.org/man/man7/netkit.7.html
39 40 Chapter 5
Experimental evaluation
In order to validate and quantify the reliability, functionality and quality of the designed solution, a set of tests were performed. These are divided into the scenarios described in section 3.1 so that one can confirm and analyze the core features of DDTP. Since the characteristics necessary for testing a distributed DT protocol (such as high latency, high error rate, link disruptions, distributed network) are difficult to emulate in laboratory, the OMNET network emulator was used to create a simulated network. To add more value to the obtained values and compare them to the current available solutions, a publicly available saratoga reference implementation was evaluated in the same scenarios. An unmodified copy of its C++ implementation1 was used at revision 56b9f1. In figure 5.1 the simulated topology used for the following tests is illustrated. Each node of the network is connected to each other by a bidirectional channel with a datarate of 10 KB/s, where links can be disconnected during a transmission, through scheduled events, or even before the simulation (forcing a transmission to follow a particular route). All nodes in the topology are running the same DDTP imple- mentation, with same the upper layer software (see figure 4.3). They do however contain different route and network configurations, since they are placed in different network positions. It should be noted that node A and B are the only two nodes that contain more than one link, and so, have the possibility of using another network route.
1https://sourceforge.net/p/saratoga/saratoga-vallona/ci/default/tree/
41 α Link Link B γ
Link δ A β Link D
C
Figure 5.1: Testing simulation topology
5.1 One to one
In order to measure the performance of both protocols in a one-to-one scenario, using the topology illustrated in figure 5.1, multiple transmissions were performed between node A and B, exclusively over link α (link β was disconnect), with multiple data payload sizes. All the values obtained in this section were captured in the network interface of node B. In figure 5.2 the DDTP and Saratoga transmissions are illustrated, where one can observe that the trans- mission time increases linearly with the payload size. In this scenario the Saratoga protocol is clearly faster, which is to be expected since it uses less metadata due to its decreased functionality compared with DDTP. However, even thought the difference is relatively small, this means that in a exclusively (combinations are possible) one-to-one scenario, a simpler protocol like Saratoga is preferable. In table 5.1, the values of the transmission are displayed, with DDTP achieving an average throughput of 8.909 KB/s and Saratoga of 9.295 KB/s.
Table 5.1: One to one transmission Time (s) Throughput (kB/s) Size (KB) DDTP Saratoga DDTP Saratoga 100 11.9 10.6 8.4 9.4 200 22.8 21.4 8.8 9.3 300 33.1 32.1 9.1 9.4 400 44.1 42.9 9.1 9.3 500 55.0 53.4 9.1 9.4 1000 111.7 109.4 8.9 9.1 2000 222.1 218.7 9.0 9.1
42 60
50
40
30 DDTP Time (s)Time Saratoga
20
10
0 0 100 200 300 400 500 600 Data payload (KiB)
Figure 5.2: DDTP vs Saratoga - One to one
5.1.1 One to one - Delay tolerant
Another feature that requires verification is the DDTP DT capabilities. Again, the transmissions were performed between node A and B (5.1), exclusively through link α (link β was disconnect), with a 200 KB data payload, and all traffic was captured at the network interface of node B. In figure 5.3 the DDTP transmission throughput is displayed in function of the time, where each point in the chart corresponds to a 1s average of the throughput. A link disruption was scheduled at the 5s mark, causing an unexpected interruption in the transmission. This forces the transmission to pause, waiting for the link re-connection, since no other routes are available. As can be observed in the mentioned figure, from the 5s mark to 53 s no data was transmitted. At that point the link reconnection is detected and the transmission is resumed, finishing at 72 s. As was desired, the property of being able to pause and resume a transmission according to the link availability was validated. This is one of the core requirements of the developed protocol. In figure 5.4 the same experience, in the same conditions, was repeated with the Saratoga imple- mentation. Here the link is once again disconnected at the 5s mark, reconnecting at 20 s. Similarly to DDTP the transmission was able to pause and resume accordingly to the availability of the link.
43 10000
9000
8000
7000
6000
5000 Route A
Throughput (B/s) (B/s) Throughput 4000
3000
2000
1000
0 0 10 20 30 40 50 60 70 80 Time (s)
Figure 5.3: DDTP DT transmission
12000
10000
8000
6000 Route A Throughput (B/s) (B/s) Throughput
4000
2000
0 0 5 10 15 20 25 30 35 40 Time (s)
Figure 5.4: Saratoga DT transmission
44 5.2 Multi to one
In order to validate DDTP abilities in a multi-to-one scenario two transmissions were performed. Using the same topology from the previous section (see figure 5.1), two transmissions of a 200 KiB data pay- load were performed simultaneously. One from node B to D, over link γ, and another from node C to D over link δ. It should be noted that in this experiment node D is receiving both transmissions at the same time. Link α and β were disconnected to ensure that no traffic goes through node A. The experiment was performed with both DDTP and Saratoga separately, as ilustrared in figure 5.5. The DDTP took a total of 44.796 s and Saratoga 42.783 s. It is to be expected that Saratoga achieves a better result in this case since, similarly to the results in section 5.1, it uses less metadata in its messages. The real advantage of DDTP will be observed in the experiments performed in more complex scenarios that will be described in the following sections.
12000
10000
8000
6000 DDTP Saratoga Throughput (B/s) (B/s) Throughput
4000
2000
0 0 5 10 15 20 25 30 35 40 45 50 Time (s)
Figure 5.5: Multi to one - DDTP vs Saratoga
5.3 One to multi
Using the topology illustrated in figure 5.6, with a 10 kB/s channel that directly connects node A, B and C, a 200 KiB transmission was performed. This transmission originated from node A to, simultaneously node B and C. This corresponds to a one-to-multi transmission as defined in section 3.1. The transmis-
45 sion is illustrated in figure 5.7. It is important to note that this case is very different from the remaining scenarios. This is because, since messages are sent in multicast, there is not confirmation from the recipients of their correct arrival (i.e. an ACK message). To solve this problem, recipients will send retransmission requests for missing or corrupt data blocks. In this particular case, two copies of the same 200 KiB data payload were sent to node A and B in a total of 20.887 s. The alternative to this woud be to perfom 2 one-to-one transmissions. Looking at table 5.1, one can see that DDTP took 22.8s to perform a one-to-one transmission of a 200 KiB payload. This means that to solve this scenario with only one-to-one transmission, it would be necessary to perform two equal transmissions taking a total of 45.6s. In this scenario DDTP’s multicast mechanism clearly offers a much more optimized solution.
B
A
C
Figure 5.6: One to multi topology
46 12000
10000
8000
6000 Throughput (B/s) (B/s) Throughput
4000
2000
0 0 5 10 15 20 25 Time (s)
Figure 5.7: DDTP - One to multi transmission
5.4 Distributed
To validate DDTP main feature, its distributed property, a transmission of 200 KiB was performed be- tween node A and D, using the topology illustrated in figure 5.1. All links and nodes were connected at the beginning of the simulation, with a link disruption scheduled for link α. Node A has two routes for reaching node D: Route A through link α; and route B through link β. Unlike what happened in section 5.1.1, where only one route was available and so node A was forced to pause the transmission, when the disruption occurs the transmission should resume through route B. In figure 5.8 this experiment is illustrated, where each point in the chart represents a 1s throughput average. When the link α is disconnected, at the 9s mark, the transmission through route A stops. At this point any current solution would proceed by simply pausing the transmission and waiting for the link restablishment. This is the behavior observed in figure 5.4 by Saratoga. Unlike Saratoga, however, the DDTP resumes the transmission through route B as soon as it detects that route A is no longer avail- able. The result of this is the full transmission finishing in under 30 s. The simpler DT approach used in section 5.1.1, with the same data payload, resulted in a 72 s transmission time, this corresponds to a huge improvement on DDTP part. It should also be taken into account that for bigger link disruption times this difference is higher, increasing linearly with the link disruption time.
47 800
700
600
500
400 Route A Route B
300 Throughput (B/s) (B/s) Throughput
200
100
0 0 5 10 15 20 25 30 Time (s)
Figure 5.8: DDTP distributed transmission
5.5 Compression
In section 3.4.5 the Pocket++ compression scheme was introduced. Since the ISTSAT-1 will compress its mission data with this software, the DDTP should be tested with it, not only to confirm its compatibility but also to measure its performance. To this effect, a sequence of 2262 spacecraft telemetry messages were transmitted between node A and B, in the topology illustrated in figure 5.1, first in their original format and then compressed by Pocket++. The original message sequence accounted to a total of 277 KiB, and the compressed version 68 KiB, resulting in a ∼4.07 compression ratio. Since the messages used correspond to samples from a spacecraft telemetry, a high compression ratio is to be expected. This is because it is normally the case for telemetry to contain repetitive or similar values over time. The results of this transmission are displayed in figure 5.9. The original messages took 30.975 s to finish while the compressed version only took 8.382 s. This test results can be extended to the, more complex, distributed scenario. This is because it makes no difference to DDTP the kind of data payload it is transmitting (whether it is compressed or not).
In the case of ADS-B messages, their content is very repetitive by nature. This is because their fields (e.g airplane ID, geographical coordinates, flight number) contain information within a fixed range of values. Moreover, this is specially true for a set of airplanes geographically close to each other, which will be the case for the ones tracked by ISTSAT-1. Due to this, it is expected that, much like the example above, Pocket++ will provide a high compression ratio when applied to ADS-B messages.
48 12000
10000
8000
6000 Uncompressed (277KB) Pocket++(68KB) Throughtput (B/s) (B/s) Throughtput
4000
2000
0 0 5 10 15 20 25 30 35 Time (s)
Figure 5.9: Pocket++ transmission
If one were to configure the ISTSAT-1 mission module to only track a particular airplane, or a small collection, the compression ratio would be even higher, since not only would some of the message field remain constant, such as the airplane and flight identification, but fields like the geographical coordinates would change in a incremental fashion. Thus increasing their recursive information.
49 50 Chapter 6
Conclusion and future work
A Cubesat is a very resource constrained type of spacecraft, with low energy reserves, small solar pan- els, and low communication and processing capabilities. Due to this, the development of a Cubesat offen requires very optimized solutions. The ISTSAT-1 mission is to perform a feasibility study of the use of cubesats for receiving ADS-B signals from traveling airplanes. A safety system that broadcast information from each airplane. It then needs to transfer the processed ADS-B information to earth, through one or more GSs. Since cubesats are typically launched in LEO, they have a very fast traveling speed (close to 8km/s), that results in a very small time window (under 20 min) to communicate with the GSs on earth. Current solutions solve this problem by operating in a DT fashion. This project proposes a new approach, the development of a distributed DT protocol, capable of per- forming the same tasks already being solved by the current solutions but in a much more optimized way, increasing the overall speed at which data can be received. This solution is not only appropriate for spacecrafts like the ISTSAT-1, that simply wish to transmit mission data to a server, but can also be used for more complex networks like spacecraft clusters with ISL and the simultaneous transmission of data to multiple receivers on earth, like a WSN. Four application scenarios were defined, and these encompass the major requirements necessary. From these four, only two are currently being solved by the existing solutions: one-to-one and multi-to-one transmissions. The developed solution was designed to not only solve the first two but also the new ones: one-to-multi and distributed transmissions. The experimental results showed that using DDTP will result in much faster transmissions in two of the scenarios. When multiple links are available (distributed scenario) DDTP is able to make use of these links and avoid completely pausing the transmission when a disruption occurs. This offers a very significant per- formance improvement over current solutions which simply operate in a DT fashion in these scenarios. This is the use case of the ISTSAT-1 project. When multiple transmission destinations are available over a single link, DDTP also manages to reduce very significantly the transmission times. While current solutions simply perform several one-to-one transmissions to solve this problem, DDTP multicast capabilities allow the transmission to reach all des-
51 tinations simultaneously. This is the use case of a WSN whose nodes are receiving a firmware update over a space link. The first two scenarios, one-to-one and one-to-multi are solved by both DDTP and the current solutions, but due to the increased metadata usage in DDTP current solutions are actually slightly faster, in the one-to-one test Saratoga was able to finish the transmission 1.8s faster than DDTP. However, purely one-to-one or one-to-multi scenarios are not the use case for which DDTP is designed for. Future work should include the testing of DDTP in real world scenarios, including its integration with the remaining systems of the ISTSAT-1 cubesat. More complex routing algorithms could also be developed, potentially increasing transmission speeds even further. One possibility would be having DDTP collect information from the module responsible for determining the ISTSAT-1 location (the Altitude Determina- tion and Control System module). This would allow DDTP routing algorithm to predict when links would become available/unavailable and make choices based on this information. DDTP power usage onboard of ISTSAT-1 should also be analyzed and, if necessary, optimized.
52 Bibliography
[1] CalPoly, “Cubesat design specification”, The CubeSat Program, California Polytechnic State . . .,
2009, ISSN: 1110662X. DOI: http://dx.doi.org/10.1155/2015/681901.
[2] Secure World Foundation, New Actors in Space. 2011, p. 159, ISBN: 9781618398055.
[3] J. Ferreira, “ISTNanosat-1 Heart”, Master’s thesis, IST - Instituto Superior Técnico, 2012. [Online]. Available: https://fenix.tecnico.ulisboa.pt/downloadFile/395144736837/MScThesis- JFerreira_vFinal.pdf.
[4] A. Mccook, “On the right path”, Nature, 2011, ISSN: 00280836. DOI: 10.1038/nj7353-667a.
[5] P. Coelho, “ISTNanosat-1 Quality Assurance , Risk Management and Assembly , Integration and Verification Planning”, no. May, p. 107, 2016.
[6] H. Weiss, “Delay-Tolerant Networking Architecture”, RFC Editor, RFC 4838, Apr. 2007, pp. 1–61. [Online]. Available: https://tools.ietf.org/html/rfc4838.
[7] A. Schlesinger, B. M. Willman, L. Pitts, S. R. Davidson, and W. A. Pohlchuck, “Delay/Disruption Tolerant Networking for the International Space Station (ISS)”, IEEE Aerospace Conference Pro-
ceedings, no. October 2010, 2017, ISSN: 1095323X. DOI: 10.1109/AERO.2017.7943857.
[8] O. N. Challa and J. McNair, “CubeSat Torrent: Torrent like distributed communications for CubeSat
satellite clusters”, Proceedings - IEEE Military Communications Conference MILCOM, 2012. DOI: 10.1109/MILCOM.2012.6415828.
[9] R. F. Afonso, “Sistema de Gestão e Determinação de Atitude do Engenharia Electrónica Júri”, p. 74, 2016.
[10] P. M. Gameiro, “Distributed Delay Tolerant Protocol”, 15th International Symposium on Wireless
Communication Systems (ISWCS), 2018, p. 6, ISBN: 9781538650059.
[11] M. C. Chuah, P. Yang, B. D. Davison, and L. Cheng, “Store-and-forward performance in a DTN”, in
IEEE Vehicular Technology Conference, 2006, ISBN: 0780388879. DOI: 10.1109/VETECS.2005. 1543275.
[12] K. Scott and S. Burleigh, “Bundle Protocol Specification”, RFC Editor, RFC 5050, Nov. 2007, pp. 1–50. [Online]. Available: https://tools.ietf.org/html/rfc5050.
[13] W. Beech, D. Nielsen, and J. Taylor, “AX . 25 Link Access Protocol for Amateur Packet Radio”, no. July, 1998.
53 [14] R. F. T. Berners-Lee and L. Masinter, “Uniform Resource Identifier (URI): Generic Syntax”, RFC Editor, RFC 3986, Jan. 2005, pp. 1–61. [Online]. Available: https://www.ietf.org/rfc/rfc3986. txt.
[15] S. E. Alaoui and B. Ramamurthy, “Routing optimization for DTN-based space networks using a temporal graph model”, 2016 IEEE International Conference on Communications, ICC 2016, 2016.
DOI: 10.1109/ICC.2016.7510733.
[16] L. Wood, C. Smith, W. M. Eddy, W. Ivancic, and C. Jackson, Taking Saratoga from space-based
ground sensors to ground-based space sensors, 2011. DOI: 10.1109/AERO.2011.5747332. [On- line]. Available: dx.doi.org/10.1109/AERO.2011.5747332.
[17] L. Wood, W. M. Eddy, W. Ivanèic, J. McKim, and C. Jackson, “Saratoga: A Delay-Tolerant Net- working convergence layer with efficient link utilization”, in 2007 International Workshop on Satel-
lite and Space Communication, IWSSC’07, 2007, ISBN: 142440939X. DOI: 10.1109/IWSSC.2007. 4409410.
[18] L. Wood, “Saratoga: A Scalable Data Transfer Protocol”, RFC Editor, RFC Draft, Mar. 2012, pp. 1– 53. [Online]. Available: https://tools.ietf.org/html/draft-wood-tsvwg-saratoga-11.
[19] “CCSDS BUNDLE PROTOCOL SPECIFICATION Recommendation for Space Data System Stan- dards PROTOCOL”, no. September, p. 94, 2015.
[20] T. de Cola, H. Ernst, and M. Marchese, “Performance analysis of CCSDS File Delivery Protocol
and erasure coding techniques in deep space environments”, Computer Networks, 2007, ISSN:
13891286. DOI: 10.1016/j.comnet.2007.04.015.
[21] CCSCS, “Space Communications Protocol Specification (SCPS)—Transport Protocol (SCPS-TP)”,
2006. DOI: CCSDS714.0- B- 2. [Online]. Available: http://public.ccsds.org/publications/ archive/714x0b2.pdf.
[22] G. Crowley, C. Swenson, C. Fish, and A. Barjatya, “Dynamic ionosphere cubesat experiment”, Tech. Rep., Oct. 2009. [Online]. Available: http://digitalcommons.usu.edu/smallsat/2011/ all2011/84/.
[23] Z. S. Gibalina, “Estimation of capabilities of cooperative CubeSat systems based on Alamouti transmission scheme”, 2018.
[24] R. ESA and S. Division, “Space Engineering - Software ECSS-E-ST-40C”, Tech. Rep., Mar. 2009. [Online]. Available: http://wwwis.win.tue.nl/2R690/doc/ECSS-E-ST-40C(6March2009).pdf.
[25] E. N. W. G. G. Camarillo, “Peer-to-Peer (P2P) Architecture RFC”, RFC Editor, RFC, 2009, pp. 1– 26. [Online]. Available: https://tools.ietf.org/html/rfc5694.
[26] Y. Song, L. Liu, and J. Chen, “A Temporal Graph Model based Power Aware Routing Algorithm in Deep-space Networks”, 2017.
[27] D. Evans, O. Chattlain, and R. Vitulli, “The ESA POCKET Housekeeping Telemetry Compression Algorithm”, no. February, p. 10, 2015.
54