UDP Protocol Specification v1.2 René Görlich, Paul Kanevsky and Peter Simpson March 2018 Gemini UDP Protocol Specification v1.2 Introduction This document defines Gemini's UDP protocol specification and establishes the processes that clients must implement in order to use the protocol. The protocol has been developed by René Görlich, Paul Kanevsky and Peter Simpson for use with the Gemini 2 telescope control system; it is not and cannot be supported by the earlier Gemini 1 hardware and v4.1 firmware. The UDP network protocol is described as are the reasons for its selection over TCP; some .NET example code is also provided in Appendix 2. For support please post on the Gemini II Yahoo group at: http://tech.groups.yahoo.com/group/Gemini-II/messages What is UDP?1 The User Datagram Protocol (UDP) is one of the core members of the Internet Protocol Suite, the set of network protocols used for the Internet. With UDP, computer applications can send messages, in this case referred to as datagrams, to other hosts on an Internet Protocol (IP) network without requiring prior communications to set up special transmission channels or data paths. The protocol was designed by David P. Reed in 1980 and formally defined in RFC 768. UDP uses a simple transmission model without implicit handshaking dialogues for providing reliability, ordering, or data integrity. Thus, UDP provides an unreliable service and datagrams may arrive out of order, appear duplicated, or go missing without notice. UDP assumes that error checking and correction is either not necessary or performed in the application, avoiding the overhead of such processing at the network interface level. Time-sensitive applications often use UDP because dropping packets is preferable to waiting for delayed packets, which may not be an option in a real-time system.[1] If error correction facilities are needed at the network interface level, an application may use the Transmission Control Protocol (TCP) or Stream Control Transmission Protocol (SCTP) which are designed for this purpose. UDP's stateless nature is also useful for servers answering small queries from huge numbers of clients. Unlike TCP, UDP supports packet broadcast (sending to all on local network) and multicasting (send to all subscribers).[2] Common network applications that use UDP include: the Domain Name System (DNS), streaming media applications such as IPTV, Voice over IP (VoIP), Trivial File Transfer Protocol (TFTP), IP tunneling protocols and many online games. UDP applications use datagram sockets to establish host-to-host communications. An application binds a socket to its endpoint of data transmission, which is a combination of an IP address and a service port. A port is a software structure that is identified by the port number, a 16 bit integer value, allowing for port numbers between 0 and 65535, that allows for multiplexing many 1 Reproduced from Wikipedia, please see document licensing section for further information. Peter Simpson, René Görlich - 7rh March 2018 - v1.2 Draft 1 applications on a host. Port 0 is reserved, but is a permissible source port value if the sending process does not expect messages in response. Comparison of TCP and UDP This comparison is based on four features of the two protocols TCP UDP Relative to UDP, a heavier weight Lightweight protocol because it is simple, protocol because it supports reliable and connectionless and does not support Lightweight ordered communications in a very reliable and ordered communication. flexible and self adaptive manner. Optimised for reliable bulk transfer Low intrinsic protocol overhead, suited to operations where multiple packet sends "chatty" applications where fast round trip can be acknowledged in one return response times are required. Responsive packet. For "highly interactive, chatty" applications, can be less responsive than UDP. TCP manages message acknowledgment, When a message is sent, it cannot be retransmission and timeout. Multiple known if it will reach its destination; it attempts to deliver the message are could get lost along the way. There is no made. If it gets lost along the way, the concept of acknowledgment, Reliable server will re-request the lost part. In retransmission or timeout. TCP, there's either no missing data, or, in case of multiple timeouts, the connection is dropped. The bytes send in a stream are handed If two messages are sent to the same over in the same order. When data recipient, the order in which they arrive segments arrive in the wrong order, TCP cannot be predicted. Ordered buffers the out-of-order data until all data can be properly re-ordered and delivered to the application. In the world of TCP/IP, with which we are most familiar, TCP provides reliable data communication streams and as application users we do not need to concern ourselves with issues such as packets arriving out of order or being lost in transit. Why UDP, why not TCP? Key requirements for the Gemini protocol are that it is: Lightweight: It places low demand on the network and Gemini hardware and the connecting operating system Responsive: Gemini receives commands and clients receive responses quickly Reliable: The protocol must detect and correct communications errors Ordered: Commands are processed by Gemini in the order in which they are sent Peter Simpson, René Görlich - 7rh March 2018 - v1.2 Draft 2 TCP based protocols are available to access Gemini, but have certain disadvantages that should be considered: 1. Lightweight: High impact on the Gemini CPU when using the built-in TCP stack. The necessary buffering limits the amount of available sockets. 2. Responsive: The TCP implementation decides how long data is buffered before sending. Slow responses when using Windows clients2 These were traced to: 1. A TCP stack needs resources to implement the features described above, impacting overall responsiveness and throughput. 2. The TCP stack decides when a data segment is sent. The default value of the Microsoft TCP stack TcpAckFrequency setting (2), resulted in a 200ms delay in the round trip time to Gemini because the stack was waiting for a second packet from Gemini before sending the ACK packet for the first packet. Gemini on the other hand was waiting for an ACK to the first packet before sending the second packet... deadlock! The deadlock was always broken after 200ms by the Microsoft stack sending an ACK, even if it had not received a second packet within that time. So the system worked reliably but with frequent delays of 200ms resulting in low throughput. When TcpAckFrequency is set to a value of 1, performance is fast, however, it was not felt appropriate to require all Gemini uses to change the Microsoft default setting and therefore to carry the risk that some other aspect of Windows would not perform as expected. Testing with UDP was much more successful, providing a higher throughput than TCP, even with TcpAckFrequency set to 1 together with lower processor impact due to the lower protocol overhead for Gemini and because it bypassed the higher levels of the Microsoft TCP stack at the client. UDP thus meets the Lightweight and Responsive Gemini protocol requirements but does not meet the Reliable and Ordered requirements; consequently we need to provide these at the application level. Implementation is straightforward since we don't need all the frills that TCP provides and just need a simple, low overhead process. The following sections describe Gemini's UDP protocol and both the client's and Gemini's roles in the error recovery process. Access to Gemini's UDP Server Gemini creates a UDP listener socket by default on port 11110; there is no security access control, login or similar concept. If you can reach port 11110 then you can immediately send commands to Gemini and receive responses. The UDP port number used by the Gemini server can be changed through the Network Settings page of the Gemini's built-in web application. 2 In testing René used a Linux client and did not see the 200ms ACK delay, this issue was only observed in the Microsoft TCP/IP stack. Peter Simpson, René Görlich - 7rh March 2018 - v1.2 Draft 3 UDP Datagram Format This diagram shows the UDP protocol fields and the Gemini protocol fields that are defined within the UDP Data field. Your UDP protocol stack will take care of the first 8 bytes (0 to7) of the UDP protocol fields, which record the source and destination IP ports, the length of the datagram and a checksum over the entire datagram. The Data field (shown in blue), starting at offset 8 is where the Gemini protocol fields are located. The Gemini protocol defines three fields within the UDP datagram Data field: Field Offset Purpose DatagramNumber 0 On sending a command, this must be a unique number for this datagram from this client. It sequentially increments by one for each datagram starting at 0. On receipt of a datagram from Gemini, this will contain the DatagramNumber of the command for which this is the response. LastDatagramNumber 4 This should be set to zero for commands to Gemini and will normally be zero in responses from Gemini. It is set to the DataGramNumber of the last received command from the client, in response to a NAK command from the client. (See error handling protocol section) GeminiData 8 Command to Gemini, using the Gemini Serial Command syntax or response from Gemini, in both cases terminated by the NULL character (0x00). The maximum size of the GeminiData field is 255 bytes; the total size of Gemini command strings, including NULL characters, should not exceed this size. Peter Simpson, René Görlich - 7rh March 2018 - v1.2 Draft 4 Client Protocol The overall Gemini client - server protocol is set out diagrammatically in Appendix 1.