An Open-Source, Extensible System for Laboratory Timing and Control Peter E

REVIEW OF SCIENTIFIC INSTRUMENTS 80, 115103 ͑2009͒ An open-source, extensible system for laboratory timing and control Peter E. Gaskell,a͒ Jeremy J. Thorn, Sequoia Alba, and Daniel A. Steck Department of Physics and Oregon Center for Optics, University of Oregon, Eugene, Oregon 97403-1274, USA ͑Received 16 July 2009; accepted 25 September 2009; published online 3 November 2009͒ We describe a simple system for timing and control, which provides control of analog, digital, and radio-frequency signals. Our system differs from most common laboratory setups in that it is open source, built from off-the-shelf components, synchronized to a common and accurate clock, and connected over an Ethernet network. A simple bus architecture facilitates creating new and specialized devices with only moderate experience in circuit design. Each device operates independently, requiring only an Ethernet network connection to the controlling computer, a clock signal, and a trigger signal. This makes the system highly robust and scalable. The devices can all be connected to a single external clock, allowing synchronous operation of a large number of devices for situations requiring precise timing of many parallel control and acquisition channels. Provided an accurate enough clock, these devices are capable of triggering events separated by one day with near-microsecond precision. We have achieved precisions of ϳ0.1 ppb ͑parts per 109͒ over 16 s. © 2009 American Institute of Physics. ͓doi:10.1063/1.3250825͔ I. INTRODUCTION ware must be run by a sufficiently primitive computer and the cost of upgrading the hardware to support modern inter- In a wide range of fields, including cold-atom physics, faces is prohibitive. experiments require complex sequences of timing and con- Other groups have worked on these problems, ranging trol signals that are both precise and repeatable. The standard from reprogramming an existing device to perform approach to meeting these needs is to control the experiment differently1 to designing a new control system.2 The general with a desktop computer outfitted with a number of inter- purpose interface bus ͑GPIB, or IEEE-488͒ has been under nally mounted ͑e.g., PCI/PCI-X͒, commercial “expansion much scrutiny, since it tends to be associated with extra hard- cards,” which are designed to produce the requisite timing ware, proprietary drivers, poor timing issues, and short, ex- pulses, control voltages, and real-time waveforms. Many of pensive cables. For example, Hall3 discussed a scheme to the available options include some data-acquisition functions improve the timing of GPIB operations, and Gao and as well, and may serve as fast interfaces for more complex 4 instruments such as cameras. This hardware is then con- Madison discussed a scheme to convert GPIB to Ethernet, trolled from the computer via proprietary libraries, and, from bypassing some of the proprietary drivers, the expansion the user’s perspective, often through high-level programming cards, and the short cable limitations. We have taken this work further, and developed hard- environments ͑e.g., LABVIEW or MATLAB͒. Despite being im- mensely popular, this approach has a number of serious ware and software for precise timing and control of laboratory experiments. We have successfully used the architecture problems: Proprietary hardware and software is generally 5,6 quite expensive, and traditional interfaces to external hard- in our ultracold-atom experiments, where we used no pro- ware ͑e.g., IEEE-488 and RS-232͒ tend to be expensive, prietary software or add-on hardware beyond the basic desk- slow, and involve limited cable distances. Hardware capabili- top workstation. Our design criteria for this architecture are ties as enumerated in advertising literature can in some cases as follows. be misleading or even differ significantly from reality. Avail- ͑1͒ Performance meets the needs of cutting-edge experi- able hardware capabilities and software support also vary ments ͑especially with ultracold atoms͒, with low con- considerably, depending on the operating system running on struction cost so that this technology may be used, for the computer. Proprietary drivers and firmware cannot be example, in teaching laboratories with modest budgets. modified to suit unusual applications not foreseen by the ͑2͒ Open-source hardware and software for longevity and vendor. Even worse, driver incompatibilities that inevitably easy customization. arise with upgrades to the operating system can paralyze an ͑3͒ Fast, inexpensive, and long-distance connectivity, which experiment until workarounds are implemented. In ultracold- we achieve with the transmission control protocol over atom experiments, it is unfortunately not uncommon for an internet protocol standard ͑TCP/IP͒ and user datagram experiment to be controlled by two interfaced computers, protocol over internet protocol ͑UDP/IP͒, both over Eth- one new and one much older, when older proprietary hard- ernet networks. ͑4͒ Decentralized, modular, and scalable hardware, where a͒Present address: Department of Electrical and Computer Engineering, timing and control hardware is divided into relatively McGill University, Montreal, Quebec. compact and physically separate “boxes,” and each box 0034-6748/2009/80͑11͒/115103/10/$25.0080, 115103-1 © 2009 American Institute of Physics 115103-2 Gaskell et al. Rev. Sci. Instrum. 80, 115103 ͑2009͒ ETHERNUT 2.1 / 3 Ethernet RAM Telnet App Streamer App INSTRUMENT CIRCUITS: Analog Out FIFOs Analog In External Clock Clock Divide Digital Out RF Out Start/Stop Start/Stop INTERFACE CIRCUIT (Input or Output) FIG. 1. Basic components of our control system. The ETHERNUT handles network communication and produces the commands for the instrument circuits. The interface circuit buffers the commands and sends them to the instrument boards with precise timing governed by an external clock, with inputs for starting and stopping the output. The instrument circuits control or read data from the experiment. is controlled by an embedded microcontroller. By plac- nication with timing and control hardware, because of the ing some “intelligence” in each box, the main computer nondeterministic nature of the standard protocols, Ethernet needs no special hardware beyond a standard network networks are not directly suited for generating precise, real- interface. Furthermore, in the case of hardware prob- time signals. Thus, all network communication in an experi- lems, the modular external boxes are easily replaced by ment occurs before the real-time sequence. Once all control spares. Replaced boxes can be repaired and used as boxes are ready, they can be triggered via the interface spares. electronics, which buffer the quasi-real-time output of the ͑5͒ Platform independence, mainly since the host computer ETHERNUT microcontrollers, allowing the output to be syn- only needs to support standard Ethernet communication, chronized to a high-precision external clock oscillator. Thus, which is now so widely implemented that it will be sup- all control signals can be precisely synchronized across ported on virtually every useful platform for many years many devices. to come. We use PERL to program the hardware, though any network-aware programming environment is suit- II. HARDWARE OVERVIEW able for this purpose. Our hardware setup is simple and modular. Each box In realizing the timing and control hardware here, we are shares an Ethernet network with at least one computer, used leveraging existing hardware projects. We use low-cost, to program and run the boxes. Each box is solely an input 7 open-source, off-the-shelf “ETHERNUT” microcontrollers to device or an output device, and contains three parts: manage the main network link between the host computer ͑1͒ ETHERNUT ͑an embedded computer͒, and the control hardware. We also use digital, analog, and ͑2͒ interface circuit, and direct digital synthesis electronics from a project at the Uni- ͑3͒ instrument circuit ͑possibly multiple copies͒. versity of Texas at Austin.8 These circuit designs are freely available, and are designed to be controlled over a digital The ETHERNUT is a simple computer that we use as an data bus by a commercial digital output board installed in the interface between the main computer ͑over an Ethernet net- host computer. The bulk of the work described here involves work͒ and the control circuits. The interface circuit buffers the design and implementation of interface electronics that the commands from the ETHERNUT with first-in-first-out allow the ETHERNUT microcontrollers to drive any circuit us- ͑FIFO͒ buffers, and passes those commands to the instru- ing the same data bus in real time. These electronics enable a ment circuits, maintaining an even timing and synchroniza- distributed, real-time control system, free of proprietary soft- tion with other boxes. The instrument circuit actually pro- ware or hardware as outlined above. The main challenge here duces the output signals that control the experiment. These is that despite the many advantages of Ethernet for commu- parts, shown in Fig. 1, will be described in Secs. II–X. All 115103-3 Gaskell et al. Rev. Sci. Instrum. 80, 115103 ͑2009͒ schematics, board layouts, and software we designed are IV. INTERFACE CIRCUIT available on our project website.9 Every device uses the same specialized interface circuit, which buffers commands to be sent to the instrument circuit. These circuits allow us to control the timing of the com- III. ETHERNUT PLATFORM AND NUT/OS mands sent to the instrument boards with high precision. The ETHERNUT reads commands from its own internal The ETHERNUT is an open-source hardware platform that runs an open-source, real-time operating system called memory, and generates commands for the instrument circuits. Using the internal memory solves the problem of send- NUT/OS. The ETHERNUT and NUT/OS were designed and are distributed by Egnite GmbH.7 ing precisely timed packets over a network, but it is difficult ETHERNUT The ETHERNUT comes in multiple hardware versions, but to precisely time multiple, fast outputs. As the firmware for this project was written for version 2.1. Our described in Sec.

Load more