Airbus Defence and Space Launches a Petascale Archive for Satellite

Image courtesy of ESA CASE STUDY Airbus Defence and Space Launches a Petascale Archive for Satellite Data with StorNext Scale-out Storage Based in the United Kingdom, Airbus Defence and Space archives critical data streaming from Earth observation satellites managed by the European Space Agency (ESA). The Airbus team selected Quantum StorNext scale-out storage to enable scale and to provide access to this fast-growing data collection. StorNext allows us to easily scale our archive so we can support new satellite missions and “collect additional data well into the future. Mark Curtis Technical Design Authority, Airbus Defence and” Space FEATURED PRODUCTS MANAGING MILLIONS OF INDIVIDUAL FILES Curtis and his Airbus colleagues selected the “ Administration costs StorNext solution to archive Sentinel satellite data The Airbus team needed to accommodate not only with StorNext are quite a large amount of data but also a huge number of in part because of previous, positive experiences low. With the Quantum files. “When there are four satellites in orbit, we’ll with Quantum storage when the team was part need to archive 40,000 files per day,” says Curtis. of Infoterra—a geospatial services subsidiary of solution, two of us can “We needed a robust file system to support tens of Airbus that was merged into Airbus Defence and easily manage this millions of files.” Space. Infoterra was a primary supplier of images growing petascale archive for Google Earth. The variability in file size presented another of satellite data.” challenge. “In addition to large files, we have to “Quantum has always listened to our requests Mark Curtis, in improving their solutions,” says Curtis. “In the store small metadata and auxiliary files,” says Technical Design Authority, Curtis. “We needed a new file system that could case of StorNext 5, Quantum incorporated input Airbus Defence and Space support a wide variety of file sizes and types from us and other organizations, which helped without any performance issues.” produce a stronger storage solution.” Image courtesy of ESA CAPITALIZING ON THE ECONOMICS OF TAPE DEPLOYING AN 18PB ARCHIVE THAT SCALES The Airbus team decided to implement a tape Using StorNext, Airbus Defence and Space initially solution for its archive largely because of the deployed an archive that could support nearly 18PB of satellite data. The team can scale the StorNext offers a wide range of storage technologies and systems for building a economics of tape. “Controlling costs is critical since we need to store tens of petabytes of data for archive up to 37PB as more satellites are launched. multi-tier environment. The platform makes it easy to integrate new solutions from the Sentinel missions,” says Curtis. “Tape is very “Our immediate goal was to support the first two economical: Each tape cartridge can store a huge satellites, but we knew we would need to start Quantum and other vendors. “ amount of data inexpensively, and tape libraries archiving data for four satellites within a few years,” Mark Curtis - Technical Design Authority, Airbus Defence and Space require less power and cooling than higher-speed says Curtis. “StorNext allows us to easily scale our storage systems.” archive so we can support new satellite missions and collect additional data well into the future.” ” The data streams that Airbus Defence and Space Airbus Defence and Space is one of few select image land masses and oceans, the Sentinel-2 stores and retrieves are also a good fit for tape. The Airbus team anticipates adding StorNext European organizations tasked with archiving mission captures multispectral high-resolution “We don’t need the random access to data that AEL6000 archives using LTO-7. “With LTO-7, we data collected by ESA’s Earth observation imaging of vegetation, soil and water cover, inland disk provides,” says Curtis. “We store data in gain more than twice the raw capacity and speed satellites. With a potential user base of millions waterways, and coastal areas. streams and then retrieve data in streams later. of LTO-6,” says Curtis. “It’s an easy decision to of Europeans, this Airbus group primarily serves move to this next-generation LTO technology as SOLUTION OVERVIEW ESA is launching two satellites for each mission, An organization monitoring the pollution levels of a Image courtesy of ESA commercial enterprises in agriculture, forestry, the archive grows.” ∙ Quantum StorNext® Scale-out Storage in staggered fashion, to maximize coverage. lake is more likely to request all data from multiple ∙ StorNext Metadata Appliances telecommunications, and insurance. years rather than a single file. For this kind of Within a few years of the program’s start, ESA will ENABLING CONTINUOUS HIGH-SPEED DATA ∙ StorNext AEL6000 Tape Archives archive, tape just makes sense.” “A company might use satellite imagery to have four Sentinel satellites in orbit—two pairs— INGEST AND ACCESS WITH STORNEXT ABOUT AIRBUS DEFENCE determine whether an area is prone to flooding or continuously sending data home. AND SPACE KEY BENEFITS BUILDING AN ARCHIVE FOR SATELLITE DATA The Airbus team must be able to continuously identify locations for building a cell phone tower,” ∙ Scales to support tens of petabytes of data The Airbus Defence and Space team had to build ON STORNEXT MULTI-TIER STORAGE ingest and retrieve satellite data. The streaming A division of Airbus Group, Airbus streaming in from Earth observation says Mark Curtis, technical design authority for performance of the StorNext file system helps satellites a new archive to house data streaming from The Airbus team selected a multi-tier StorNext Defence and Space is Europe’s Airbus Defence and Space. “In many cases, users ∙ Serves millions of potential users, Sentinel satellites. “Each satellite streams 250MB solution for its archive. The solution includes meet that requirement. “Our current configuration leading provider of solutions and providing access to satellite data want to track changes over time by comparing of data per second, which is more than 1PB each StorNext AEL6000 tape archives and StorNext supports 1.1GB-per-second performance with the services for the defense and space ∙ Manages millions of files, of all sizes, sets of images from one period to another. The using a high-speed shared file system with per year. And each satellite typically has a design metadata appliances, as well as a primary disk StorNext file system,” says Curtis. “As a result, we industries. It is the second-largest satellite imagery gives them the information they a policy-driven tiered archive life of 5 years. Sometimes as many as 15 years,” cache. The Airbus team powers the entire solution can rapidly ingest all the raw and processed data space business worldwide and ∙ Safeguards archived satellite data need to make precise, data-driven decisions.” among the top ten global defense with built-in data protection capabilities says Curtis. “We archive not only the raw, level 0, with StorNext data management software. we receive, and quickly retrieve it when necessary.” ∙ Enables a small staff to manage a big instrument data but also data processed by ESA’s enterprises. Airbus Defence and archive, with features that streamline ARCHIVING PETABYTES OF SATELLITE DATA “Quantum offers a wide range of storage options In most cases, the Airbus team can retrieve Payload Data Ground Segment, which generates Space is responsible for archiving administration In 2013, Airbus Defence and Space won a bid to for building a multi-tier environment,” says Curtis. individual files from the Sentinel archive in under level 1 and level 2 products. To provide a long- a tremendous amount of data archive data from ESA’s Sentinel satellites, which “The StorNext platform makes it easy to integrate a minute. StorNext support for Fibre Channel continuously streaming in from term archive for all that data, we needed a storage are part of the agency’s Copernicus program. new solutions from Quantum and other vendors.” enables the team to transfer even large data ESA’s Sentinel satellites. solution that could scale to tens of petabytes.” Each Sentinel mission has a specific objective. While the Sentinel-1 mission uses RADAR to www.quantum.com/customerstories www.quantum.com/customerstories CASE STUDY Image courtesy of ESA sets quickly. “If a user requests all the lost,” says Curtis. “The policy-based EDLM to implement a relay satellite system data from a particular year, we might features of the StorNext tape archive help that will increase storage needs. “Future need to deliver 1.2PB of data,” says Curtis. proactively maintain the integrity of data on relay satellites shall enable a much larger “Fibre Channel enables us to get data the tapes over the long term. If a problem coverage area and more download per orbit from tape to block storage fast, so it can develops with a tape, StorNext automatically even when the primary satellite cannot see a be delivered rapidly.” migrates data to better tape, without our ground station,” says Curtis. administrators having to lift a finger. EDLM Because the relay system will help SUPPORTING A FULL RANGE OF provides a huge advantage.” OPERATING SYSTEMS ensure data ultimately reaches ground stations, Airbus Defence and Space will The heterogeneous StorNext solution MANAGING A LARGE ARCHIVE WITH A need to store more data than previously supports a range of different operating SMALL STAFF anticipated. “Fortunately, StorNext enables systems. “Many of our users have Linux StorNext data management capabilities us to easily scale our environment. We can systems, but we can’t ignore the Windows help streamline administrative tasks. As a add new archive systems—and move to users,” says Curtis. “With the StorNext result, Airbus Defence and Space can grow LTO-7—to meet these new data collection file system, we can support a full range of the archive without having to hire more staff.

Load more