Computing at the Limit Page 84 Research Campus Modal

ZIB 2016 Annual Report

SHAPE-BASED RESEARCH DATA ANALYSIS – CAMPUS MODAL: SUPER- BIGGER DATA, METHODS AND REPORT FROM COMPUTING BETTER HEALTH RESULTS THE GASLAB AT THE LIMIT PAGE 30 PAGE 44 PAGE 52 PAGE 84

Preface PREFACE

The year 2016 was the 75th anniver- The anniversary was an ideal occasion sary of the invention of the computer for ZIB to reflect upon its strategic by Konrad Zuse, after whom ZIB is position and its research activities in a named. On May 12, 1941, Konrad Zuse world where digital technology is expo- presented the first fully functional nentially growing, getting more complex, digital computer, his Z3, in Berlin promising unprecedented insights into and the digital age began. ZIB cele- nature, economy, or society at large, but brated this historic breakthrough by also into our personal lives. As Berlin’s organizing a series of public events in competence center for Digital Science, which the computer as an instrument, with its rich portfolio of collaborative computing, and digital science in gen- projects in Applied Mathematics and eral were presented in many of their Applied Computer Science, its regional, fascinating aspects. The main event, national, and international partners, the international conference “The and its expertise and capacities in data Digital Future – 75 Years Zuse Z3 and and information science, ZIB seems to the Digital Revolution” assembled be ideally placed, both scientiﬁcally and world-renowned experts and more geographically, as a part of Berlin’s thriv- than 1,600 participants in Berlin (see ing research and technology landscape. page 16 for details). The members of ZIB are very proud of their institute and its outstanding performance, and strongly believe that success is only sustainable through continuous adaptation and development. In 2016, the initiation of the next round of the German Excellence Initiative presented a perfect platform for discussing ZIB’s future research strategy together with its closest partner institutions. Debates began in almost all research fields regarding the respective grand challenges and related research oppor- tunities. Members of ZIB participated in many of these discussions, and many new initiatives and partnerships origi-

4 2016 Annual Report nated from them. Through this process, pain-relief drug, which has been devel- the GasLab” reviews progress and chal- ZIB has identified several new research oped purely based on computational lenges in the largest of ZIB’s projects with priorities and, in turn, several of ZIB’s methods in a cooperation between ZIB industry that may even be the largest research groups are part of Berlin-based and Berlin’s university hospital Charité industry-related research project in all of Clusters of Excellence initiatives. One of within the last five years, has success- German mathematics. Switching fields the consequences of these developments, fully passed preclinical trials, has led to a again, “Supercomputing at the Limit” for example, will be that ZIB strengthens publication in Science, the formation of a explains why software modernization is its research activities regarding digital new spin-off company, and the beginning one of the essential factors for the future humanities and computational social of clinical tests (see page 22 for details). of supercomputing and how this topic sciences. is addressed at ZIB. Finally, the article This annual report provides insight into a “Performance in Data Science” reports Apart from strategic planning, ZIB variety of other success stories and gives on ZIB’s strategy to increase the perfor- continues to be a place booming with a general overview of ZIB’s organization mance of data-analysis frameworks by excellent research and first-rate scien- and key factors for its successful devel- monitoring and improving data-access tific services and infrastructure. The opment. In particular, six feature articles pattern, layout, and placement. year 2016 again broke several all-time highlight aspects of our work: “Bigger records. For example, in total, Data, Better Health” gives an overview Obviously, these feature articles can of ZIB-based data-driven research only be appetizers. Readers interested and related Web-based deployment of in the specifics of ZIB’s activities can find 8 MILLION EUROS’ data-analysis and data-management further articles and the full technical tools for applications in biomedicine and details of our research and services, of WORTH OF THIRD- health care. “Shape-Based Data Analysis our projects, and the statistics about – Methods and Results” features our publications, lectures, and so on, on ZIB’s research activities around shapes and Web pages: the starting point is www. PARTY FUNDING geometry reconstruction with its diver- zib.de. sity of applications ranging from medical was acquired, which marked an increase Last but not least, ZIB is grateful for therapy planning to digital humanities. for the fifth year in a row and a new having many supporters and friends. The article “Computing the Truth” record in ZIB’s history! With more than Thanks to all those who provide us with takes a completely different perspective one hundred ongoing research projects, encouragement, assistance, and advice. by addressing the question of whether ZIB grew to have over 200 employees, absolute evidence can be computed, at complemented by more than 70 research least in the realm of mathematical proofs. fellows and long-term guests. In several Berlin, May 2017 Turning back from philosophical ques- cases, the rewards of long-term efforts Christof Schütte tions to real-life applications, the article could be reaped. For example, a nontoxic President of ZIB “Research Campus MODAL: Report from

Zuse Institute Berlin 5 30 BIGGER DATA, BETTER HEALTH Generating Medical Knowledge from 4 Vast Amounts of Data ZUSE INSTITUTE 44 BERLIN Preface | Executive Summary | SHAPE-BASED DATA Organization | ZIB Structure | ZIB in Numbers | The Digital Future – ANALYSIS – METHODS 75 Years Zuse Z3 and the Digital Revolution | The Hall of Fame | AND RESULTS Core Facility@ZIB | Computational Design of a Nontoxic Painkiller | Towards Understanding Empirical Economic Situation | Spin-Offs | Shape Collections Number of Employees CONTENTS 52 72 RESEARCH CAMPUS MODAL: REPORT PERFORMANCE FROM THE GASLAB IN DATA SCIENCE Digging Deep to Uncover Treasures Meeting the Challenges of Transporting Natural Gas 64 COMPUTING THE TRUTH

A Man-Machine Search for the Absolute Evidence 84 104 SUPERCOMPUTING ZIB PUBLICATIONS AT THE LIMIT REFERENCES IMPRINT How Technology Innovations Impact Software Design Executive Summary EXECUTIVE SUMMARY MATHEMATICS FOR LIFE AND FOR MATERIAL SCIENCES

The focus shift towards data-driven on the upcoming excellence initiative research gained momentum during the guiding many new developments, last year, with many activities in mod- especially in new topics such as model-based data-analysis, inverse prob- eling and simulation of opto-molecular lems, data-driven and network-based interactions and tissue-scale processes, modeling of multiscale processes, or agent-based modeling in humanities uncertainty quantification, and geo- and social science. Furthermore, during metric image analysis for extracting the last year, the division’s research information and deriving scientific activity was shaped by an institute-wide knowledge from data. In this context, trend towards the development and uti- considerable effort went into the devel- lization of advanced machine learning opment of Web-based deployment of techniques. data-analysis and management tools; The continuation of established MSO the collaboration with the FU Core activities in optical nanostructures, Facility BioSupraMol on microscopy metabolic networks, and musculoskel- and the MODAL smart health platform etal diseases led to many achievements, are two prominent examples. with the extension of the DFG-CRC 787 The successful extension of the Einstein as just one example. Center for Mathematics enabled a focus

8 2016 Annual Report MATHEMATICAL OPTIMIZATION PARALLEL AND AND SCIENTIFIC DISTRIBUTED INFORMATION COMPUTING

The last year was marked by consid- 75 years after Konrad Zuse’s invention erable progress in the optimization of his Z3, the first digital, programmable department's ability to deal with computer, we prepared the procurement problems that require the integration of the next HLRN supercomputer which of methods from different mathematical will be more than 100,000,000,000,000 fields. Such problems are of paramount times faster than the Z3. Computing importance from both a theoretical technology is still advancing at impres- and a practical point of view. Examples sive speed, recently through technolog- of such developments are successes ical advancements such as many-core in interactive conbinatorial theorem CPUs, deeper memory hierarchies and proving using exact integer program- wider SIMD vector units. Consequently, ming (see the respective feature article), the difficult process of procuring a first steps towards discrete-continuous supercomputer has to be combined aircraft trajectory routing algorithms, with further improvements of our HPC and striking results for the optimiza- consultancy, which is a focal point in the tion of gas network operations using efficient use of modern supercomputers. demand forecasts based on machine This includes code modernization in learning; the latter project was awarded tight cooperation with application devel- the prestigious EURO Excellence opers to achieve better performance on in Practice Award 2016. ZIB also new system architectures. hosted the International Congress on Handling large data volumes plays an Mathematical Software (ICMS 2016), increasingly important role in high a high-level conference that aims at an performance computing and large-scale integration of mathematics and com- data analytics. Within our two research puter science. projects GeoMultiSens and Berlin Big Promoting open-access to scholarly Data Center, for example, we develop research remained a major task for methods to improve the performance of the scientific information department. data-analysis pipelines. In some cases, The DeepGreen project was started better data placement, improved sched- with the aim to automatically transfer uling, and enhanced code optimization scientific publications into open-access workflow resulted in tremendous run- repositories. The release of version 4.5 time reductions. of the open-source repository software OPUS 4 marked a cornerstone of the department´s ongoing commitment to openness. KOBV and digiS launched the digital preservation system EWIG which combines strategies and actions that address the still unresolved issue of ensuring long-term access to digital cultural heritage.

Zuse Institute Berlin 9 Organization

ADMINISTRATIVE SCIENTIFIC BODIES ADVISORY BOARD

The bodies of ZIB are the President and DR. JUTTA KOCH-UNTERSEHER The Scientiﬁc Advisory Board advises the Board of Directors until September 18, 2016 ZIB on scientiﬁc and technical issues, (Verwaltungsrat). Senatsverwaltung für Wirtschaft, supports ZIB’s work, and facilitates Technologie und Forschung ZIB’s cooperation and partnership with President of ZIB is since September 19, 2016 univer sities, research institutions, and PROF. DR. CHRISTOF SCHÜTTE Der Regierende Bürgermeister von industry. Berlin Senatskanzlei Wissenschaft und Vice President is Forschung The Board of Directors appointed the N.N. following members to the Scientific N.N. Advisory Board: The Board of Directors was composed in since September 19, 2016 PROF. DR. JÖRG-RÜDIGER SACK 2016 as follows Senatsverwaltung für Wirtschaft, Carleton University, Ottawa, Canada Energie und Betriebe PROF. DR. PETER FRENSCH Vice President, Humboldt-Universität PROF. DR. ALFRED K. LOUIS STS. STEFFEN KRACH zu Berlin (Chairman) Universität des Saarlandes, until September 18, 2016 Saarbrücken Senatsverwaltung für Bildung, Jugend PROF. DR. CHRISTIAN THOMSEN und Wissenschaft President, Technische Universität Berlin PROF. DR. RAINER E. BURKARD (Vice Chairman) Technische Universität Graz, Austria PROF. MANFRED HENNECKE Bundesanstalt für Materialforschung PROF. BRIGITTA SCHÜTT PROF. DR. MICHAEL DELLNITZ und -prüfung (BAM) Vice President, Freie Universität Berlin Universität Paderborn

THOMAS FREDERKING LUDGER D. SAX Helmholtz-Zentrum Berlin für Grid Optimization Europe GmbH Materialien und Energie (HZB)

PROF. DR. ANNA SCHREIECK PROF. DR. HEIKE WOLKE BASF SE, Ludwigshafen Max-Delbrück-Centrum für Molekulare Medizin (MDC) PROF. DR. REINHARD UPPENKAMP Berlin Chemie AG, Berlin The Board of Directors met on May 27, 2016, and December 9, 2016. PROF. DR. KERSTIN WAAS Deutsche Bahn AG, Frankfurt am Main

The Scientiﬁc Advisory Board met on July 4 and 5, 2016, at ZIB.

10 2016 Annual Report THE STATUTES

The Statutes, adopted by the Board of Directors at its meeting on June 30, 2005, define the functions and procedures of ZIB’s bodies, determine ZIB’s research and development mission and its service tasks, and decide upon the composition of the Scientiﬁc Advisory Board and its role. ORGANIZATION

SCIENTIFIC BOARD OF DIRECTORS CHAIRMAN: PROF. DR. PETER FRENSCH ADVISORY BOARD Humboldt-Universität zu Berlin (HUB) CHAIRMAN PROF. DR. JÖRG-RÜDIGER SACK | Ottawa PROF. DR. RAINER E. BURKARD | Graz PROF. DR. MICHAEL DELLNITZ | Paderborn PROF. DR. ALFRED K. LOUIS | Saarbrücken LUDGER D. SAX | Essen PROF. DR. ANNA SCHREIECK PRESIDENT | Ludwigshafen PROF. DR. CHRISTOF SCHÜTTE PROF. DR. REINHARD UPPENKAMP | Berlin PROF. DR. KERSTIN WAAS VICE PRESIDENT | Frankfurt am Main N.N.

MATHEMATICS FOR LIFE AND MATHEMATICAL OPTIMIZATION AND PARALLEL AND ADMINISTRATION MATERIAL SCIENCES SCIENTIFIC INFORMATION DISTRIBUTED COMPUTING AND LIBRARY Prof. Dr. Christof Schütte Prof. Dr. Ralf Borndörfer Prof. Dr. Alexander Reinefeld Annerose Steinke Prof. Dr. Thorsten Koch

Zuse Institute Berlin 11 ZIB Structure

MATHEMATICS FOR LIFE MATHEMATICAL OPTIMIZATION PARALLEL AND DISTRIBUTED COMPUTING AND MATERIAL SCIENCES AND SCIENTIFIC INFORMATION A. Reinefeld C. Schütte R. Borndörfer, T. Koch

NUMERICAL VISUAL DATA MATHEMATICAL SCIENTIFIC DISTRIBUTED SUPERCOMPUTING MATHEMATICS ANALYSIS OPTIMIZATION INFORMATION ALGORITHMS T. Steinke M. Weiser H. Hege R. Borndörfer, T. Koch (B. Rusch) F. Schintke T. Koch

COMPUTATIONAL VISUAL DATA MATHEMATICS OF WEB DISTRIBUTED DATA HPC CONSULTING MEDICINE ANALYSIS IN TRANSPORTATION TECHNOLOGY MANAGEMENT T. Steinke M. Weiser, SCIENCE AND AND LOGISTICS AND MULTIMEDIA F. Schintke S. Zachow ENGINEERING R. Borndörfer W. Dalitz H. Hege

COMPUTATIONAL IMAGE ANALYSIS MATHEMATICAL DIGITAL PRESER- SCALABLE HPC SYSTEMS MOLECULAR IN BIOLOGY OPTIMIZATION VATION ALGORITHMS C. Schimmel DESIGN AND MATERIAL METHODS W. Peters-Kottig T. Schütt M. Weber SCIENCE A. Gleixner S. Prohaska, D. Baum

COMPUTATIONAL THERAPY MATHEMATICS OF SERVICE CENTER MASSIVELY ALGORITHMS NANO-OPTICS PLANNING TELECOMMUNI- DIGITIZATION PARALLEL DATA FOR INNOVATIVE F. Schmidt S. Zachow CATION BERLIN ANALYSIS ARCHITETURE R. Borndörfer A. Müller F. Schintke T. Steinke

COMPUTATIONAL BIOINFORMATICS ENERGY KOBV LIBRARY SYSTEMS BIOLOGY IN MEDICINE NETWORK NETWORK – S. Röblitz T. Conrad OPTIMIZATION RESEARCH AND J. Zittel DEVELOPMENT B. Rusch

UNCERTAINTY KOBV LIBRARY QUANTIFICATION NETWORK – T. Sullivan OPERATING S. Lohrum

FRIEDRICH- ALTHOFF- KONSORTIUM U. Kaminsky

CORE FACILITY IT AND DATA SERVICES C. Schäuble

12 2016 Annual Report ADMINISTRATION AND LIBRARY A. Steinke ZIB STRUCTURE

ZIB is structured into four divisions, three scientific divisions, and ZIB’s administration. Each of the scientific divisions is composed of two departments that are further subdivided into research groups (darker bluish color) and research service groups (lighter bluish color). Click on the respective box to get the unit of interest:

LEGEND SCIENTIFIC DIVISIONS AND DEPARTMENTS RESEARCH GROUPS

RESEARCH SERVICE GROUPS BRAIN BERLIN RESEARCH AREA CORE FACILITY INFORMATION NETWORK C. Schäuble

Zuse Institute Berlin 13 ZIB in Numbers ZIB IN NUMBERS 12,903 PROMOTION OF YOUNG SCIENTISTS: DISSERTATIONS SEMINARS DIPLOMAS GIVEN BY ZIB SCIENTISTS AT UNIVERSITIES MASTER’S 13 SCIP

11 OF MIP SOLVER SCIP LECTURES GIVEN BY ZIB SCIENTISTS AT UNIVERSITIES 1,601 VISITORS PROFESSORSHIPS LONG NIGHT OF THE SCIENCES OFFERED TO ZIB 3RESEARCHERS SCIENTIFIC 116 311TALKS PEER-REVIEWED PUBLICATIONS IN INTERNATIONAL SCIENTIFIC JOURNALS 61 DISTINGUISHED 150 INVITED

14 2016 Annual Report DATA ARCHIVE AT ZIB TOTAL CAPACITY ON 50 PB 16,600 TAPES

OUTREACH EVENTS FOR SCHOOL CLASSES AND 67 THE GENERAL PUBLIC ¤7,981,000 ¤5,487,230 PROJECT-RELATED, PUBLIC THIRD-PARTY FUNDS ¤2,493,770 INDUSTRIAL THIRD-PARTY PROJECTS

21 7,350 105 INTERNATIONAL GUESTS CONFERENCES AND AT ZIB IN 2016 WORKSHOPS AT ZIB

IN JUNE 2016: ZIB SUPERCOMPUTER IS NO. 95 IN TOP-500 LIST

Zuse Institute Berlin 15 Zuse 75 THE DIGITAL FUTURE – 75 YEARS ZUSE Z3 AND THE DIGITAL REVOLUTION

Celebrating the 75th Anniversary of the Invention of the Computer; a Perfect Occasion for Science Communication and Public Outreach

On May 11, 1941, Konrad Zuse this digital revolution is seen as a sciences, and therefore decided to uti- presented the first fully functional curse or blessing very much depends, lize the organized 75th anniversary of digital computer, his Z3, in Berlin and however, on the perceptions and actions the invention of the Z3 for organizing a the digital age began. of humans: Do we possess the tools to series of related public events. utilize it for our goals, or will we be over- 75 years later, the world has changed The main event, the scientific confer- whelmed and even controlled by these completely: Konrad Zuse’s invention ence “The Digital Future – 75 years advances? The challenge of mastering started the digital revolution that Zuse Z3 and the Digital Revolution” this transformation affects individuals brought computers into almost a billion took place in Berlin on May 11, 2016, as well as companies or organizations households, created the Internet, mobile with more than 1,600 participants and but also scientific fields and disciplines. communication and smart devices, high a series of more than 40 presentations performance computing, big-data ana- The Zuse Institute Berlin, feeling com- by internationally renowned experts in lytics, as well as, with it, the information mitted to the legacy of its name patron, the following five sessions: society and an avalanche of other inno- aims at supporting debates between vations and transformations. Whether the public and the progressing digital

Horst Zuse and Governing Mayor of Berlin Michael Müller

16 2016 Annual Report “The Digital Future 2016”

1. Simulation, Optimization, the German Chancellery and Federal the effect of the conference: on each Visualization Minister for Special Affairs, Peter of the 75 days before the conference, 2. Data Analysis, Big Data, and Altmaier. ZIB’s media partner Der Tagesspiegel, Security/Privacy one of Germany’s largest newspapers, The conference was closely followed 3. The Future of Computing published a portrait of a pioneer of by the media. For example, 25 German 4. Networks and Mobility the digital age and of their scientific and international newspapers published 5. Communication, Digital Society, and achievements. These portraits were articles about the conference, the anni- Gaming written by science journalists based on versary, and ZIB’s work, and several TV information provided by ZIB, were pub- The scientific program was comple- channels broadcasted related reports, lished on the second most read page of mented by addresses of the Governing documentaries, and interviews. the newspaper, and created a signiﬁcant Mayor of Berlin, Michael Müller, the In terms of outreach into the public, awareness for the scientists presented Head of the Government of the State another media activity even outclassed as well as for ZIB. of Berlin, and by the Chief of Staff of

Left to right: Christof Schütte, Horst Zuse, Leslie Greengard, Michele Parrinello, Governing Mayor of Berlin Michael Müller, Klaus-Robert Müller, Martin Grötschel

Zuse Institute Berlin 17 Zuse 75

The digital pioneers presented in this son Horst Zuse accepted the award on matics (K. Reinert), women pioneers in article series, starting with Konrad his father’s behalf, and gave a presenta- computing (S. Krämer), digital human- Zuse and Alan Turing but also includ- tion about the life of the inventor of the ities (V. Lepper), supercomputing (A. ing technology drivers like Bill Gates computer. Reinfeld), computational drug design or Steve Jobs, were all selected and (M. Weber), efficient data transfer and In addition to the conference, ZIB inducted into the “Hall of Fame of the video compression (T. Wiegand), and and Freie Universität Berlin together Digital Age” by an academic jury. In its autonomous driving (R. Rojas). The organized a lecture series of 12 public electronic form, this Hall of Fame can 90-minute-long talks were all appro- presentations by renowned experts in be found at www.zib.de/de/hall-of-fame. priate for a broad public audience and the course of the 2016 summer semester. attracted more than 2,000 participants At the beginning of the conference, This series covered essential topics of in total. Most of the presentations were Michael Müller, Governing Mayor of digital science: scientific visualization followed by long discussions between Berlin, honored four digital pioneers for (C. Hege), machine learning (K.R. the speaker and the audience that made their remarkable achievements: Michele Müller), optimization (M. Grötschel), the lectures particularly successful as Parrinello, Leslie Greengard, Klaus- digital health (E. Böttinger), digitization communication events between scien- Robert Müller, and Konrad Zuse – who’s of cultural heritage (T. Koch), bioinfor- tific experts and the wider public.

18 2016 Annual Report Zuse Institute Berlin 19 Core Facility CORE FACILTY

In 2016, the IT department of ZIB was restructured to become the new Core Facility “IT and Data Services.” Carsten Schäuble was hired as the head of the Core Facility. The reor- ganization merged several working groups and competence sets and aims at making ZIB’s IT infrastructure ﬁt for the increasing requirements of the digital revolution. The ultimate goal of the Core Facility is to build and offer an IT environment that enables Open Science for the members of ZIB and collaborating institutions.

One way to handle 50 PB of data

20 2016 Annual Report @ZIB

The IT service portfolio will be infrastructure will be modernized improved and extended. This includes and included in an institute-wide a new identity management, which will management system. This also allow better self-service and automa- includes the BRAIN (Berlin Research tion of common tasks such as work- Area Information Network) system. group communication and management Here, installation of the new 100 Gbit services. Internally, new security router was ﬁnished at the end of 2016. standards will be implemented, mainly Until May 2017, BRAIN will take the effecting backup, storage, and ﬁrewall Berlin research network to the next technology. Especially the access layer performance level, raising the possible for the internal ZIB networks will be network throughput from 10 GBit/s up newly designed and installed. A new to 100 GBit/s. software-defined firewall system will The modernization of services and allow the creation of two security zones infrastructure will lead to an innova- and virtual private networks (VPNs) in tive service portfolio that the new core which mission-critical and experimen- facility can deliver and offer to internal tal services can be placed and run with members and external groups and © Zuse Institute Berlin Institute © Zuse different access patterns from the World institutions. Ultimately, this will enable Wide Web. more and advanced projects running On the infrastructure side, ZIB’s within ZIB’s data center. As one of the data-center will be fully modernized. first steps, the Europe-wide authenti- New power-efficient server systems cation infrastructure DFN AAI AAI and racks will be installed, including – Identity Federation was implemented new technology for climate, power which allows participating institutions supply, and distribution. Based on this, to use ZIB services and enables ZIB all internal resources such as compute members to use external DFN services and virtualization servers, HPC, and with their normal ZIB account. storage systems, as well as the network

Zuse Institute Berlin 21 Computational Molecular Design COMPUTATIONAL DESIGN OF A NONTOXIC PAINKILLER

ZIB’s research group “Computational Molecular Design” successfully designed a candidate for a pain-relief drug. The design strategy was created in cooperation with clinical researchers at Berlin’s university hospital Charité but was performed purely computationally. The drug candidate very successfully passed all preclinical tests; clinical trials will follow. © Zuse Institute Berlin Institute © Zuse

One of the inventors, Marcus Weber, in front of ZIB’s supercomputer that was used for parts of the development

22 2016 Annual Report © Zuse Institute Berlin Institute © Zuse

NFEPP inside the binding pocket of the -opioid receptor

Worldwide, more than 1,000,000,000 In a long-lasting cooperation between The computations resulted in the design people suffer from significant pain. ZIB and Charité, the idea was created to of a fentanyl derivate (NFEPP). On the In 50% of these cases, the pain is not change the molecular structure of opioids one hand, quantum chemical simulations adequately treated. Although effective in such a way that the painkillers are only showed that NFEPP would allow for the analgesic drugs are available, their active in inflamed tissue and inactive in desired discrimination between healthy use is limited due to severely adverse other parts of the body, especially the and inflamed tissue by pH values, while side effects. Opioids, the most effective brain or gut. This kind of inactivity would docking simulations using classical painkillers, produce sedation, apnea, result in an absence of the adverse side molecular dynamics in combination addiction, and constipation mediated effects, that is, in a nontoxic painkiller. with conformation dynamics approaches in the brain or gut. In the USA, the rates developed at ZIB showed that NEEPP The search for an appropriate structural of overdose on prescription opioids would be a good agonist of the -opioid change was performed with purely com- increased 4-fold in the last decade, with receptor [CMD2]. After synthesizing putational means: under the working almost 19,000 overdose deaths associ- NFEPP, it was used for animal tests at hypothesis that inflamed and healthy ated with prescription opioids in 2014. Charité that confirmed the properties tissue exhibit different pH values, we The famous singer Prince died from such predicted computationally [CMD1]. In screened for molecules that are good ago- an overdose in 2016. Similar alarming 2016, a ZIB spin-off (DoloPharm UG) was nists of the -opioid receptor for normal trends can also be observed in Europe. founded with the aim to organize clinical pH but fail to do so for reduced pH values. trials of this drug candidate.

[CMD1] V. Spahn, G. Del Vecchio, D. Labuz, A. Rodriguez- [CMD2] C. Stein, M. Weber, C. Zöllner, O. Scharkoi: Gaztelumendi, N. Massaly, J. Temp, V. Durmaz, P. Fentanyl derivatives as pH-dependent opioid receptor Sabri, M. Reidelbach, H. Machelska, M. Weber, C. agonists. USA patent US 14/239,461 (2015). Stein: A nontoxic pain killer designed by modeling of pathological receptor conformations. Science, publication submitted August 2016, to appear in March 2017.

Zuse Institute Berlin 23 Economic Situation in 2016

In 2016, the total income of ZIB com- prised 18.5 million euros. The main part of this was made available by the State of Berlin as the core budget of ZIB (8.4 million euros) including investments and Berlin’s part of the budget of HLRN at ZIB. A similarly large part resulted from third-party funds (8.0 million euros) acquired by ZIB from public funding agencies (mainly DFG and BMBF) and via industrial research projects. This was complemented by a variety of further grants such as the budgets of BRAIN (State of Berlin) and KOBV (mixed funding) as well as the part of the HLRN budget made available by other German states. ¤2,100,000

¤8,400,000

¤8,000,000

24 2016 Annual Report ECONOMIC SITUATION IN 2016

ZIB INCOME

45% Core budget by State of Berlin 43% Third-party funds 12% Further grants

Zuse Institute Berlin 25 Economic Situation in 2016

ECONOMIC SITUATION IN 2016

The Zuse Institute Berlin finances its scientific work via three main sources: the core budget of ZIB provided by the State of Berlin, third party funds from ¤9,000,000 public sponsors, and those of industrial- cooperation contracts. In 2016, ZIB raised third-party funding for a large number of projects. Project-related public third-party funds ¤8,000,000 decreases slightly by 4 % to 5,487 k€ in 2016, and industrial third-party projects increased by more than 18% from 2,106 k€ in 2015 to 2,494 k€ in 2016. ¤7,000,000 In total, 7,981 k€ third-party funding marked a new record in ZIB’s history; an increase for the fifth year in a row! ¤6,000,000

¤2,493,770 ¤2,720,670 ¤5,000,000 Industry BMBF incl. FC MODAL

¤4,000,000

¤1,626,400 ¤1,028,770 Other public funds DFG ¤3,000,000

¤111,840 EU ¤2,000,000 ZIB THIRD- PARTY FUNDS ¤1,000,000 BY SOURCE

2% EU 13% DFG ¤0 20% Other public funds 2007 2008 31% Industry 34% BMBF incl. FC MODAL

26 2016 Annual Report ZIB THIRD- PARTY FUNDS IN EUROS

INDUSTRY

PUBLIC FUNDS

2009 2010 2011 2012 2013 2014 2015 2016

Zuse Institute Berlin 27 Spin-Offs | Number of Employees

COMPUTING IN BIT-SIDE GMBH LAUBWERK GMBH TECHNOLOGY GMBH 2000 | www.bit-side.com 2009 | www.laubwerk.com (CIT) Telecommunication applications Construction of digital plant models 1992 | www.cit-wulkow.de and visualization Mathematical modeling and develop- 1000SHAPES GMBH ment of numerical software for technical DRES. LÖBEL, BORNDÖRFER & 2010 | www.1000shapes.com chemistry WEIDER GBR Statistical shape analysis 2000 | www.lbw-berlin.de RISK-CONSULTING Optimization and consulting in TASK – Berthold Gleixner Heinz Koch PROF. DR. WEYER GMBH public transport GbR 1994 | www.risk-consulting.de 2010 Database marketing for insurance LENNÉ3-D GMBH Distribution, services, and consulting for companies 2005 | www.lenne3d.com ZIB’s optimization suite 3-D landscape visualization, INTRANETZ GMBH software development, and services QUOBYTE INC. 1996 | www.intranetz.de 2013 I www.quobyte.com Software development for logistics, JCMWAVE GMBH Quobyte develops carrier-grade storage database publishing, and e-government 2006 | www.jcmwave.com software that runs on off-the-shelf Simulation software for hardware AKTUARDATA GMBH optical components 1998 | www.aktuardata.de KEYLIGHT GMBH Development and distribution of risk- ONSCALE SOLUTIONS GMBH 2015 I www.keylight.de evaluation systems in health insurance 2006 | www.onscale.de Keylight develops scalable real-time Web Software development, consulting, and services and intuitive apps. The focus VISAGE IMAGING GMBH services for parallel and distributed is on proximity, marketing, iBeacon, (Originating from a spin-off of Visual storage and computing systems and Eddystone for interactive business Concepts GmbH) models 1999 | www.visageimaging.com Visualization and data-analysis DOLOPHARM UG (Amira etc.), especially 2016 medical visualization A specialty pharmaceutical company focused on the clinical and commercial ATESIO GMBH development of new products in pain 2000 | www.atesio.de management that meet the needs of Development of software and consulting acute and chronic care practitioners and for planning, conﬁguration, and optimi- their patients zation of telecommunication networks SPIN- OFFS

28 2016 Annual Report In the year 2016, 236 people were employed at ZIB; of these, 172 positions were ﬁnanced by NUMBER third-party funds. OF EMPLOYEES 1/1/2016 1/1/2017 303303MANAGEMENT 18 87 105 15 99 114 SCIENTISTS 34 13 47 38 10 48 SERVICE PERSONNEL 7 8 15 8 8 16 KOBV HEADQUARTERS 0 54 54 0 55 55 STUDENTS 62 162 224 64 172 236 TOTAL Temporary Temporary Permanent Permanent TOTAL TOTAL

Zuse Institute Berlin 29 Prof. Dr. Tim Conrad | [email protected] | +49-30-84185-250 BIGGER DATA, BETTER HEALTH

Bigger Data, Better Health

1 DO YOU REMEMBER THE LAST TIME A MEDICAL DOCTOR TOOK SOME DATA FROM YOU?

Chances are good it did not hurt at all – ﬁnally – its deep analysis. Making use and you might not even have noticed. of these data will support medical doc- However, if you take a moment and tors in the future to make more informed think about it, you probably remem- decisions. For example, comparing their ber an x-ray being taken or some own patients to millions of other cases of blood test performed. Maybe you even patients having the same conditions and remember that your body mass index (early) symptoms would allow to recog- was calculated and written down, or nize a disease in a very early stage and simply your weight being measured. select the best personalized treatment It's possible you also underwent some option. gene testing procedure to check which At ZIB, we support the rise of this new drug you would react to best for a par- type of information-based medicine in ticular treatment. two ways. First, we develop IT infra- One of the most noteworthy changes structure needed for secure storage and during the last years is that hospitals fast retrieval of these very large data started to routinely collect and store all collections. Second, we work on new these data. What is yet missing though algorithms and methods to analyze these is the connection of all these data within data and turn it into medical knowledge. and across hospitals, the aggregation, and

1 A schematic representation of a DNA molecule and virus cells. The challenge is to find changes induced by these tiny intruders that can help identifying a particular disease.

32 2016 Annual Report DETECTING DISEASES kjpargeter / Freepik © Designed by FROM LARGE-SCALE OMICS DATA Bigger Data, Better Health

Early detection of a disease usually leads to a significantly better outcome for the patient, compared to the same treatment given at a later time. This is true in particular for some types of cancer or cardiovascular diseases, where hours or days lost can make a substantial difference. Many traditional detection approaches based on molecular tests often fail in early etectiond due to a lack in efficiency. In many cases these methods simply fail because changes in the traditional, measurable biomarkers are simply too subtle.

BIOLOGY HAPPENS Changes in the human body during pro- THE CENTRAL DOGMA OF BIOLOGY gression of a disease happen on many biological levels, such as the genome (what can happen?), the transcriptome (what seems to happen next?), the pro- DNA teome (what is currently happening?), What can happen? or the metabolome (what happened?). Following the central dogma of molecular biology and its extensions, these levels are highly interconnected and depend on each other. Thus, tests for diagnosing RNA a condition or classifying a disease’s What appears to be happening? state should take all available data into account to build more reliable diagnostic predictors. This can be especially crucial given the nature of biological data which are PROTEIN usually very noisy and contains errors What makes it happen? (e.g. missing data). In this situation, multiple data types may compensate for missing or unreliable information in any one data type and thus yielding a more robust method, with regards to METABOLITE biological uncertainties. Further, differ- What has happened and is happen- ent sources of data that point to the same ing? gene, protein, or metabolite are less likely to be false positives and could indicate functionality.

34 2016 Annual Report © ktsdesign – Fotolia

BIG DATA IN DISEASE DIAGNOSIS

Researchers at ZIB work on new methods based on the combination of multiple biomarkers, potentially from different biological omics levels and including new data types such as environmental factors or tracking devices. However, the more variables can be measured and hence enter the modeling equations, the more data is needed to determine their correct values. Or – in other words – to REDUCING prevent overfitting. Until recently, acquisition of these large data sets was COMPLEXITY very time-consuming and certainly very expensive. Only large international sci- The main problem when using large- entiﬁc collaboration efforts could afford scale data is that possible analysis these large studies. Thanks to today’s results are also large scale. The goal is high-throughput omics technologies, thus to generate answers that are of low formerly hardly imaginable omics data complexity. Researchers at ZIB have volumes can be generated within days found a new way to only extract the with a fraction of the budget needed some relevant (low-complexity) information years ago. Using these technological from very large biological data sets. For more details about efficient advances, vast amounts of data have been Using these new methods will allow us large-scale data-management created, providing a very detailed view on to generate solutions that can be easily methods developed by ZIB, see the diseases and their diagnosis. understood by humans, and not only by feature article “Performance in Data large supercomputers. Today, we have all this information avail- Science.” able in large databases and the computing power for detailed analysis. What is still n missing are efficient methods and algorithms to extract knowledge. Knowledge, max y (x ␻) subject to ͉͉␻͉͉ р ␭ ͉͉␻͉͉ р ⌺ i i1 1 and 2 1 that is interpretable for practitioners and ␻⑀Rd ultimately supports medical doctors to i=1 make better diagnoses.

Zuse Institute Berlin 35 Bigger Data, Better Health DATA ANALYSIS FOR UNDERSTANDING NEURONAL DEVELOPMENT AND MAINTENANCE

Stochastic mechanisms play an By applying image analysis, we are able to important role in the brain. A better reconstruct and analyze particle dynam- understanding of the principles cre- ics, which can then reveal differences ates insight in disease mechanisms, between cell mutations. (see figure 2). By which will contribute to developing applying stochastic modeling, we hope to effective therapies of neurological explain the observed transport processes disorders. ZIB has been working and understand how they lead to robust, together with NeuroCure on under- long-term maintenance of synapses, standing principles of neuronal whose underlying mechanism are yet development and maintenance. Two unknown. specific examples are the analysis of The brain develops complicated neuronal protein trafficking during synaptic circuits. Long-term, high-resolution live maintenance and filopodia dynamics b imaging can be applied to visualize neu- in neuronal growth cones. ron growth during brain development. Understanding the mechanisms that Growth cones exhibit seemingly random cause neurological disorders such as movements of finger-like structures, multiple sclerosis and epilepsy could called filopodia, which eventually form help developing therapies. Insight in the connections with neighboring neurons. general principles are key to understand- It is a remarkably robust process that ing specific disorders. We have been reproduces complex neuronal circuits developing methods for image analysis in individual brains. We have been and mathematical modeling that enable developing image analysis methods that understanding stochastic mechanisms allow reconstructing the geometry of of maintenance in the adult brain and growth cones from microscopy images. development in the growing brain. Automatic image analysis is applied to create a geometric representation of the Synapses are the neuronal connections filopodia. The geometry is then manually in the brain. They remain functional over corrected and fed back to the automatic long periods, possibly the entire life span analysis to incrementally refine the rep- of an organism. The understanding of resentation. The results are visually and maintenance processes that enable the statistically analyzed into gain insight in longevity of synapses is key to under- the neuron growth dynamics over time standing the longevity of the brain as a (see figure 2). The information is also key whole. Seemingly random trafficking of The soma a collects the input signal to validating stochastic mathematical proteins and cell organelles along axons from the dendrites b and generates models of the growth processes. can be observed using light microscopy. a specific output signal. The axon c is the only output of a neuron. Its characteristic electrochemical signal is forwarded toward the axon terminals d which are connected via synapses with the dendrites of a succeeding neuron.

36 2016 Annual Report 2 Axonal trafficking. The microscopy image (left) shows transport vesicles (red), which are transported along the axon. The diagram (right) depicts a statistical analysis of particle trajectories. The color coding indicates the strength of directional transport towards the soma and the terminals (top and bottom, resp.) d under certain mutations.

2 3 a m] ␮

c [ Length Time steps [min] steps Time

Orientation [°]

3 Growth-cone dynamics. The image (top) illustrates a growth cone with a geometry representation of the filopodia (green lines and yellow dots). The geometry allows a simplified estimation of length and orientation of the filopodia. The change of length over time is color coded in the diagram (bottom). The plot indicates the life span of a filopodium (y-axis) and the orientation (x-axis).

Zuse Institute Berlin 37 Bigger Data, Better Health

DIAGNOSTIC SCORES FROM MRI

Osteoarthritis, a degenerative disease In clinical practice, cartilage damage scores for several hundred data sets, of articular cartilage, is prevalent in is graded by expert readers employing deep neural networks will be trained to our aging society, mostly affecting the semiquantitative scoring methods. The automatically classify and localize the knee joint. While a strong inﬂuence of cartilage damage is usually assessed in cartilage damage. Jointly with Charité mechanical loading – in particular of MRI data considering percentages of Berlin, a prospective study started in high impacts such as those occurring cartilage thickness loss and cartilage February 2016 with up to 240 patients in extreme sports – on osteoarthritis area affected by degradation. We are for two years. These patients perform onset and progression is evident, the developing automated methods for local- different levels of regular exercise under mechanisms of cartilage degradation ization and grading of cartilage damage. psychological supervision. Based on are not well understood, despite First, using multi-object graph-cutting image data of this study in combination decades of clinical research. Within methods, a region of interest of the fem- with deep learning-based methods for the musculoskeletal research net- oral and tibial cartilage is automatically automated assessment of cartilage dam- work OVERLOAD/PrevOP, we are determined in the MRI data yielding a age, we are aiming to evaluate whether or working together with Charité Berlin 3-D segmentation. Afterwards, using not regular exercise slows down the rate on improving understanding. both MRI data and the expert readers’ of osteoarthritis progression.

38 2016 Annual Report CARTILAGE LOADING FROM GAIT DATA

While the importance of mechanical are reproduced allows the computation loading is generally accepted, correlating of 3-D distributed stress and strain pat- disease progression with local stress and terns within the cartilage (see figure 5). strain states of loaded cartilage is hard These can be correlated to clinical due to the difficulty of measuring these findings, variations of the anatomy, or dynamic quantities in vivo. An approach frequent lesions like loss of the anterior we pursue is to estimate the local crucial ligament. mechanical state of knee cartilage from Solving the identification problem gait data by coupling a global dynamic involves a combination of finite element multibody model of the lower limb to an methods, time-stepping schemes, non- elastomechanical contact model of the linear solvers, and constrained optimiza- cartilage. Adjusting the active muscle tion. In particular, the intricate geometry forces during motion such that the 4 of cartilage surfaces introduces high laws of mechanics are satisfied on the nonlinearity into the problem of force patient-specific anatomy and the mea- equilibration. The focus is therefore on surements such as skin-marker positions robustness of the solver, as is required (see figure 4) and ground reaction forces for investigating larger patient cohorts. © Julius Wolff Institute

4 Dynamic multi-body models of the lower extremities relate measurable quantities, such as marker positions, to bone displacements and forces acting 5 on the cartilage. 5 Stress distribution in articular cartilage computed from forces provided by a multi-body gait model. Spatially resolved stresses allow the relation of anatomy and lesions to clinical findings. STATISTICAL GAIT AND EXERCISE ANALYSIS The statistical interpretation of exercise data for different patients is complicated and time-consuming due to its high diversity. To address these problems, we are developing and using interactive and automated mathematical methods on time-dependent gait and © Zuse Institute Berlin exercise data (in vivo and in silico) to extract comparable sets of parameters and features.

Zuse Institute Berlin 39 Bigger Data, Better Health INTERPRETING MEDICAL IMAGES

DEEP LEARNING FOR ENDOSCOPIC LANDMARK CONCHA NASALIS MEDIA SURGERIES DETECTION

With advancement in surgical tech- To facilitate a MIS procedure, the therapy niques, modern procedures are driven planning group of ZIB is working on the more towards minimally invasive development of a multimodal framework surgeries (MIS). Nowadays, a surgeon for determining the anatomical position may treat a lesion inside the human of the endoscope by using real-time body by making small incisions or image, sensor, and process information. navigating through natural orifices It would assist the surgeon in guiding the with the assistance of a video camera endoscope through rich visualization and and several thin instruments. Despite alert them about the risk structures in 6 challenges due to limited view of the close proximity. A possible visualization surgical site, rotating images and scheme can be seen in ﬁ gure 6. impaired hand-eye coordination, MIS procedures such as laparoscopic cholecystectomy (for gallbladder removal) and endoscopic endonasal (for skull-base surgery) are becom- ing the de facto surgery standards. A promising way to support MIS procedures is identifying the current position of the endoscope and detecting the surgical tools over the endoscopic video stream. The key beneﬁ t is robotic assistance in image- guided surgery and assistance during navigation.

6 Image-based navigation support for endonasal use case scenario. At the top, possible paths inside the nose are represented by a simplified graph, where a green circle represents the current position of the endoscope and green text represents the name of the corresponding landmark. At the bottom, a reference model shows the overall frontal structure of the nose. It is cut to the corresponding position for better visibility.

40 2016 Annual Report CURRENT STATUS

The utility of ZIBNet has been evaluated for cholecystectomy (gallbladder removal) procedures. In particular, ZIBNet performed second best during the M2CAI16 tool-presence-detection challenge. Currently, ZIBNet is the state- SURGICAL TOOL of-the-art method for tool-presence detection in cholecystectomy DETECTION MULTI-TOOL PROBLEM procedures. A joint tool and landmark-detection method is currently in progress. A novel Convolution Neural Network Since multiple surgical tools can be architecture – named ZIBNet – has been visible in an image, ZIBNet focuses on developed, which is speciﬁcally designed learning the tool-pair patterns (tools that for surgical tool detection tasks. It takes are commonly used in conjunction with endoscopic video frames as an input and other tools) to map each image to its cor- learns to detect the presence of the surgi- responding tool pair (ﬁgure 7). Moreover, cal tools visibleisible in each frame. The main ZIBNetZ employs an enhancement idea is hierarchicalrarchical representation off approacha (called temporal smoothing) the image (similar(similar to the humanhuman to suppress run-time false detections. visual cortexrtex system) where high-level rrepresentationepresentation is IRRIGATOR derived fromom precedingprecedingg low-level rerepresenta-presenta- 7 tion. Duringng llearning,earning, thethe network triesries to cacapturepture CLIPPER these multipleltiple levels of abstractionon to recognizerecognize SCISSORS the appearancerance of thethe SPECIMEN BAG tools. HOOK

BIOPOLAR

7 Chord diagram of second-order tool co-occurrences GRASPER in gallbladder-removal surgery. Common tool pairs are connected by a ribbon. Rare pairs are marked by red and frequent ones by green.

Zuse Institute Berlin 41 Bigger Data, Better Health A PROTOTYPICAL PLATFORM FOR SMART HEALTH MANAGEMENT

Managing your health in the current has already gone wrong (such as catching put into perspective: information about health-care system is not easy. With a bad cold). It also means to recognize our genetic setup, type, and concentra- changing health plans, multiple and when our body needs more sleep, more tions of proteins and metabolites that disconnected practitioners, and the fitness sessions , or a particular vitamin. are floating through our blood, names of wish to focus on prevention, we need drugs we are taking, and tracking infor- All these findings are based on the to take responsibility for our own mation such as step counters or blood data we feed into the analysis system. health. pressure throughout the day. Using this What we make out of this eventually variety of complementary information Smart Health Management means depends on us. If the data show – based will make it possible to understand and to change the perspective about how on a metabolomics analysis – that we are manage our body better. New analysis we look at our health and our body. The lacking some vitamins or other essential algorithms that can deal with the size underlying philosophy is that we can col- nutrients, we can choose to change our and complexity of this (big) data will be lect all possible information (data) about food based on the recommendation. If the able to find hidden signals and patterns. ourselves and use them to adjust our data show that – based on our genes – we They will find whether a correlation lifestyle in a preventive way. It means to should give favor to a particular drug to exists between your low blood pressure, constantly collect, integrate, and analyze treat that nasty headache, it is on us to get sleep habits, and the weather. Based on data that our body are producing. Smart that drug or go with the old one. this extracted knowledge, smart services Health Management is based on the Researchers at ZIB are working on a sys- will be able to suggest what, where, and idea that we can (and want to) see early tem to enable Smart Health Management how things can be improved and what is signals that something might go wrong. for everyone. This platform allows secure the expected effect on us. Most impor- And then take appropriate action. Thus storage of all relevant data and provides tantly: the system will also alert us when – unlike traditional approaches – it is not smart-analysis services. At the very core, it is better to see an actual human doctor. only focused on diagnosing what what all data sources will be integrated and

More than 70% of the data needed Your blood cells regenerate every for health management is in your 120 days, so you can quickly measure blood. It contains information about significant improvements from your health status, optimal disease lifestyle changes such as nutrition treatment, and prevention. intake.

42 2016 Annual Report 8

8 A glimpse into the ZIB Smart Health Management platform. Top: Visualization of the blood proteome and possible diagnoses. Bottom: Integration of step-tracking data over one day.

Geometric Shapes in Computers 3-D shapes are all around us and our visual system is very well trained in recognizing, comparing, and categorizing them. But how can we teach this to a computer? What are good shape representations and how can one do statistics on shapes? And can we exploit prior shape knowledge to solve otherwise insoluble problems?

Shape-Based Data Analysis – Methods and Results

MATHEMATICAL MOTIVATION FRAMEWORK

We live in a world of 3-D objects, each Shapes and shape manifolds. cal shape modelel (SSM). It encodes characterized by its material properties Considering a physical object in 3-D the probabilityy of occurrence and its geometric form. In this article, space, its shape is defined by infinitely of a certain shapeape within a we focus on the latter, the geometric many points that constitute its boundary. given ensemble ooff shapes. shapes, from a computer science per- A boundary surface can be considered as Given a statisticalal shape spective. Traditionally, the main topics a single point in an infinite-dimensional model, one can,, for in computerized processing of geometric “configuration space,” comprising the example, computee a shapes were the construction and visu- coordinates of its points. The shape is mean shape or quan-- alization of shapes. The techniques that independent of rotation, translation, and tify the variabilityy were developed later found their way sometimes also scaling. Mathematically, of shapes (Smalll into, for example, CAD and computer this means that a shape is an equivalence 2012; Dryden et al. 2016;016; Srivastava and animation software. Nowadays, with the class of boundaries with these transfor- Klassen 2016). availability of techniques for capturing mations filtered out; furthermore, the Correspondence problem. One of the 3-D shapes, the focus has shifted to space of all possible shapes, called “shape fundamental problems in shape anal- shape analysis and exploitation of the space,” is a quotient space (Kendall 1989). ysis is to find a meaningful relation (or acquired form knowledge. For example, While the configuration space is a linear mapping) between their points; this is in archaeology, one aims at quantifying space, the shape space is curved and called the “correspondence problem,” see similarities of shapes in order to classify therefore also called “shape manifold.” figure 1. Depending on the application, and stratify archaeological finds, or at Discretization. To process shapes in “correspondence” just means geometric using shape knowledge to restore dam- the computer, they have to be turned into correspondence, defined, for example, by aged and incomplete objects; in biology, finite-dimensional objects. There are geometric similarities; but often it also one is interested in grouping anatomical various ways to do this. In this article we means semantic correspondence. Then shapes to understand relations between focus on representing them as discrete semantic information has to be consid- anatomical and, for instance, genetic objects. Examples are representation by ered, too. The task is thus to determine variations; in medicine, one would finitely many points (point clouds), by all homologous locations on all shapes like to characterize “normal” anatomy a few specific points (landmarks), or by in a given set of shapes (Van Kaick et including “normal” variation to identify discrete meshes that connect points to al. 2011). At ZIB, we have developed and classify pathologies or malforma- form polygons and polyhedra. various approaches to establish highly tions, or to utilize the shape knowledge in accurate and dense correspondences for surgical reconstructions; in autonomous Statistical shape models. In many anatomical structures. This ranges from driving, scene recognition from incom- applications, sets of shapes, like collec- techniques that integrate fundamental plete information becomes possible by tions of clay jugs found in an archaeo- expert knowledge using interactive utilizing prior information about shapes logical field, must be analyzed. Then annotation techniques (Lamecker and appearances of potentially occur- statistics comes into play. The sets of 2008) up to fully-automated techniques ring objects. These examples indicate shapes can be considered as arising (Günther et al. 2013; Grewe and Zachow that algorithmic techniques for empir- from an underlying probability density 2016). Based on this methodology, we ical analysis of shapes and for utilizing function that is defined on some shape have built a software pipeline that allows shape knowledge have great potential in manifolds. An estimate of this shape us to create SSMs for a wide range of a broad range of applications, including manifold and the probability density applications. humanities, natural sciences, medicine, function, determined from empirically and engineering. observed occurrences, is called statisti-

46 2016 Annual Report 1 Point correspondence between the surfaces of two snake bones. 1 Some corresponding point pairs are shown explicitly by lines. The rest is shown implicitly by color.

2 Mean shapes (opaque) and first principal geodesic curve (transparent at ±0.75 and ±1.5 standard deviations) for a data set of 100 human body shapes computed in a curved (left) and in a euclidean shape space (right).

Commonly, manifold-valued generaliza- tions are obtained by replacing straight lines with geodesic curves in the problem formulation, for example, yielding the principal geodesic analysis (see figure 2). 2 Similarity of shapes. Another fundamental problem is to measure distances between shapes in order to quantify their dissimilarity; this can be considered as measuring distances in the shape manifold, either “intrinsically” along geodesics in the curved manifold, or “extrinsically” along straight lines in the ambient euclidean space in which the shape space is embedded. From an application point of view, however, the similarity and dissimilarity of shapes often is problem-specific; a suitable distance metric thus has to be induced from problem-specific conditions.

Geometric variability and mean. Despite the many methods for capturing the geometric variability in a population, principal component analysis (PCA) and its manifold extensions remain a workhorse for the construction of statistical shape models. PCA, a traditional multivariate statistical tool, determines a hierarchy of major modes explaining the main trends of data variation. As the shape spaces frequently have nontrivial curvature, the statistical analysis needs to account for the nonlinearities in the model.

Zuse Institute Berlin 47 Shape-Based Data Analysis – Methods and Results

GEOMETRY RECONSTRUCTION UTILIZING STATISTICAL SHAPE MODELS

Shapes from images. Often, geometric structures is to segment the respective et al. 2013, 2015) (see figure 3). The local shapes are not captured by geometric anatomy from medical image data and uncertainty quantification that is possi- measurements but are instead recon- to represent the shapes by surfaces that ble due to the shape statistics encoded in structed from image data. Acquiring separate differently labeled volumes. SSMs does not only give an indication of highly accurate image data in, for Although the general shape of an organ is the reconstruction accuracy, but allows example medicine, biology, or archeol- known, individual organs differ in shape, one to utilize methods of experimental ogy, is often tedious, costly, destructive, either within a normal range of variation design to optimize the imaging setup, or requires patients to be exposed to a or due to a pathology. To fully automate defined by number, direction, and dose of significant amount of radiation. To an often labor-intensive process of tissue radiographs taken, such that a requested minimize such negative effects during delineation, advanced algorithms employ geometrical accuracy is achieved with a data acquisition, highly sensitive com- SSMs as geometric priors to identify the minimal radiation dose. Since the recon- puter-aided approaches are required that organ of interest within the image data structed shapes are restricted to the low extract as much valuable information and to geometrically fit the model to dimensional space covered by the SSM, as possible from the measured data. the individual data (for an overview see pathological cases cannot necessarily Challenges for automatic and reliable Lamecker and Zachow 2016). be addressed with this method; this data processing arise from (a) the high requires further research. 3-D anatomy from 2-D radiographs. variability in shape and appearance in While MRI is expensive and less suited Facial shapes and expressions. real-world data, (b) measurement noise for imaging bones, CT incurs a signif- For the planning of facial surgery and due to fast and low-dose acquisition icant radiation dose. Using SSMs as the development of tools for diagnosis techniques, and (c) the ill-posed recon- strong prior allows the shape of skeletal and treatment of psychopathologies, it struction of 3-D structures from 2-D structures to be reconstructed from few is of interest to extract the significant measurements. planar radiographs with low doses (Ehlke morphological patterns in facial shape Prior shape knowledge. At ZIB, we develop algorithms that allow the reconstruction of highly accurate geometric representations of medical, biological, and artificial structures as surfaces and volumes from image data. To address the aforementioned challenges, one key strategy is to incorporate prior knowledge on statistical variation of shape and appearance of the measured structures, utilizing statistical shape models and Bayesian inference (Bernard et al. 2017). 3 In the following, we describe three cases of anatomy reconstruction, where algorithmic utilization of shape knowledge has been instrumental:

Medical image segmentation. One way to create digital shapes of anatomical

48 2016 Annual Report 4

ANALYSIS OF SHAPE ENSEMBLES

and appearance from large-scale 3-D Shape descriptors. Given a SSM, mem- 3 Employing statistical shape models face databases. SSMs not only provide a bers of a set of shapes can be uniquely allows individual 3-D geometry to convenient way to analyze the statistical encoded within the basis of principal be estimated from few planar X-ray variation of geometric and photometric modes of variation. This yields low-di- images with low radiation exposure. features; they also allow one to reliably mensional representations that can serve acquire and process large amounts of to derive statistical shape descriptors. 4 Matching of a facial surface S to measured data to capture the great vari- These shape descriptors can be utilized the reference R: parametrizations ␾ ␾ ety in human faces fully automatically. to perform many shape-analysis tasks, S and R are computed and 3-D facial geometry can thus be recon- such as investigating the similarity of photometric as well as geometric structed with the highest resolution individual shapes. Shape descriptors features are mapped to the without the need for manual interven- not only allow one to compute a distance plane. The dense correspondence tion. Accurate dense correspondence between individual shapes, but they also mapping ⌿␾ ␾ accurately S R even under high deformations due to allow the changes to be categorized into registers photographic and facial expressions can be computed up to different modes of variation that can geometric features from S and R. the level of skin pores by simultaneously be studied separately, for example, via exploiting geometric and photometric morphing with respect to selected modes 5 Variation of the shapes of a PDM features (see figure 4). This permits of variation (see figure 5). Thus, shape generated from 225 input snake the capture of even the ﬁnest details of descriptors enable a detailed study of bones along the two first PCA morphological patterns in human faces differences between individual shapes. modes. and enables statistical examination of In the following, we shortly describe two the complex deformations of facial tissue examples of application. on all levels of detail (Grewe and Zachow 2016). PCA MODE 2

5 PCAP MODE 1

Zuse Institute Berlin 49 Shape-Based Data Analysis – Methods and Results

HEALTHY DESEASED

6 Low-dimensional visualization (Sammon projection) of our Riemannian statistical shape descriptor revealing a high degree of separation between healthy and diseased distal femora.

Evolutionary biology. In collaboration Musculoskeletal medicine. In A general conclusion from many studies with the Museum für Naturkunde in order to analyze characteristic shape performed at ZIB is that the result of Berlin, more than 300 data sets of a single changes incident to radiographic knee any shape analysis might be severely head bone of 18 species of the snake genus osteoarthritis, data of a longitudinal influenced by the type of statistical Eirenis were investigated using shape study (performed by the Osteoarthritis shape model being used. This means, one clustering. In particular, hierarchical Initiative of the NIH, USA) have been has to take care that the model used is clustering was applied using the euclid- analyzed. For these, data shape models mathematically appropriate and, equally ean distance of the shape descriptors and shape descriptors have been derived as important, that the metric, which resulting from a point distribution model. (see ﬁgure 6 for a visualization). Utilizing typically is application-specific, has to The clusterings yielded trees comparable principal geodesic analysis within a be chosen with the particular analysis to phylogenetic ones, though clear differ- novel Riemannian framework based on question in mind. ences were observable indicating that the Lie groups of differential coordinates, shape morphologies are not only influ- we achieved a highly accurate separation enced by genetic but also by functional between subpopulations of healthy and relations (Baum et al. 2014). diseased patients (von Tycowicz et al. 2016).

50 2016 Annual Report 8

8 The analysis of the evolution of shapes can also be utilized for the stratification of archaeological finds, for example, in application to classification of ancient CURRENT AND pottery and different aspects of shape evolution throughout history. FUTURE WORK

Major algorithmic advances in shape Problem-adapted descriptors and trajectories. Furthermore, analysis of processing have been achieved over the metrics. Often one is confronted with anatomical changes in longitudinal last few years, allowing us to address the task to quantify certain subtle (local- studies will help to characterize pro- many applications. Yet, a number of ized or extended) shape variations while gression of diseases. Time-dependent problems still need to be solved. ignoring others. This is related to the shape analyzes will become even more particularly important aspect of defin- interesting when covarying factors Varying topology. Some applications ing suitable problem-adapted metrics are considered. Another example is the require analyzing sets of shapes with in shape manifolds. Suitable notions of analysis of the evolution of shapes, for different topologies – which are either distance (beyond general concepts like example in pottery in the ancient world intrinsic or emerge as artifacts due to elastic deformation energies or norms of (see figure 8). Currently, we are in close noise and outliers in the acquisition flow fields) are hard to conceive. Here, we discussion with archaeologists about process. expect advances from other techniques analysis tasks that could be supported Uncertainties. Another requirement is for nonlinear dimensionality reduction by mathematical shape analysis, such to consider uncertainties in a coherent and from interactively controlled metric as stratification of archaeological finds. way – starting with measurement errors learning. In general, we expect shape-trajectory during acquisition, to numerical errors analysis to bear great potential for all Shape trajectories. An important during shape reconstruction and analy- areas of digital humanities that deal with topic is parameter-dependent shapes, sis, up to statistical inference. 3-D objects and their changes throughout especially 1-D shape trajectories, since history. Multiscale representation. A largely shapes varying over time occur in many unaddressed topic is the multiscale rep- applications. Think, for example, of a Computational efficiency. A last resentation of shapes and corresponding beating heart, or transitions between but important topic is the reduction techniques of analysis. One such example facial expressions (see figure 7). Working of computational effort. The handling is the analysis of a collection of shapes with complete shape trajectories instead of nonlinearities of shape manifolds representing the outer surfaces of com- of single snapshots will improve the (“Riemannian computing”) is compu- plex insects in high detail. segmentation of 4-D medical images tationally expensive. Therefore, it is with low resolution, or enable certain important to transfer computation-sav- functional defects to be identified by ing concepts of numerical mathematics, a classification of the deformation such as adaptive discretizations and multilevel solvers, to shape computing.

7 As high-resolution facial geometries can only be 7 acquired at high latencies, facial trajectories have to be reconstructed from noisy and low-resolution video data in a manner consistent with physical and dynamical constraints.

Zuse Institute Berlin 51 Dr. Janina Zittel | [email protected] | +49-30-84185-146 RESEARCH CAMPUS MODAL: REPORT FROM THE GASLAB © Shutterstock Research Campus Modal: Report from the GasLab DEREGULATION OF THE ENERGY MARKET – NEW CHALLENGES FOR NATURAL GAS TRANSMISSION OPERATORS

About 20% of the German (and nies to ensure discrimination-free access traders. Since then, they are independent European) energy demand is met by to the transport network for all traders and need to plan under the uncertainty natural gas. Until 2005, gas transport (Geißler et al. 2015). This changed the regarding the gas-ﬂow situations result- and supply in Europe was provided operation and business model of the gas ing from trading. This leads to several by a handful of companies, owning transmission system operators (TSOs) challenges which require a complex deci- and operating the natural gas trans- that – due to their high investment cost sion support system, including technical mission system to do so. To establish for natural gas pipeline networks – are operation support and decision support a European gas market, in 2005, the mostly a natural monopoly. Before this, on various nontechnical intervention European Union legislated that gas TSOs were part of an integrated organi- options. trading and transport had to be done by zation and could plan the network oper- mutual completely independent compa- ation and expansion together with the

NATURAL-GAS Denmark Baltic Sea TRANSMISSION- North Sea

SYSTEM (edited) ©: Open Grid Europe Netherlands Poland The Research Campus Modal industry partner Open Grid Europe GmbH (OGE), located in Essen, Germany, operates about 12,000 kilometers of natural gas pipelines in Germany, a network of pipes roughly as long as the German autobahn network. About two thirds of the natural gas transported in Belgium the network is supplied by or consumed Czech Republic in other countries. Therefore, securing Luxemburg the transport requirements in Germany is key for the European cross-border natural-gas supply. France

Austria

Switzerland Property of Open Grid Europe

Operated with Open Grid Europe Share

Other Transmission-System Operators

54 2016 Annual Report THE VIRTUAL TRADING POINT

In a simpliﬁed view, the EU regulations guaranteeing the right to supply/extract stipulate that the so-called entry/exit gas to/from the network with different model with a virtual trading point is constraints. The preferred capacity prod- the basis of the capacity market. The uct sold by the TSO is freely allocable, virtual trading point is an abstraction which basically means that the trans- of the physical gas network, decoupling port customer can trade with any other the direct local connection between a market participant up to the specified customer demanding natural gas and a amount of gas. After trading, the amount seller supplying natural gas. Both parties, of capacity used is communicated to the the buyer and the seller, are regarded as TSO (the nomination). The requirement transport customers of the TSOs that for the TSO is to be able to transport operate the network represented by the this gas. In contrast, employing another virtual trading point. These transport product, the interruptible capacity, customers need to buy capacity from the transport customer buys the right the transmission system operator to for transport, which, however, can be access the virtual trading point, that interrupted by the TSO if the network is, suppliers need to buy entry capacity, situation does not allow the respective demand customers need to buy exit transport. capacity. Thus, a seller can supply gas at A TSO may only sell capacity rights for any physical point of the network that which it can guarantee that each “likely a buyer demands at any other physical and realistic” (Gas, 2010, section 9) gas point of the network. ﬂow complying with the capacity rights There are different capacity products booked by all transport customers can available to the transport customers, technically be realized.

Zuse Institute Berlin 55 Research Campus Modal: Report from the GasLab ©: ZIB

1 The GasLab team – the GasLab team includes researchers at different career stages led by Prof. Thorsten Koch, experts from OGE, THE MODAL GASLAB and software developers at Soptim. FROM SCIENCE TO A DECISION SUPPORT SYSTEM

The MODAL GasLab aims at the The GasLab does the whole chain from The MODAL AG acts as a bridge realization of fundamentally new fundamental research to implementa- between industrial needs, such as possibilities for transmission net- tion. Therefore, in the framework of the maintainability and stability of work operations beyond current Research Campus MODAL, the GasLab software solutions, and priorities of feasibilities. The software developed involves several cooperations. First of all, ZIB, such as basic research on funda- within the GasLab addresses the main the cooperation with the industrial part- mental problems and scientific pub- challenges of natural-gas dispatching ner Open Grid Europe (OGE) forms the lication of the corresponding results. by providing foresighted decision basis for all research and development in Likewise, the MODAL AG helps to support. To achieve this ambitious the GasLab. OGE experts provide indus- separate scientific and economic goal, fundamental research is needed. try-specific knowledge on the technical risks, creating a win–win situation for The research focuses on: and nontechnical operation measures both ZIB and its industrial partners. and how these measures are used for Ċ Mathematical models integrating The incorporation as one lab of the efficient operations by the dispatchers. the technical network description, Research Campus MODAL is fundamen- Insights on specific knowledge of tech- such as the transient behavior of the tal for the success of the project due to the nical elements, such as compressor sta- gas transmission network and its synergies with the other labs creating tions, are fundamental for the definition technical elements, and nontechnical methodological advances. For example, of mathematical models. Furthermore, operation measures, such as capacity joint research on machine learning OGE provides real-world data used as limitations. methods together with the MedLab research data sets, and test data for new stipulates advances in prediction of gas Ċ Machine learning solutions to predict methods insures the applicability of any flows and medical diagnostics alike and gas supply and demand in the opera- developments of the MODAL GasLab. In can be generalized for multiple purposes. tion horizon of up to two days. addition, OGE subcontracted the soft- Likewise, joint work with the SynLab on ware-development company Soptim to Ċ Solution algorithms for the respective solving the optimization problems of the implement an industry product including optimization problems for efficient GasLab creates advances on solvers. On interfaces to data source systems of real-time decision support. one hand, the GasLab provides challeng- OGE, a graphical user interface for the ing problems, driving progress in math- dispatchers, and a test suite to support ematical programming solvers. On the the development of the MODAL GasLab. other hand, advanced solver technologies reflect back on the GasLab.

56 2016 Annual Report SOFTWARE SOLUTIONS TO SUPPORT THE BALANCING TOOL DECISION-MAKING (BILANZTOOL)

The changing market environment The TSO may assume that over a full Commercially, this is organized by raises several new challenges for the day the result of market operation is grouping network nodes into “balancing TSOs from everyday operations up balanced in the sense that the amount zones” (Bilanzkreise) for which in- and to strategic network planning. Three nominated at entries is a sum equal to outflows are supposed to be balanced. key challenges for operations are the amount nominated at exits. However, An important task of the operator of a tackled in the MODAL GasLab, pro- temporal and spatial lacks may appear. balancing zone is to proactively manage viding foresighted decision support For example, gas may be extracted in the the transport of the zone in order to avoid on: morning in the south and injected in the the demand for balancing energy. One evening in the north. In order to satisfy possibility is to exchange surpluses with Ċ Efficient technical operations of the the transport requirements, the TSO is other balancing zones. Any additional gas-transport network. first obliged to exploit all technical oper- and unavoidable balancing energy Ċ Ordering additional gas to overcome ation measures. Only, when this does not demand is bought or sold at the entries temporal imbalances in the network. suffice, several nontechnical measures and exits of the virtual trading point (see can be taken. box) in a nondiscriminatory manner. Ċ Employing a new capacity product for secure supply to power stations of One of the main problems arises from The MODAL GasLab supports these systemic importance. the varying distribution of supply and decisions by solving several mathemat- demand in space and time. Before the ical problems: The aim is to integrate technical and deregulation of the European gas market, nontechnical operation measures into Ċ Forecasting temporal and spatial such problems were solved by storage one tool for efficient network operations. imbalances of in- and outflows of the units controlled by the gas company. However, solutions for the nontechnical network. Nowadays, to secure the transport operation measures may be developed requirements in these situations, there Ċ Computing gas flows in the network separately in a first step. Later on, they are two options for TSOs: to use the line to tracking down temporal and spatial will be integrated in one holistic tool. buffer or to order balancing energy. line-buffer capacities.

Ċ Line buffer: To a limited extent, imbal- Ċ Determining the unavoidable amount ances can be buffered using the gas of balancing energy needed. that is already in the network.

Ċ Balancing energy (Regelenergie): When line buffer is not sufficient to secure network operations, the TSO can buy or sell gas to balance physical differences between supply and demand.

Zuse Institute Berlin 57 Research Campus Modal: Report from the GasLab

1 Pressure-induced flow bound of 2 units 3 2 3

Ausgleichsentry

2 The sketch shows a simplified network situation in which the demand (3 units at the power plant) and supply (three entries with 1 unit each) is globally balanced. However, due to construction at one pipe and a capacity bound of 2 THE KWP TOOL units at an alternative pipe, there is no feasible transport solution to the One particular set of exits of the network the gas supplier at the Ausgleichsentry power plant (left image). Once the corresponds to gas-fired power stations. is obliged to reserve sufficient capaci- power plant is restricted to obtain Some of these stations are system-rele- ties. When several of these restrictions its supply from the Ausgleichsentry, vant for the electricity system, especially would resolve the bottleneck situation, there is a feasible transport solution due to the increasing amount of renew- the actions of the TSO have to be nondis- (right above). ables in electricity generation related to criminatory among the available power the German “Energiewende”. Gas-fired plants. power stations have high use of gas The GasLab aims at the development of during their hours of operation, which is a tool – the KWP Tool – that calculates a highly variable depending on other elec- foresighted, nondiscriminatory decision tricity-generation capacities. In general, suggestion for the TSO for which power the supply of gas to the stations may not stations the restriction needs to be exe- be met with freely allocable capacities. cuted for the upcoming gas day. One solution to this problem would be to build new pipelines and compressor To tackle this challenge, several mathe- facilities to ensure supply to this critical matical problems need to be solved: infrastructure. Ċ Predicting the gas flow at the entries However, to avoid these infrastructure and exits of the network for planning investments, the TSOs will soon have horizon. a new regulatory option to ensure the Ċ Identifying bottleneck situations from secure operation, related to the so-called the spatial and temporal distribution “Kraftwerksprodukt” (KWP). This of the gas in the network. option may be executed only in bottleneck situations. In these situations, the TSO Ċ Analyzing whether or not a subset of can force the customer corresponding to power plants exists, whose restrictions the power station to buy its gas from one admit a feasible network flows. specific entry (Ausgleichsentry) instead Ċ Finding a smallest subset of power of any entry point of the virtual trading plants, w.r.t. the cardinality, whose point. The appropriate Ausgleichsentry is restrictions admit a feasible network specific for each power station and spec- flow. ified in the capacity contract. Likewise,

58 2016 Annual Report THE NAVI

Gas network operation is a challenging compressor station, lead to continuous For the decision support system Navi task. In practice, dispatchers operate the operation options, while some of them, that reflects the technical details, the network and are responsible for differ- for example opening or closing a valve, hierarchical nature of nontechnical ent parts of it. Dispatchers carry great lead to discrete operation options. interventions, and the additional objec- responsibility, as many technical, legal, Additionally, rules apply to the order tive functions, basic research on novel and regulatory constraints have to be of interventions. First, the dispatcher mathematical models and solutions fulﬁlled. Nowadays, the job requires long should use all technical interventions, approaches has to be the first step. To experience assisted by limited software then, if necessary, balancing energy. If face these challenges, several different solutions mainly based on simulations. this still is not sufficient to secure opera- approaches should be considered and tions, capacity limitations may be applied evaluated. The research involves, but is To support the dispatchers at the TSO as a last resort, ranging from KWP to not limited to, the following: during everyday operations, a com- cutting supply or demand on certain prehensive decision support system is Ċ Predicting future gas flow for fore- entries or exits. needed. This tool is supposed to assist sighted operations. the dispatchers by addressing nontech- For efficient operations, the dispatcher Ċ Finding a set of technical and nontechnical measures and technical decisions has to meet additional objectives. Just nical measures sufficient to fulﬁll the providing the recommendations for like the driver of a car has secondary transport requirements respecting the changes in the network-configuration. goals such as the fastest or shortest route, regulatory terms. On one hand, it is expected to support efficient transmission-system operation everyday operations; on the other hand, involves several goals, such as: Ċ Computing optimal measures that it is a useful tool to decrease the training achieve the different operation Ċ Operation as far away from technical period of new dispatchers. objectives. bounds as possible. In the MODAL GasLab, the dispatcher Ċ Robustness to late changes in the Navi – like a navigation system for gas transport requirements. network operators – is being developed. Mathematical models serve as the basis Ċ Minimal fuel use of compressor for optimization solutions, describing stations. the technical elements of the network Ċ Restricting the number of and their control options. Some of them, interventions. for example the outgoing pressure of a

Zuse Institute Berlin 59 Research Campus Modal: Report from the GasLab

THE TIP OF THE ICEBERG – REAL- WORLD DATA FOR RESEARCH AND DEVELOPMENT

Research in the MODAL Labs relies on specifications. Moreover, due to aging real-world data. In the context of the and modernizations, the technical spec- short-term operations of gas networks, iﬁcation is known at the time of delivery data from very different areas have to be of the hardware, but changes over the considered and combined. This includes: years. Taking all these possibilities into account in new data formats is a major Ċ technical network speciﬁcations. challenge that is iteratively approached Ċ real-time measurements of physical in the MODAL GasLab. gas properties. One major challenge was the integration Ċ real-time network-configuration of data from several sources. These data information. sources were initially deployed for different purposes in the software landscape of Ċ various nontechnical intervention the industrial partner, and therefore use options different time scales and abstractions of Gas networks are complex structures the network. Within the GasLab coherent consisting mainly of pipelines but con- structures for both, stationary data, such taining many special devices such as as the network topology, and transient compressor stations. Physics of pipelines data, such as the simulated or measured with their relevant parameters is under- current state of gas properties, have stood in theory. In practice, however, been developed including transforma- these parameters can often only be esti- tion rules for the different abstractions mated or derived from measurements, of the network. A new comprehensive and vary over time. Compressor stations data model is the result. A great effort are typically the most complex facilities is needed to integrate this data model dispatchers have to handle. They often into the legacy systems at OGE to get not only comprise several machines, but real-world data for the project. Powerful also extensive piping and other special interfaces to the dispatching system network equipment that is sometimes of OGE were implemented to generate unique in the network. These stations research data sets, test new methods on have been built and extended over time, live data, and later productive use of our and thus the equipment within one sta- tools. tion often has very different technical

60 2016 Annual Report EVALUATING THE STATE OF THE NETWORK 3 Graphical representation of a sample solution for a part of the OGE network in nothern Germany. Thicker lines represent higher flow; colors illustrate pressure, with warmer colors representing higher pressure. © Shutterstock

3 Compressor station

Zuse Institute Berlin 61 Research Campus Modal: Report from the GasLab

RESEARCH ADVANCES IN FORECASTING GAS FLOW

The need to forecast gas in- and outflows applies. All of these different characteris- Taverna (University of Milan, Italy) is common to all tasks tackled in the tics and constraints have to be taken into applied several outlier detection tech- MODAL GasLab. Expert knowledge and account in a forecast model. niques. Dr. Milena Petkovic (University historical data are the basis for such a of Novi Sad, Serbia) used support vector Developing the forecast model is in itself forecast system. Both support the anal- machines to predict days where certain a demanding research project, which we ysis of the complex market structure, entries and exits are not used at all, a have tackled with an international group which includes different behavior of the question relevant for, for example, big of researchers. Together we investigated various types of customers. Only about power plants or cross-border connection the application of several methods from a third of the gas in the network is con- points. Currently, we are working on a neural networks over time series analysis sumed in Germany for different purposes joint publication comparing the results to optimization-based multi-regression and with very different demand profiles. of the different methods and useful and combinations of these methods to Domestic heating depends mainly on combinations to increase the quality come up with the best forecast model. For outside temperatures. The consumption of gas-flow forecasts. The optimiza- example, Prof. Ying Chen (University of of industrial complexes varies from high tion-based multi-regression (see box) is Singapore, Singapore) visited ZIB for a during working hours to low on weekends implemented in software integrated at month to apply an adaptive functional or public holidays. Gas-fired power plants OGE to test its performance on live data. autoregressive model, previously used for depend on the availability of other energy Further development of this method and forecasting electricity prices (Chen and sources, in particular renewables. The the integration of additional methods Li, 2015), to the gas flow forecast prob- remaining two thirds of the gas are trans- will be researched and implemented in lem. Several scientists visited ZIB for ported through the network, for example the remainder of the project. short-term scientific missions supported from Eastern Europe to France. These by the COST Action TD1207. Dr. Selini The challenges, described in this fea- transport customers are obliged to “nom- Hadjidimitriu (University of Modena and ture article, present only some of the inate,” that is declare the capacity needs Reggio Emilia, ICOOR, Italy) employed aspects tackled in the MODAL GasLab beforehand, but might “renominate,” that neural networks using external vari- on our way toward a comprehensive is change the declared amount, during ables such as European temperature decision support system for gas network the course of the day. For nominations data to predict gas demands. Dr. Andrea operations. and renominations, a complex set of rules

62 2016 Annual Report OPTIMIZATION-BASED MULTI-REGRESSION

This prognosis method is a regression approach with a rolling horizon. The prediction model is trained using a comparably short historical time period, which moves when we are moving forward in time.

In the training phase, we calculate a function which describes the given historical time series best.

This is done by optimal choice of coefficients (weights) of a function with predeﬁned terms (features), which lead to the smallest error over the training horizon.

For instance, the predefined terms could be the value of a former hour (such as 24 hours ago), the mean of the values of a former day (such as yesterday), or the ratio between different values (such as the ratio between the first value of yesterday and the first value of the day before yesterday).

Then the future values of the time series are predicted using the calculated function.

However, when using too many terms with a small training data set, the function can become overﬁtted, that is, it perfectly represents the past but is very poor in predicting the future. Since it is difficult to choose the terms a priori, we plan to extend the method by adding more terms and forcing the training algorithm to choose the coefficients in a way such that, at most, k-coefficients are nonzero. Although this makes the training problem more difficult, we hope that it leads to considerably better results.

The optimization-based multi-regression can be used for predicting all kinds of time series.

A special case for the GasLab are time series, which are somehow connected (e.g. time series of demands and supplies at several entries and exits of the gas network). Here, it is promising to also take information of time series of other entries/exits (i.e. boundary nodes of the network) into account when forecasting the time series of a speciﬁc node. ©: ZIB

Example of gas flow forecasts using optimization-based multi-regression.

Schematic plot (without scale) depicting the forecast (orange) for a specific exit of the gas network supplying an industrial customer compared to measured values (green); hourly forecasts started at the beginning of each day (vertical grey lines); weekends displayed with gray background shade.

Zuse Institute Berlin 63 Prof. Dr. Ralf Borndörfer | [email protected] | +49-30-84185-243

A Man–Machine Search for the Absolute Evidence Jonad Pulaj © BenFrantzDale, commonswiki © BenFrantzDale, COMPUTING THE TRUTH Computing the Truth

Using computers to prove results it is very likely that only a minority has of results building upon previous results. in pure mathematics is not a new carefully read the whole proof (many may The Four-Color Theorem is the first endeavor, although it still remains have read portions of it). Indeed, we may major result that was proved via a controversial amongst some math- safely assume that situations like this are computer-assisted method in the 1970s ematicians. After all, how are we to normal, as progress measured in the tra- by Appel and Haken. Its statement is trust results if they cannot be directly ditional mathematical way often consists checked by hand? This question quickly turns into a philosophical one and has little to do with mathematics itself. Suppose that you are reading a proof of a statement, that is, a series of connected logical arguments that begin with some assumptions and end with the desired result. Is it possible to know with absolute certainty that no mistakes have been made? Arguably, for proofs that are quite complicated and go on for many pages, the answer is simply no. However, it is often enough to be “sufficiently convinced.” For example, the theorem which classifies finite simple groups is over ten-thousand pages long. Reading the entire proof would take a long time, and a great deal of optimism is necessary to assume that absolutely no mistake could be made in the writing, reading, and understanding of each crux in the proof. Nearly all mathematicians accept the result as true, although rationally speaking

66 2016 Annual Report deceptively simple: any map in the plane while numerous attempts date back to the such as Coq are partly based on type the- can by colored with four colors such that 1850s. Indeed, in 1879, Kempe published ory invented by Russell at the beginning the regions sharing a common boundary his first “proof” of the four-color conjec- of the 20th century in order to deal with (other than a single point) do not share ture. As a result, the English lawyer and the logical paradoxes of naive set theory the same color. Nevertheless, a “simple” mathematician was elected a fellow of as a foundation for mathematics. Thus, proof of the statement is still missing, the Royal Society and even knighted at interactive theorem provers serve as a a later date. However, in 1890 Heawood rigorous framework for the formalization showed that Kempe's arguments con- of the discipline. Of course, in practical tained a nontrivial flaw, while at the terms, such rigor comes with significant same time proving that five colors are costs that may alienate the average sufficient for every map. Throughout the mathematician from this approach in decades, many other mathematicians the first place. Formalizing results in worked on the four-color conjecture, interactive theorem provers is incredibly showing that the problem could be time consuming (note the nearly 30-year reduced to some finite, albeit large, num- gap between the result of Gonthier and ber of cases. Finally, this culminated in that of Haken and Appel) since the imple- the computer-assisted proof of Haken mentations in functional programming and Appel, which took over 1,200 hours can be particularly daunting for those not of machine time. The proof was initially familiar with this paradigm. met with skepticism, but was eventually Still, for the new generations of math- (mostly) accepted by the mathematical ematicians, Coq and Isabelle/HOL are community. Several other implemen- considered (somehow) well established tations and computer-assisted verifi- but rather tedious tools. Interactive cations have reaffirmed the Four-Color theorem provers have shown consid- Theorem. Along these lines, the final erable flexibility, ranging from the verification in 2005 by Gonthier [5], via formalization of Kepler's conjecture to the interactive theorem prover Coq, put results beyond the realm of traditional (nearly) all mathematical fears to rest. mathematics. Indeed, the formalization Indeed, in some sense, interactive theo- of the ontological proof of God's existence rem provers such as Coq or Isabelle/HOL [1], which can be traced back to Leibniz, represent the highest levels of trusted is a striking example of computational code for computer-assisted proofs. This metaphysics and the far reach of inter- is because interactive theorem provers active theorem proving.

Zuse Institute Berlin 67 Computing the Truth

68 2016 Annual Report and operations research, its use in pure mathematics is very limited. Given the high burden of proof in establishing theoretical results, the majority of integer programming solvers are simply not good enough without further safeguards or verification routines. This is because most solvers suffer from a dual curse: the possibility of ill-conditioned matrices and floating-point arithmetic can lead to wrong results due to rounding errors, in addition to the ever-present possibility of a programming error. This situation has led to the extensive In theory, the methodology of integer constraints in the considered decision use of SAT solvers by researchers in programming can also be used to (com- variables. The objective function is not pertinent areas of pure mathematics. putationally) prove suitable results of important, since we are only interested SAT solvers provide an even higher level interest. This is due to the flexibility in a feasible solution. By identifying of trust by producing certificate formats of integer programming as a modeling conditions that need to be shown feasi- that can be checked by interactive the- paradigm. Consider, for example, a bly or infeasibly in a theoretical problem orem provers. Furthermore, in the past challenging sudoku puzzle. Given the of interest, we may consider similar few years, all major SAT competitions nine-by-nine square grid with some fixed “abstract” sudoku puzzles. As we will see require participating solvers to produce values, the key idea is to consider a cor- later on, such conditions are sometimes a certificate of infeasibility which can responding cubic nine-by-nine-by-nine easy to identify but difficult to compute, be checked with DRAT-trim [8]. Perhaps array of binary values. We can visualize as is the case with most Ramsey theo- the most spectacular and controversial this as nine square grids stacked on top of retic numbers. Yet, in other contexts, use of SAT solvers is the recent proof (in each other. The top grid will be assigned such conditions are highly nontrivial to the negative) of the boolean Pythagorean a one whenever the solution has a one in identify in the first place, as is the case triples [4]. The problem asks whether it the corresponding square. The grid right with questions related to Frankl's conjec- is possible to color the positive integers below it will be assigned a one whenever ture. Nevertheless, almost any situation red and blue such that no three integers the solution has a two in the correspond- of interest can be modeled by an integer a, b, and c that satisfy the Pythagorean ing square, and so on. Thus, we arrive at program. theorem are of the same color. With a 0-1 decision variables that correspond Although the use of computational proof size of about 200 TB, this result to each empty square in each of the integer programming is well estab- must keep many a pen-and-paper tradi- nine grids, and it is straightforward to lished in applied discrete mathematics tionalist up at night! encode all the rules of the game as linear

Zuse Institute Berlin 69 Computing the Truth

In the optimization department of Zuse picture above. Suppose we are given the boxes are the colors. For example, Institute Berlin, researchers have long the nine boxes above and want to know given the colors red, blue, green, and been interested in safe numerical com- how many pigeons it takes before a box yellow, we may color 1 as red, 2 as blue, 3 putations. This led to the development of contains more than one pigeon? Clearly, as green, and finally 4 as yellow. Thus, it an exact rational integer programming the answer is ten. This illustrates a well- is clear that any coloring of the first five solver [3] that avoids the trouble that known mathematical notion, namely the integers with four colors cannot avoid a comes with floating-point arithmetic. pigeonhole principle. As simple as this monochromatic arithmetic progression Yet, the possibility of a programming principle may sound, its applications are of two integers. Unfortunately, these error remains. In this regard, the recent far reaching and far from trivial. were already all the nontrivial cases. development of VIPR [2], a tool which Only seven other van der Waerden num- Ramsey theory may be thought of as a verifies the branch and bound tree bers are known, and as other Ramsey generalization of the pigeonhole prin- produced by an integer programming theoretic numbers they are notoriously ciple, and van der Waerden numbers solver, provides solver-independent ver- difficult to compute. Here is what the late are classical objects in Ramsey theory. ification of integer programming results Paul Erdös had to say about the compu- W(r,k) is the smallest integer M such that and needed redundancy for theoretical tation of Ramsey numbers, which also any coloring of {1,2,…,M} with r colors, results. All these tools are crucial in the applies to van der Waerden numbers: contains a monochromatic arithmetic results that we feature in the next few progression of k integers. It is easy to “Suppose aliens invade the earth and paragraphs. see that W(1,k) is simply k, since the threaten to obliterate it in a year's So what kind of problems in pure math- only possible coloring is the trivial one. time unless human beings can find the ematics do researchers at ZIB work on? Furthermore, W(r,2) = r+1, since if we Ramsey number for red five and blue Currently, the area of focus is extremal are to avoid a monochromatic arithmetic five. We could marshal the world's best combinatorics. Roughly speaking, progression of two integers, we must minds and fastest computers, and within extremal combinatorics is concerned color each integer with only one of the a year, we could probably calculate the with how large or small a collection of given colors. Thus, given a sequence of value. If the aliens demanded the Ramsey objects can be before a certain pattern of r+1 integers, two of them must have the number for red six and blue six, however, interest appears. The simplest example same color by the pigeonhole principle, we would have no choice but to launch a of this notion can be explained by the where the pigeons are the integers and preemptive attack.”

70 2016 Annual Report © Vimala D/O Ganavalu

Using an integer programming approach, researchers at ZIB are able to give the following, previously unknown result: W(7,3) >= 258 (Pulaj 2015). The picture below is a visualization of a coloring that verifies this lower bound. Integer programming for van der Waerden numbers, and extremal combinatorics in general, is benefi cial because polyhedral theory is well-developed but not typically used in this context. Thus, the geometry of optimization problems may lend its own structure to questions of interest and shed light on otherwise hidden structures. Although, in this article, we lay out a strong case for the eff ectiveness of computer-assisted theorem proving such an approach is by no means guaranteed to yield answers to all mathematical questions. Indeed, the limits of formal Another problem of interest is Frankl's systems with basic arithmetic were conjecture. Frankl's conjecture concerns already clearly shown by Gödel in 1931. union-close families of sets. A nonempty Simply put, Gödel shows that there will finite family of distinct finite sets is always be true statements in mathemat- union-closed (UC) if, and only if, for ics that cannot be proven. Furthermore, every two sets in the family, their union Turing showed that it is not the case that is also in the family. Frankl's conjecture all tasks that are mathematically well states that any UC family containing an defi ned can be performed by a computer element that is in at least half the sets of – not even in theory. the given family. The conjecture appears to have little structure and has evaded all Although Turing proved that comput- attempts at a solution for nearly 40 years. er-assisted (mathematical) super-opti- Frankl's conjecture was fi rst noticed by mism is false, it is clear that applications Peter Frankl, a well-known mathemati- of such methods are still in their infancy. cian and juggler that currently has some Therefore, it stands to reason that com- celebrity status in Japan. puter-assisted methods will only become more commonplace in theoretical mathematics, and, a few decades from now, the use of computers for either checking correctness or discovering new mathematics will gain wider acceptance in the community. It is not diffi cult to imagine that one day the use of computers in all mathematics will be as ordinary as using LaTex. For all the future mathematical purists who believe that computers have little to contribute to the fi eld, we highly recommend a second specialization like that of Peter Frankl, just in case computers prove them wrong.

Zuse Institute Berlin 71 Florian Schintke | [email protected] | +49-30-84185-306 PERFORMANCE IN DATA SCIENCE © pixabay

Dig Deep to Uncover Treasures

Data handling is often the main performance bottleneck in big-data applications. Understanding the distributed data-access deep inside the architecture of a data-analysis workflow usually reveals possible improvements of the data layout and distribution. In two practical use cases from satellite imagery and blood spectrography, we illustrate our approach to monitor and improve the data-access patterns, layout, and placement. This provides interesting insights into the data management of analysis frameworks and allows a reduced time to solution. Performance in Data Science

BIG DATA © NASA FROM SPACE

DATA-ANALYSIS PIPELINE

Several earth observation missions The data analysis in GeoMultiSens conditions, eliminates shadows, detects have been collecting satellite imagery is based on the Apache Flink data- clouds, takes observation angle into of our planet for many years, such as analytics framework, which is used at account, and so on. Instead of choosing Landsat, Sentinel, SPOT, RapidEye, different levels throughout the pipe- the coarsest resolution of all involved or WorldView, to just name a few. line. When the user has selected the sensors, extrapolation with a separate Their sensors differ in the spatial region and data sources of interest in error-probability layer is used. This resolution, the explored frequency a Web interface, the raw imagery data results in a homogeneous data cube that bands, the repeat-pass intervals, and are downloaded from the data servers represents a virtual sensor consisting of their spherical projection model, of the particular missions, if it has not the data of multiple input sensors over making data analysis across multi- already been done for a previous analy- time. It can be used for the actual data ple sensors a challenging task. In the sis job. Then, the data are homogenized analysis, such as the classification of BMBF-funded project GeoMultiSens, and prepared for the analysis by a ﬁrst land usage and its change over time – GFZ Potsdam, Humboldt-Universität multistep Flink job that geometrically which is another Flink job. Finally, zu Berlin, and ZIB have built a big aligns and stitches the tiles, corrects the results can be explored in a visual- data-processing pipeline for auto- atmospheric distortion, uniﬁes lighting data-analytics Web interface. mated multisensoral data analysis. The pipeline includes the management of large amounts of data in the tera-to-petabyte scale, the integration of heterogeneous geospatial data in a common reference model, and the parallel analysis of large geospatial data. It is framed by visual exploration tools for the available input data and analysis results to facilitate the correct detection and assessment of spatial and temporal changes of the earth's surface.

74 2016 Annual Report SOFTWARE ARCHITECTURE 1

output output output output

REDUCE REDUCE REDUCE REDUCE sort sort sort aggregate

REDUCE REDUCE REDUCE The analysis tasks are user-defined sort sort sort REDUCE functions written in Python and Flink create cube jobs that express the required data ﬂow. COMBINE COMBINE COMBINE This description is independent of the part sort part sort part sort number of nodes and enables horizontal MAP scalability of the system. The execution create L2 MAP MAP MAP environment is a Flink compute cluster pipeline pipeline pipeline where each node consists of some CPUs, RAM, local disks, and a network inter- MAP face to the other nodes. Typically, such create L1 MAP MAP MAP pipeline pipeline pipeline clusters use the Hadoop File System (HDFS) that spreads the data across the disks of the cluster nodes. In the case of

input input input input GeoMultiSens, we use ZIB's distributed ﬁle system XtreemFS at the lowest level. As XtreemFS offers [2] both a POSIX- compliant interface and the HDFS interface, the input data do not have to be staged in before starting the job nor do the output data have to be staged out after the job is ﬁnished.

Object Storage Devices (OSDs) and Apache Flink Nodes 1011010011101101011101101001110110 0100111100101011110110110010111011 10101110010101001001001110110100 data 1000110011001111001011101011010111 FILE SYSTEM

sample.bsq Metadata and 1,365 Mbytes Replica Catalog (MRC)

1 Data-storage scheme and software architecture of GeoMultiSens based on meta data Apache Flink.

Zuse Institute Berlin 75 Performance in Data Science

2 node 1 node 2 node 3

assigned input splits 32TQU 32TKS 32TPT

files stored 32TQU/file_1 32TKS/file_2 32TPT/file_3 on local hard drive

32TKS/file_1 32TQU/file_2 32TKS/file_3

32TPT/file_1 32TPT/file_2 32TQU/file_3

INPUT SPLITS

In data-analysis frameworks, an input split denotes the smallest portion of data POSIX VS. HDFS LOCALITY PARADIGM whose processing cannot be parallelized, that is, must be processed on the same INTERFACE Clusters built for big-data applications node. Typically, one such split is one typically use commodity hardware; sin- file but must be written once and never The HDFS interface is a file-system gle nodes serve as storage and compute written again. interface especially designed for large- nodes. The questions “Where to store In GeoMultiSens, however, satellite scale distributed data processing and file x?” and “Where to process file x?” images are grouped into input splits compatible with most MapReduce imple- come up. As the network layer is often that correspond to the same geographic mentations. Our file system XtreemFS the main bottleneck, the data-affinity region. Each image is cut into multiple recently received support for this inter- paradigm is fundamental in the design of files; more images may be added later. To face. Unfortunately, the libraries to work efficient data-processing pipelines: files achieve the aforementioned data affinity, with satellite data used in GeoMultiSens should be processed where the data are it is required that all files associated require byte-level access to files, which is stored. In general, data should be trans- with a certain region are located on the only supported by the POSIX interfaces ferred between nodes as little as possible. same node. Together with the possibility and not the HDFS interface. Hence, our to add further data later on, the proper solution has to take this into account. placement of the input data becomes a challenging task that requires new techniques. 2 Random layout shows file accesses as provided by a naive XtreemFS setup.

3 Good layout depicts the current state of development.

76 2016 Annual Report DATA PLACEMENT FOR BETTER PERFORMANCE

3 node 1 node 2 node 3

assigned input splits 32TQU 32TKS 32TPT

32TQU/file_1 32TPT/file_1 32TKS/file_1 files stored on local hard drive 32TQU/file_2 32TPT/file_2 32TKS/file_2

32TQU/file_3 32TPT/file_3 32TKS/file_3

REALIZATION IN XTREEMFS

We have enhanced the XtreemFS servers input splits and automatic rebalancing first step of improvement. The next step with a new data-placement mechanism of existing data. is to schedule the Flink tasks near the based on path prefixes. This way, one can node storing the corresponding data, With the described grouping of data files, ensure that all files in a directory and its which will further improve the speedup. we were able to improve the overall job subdirectories are stored on the same The presented grouping of data was a run time from 80 minutes to 72 minutes node. For the client side, we have devel- necessary preparatory step to exploit the for around 600 GB of input data distrib- oped a placement-management tool that full performance potential. uted over 16 nodes. But this was just the allows automatic load balancing for new

Zuse Institute Berlin 77 Performance in Data Science central storage infrastructure, for exam- infrastructure, storage central electronic an in stored also is data the all consents, apatient If monitoring. therapy orfor itfor diagnostics using example, for patients, their treating for directly acquired medical patient data not only use hospitals that recently only It started 78 DATA ANALYSIS MEDICAL DISTRIBUTED FOR NEED THE WORK: THE SHARE 2016 Annual Report 2016 Structured information is easy to col- information. structured so-called the is type), blood (e.g. parameters or simple lists, gender, medication or ethnicity), (age, data demographic such as mation, purposes. Most of this clinical infor- ple, for research-oriented data-mining © Photo: Designed by Jannoon028 / Freepik their daily routine, there is yet another another yet is there routine, daily their during doctors medical help certainly although these pieces of information However, understandable. computer a common database system, and thus in stored (typically) standardized, it is because query to and exchange, to lect, NEW DATA SOURCES WHAT TO DO WITH DELIVER BIG DATA ALL THESE DATA?

DNA sequencer and mass spectrometer The point of collecting all the data is of are modern medical devices that can course to later analyze them. This has extract information about genes and at least two parts: First, the data need proteins from a biological sample. A to be copied to the compute device that single drop of blood contains informa- is assigned for the actual analysis. And tion about the full genetic setup (the second, the computationally heavy building blocks) and all active proteins analysis has to be performed on some (the working horses in a cell) of a person. compute infrastructure. Both tasks Detailed knowledge about all of a person’s are quite well understood and several genes and proteins allows the status to be frameworks are available to do the job. characterized in ways that have not been However, to really do this in a modern possible before. Was the person digest- IT infrastructure environment takes ing, stressed, or carrying a disease? What quite some time: just to transfer 1 TB are the odds of developing a major disease of patients’ data takes about 2.5 hours in the next years? However, to even start on a standard gigabit-Ethernet (1,000 developing new medical tests based on Mbit/s) intranet network. The standard these data, one has to first understand analysis (remember: going from unstruc- the raw information generated by these tured to structured) of that data takes machines. One of the first problems is another ten hours on a normal desktop the sheer size: a raw data set can easily PC. Now, multiplying that with the 1,000 consume up to 1 TB (one terabyte) of disk study-participants’ data sets gives some space. This is the size of about 300,000 uncomfortable numbers: one would have images or 40 days of video material. This to wait about 1.5 years until the results seems like a lot of data, but somehow still are available. manageable. However, for a meaningful For more details about new analysis scientific study, not one single data set methods developed by ZIB, see the feature is needed, but several hundred, better article “Bigger Data, Better Health." thousand participants. Now, storing (and watching) 300 million images or 40,000 days of video seems much more like a problem. type of information that is at least as 4 important. This other type is often called unstructured information. The main feature of unstructured data is STORE AND ANALYZE that it is not computer understandable. Simply put, this means that a human is needed for proper interpretation and – ultimately – translation into (computer-understandable) structured data. Examples for medical unstructured data are images (think of x-ray or MRT images), documents, or hand-written reports. Patient attributes (structured information), medical history reports, and x-ray images (unstructured information) certainly already provide a 4 Medical pipeline. medical doctor with many pieces of the puzzle that characterize a patient. Nevertheless, modern medical devices can deliver even more.

Zuse Institute Berlin 79 Performance in Data Science SCALABILIT SIMPLE BIG-DATA ANALYSIS MADE CALL THE CAVALRY IN BERLIN

A simple yet very effective way of dealing with this kind of scaling problem is known: divide and conquer. This essentially means to split up the problem into smaller independent 5 pieces. These small tasks can then be distributed over a whole army of compute nodes and solved in parallel. Obviously, this is needed twice: once for the storage problem and again for the analysis part. Divide-and-conquer strategies are well known in computer science. However, the hard part is to adjust the quite general approach to the actual storage and analysis problem at hand. However, once this is done, the idea needs to be implemented such that it can be executed on a distributed compute cluster. Researchers at ZIB together with partners from the Berlin Big Data Center have found very efficient ways to implement algorithms The development and ubiquitous employ- which aims at opening up machine- for the distributed analysis of geno- ment of sensors of all kinds, collection learning-based, scalable data analytics mics and proteomics data. This effort of usage statistics, and refinement of for a large audience of data scientists contributed to the development of the scientific equipment – such as DNA through a concise declarative API. The software frameworks Apache Flink sequences and mass spectrometers – intricacies of distributed systems devel- and XtreemFS. The new approach now are only a few recent developments that opment are hidden from the programmer allows the distributed analysis of very contribute to the ever-growing amount of by providing an execution engine that large data sets in signiﬁcantly less time data that is being collected for analysis. transparently scales the analysis from than before, thus enabling the analysis Managing the three Vs of big data – large local development to cluster deployment. of clinically relevant amounts of data volume, great variety, and high arriving Queries are expressed declaratively with cohorts of thousands of patients. velocity – call for specific application- using well-understood concepts of sets domain expertise, knowledge of machine and transformations. With applications learning techniques, and scaling their in information market places, materials implementations to systems of thou- science, and medical imaging, we enable sands of nodes. Funded by the BMBF, data scientists to formulate scalable and researchers from Beuth Hochschule, effective programs using approaches DFKI, Fritz Haber Institute, Technische known from SQL without being experts Universität Berlin, and ZIB are joining in distributed systems development or efforts within the Berlin Big Data Center machine learning. (BBDC) to develop “Technology X,”

80 2016 Annual Report Y AT ALL LEVELS

STATISTICS FILE FOLLOWING THE BYTE SYSTEM

Most of these applications require tera- understand and trace data-access at all At the highest level, SFS wraps around bytes of input data, which usually do not abstraction levels: the application itself, HDFS by logging every call and passing fit into a single machine. Therefore, dis- the execution framework it is run on, the the request on to HDFS, measuring tributed file systems (such as Hadoop’s framework’s distributed-file-system execution time, relevant parameters, Distributed File System HDFS) are abstraction and the actual low-level file and number of bytes accessed, among used to incorporate many machines system. To this end, we have built the others. At the lowest level, SFS uses and their disks into one logical file Statistics File System SFS, which can bytecode instrumentation to intercept all system. When an application reads be plugged into any HDFS-compatible native I/O calls leaving the Java Virtual from HDFS, it incurs read and write big-data-analytics engine at the highest Machine JVM, logging similar statistics. operations to potentially many disks in and lowest levels to collect various I/O This way, low-level disk access can be many machines. It is therefore vital to operation statistics. correlated with high-level HDFS interactions, and unintentional I/O can be identified. Cross-checking the SFS statistics with the underlying file system’s statistics – as found in Lustre, XFS, and others – allows for precise determination of the application’s share on total I/O on 6 the disk.

5 Construction and spirit of the Berlin Big Data Center.

6 The Statistics File System wraps distributed file systems on different abstraction levels, gathering vital I/O usage and performance data.

Zuse Institute Berlin 81 Performance in Data Science

XFS Total SFS Yarn SFS HDFS SFS Flink SFS Totol Write 417 3 202 207 415 Read 545 0 101 332 534 (All numbers in gigabytes)

7 ANALYSIS OF (SOME) SORT

Depending on the job, however, I/O can become a lot more involved. We have analyzed TeraSort as a common big-data benchmark, using Hadoop and Flink on PEEK INTO PEAK YARN and HDFS on 16 nodes. First, 1 TB worth of 10-byte-key/90-byte-value pairs of random numbers is generated IDENTIFICATION into HDFS using TeraGen. Then the numbers are globally sorted on the keys. We used SFS on an Apache Flink appli- Because we instrument each JVM sep- cation analyzing 205 GB worth of blood- arately, with SFS, we are able to report Ideally, we expect 2 TB of writes – 1 TB mass-spectrography data to obtain each component’s share of the overall during input generation, 1 TB during detailed I/O information. First, all data I/O. For HDFS, the numbers reported by output generation – and 1 TB of reads are staged into HDFS. Next, it is analyzed SFS meet our expectations. Motivated during sorting. For Hadoop, 1 TB of reads for specific peaks to identify diseases. by the unexpected I/O for Flink, we and writes each occurs during the shuffle These peaks are then written back to identified the cause to be a mass- spec- phase. Again, our expectations are not HDFS (just a few kilobytes). The appli- trometry-data-processing library which met: XFS reports 4.1 TB of writes and cation is run on YARN and HDFS on 16 does not support HDFS directly, and 6.4 TB of reads for Hadoop. Using SFS, nodes. Without SFS, the only available therefore downloads all data to local we are able to match these numbers to I/O statistics are those of the underlying storage first before (repeatedly) reading each component involved over time (see file system’s counters, XFS in this case: it. The gap between the total XFS and figure 8). 417 GB of writes to and 545 GB of reads total SFS numbers is largely accounted from the file system (see table 7). However, for by loading of libraries during JVM we would expect 205 GB of writes during startup, which we have not explicitly staging, 205 GB of reads during process- reported in this case, as there is also ing, and a few kilobytes of writes for the unpredictable file-system caching output – where does the I/O come from? involved.

yarn: jvm: read yarn: jvm: write hdfs: jvm: read hdfs: jvm: write map: sfs: read map: sfs: write map: jvm: read 7 File I/O recorded by XFS and detailed I/O per map: jvm: write component recorded by SFS. reduce: jvm: read reduce: jvm: write 8 I/O profiles for the TeraSort benchmark.

82 2016 Annual Report In the first 20 minutes, TeraGen gener- incurs more than 3.6 TB of reads;2) the ates the data using map tasks, and the reducer JVMs write 1 TB of data without MIND THE DISK writes in HDFS at JVM level correlate corresponding SFS requests; and 3) over with the writes issued by these map the entire TeraSort run, the HDFS JVMs Tracing file system calls from the highest tasks at SFS level, accumulating to 1 TB. read 600 GB without corresponding SFS to the lowest abstraction level allows Following that, the TeraSort map tasks requests. For Flink, XFS reports 3 TB of the requests to be matched to I/O, which read the data at SFS level, and HDFS writes and 3.2 TB of reads. The TeraGen allows the detection of bottlenecks in serves these requests with 1 TB of reads step is the same as before, and all com- disk access, especially in highly concur- at JVM level. The map tasks write all ponents except two behave as expected: rent contexts. We have learned about their data to local disk for the shuffle again, the HDFS JVMs read 200 GB more unexpected I/O incurred by commonly phase, causing 1 TB of writes in their data than are requested through SFS, used big-data-processing engines, only JVM. Next, the reducers read this data and Flink adds 1 TB of reads and writes part of which can be explained by spilling from disk, and write the output to HDFS, each (figure 8). Note that YARN incurs because of memory exhaustion. Given incurring 1 TB of writes at SFS and JVM almost no overhead because instead of that thousands of terabytes of data are level. scheduling thousands of mappers and being processed worldwide using these reducers, only few components need to So far, the analysis agrees with our frameworks, surprisingly little is pub- be deployed. To close the gap between expectations for TeraSort on Hadoop. To lished about their underlying disk access. the SFS and XFS counters, 1 TB of reads explain the gap between expectations and With our work, we are able to shed some is still unaccounted for, which we are XFS counters, we point out three facts: light in the dark, and, using the entry- currently investigating. 1) over the entire reduce phase, YARN points we have at the highest and lowest levels, can make these frameworks more disk-aware.

I/O for TeraSort on Flink I/O for TeraSort on Hadoop 8 Cum. Data (GB) Cum. Data (GB)

Time (minutes) Time (minutes)

Zuse Institute Berlin 83 Dr. Thomas Steinke | [email protected] | +49-30-84185-144 SUPERCOMPUTI AT THE LIMIT How Technology Innovations Impact Software Designs

Innovative technologies are changing the way of supercomputing. Many-core CPUs 1 and accelerators are the answer to the ever- growing performance demands while staying within power budget constraints. High-bandwidth memory of processor devices extends the memory hierarchy, and high-capacity, persistent storage is seen near the processors. But how can we cope with the resulting challenges for software design? NG 1 Waver with CPUs manufactured with Intel's improved 14nm+ process. © Intel Corp., 2017 Supercomputing at the Limit TECHNOLOGY INNOVATIONS CHALLENGES THE CODE DEVELOPERS ON SUPERCOMPUTERS

Moore's law from 1975, which On the other hand, the application legacy code still in use that needs to proclaims the exponential growth and library software running on these be modernized for every new hard- of processor complexity and thus systems is rather long lived. Some codes ware architecture. compute power, is still valid today. date back to the 1970s and beyond. While Modernization in this context means Consequently, hardware is outdated software engineering in general can be to adapt the code such that it uses the rather fast and the typical lifetime fast paced, the scientific computing com- new hardware efficiently. The earliest of a supercomputer is about five munity is rather conservative in adapting computers would sequentially execute years. After this time, it is no longer the merits of modern programming tech- a stream of instructions provided by economical to continue running niques. This is a consequence of scientific the programmer on a single processor the system, as the performance per diligence, but also of code being written core. Increasing the frequency at which energy invested is no longer compet- mostly by noncomputer scientists whose these processors work would increase itive with what current hardware focus is on computational results rather the application performance. Today’s would deliver. Another reason is an than modern code. Many codes are still supercomputers are networks of work- increased component failure rate written in FORTRAN, a programming station-like computers (nodes) that solve due to aging effects and thus higher language whose development started in problems in parallel. Each node has one maintenance cost. the mid-1950s that is not found outside or more processors that has tens of com- the domain of scientific computing any- pute cores working in parallel, each of more. This builds on a huge amount of which has multiple forms of instruction

86 2016 Annual Report 2 Cray XC blade with four compute nodes (from left) each of them with one Intel Xeon Phi 7250 CPU (Knights Landing) having 68 cores and 16 GiB high-bandwidth MCDRAM, and the Cray Aries system-on-a-chip device (right). 2

TEST PLATFORM WITH MANY-CORE CPUS AND BURST BUFFER

Our Cray XC40 test and development system (TDS) is configured with 80 compute nodes, each of them with one Intel Xeon Phi 7250 processor – code- named Knights Landing and 68 cores, 16 GiB MCDRAM (high-bandwidth memory), and 96 GiB DDR4 main memory. A globally accessible burst buffer which supports I/O intensive workloads is built by 10 Cray DataWarp nodes level parallelism. Additionally, each node equipped with 3.2 TiB SSD capacity each and providing a sustained I/O might have accelerator devices to which bandwidth of 3 GiB/s locally. All nodes are connected through the Cray parts of the computation can be out- Aries interconnect. In summary, the Cray TDS features the following total sourced for faster, more energy-efficient performance characteristics: processing – in parallel, of course. For Total theoretical peak performance: 244 TFLOPS. each of the components, there exist multi- Total DRAM memory capacity: 7.6 TiB DDR4. ple vendors, architectures, programming Total high-bandwidth memory capacity: 1.2 TiB MCDRAM. models, and software tools. Writing a Total burst-buffer capacity: 32 TiB SSD. correct scientiﬁc simulation program for a given mathematical model, that keeps all these resources busy in parallel, while handling communication is a hard problem. Transforming an existing code, that has been developed for decades, and uses a bunch of software libraries with an even longer history, to a new supercomputer system can be even harder.

Zuse Institute Berlin 87 Supercomputing at the Limit MODERNIZING CODE MODERNIZATION FOR MANY CORES A LEGACY CODE: Code modernization is one of the main tasks for a provider of supercomputing resources like ZIB. It is a collaborative effort that should be addressed hand in hand with the code developers and indus- THE PALM try partners. It starts long before the call for bids for a new system procurement, as the codes should ideally be ready when a new production system goes online. At ZIB, one such effort is the Cray XC40 test-and-development system (TDS) that SIMULATION features Intel's newest Xeon Phi processors (Knights Landing) and Cray's recent DataWarp technology. It serves as a platform for code-modernization efforts within the HLRN and other European compute center through the Intel Parallel PROGRAM Compute Center collaboration network.

4 3

3 Intel Xeon Phi processor of the 4 Simulation of a densely built-up 7200 series (Knight Landing) as artificial island off the coast of installed in our Cray XC40 test- Macau. and-development system.

88 2016 Annual Report 5 Germany bniz Universität Hannover, Meteorology and Climatology, Lei Meteorology and Climatology, © PALM group, Institute of group, Institute © PALM 5 Simulation of dust devil with PALM.

THE PALM CODE

PALM – short for “A Parallelized Large- lion operations per second (1 PFLOPS). Eddy Simulation Model for Atmospheric PALM is written in Fortran using the and Oceanic Flows” – is one of the main 95 and 2003 language standard, with production codes on the HRLN-III, and around 140 thousand lines of code, struc- thus a priority target for modernization. tured in 79 modules and 171 source files. It computes oceanic and atmospheric Parallelization and communication build flows, and has a variety of applications in on the MPI and OpenMP standards. It research. They range from studying nat- has proven to be highly scalable, for up ural phenomena such as dust devils or to 43,200 cores. cloud physics, over technical applications In the following, we will describe the such as wind energy or the interaction process and challenges of porting this between aircraft and the atmosphere, to code to the Intel Many Integrated Cores modeling and simulating air flows within (MIC) architecture. This architecture whole cities. For instance, figure 4 shows is the foundation of the Intel Xeon Phi the simulation of a densely built-up arti- processor that powers recent supercom- ficial island off the coast of Macau. The puter installations in the USA, Asia, results allow studying the effects of such and Europe, respectively. At ZIB, we are a mega-project on the air flows within the working on our Cray XC40 test-and-de- existing city in the early planning phase. velopment system (TDS) with 80 Intel The PALM code has been being devel- Xeon Phi (KNL) nodes. oped by the Siegfried Raasch’s research A three-day hackathon meeting group at the Institute for Meteorology with members of the PALM devel- and Climate at Leibnitz Universität opment team and the Algorithms for Hannover since 1997. In that year, the Innovative Architectures Group of fastest computer in the world provided ZIB's Supercomputing department are a computational power of around one getting the process of code moderniza- trillion floating-point operations per section started. The schema on top of the ond (1 TFLOPS). The current HLRN-III next page illustrates the main steps Konrad system at ZIB provides roughly performed. a thousandfold of that, or one quadril-

Zuse Institute Berlin 89 Supercomputing at the Limit

get operational define a set of measure baseline on KNL benchmarks on Xeon

GETTING OPERATIONAL SETTING REALISTIC The first step is to get operational on the new system. This means the code has to be successfully compiled and executed, EXPECTATIONS not yet looking at performance numbers. Before starting to look at the performance Since the Intel Xeon Phi many-core chip impact of different system settings and has a very similar programming model code changes, it is beneficial to define a compared with the existing multi-core DEFINING THE baseline to compare with later, and to Xeon processors, the same tool chain build some expectations of what can be can be used with different settings. This BENCHMARKS achieved. As a baseline, we measured renders building an executable program the runtime of the defined benchmarks from the source code a relatively easy The next step in the process is to define on the HLRN-III production system. A task. However, executing the program for a whole series of benchmarks covering maximum speedup over that on the new the first time, and validating the results, different problems and problem sizes system can be estimated by looking at revealed some bugs in the code that had that represent typical use cases of PALM. the theoretical performance of both, the not surface on any other architecture These benchmarks can then be run to production, and the test-and-develop- before. This is a common observation, determine optimization success and ment system. as complex software cannot be formally compare the effects of different changes validated to prove its correctness in a on the program’s performance. At this One compute node of the HLRN-III mathematical sense. It can only be tested stage, it is important to not focus just on system (phase two), with its two within reasonable effort to minimize a single test case to avoid overoptimizing 12-core Xeon processors (E5-2680v3) code defects causing unexpected behav- for just a single execution path through provides 960 GFLOP of computational ior at runtime (i.e.bugs). Fixing these the program's modules that might not performance, and 136 GiB/s of memory problems almost took the entire first day represent the typical production work- bandwidth between the processor and of the hackathon, at the end of which, we load. We selected three differently sized main memory. A node on the TDS has had a running program that produced the benchmarks using 4, 8 and 16 compute a single 68-core Xeon Phi processor expected results for a simple test case. nodes of the Cray XC40 TDS. (Intel Xeon Phi 7250) that has a peak of

90 2016 Annual Report MCDRAM and MPI processes same optimization boot mode vs. OpenMP workflow as on threads multicores

MULTIPLE OPTIONS FOR ACCESSING THE HIGH-BANDWIDTH MEMORY ON INTEL KNL

To support a wide range of applications, the processors of the Intel Xeon Phi 7200 series can be booted in a series of modes, which affect how the main memory, 2611 GFLOPS in compute performance, MCDRAM, and caches are configured and 115 GiB/s of bandwidth to the main and exposed to the software. The most memory. To balance this huge compute important decision here is how to con- power with the rather low main-memory The cache mode has the advantage of figure the aforementioned MCDRAM. bandwidth, it has an additional 16 GiB not requiring any further action to make of MCDRAM (Multi-Channel DRAM), There are three options: use of the MCDRAM. However, the which is a high-bandwidth memory cache protocol causes overhead. Since Ċ It can be used as a last-level cache located on the processor package show- the MCDRAM is much smaller than the (cache mode), meaning it will not be ing a sustained bandwidth of 490 GiB/s. main memory it caches, that is, old data exposed to the software at all, and all are constantly moved out to make room Based on these numbers, an application data in the main memory go through it. for new data, and data that are highly whose performance is limited by the Thus, subsequent accesses to the same reused might get evicted for data that are computational throughput (compute data are sped up by the higher memory only used once and thus does not benefit bound), can run up 2.7 times faster than bandwidth of the MCDRAM over the from the caching. on the production system, while an appli- main memory. cation limited by data transfers (memory The explicit-usage model allows placing Ċ Another mode is to expose it as a bound) can be sped up by a maximum of data that are known to be frequently separate region of memory that has to 3.6 times – given the newly introduced accessed into the limited MCDRAM. be explicitly used (flat mode). MCDRAM is used. In practice, these This, of course, requires carefully upper bounds are not likely to be reached, Ċ The third option is a hybrid mode, con- analyzing the application and adapting but they provide a means to assess what- figuring part of the memory as a cache, the code to allocate certain parts of the ever is measured later. and the other part for explicit use. memory in MCDRAM.

Zuse Institute Berlin 91 Supercomputing at the Limit 6 THE STRATEGY FOR Boot flat mode DETERMINING Run in DDR

p 1SPCMFN(J# MCDRAM Run in MCDRAM p 6QQFSCPVOEGPS gain from MCDRAM

Boot cache mode PERFORMANCE

p (PPEFOPVHI Run again Decide about explicit placement IMPACT

First, the flat mode is used. Running the between these numbers and the DRAM- benchmarks in this mode provides tim- only numbers cause the speedup due to ings for only using the slow DRAM main using the MCDRAM as a cache – also 6 The strategy for determining memory. In a second step, still in flat for larger problem sizes that fit into the MCDRAM performance impact. mode, the application can be run using MCDRAM. If the difference between the a special tool (numactl) to make it use MCDRAM-only and cache-mode speed- MCDRAM only without changing the ups are small, the cache mode is a viable code. This requires problem sizes that option. entirely fit into the MCDRAM. Now we If not, the code needs to be adapted for have two timings completely run in the explicitly using the MCDRAM. main memory (DRAM), and completely run in MCDRAM. The ratio provides a Figure 7 shows the data for PALM. The maximum speedup due to MCDRAM gain due to MCDRAM ranges from 25% usage for that application. Now the to 41%, while the cache mode is less than system is booted up in cache mode, and 3% slower than the MCDRAM only. So, the benchmarks are run again. The ratio the cache mode works well for PALM.

92 2016 Annual Report BALANCING PROCESSES AND THREADS IN HYBRID WORKLOADS

Since PALM is a hybrid code, using MCDRAM Usage Comparison OpenMP for thread parallelism and MPI 7 for communication, the “right” number 160 of threads and processes needs to be quad_flat, DDR4 ﬁgured out. A process is one instance of 140 quad_flat, MCDRAM quad_cache, both the program being run in an executable 120 fashion. It has its own memory space and can use multiple parallel threads of 100 1.25 1.22 execution to utilize multiple processor 80 1.41 1.39 cores. MPI enables communication

Runtime[s] 1.41 1.38 60 between such processes, either over the network between compute nodes 40 or between processes running on the 20 same node by copying memory between them. OpenMP handles the threads 0 within each process, which share the Small Medium Large same memory and thus do not need to communicate data between each other, but instead only synchronize concurrent 7 Impact of different MCDRAM usage modes on runtimes and speedups for accesses to the same memory region. the three PALM benchmarks “small,” “medium,” and “large.” Shown are data for main memory only (DDR4, purple bars) as baseline vs. MCDRAM only While naturally, there would be one MPI (green bars), and cache mode (blue bars) usage. process per compute node that uses as many threads as there are processor cores available, it is often beneﬁcial to have multiple MPI processes on each compute node that use less OpenMP threads each. One reason is that the fast, internal network of a supercomputer is easier to saturate by having multiple MPI processes feeding it with data. Another reason is the historic development of MPI being there before OpenMP, such that MPI parallelism is THE INTEL XEON PHI USER’S GROUP (IXPUG) often better implemented than OpenMP thread parallelism – even though it causes The Intel Xeon Phi User’s Group (www.ixpug.org) is an independent nonprofit avoidable memory copies. organization whose mission is to provide a forum for the free exchange of information that enhances the usability and efficiency of scientific and technical applications Assuming we want to use 64 cores per running on large High Performance Computing (HPC) systems using the Intel Xeon Xeon Phi compute node, we could start Phi processor. IXPUG is administered by representatives of member sites that operate a single MPI process per node using 64 large Xeon Phi-based HPC systems. Thomas Steinke from ZIB is cofounder of IXPUG OpenMP threads each, or 64 MPI pro- and served as its vice president from 2014 to 2017. He now helps as senior advisor on the cesses per node using a single OpenMP Steering Group of IXPUG. thread each – or anything in between.

Zuse Institute Berlin 93 Supercomputing at the Limit

THE CODE OPTIMIZATION WORKFLOW

Which setting works best highly depends Now that the basics are determined (i.e. on the application at hand, and is hard to the MCDRAM-usage mode and MPI/ predict. The easiest way to figure it out is OpenMP configuration), the actual to run a parametric study for a series of work with the code can start. The corre- settings to see what works best. sponding workflow is shown in figure 9. It is basically the same as on multi-core Figure 8 shows the results for PALM. architectures such as the Xeon proces- While the fastest setting is a different sors of the HLRN-III production system. one for each benchmark, there is one setting that performs reasonably well for At first, for the whole application, a all of them (i.e. 16 MPI processes [often runtime profile is generated using a tool called ranks] per node with four OpenMP like Intel VTune Amplifier. This profile threads each). Comparing the fastest reveals which parts, or kernels, of the 8 Parametric study for the impact of with the slowest tested configuration code consume the most runtime, and the number of MPI processes (ranks) reveals a factor of 2.3 between the two, thus are the best target for optimization vs. OpenMP threads per compute underlining how crucial this step is on efforts. This kernel is then benchmarked, node. Shown are runtime results of the way to optimizing for performance analyzed, and modified in a loop until the three PALM benchmark cases on that architecture. the performance matches expectations. as a heat map. Which changes to perform are guided by experience, compiler optimization 8 reports, and special analysis tools.

MPI Ranks vs. OpenMP Threads per KNL Node Ranks 64 32 16 8 4 2 1 Threads 1 2 4 8 16 32 64 Small X Medium X Large X

≤ 2% off from best Worst

≤ 15% off from best X Fastest

94 2016 Annual Report 9 Unoptimized kernel Benchmark

Tune Identify hotspots Analyze

Optimized kernel Done no

9 General code-optimization workflow.

On the Xeon Phi, the most important chip area and per energy consumed. Not different memory locations and instead optimization is to make sure the code using the SIMD units costs a potential to load them as on contiguous piece of is vectorized, as most of the computa- factor eight in application performance. memory. However, this is a bigger change tional power comes from so-called SIMD and part of the ongoing efforts with that For PALM, the compiler report revealed (single-instruction multiple data) units. code. the currently implemented way of They can perform arithmetic operations, numeric error handling completely From the different things we have tested like multiplying two inputs into an out- prevents vectorization and requires so far, introducing the CONTIGUOUS put, on vectors of data of given length in adapting the application design. The cur- keyword of the Fortran 2008 language to one step instead of using single scalar rently used exception handling needs to tell the compiler explicitly that an array data arguments requiring multiple steps. be disabled, and instead regular, manual is contiguously allocated in memory has The Intel Xeon Phi works with 512-bit checks on the data need to be introduced, had the biggest impact. It prevents the vectors, which can ﬁt eight double pre- for example, when checkpointing the compiler from making the pessimistic cision floating-point numbers. On the application state. With that change, the assumption of noncontiguous memory. hardware side, this means producing compiler still did not generate signifi- Beside the Intel Fortran compiler, we also eight results at once without introduc- cantly vectorized code. Further analysis have tested the Cray Fortran compiler ing additional transistors for the control revealed that the current memory layout which produced faster machine code logic, as compared with eight scalar needs to be adapted for efficient vector- due to better performing optimization multiplication units. This design results ization too. This is necessary to avoid techniques for this application. in more computational performance per gathering vector elements from eight

Zuse Institute Berlin 95 Supercomputing at the Limit

10 Projected speedups, that is, without one-time initialization cost, for PALM on Cray XC40 with Intel Xeon Phi 7250 (KNL) over the original code on the HLRN-III, a Cray XC40 with Intel Haswell nodes. For the Intel Xeon Haswell CPU, the code is generated by the Intel compiler. For the Intel Knights Landing CPU, the Intel and Cray compiler were used.

MANY-CORE RÉSUMÉ

Figure 10 shows the results obtained Compared to accelerator architectures as a new layer in the memory hierarchy, after the three-day hackathon. The such as GPUs (Graphics Processing and the mandatory use of the SIMD speedup over the production system is Units) or the accelerator version of the units. Fortunately, optimizations for up to 1.45. Xeon Phi, porting code from multi-core the Xeon Phi are usually beneficial on CPUs like the Intel Xeon to the current Xeon as well because the latter has SIMD generation many-core Xeon Phi is rel- units (currently 256-bit vectors) too. The atively easy. Code does not need to be ﬁ rst results for the PALM code are very offl oaded from the host processor to the promising, but getting the code to better accelerator, and no data have to be trans- use the processor’s SIMD units is a larger ferred between their memories. Also, the eff ort still in progress. tool chain remains the same. Pushing the limits of scientiﬁ c high-per- However, getting performance out of formance computing means not just buy- the Xeon Phi can be challenging due ing faster computers every few years, but, to their new architecture features: the most of all, continuously modernizing increased level of parallelism and slower the applications to make best use of them. single-core performance, the MCDRAM 10 Projected Speedups (without init) 2 HSW, Baseline HSW, Current, Intel KNL, Current, Intel KNL, Current, Gray 1.5 1.45 1.25 1.25 1.23 1.10 1.14 1.11 1.00 1.00 1.00 1.00 1.06 Speedup 1

0.5

0 Small Medium Large

96 2016 Annual Report STORING DATA BECOMES MULTIFACETED

© Cray Inc.,2017 11 Cray DataWarp node a with Beginning in the 1950s, the memory and two SSD slots b to integrate storage hierarchy in computing devel- a burst buffer into a Cray XC oped into a hierarchy of three tiers: main system. c State-of-the-art SSD memory for fast access to temporary storage solutions for new data data; secondary memory for storing data tiers providing low-latency, high persistently on disks; and tape-based throughput and ultraendurance archive storage for long-living data in data centers. Shown is an Intel archives. This traditional three-level Optane SSD DC P4800X series. storage hierarchy is now going to change b into a more complex one. Solid-state drives (SSD) are increasingly integrated in any computer system to replace and enhance the traditional disk-based storage solution. Recently, in the new generations of many-core CPUs and GPUs, high-bandwidth memory (HBM) is stacked onto the CPU or GPU device substrate and thereby integrated into the device package. With that, HBM deepens the memory hierarchy as it is functionally placed between the on-chip cache and the external DRAM (main) memory. HBM is c not faster, but is wider and tackles today’s © Intel Corp., 2017 increasingly critical “memory wall” – the restricted compute performance due to the limited bandwidth to data residing 11 in the main memory.

Zuse Institute Berlin 97 Supercomputing at the Limit

CPU CPU

Near Memory (HBM) Main Memory Main Memory

(DRAM) (DRAM) Performance Capacity Far Memory (NVDIMM)

Storage Network NV Memory (HDD) (SSD) MidStorage (HDD)

Archive Storage Archive Storage (Tape) (Tape)

Current Next-Generation Production Systems Production Systems 12 (HLRN-III) (HLRN-IV)

The next technology innovation in the Complex HPC workflows are orches- 12 The memory and storage hierarchy memory segment is visible on the hori- trated as multiple-data-processing and of the current HLRN-III production zon: nonvolatile dual in-line memory data-management steps. Each access to system (left), consists of the three modules (NVDIMM). It is expected high-volume or nontrivially structured tiers of main memory in DRAM: that NVDIMM and HBM become data must be adapted to take advantage persistent storage using hard-disk widely available in the next years and of these new storage technologies and drives (HDD), and an archive using complement existing technologies such their different characteristics, such as tapes. For the next-generation HPC as DRAM and SSD. A possible scenario bandwidth, access time, and capacity. At system HLRN-IV (left), we expect for, but not limited to, supercomputer ZIB, we are building the “I/O Test Bed” high-bandwidth memory (HBM) conﬁgurations is a memory and storage platform helping us to explore future and nonvolatile DIMMs (NVDIMM) hierarchy consisting of six tiers as shown technologies for data management in extending the main memory by in ﬁgure 12. supercomputer systems. near memory and far memory, respectively. NVDIMMs and SSDs as network-attached nonvolatile memory can extend disk-based file- system functionality.

98 2016 Annual Report ARCHITECTURE OF PARALLEL HPC FILE SYSTEMS

Parallel file systems are designed for example, thousands of compute nodes, manage the metadata of the file system, storing data persistently to meet the communicate with multiple servers whereas the storage servers save and demands of supercomputing workloads: simultaneously, enabling parallel access serve stripes of user files contents. The access with high bandwidth and reli- to data by the workloads running con- file system is mounted on all compute ability. To circumvent the limited I/O currently on the supercomputer. Ideally, nodes as a network file system. Current performance of a single storage server, the I/O performance and the file-system parallel file-system installations have the principle of parallelism is also applied capacity scales with the number of thousands of clients, a capacity in to the storage system in today's super- installed storage servers. the petabyte range, and deliver an I/O computer installation. Multiple storage streaming throughput in the range Scalability is achieved by separating servers are combined to offer parallel from hundreds of gigabytes per second file-system metadata such as file names data paths in a network such that the load up to multiple terabytes per second. and directory information from the con- is distributed over many storage servers Commonly used parallel file systems are tent of a file. So-called metadata servers (see figure below). Many clients, for Lustre and BeeGFS.

Clients 13 Metadata Servers (Fast) Network (Fast)

Storage Servers

13 The basic structure of a parallel file system: clients (e.g. compute nodes), metadata and storage servers are connected through a high-bandwidth network to provide multiple data paths to data in a file system.

Zuse Institute Berlin 99 Supercomputing at the Limit

EXPLORING THE CAPABILITIES OF PARALLEL FILE SYSTEMS

Operating a large supercomputer production system like the current HLRN-III requires a detailed understanding of the configuration of the installed storage system. Parallel file systems support a variety of installation requirements. This desired flexibility is addressed by METADATA many parameters which control the file-system configuration. Primarily, PERFORMANCE IN parallel file systems can be tuned for best performance to minimize unnecessary COMPARISON slowdown of the production runs with their often complicated data flows. OUR STORAGE For system configurations with thousands of compute nodes, the metadata Data on a parallel file system is globally performance of a parallel file system accessible from all compute nodes, and TESTBED becomes a bottleneck for trivial I/O usually only one or a few instances of operations, such as creating a new file or such a file systems are part of a super- First, our storage testbed acts as a test retrieving the metadata of existing files. computer installation. Therefore, a high infrastructure for different configuration Recent versions of parallel file systems availability is needed, that is, data on and tuning settings, optimizations, and support multiple metadata services to the parallel file system is expected to file-system updates. New file-system distribute the load. We measured the be accessible all the time. An unreliable features, such as multiple metadata serv- scaling of the metadata performance storage system would become the weak ers, file system snapshots, access-control of the BeeGFS and Lustre file system spot of any supercomputer system, ren- mechanisms, and resource-usage poli- as a function of the number of active dering the whole system unusable during cies, are tested before their deployment metadata servers. For this benchmark, downtimes. on the production system. we used the widely known “mdtest” tool. We explore the configuration space of Second, the testbed serves as a platform As shown in figure a , the retrieving of state-of-the-art parallel file systems for for the development of tools for the I/O metadata (“file stat”) scales well with high-performance computing by means profiling of workloads, and for file-sys- the number of metadata server for both of synthetic I/O benchmarks and real- tem monitoring. Successfully tested, BeeGFS and Lustre with advantages for world, I/O-intensive workloads from the tools can be subsequently rolled out the former. In contrast, the “file creation” different scientific fields on our storage on the larger production supercomputer performance scales weakly, and is simi- testbed. system with hundreds of users. lar for both file systems.

100 2016 Annual Report THE I/O TESTBED

The I/O testbed consists of five file servers each equipped with one Intel Xeon Haswell processor E5-1620v3, 64 GB RAM, a hardware RAID controller with two 120 GB SSDs and 10 SAS drives of 2 TiB capacity each for the parallel file system, and two FDR InfiniBand cards for fast network connections. The testbed also contains eight compute nodes with two Intel Xeon Haswell E5-2620v3 processors, 64 GB RAM, and one InfiniBand card. The operating system is CentOS 7. The testbed is integrated into the InfiniBand fabric of the Cray TDS and HLRN-III production IMPROVING I/O system “Konrad” so that scaling experiments with a large number of nodes can be realized. PERFORMANCE BY 14 BURST BUFFER A burst buffer is an additional storage ing bandwidth for writing and reading Scaling of Metadata Performance on I/O Test Bed layer between the node memory and data on Cray DataWarp is twice as much 500,000 the disk-based parallel file system that compared to the Lustre performance BeeGFS File stat 450,000 Lustre File stat accelerates I/O operations of the appli- (see figure b ). But can real-world I/O- 400,000 BeeGFS File creation Lustre File creation 350,000 cations on the supercomputer system. intensive workloads benefit from a burst 300,000 Nowadays, implemented as nonvolatile buffer? 250,000 storage using SSDs, it provides a signifi- 200,000 To answer that question, we ran 150,000 cant performance boost for a broad range TURBOMOLE (reference 3), a well- Metadata Performance [1/s] 100,000 of file-access patterns – from small-sized 50,000 known ab-initio quantum chemistry block I/O up to streaming high-volume program package. We selected the 0 1 2 3 4 5data sets. Number of Active Metadata Servers TURBOMOLE implementation of the a We used the Cray DataWarp technology RI-MP2 method which is often used by which implements a burst buffer as scientists on the HLRN system. This network-attached memory. In the Cray method includes I/O-intensive opera- Synthetic IOR Benchmark Lustre versus Cray DataWarp XC system, a pool of fast SSDs configured tions on temporary data. Within our 20 in dedicated I/O nodes are made acces- benchmark, almost 80 GiB were written 18 sible on each compute node via the Aries to the Lustre file system and/or the burst 16 14 network as one additional SSD-based tier buffer, respectively, and read back. The 12 with a high aggregated bandwidth. measured wall times clearly indicated 10 the advantage of the burst-buffer tech- 8 We compared the performance of the Bandwidth [GiB/s] 6 nology for the I/O pattern in the workload DataWarp read Cray DataWarp technology with our 4 DataWarp write as shown in figure c . For all test cases, 2 Lustre read disk-based Lustre file system. We used a Lustre write the run time with scratch I/O on SSD- 0 synthetic benchmark and a typical HPC 2 4 8 16 32 64 128 based burst buffers is significantly lower workload. CPU Cores than on disk-based Lustre file systems. b First, we measured the scaling of the The performance gain is more pro- streaming bandwidth using a synthetic nounced for higher node counts, which Application Benchmark TURBOMOLE Lustre versus Cray DataWarp and widely known IOR benchmark. is important for scalable workloads. With 128 processor cores, the stream- 10,000 Lustre 9,000 DataWarp 8,000 7,000 6,000 5,000 14 a Scaling of metadata performance for the “file stat” and “file creation” 4,000 Total Time [s] 3,000 operation as function of the number of active metadata servers. 2,000 1,000 b Streaming performance for read and write access to the disk-based Lustre file 0 1 2 4 8 16 system and Cray DataWarp burst buffer. CPU Cores (1 per Node)

c Total runtimes of the RI-MP2 method as implemented in TURBOMOLE using up c to 16 nodes by performing the scratch I/O on the Lustre file system or on the burst buffer (Cray DataWarp), respectively.

Zuse Institute Berlin 101 Supercomputing at the Limit

A FUTURE WITH OPEN POSSIBILITIES

Innovative data-processing technologies such as many-core CPUs and GPUs provide the capabilities to match demands for computational power and lower energy consumption. The key architec- tural features of CPUs are wider SIMD units – up to 512 bit – or wide SIMT (Single Instruction Multiple Threads) in GPUs. To exploit this performance potential, existing legacy codes need to be adapted in code-modernization efforts. Developing best-practice solutions in collaboration with a strong community of early adopters (e.g. the Intel Xeon Phi User’s Group), are essential so that domain scientists can design state-of- the-art simulations codes.

102 2016 Annual Report

ZIB Publications ZIB PUBLICATIONS

PEER-REVIEWED (116)

Aimi Abass, Philipp Gutsche, Bjorn Maes, Time Dependent Shortest Path Problems on Ralf Borndörfer, Markus Reuther, Thomas Carsten Rockstuhl, Emiliano R Martins (2016). Airway Networks Using Super-Optimal Wind. Schlechte, Kerstin Waas, Steffen Weider Insights into directional scattering: from cou- In 16th Workshop on Algorithmic Approaches (2016). Integrated Optimization of Rolling pled dipoles to asymmetric dimer nanoanten- for Transportation Modelling, Optimization, Stock Rotations for Intercity Railways. Trans- nas. Opt. Express, 24(17):19638-19650. and Systems (ATMOS 2016), volume 54 of portation Science, 50(3):863-877. OpenAccess Series in Informatics (OASIcs). Carlo Barth, Jürgen Probst, Sven Herrmann, Ralf Borndörfer, Sebastian Schenker, Mar- Martin Hammerschmidt, Christiane Becker Ralf Borndörfer, Armin Fügenschuh, Torsten tin Skutella, Timo Strunk (2016). PolySCIP. In (2016). Numerical characterization of symme- Klug, Thilo Schang, Thomas Schlechte, Hanno G.-M. Greuel, Thorsten Koch, Peter Paule, try properties for photonic crystals with hex- Schülldorf (2016). The Freight Train Routing Andrew Sommese, editors, Mathematical agonal lattice. Proc. SPIE, 9885:988506. Problem for Congested Railway Networks Software – ICMS 2016, 5th International Con- with Mixed Traffic. Transportation Science. ference, Berlin, Germany, July 11-14, 2016, Carlo Barth, Sven Burger, Christiane Becker Proceedings, volume 9725 of Lecture Notes in (2016). Symmetry-Dependency of Anticross- Ralf Borndörfer, Guillaume Sagnol, Thomas Computer Science, pages 259-264. ing Phenomena in Slab-Type Photonic Crys- Schlechte, Elmar Swarat (2016). Optimal duty tals. Opt. Express, 24:10931. rostering for toll enforcement inspectors. Ralf Borndörfer, Heide Hoppmann, Marika Annals of Operations Research. Karbstein, Fabian Löbel (2016). The Modulo Gernot Bauer, Nadezhda Gribova, Alexander Network Simplex with Integrated Passenger Lange, Christian Holm, Joachim Gross (2016). Ralf Borndörfer, Heide Hoppmann, Marika Routing. In Operations Research Proceedings Three-body effects in triplets of capped gold Karbstein (2016). Passenger routing for peri- 2016. (accepted for publication on 2016-10-21) nanocrystals. Molecular Physics, 1-10. odic timetable optimization. Public Transport. Christopher Brandt, Christoph von Tycowicz, Daniel Baum, Jürgen Titschack (2016). Cavity Ralf Borndörfer, Heide Hoppmann, Marika Klaus Hildebrandt (2016). Geometric Flows of and Pore Segmentation in 3-D Images with Karbstein (2016). Separation of Cycle Inequal- Curves in Shape Space for Processing Motion Ambient Occlusion. In EuroVis 2016 – Short ities for the Periodic Timetabling Problem. In of Deformable Objects. Computer Graphics Papers. 24th Annual European Symposium on Algo- Forum, 35(2). rithms (ESA 2016), volume 57 of Leibniz Inter- Isabel Beckenbach, Ralf Borndörfer (2016). An national Proceedings in Informatics (LIPIcs). Karl-Kiên Cao, Ambros Gleixner, Matthias Approximation Result for Matchings in Parti- Miltenberger (2016). Methoden zur Reduktion tioned Hypergraphs. In Operations Research Ralf Borndörfer, Guillaume Sagnol, Stephan der Rechenzeit linearer Optimierungsmo- Proceedings 2014, 31-36. Schwartz (2016). An Extended Network Inter- delle in der Energiewirtschaft – Eine Perfor- diction Problem for Optimal Toll Control. In mance-Analyze. In EnInnov 2016: 14. Sympo- sium Energieinnovation 2016 Kaspar Bienefeld, Fred Zautke, Pooja Gupta INOC 2015 – 7th International Network Opti- . (2016). A novel method for undisturbed long- mization Conference, volume 52 of Electronic term observation of the honey bee (Apis melli- Notes in Discrete Mathematics, pages 301- Tim Conrad, Xintian You (2016). Acfs: accurate fera) behaviour – illustrated by the hygienic 308. circRNA identification and quantification from behaviour towards Varroa infestation. Journal NGS data. Nature Scientiﬁc Reports, 6. of Apicultural Research, 54(5):541-547. Ralf Borndörfer, Marika Karbstein, Julika Mehr gahrdt, Markus Reuther, Thomas Natasa Djurdjevac Conrad, Marcus Weber, Marco Blanco, Ralf Borndörfer, Nam Dung Schlechte (2016). The Cycle Embedding Prob- Christof Schütte (2016). Finding dominant Hoang, Anton Kaier, Adam Schienle, Thomas lem. In Operations Research Proceedings structures of nonreversible Markov pro- Schlechte, Swen Schlobach (2016). Solving 2014, 465-472. cesses. Multiscale Modeling and Simulation, 14(4):1319-1340.

104 2016 Annual Report Marta Costa, James D. Manton, Aaron D. Rainald Ehrig, Thomas Dierkes, Stefan Schäfer, Patrick Gelß, Sebastian Matera, Christof Ostrovsky, Steffen Prohaska, Gregory S. X. Susanna Röblitz, Enrico Tronci, Toni Mancini, Schütte (2016). Solving the master equation E. Jefferis (2016). NBLAST: Rapid, Sensitive Ivano Salvo, Vadim Alimguzhin, Federico Mari, without kinetic Monte Carlo: Tensor train Comparison of Neuronal Structure and Con- Igor Melatti, Annalisa Massini, Tillmann H. C. approximations for a CO oxidation model. struction of Neuron Family Databases. Neu- Krüger, Marcel Egli, Fabian Ille, Brigitte Leen- Journal of Computational Physics, 314:489- ron, 91(2):293-311. ers (2016). An integrative approach for model 502. driven computation of treatments in repro- Fabio D’Andreagiovanni, Ambros Gleixner ductive medicine. In BIOMAT – Proceedings Ambros M. Gleixner, Timo Berthold, Benjamin (2016). Towards an accurate solution of wire- of the 15th International Symposium on Math- Müller, Stefan Weltge (2016). Three Enhance- less network design problems. Proceedings of ematical and Computational Biology, Rorkee, ments for Optimization-Based Bound Tight- the 4th International Symposium on Combina- India. ening. Journal of Global Optimization. torial Optimization (ISCO). David Eisenhauer, Klaus Jäger, Grit Köppel, Ambros M. Gleixner, Daniel E. Steffy, Kati Fabio D’Andreagiovanni, Fabian Mett, Jonad Bernd Rech, Christiane Becker (2016). Opti- Wolter (2016). Iterative Refinement for Linear Pulaj (2016). An (MI)LP-based Primal Heu- cal Properties of Smooth Anti-reflective Programming. INFORMS Journal on Comput- ristic for 3-Architecture Connected Facility Three-dimensional Textures for Silicon Thin- ing, 28(3):449-464. Location in Urban Access Network Design. film Solar Cells. Energy Procedia, 102:27-35. In EvoApplications: European Conference on Leonid Goubergrits, Jan Osman, Ricardo the Applications of Evolutionary Computation. J. Fajerski, M. Noack, A. Reinefeld, F. Schintke, Mevert, Ulrich Kertzscher, Kai Pöthkow, Applications of Evolutionary Computation. T. Schütt, T. Steinke (2016). Fast In-Memory Hans-Christian Hege (2016). Turbulence in 19th European Conference, EvoApplications Checkpointing with POSIX API for Legacy blood damage modeling. The International 2016, Porto, Portugal, March 30 – April 1, 2016, Exascale-Applications. In SPPEXA Symposium Journal of Artiﬁcial Organs, 39(4):147-210. Proceedings, Part I , volume 9597 of Lecture 2016. Notes in Computer Science , pages 283-298. Carl Martin Grewe, Stefan Zachow (2016). Bálint Fodor, Peter Kozma, Sven Burger, Fully Automated and Highly Accurate Fabio D’Andreagiovanni, Giovanni Felici, Miklos Fried, Peter Petrik (2016). Effective Dense Correspondence for Facial Surfaces. Fabrizio Lacalandra (2016). Multiband Robust medium approximation of ellipsometric Computer Vision – ECCV 2016 Workshops, Optimization for optimal energy offering response from random surface roughness 9914:552-568. under price uncertainty. In Proc. of ROADEF simulated by finite-element method. Thin Solid Films 2016. , 617:20. Boris Grimm, Ralf Borndörfer, Markus Reuther, Thomas Schlechte, Stanley Schade (2016). M. N. Dean, R. Seidel, D. Knötel, K. Lyons, D. Claudia Färber, Jürgen Titschack, Christine Regularity patterns for rolling stock rotation Baum, J. C. Weaver, P. Fratzl (2016). To build a H. L. Schönberg, Karsten Ehrig, Karin Boos, optimization. In 8th International Conference shark: 3-D tiling laws of tessellated cartilage. Daniel Baum, Bernd Illerhaus, Ulla Asgaard, on Applied Operational Research, Proceed- In Abstract in Integrative and Comparative Richard G. Bromley, André Freiwald, Max ings, volume 8 of Lecture Notes in Manage- Biology; conference Society of Integrative and Wisshak (2016). Long-term macrobioero- ment Science, pages 28-32. Comparative Biology annual meeting, Janu- sion in the Mediterranean Sea assessed by ary 3-7, 2016, Portland, USA Biogeosci- , 56 (suppl 1). micro-computed tomography. Philipp Gutsche, Raquel Mäusle, Sven Burger ences , 13(11):3461-3474. (2016). Locally Enhanced and Tunable Optical Jason Dunlop, Dmitry Apanaskevich, Jens Chirality in Helical Metamaterials. Photonics, Lehmann, Rene Hoffmann, Florian Fusseis, Gerald Gamrath, Thorsten Koch, Stephen 3:60. Moritz Ehlke, Stefan Zachow, Xianghui Xiao Maher, Daniel Rehfeldt, Yuji Shinano (2016). (2016). Microtomography of the Baltic amber SCIP-Jack – A solver for STP and variants with Philipp Gutsche, Raquel Mäusle, Sven Burger Mathematical Pro- tick Ixodes succineus reveals affinities with the parallelization extensions. (2016). Tailoring local optical chirality in helical gramming Computation modern Asian disease vector Ixodes ovatus. , 1-66. metamaterials. In 2016 10th International Con- BMC Evolutionary Biology, 16(1). gress on Advanced Electromagnetic Materials in Microwaves and Optics, IEEE, pages 73-75.

Zuse Institute Berlin 105 ZIB Publications

Philipp Gutsche, Lisa V. Poulikakos, Martin Gregor Hendel (2016). Exploiting Solving Nikolay Ledentsov, Jr., Vitaly A. Shchukin, Hammerschmidt, Sven Burger, Frank Schmidt Phases for Mixed-Integer Programs. In Oper- Jörg-R. Kropp, Sven Burger, Frank Schmidt, (2016). Time-harmonic optical chirality in inho- ations Research Proceedings 2015, 3-9. Nikolay N. Ledentsov (2016). Direct visual- mogeneous space. Proc. SPIE, 9756:97560X. ization of the in-plane leakage of high-order Gunter Hermann, Vincent Pohl, Jean Chris- transverse modes in vertical-cavity sur- Tobias Günther, Alexander Kuhn, Hans-Chris- tophe Tremblay, Beate Paulus, Hans-Christian face-emitting lasers mediated by oxide-aper- tian Hege, Markus Gross, Holger Theisel Hege, Axel Schild (2016). ORBKIT – A modular ture engineering. Proc. SPIE, 9766:976608. (2016). Progressive Monte-Carlo Rendering Python toolbox for cross-platform post-proof Atmospheric Flow Features Across Scales. cessing of quantum chemical wavefunction Leonardo Agudo Jácome, Hans-Christian In 69th Annual Meeting of the APS Division of data. Journal of Computational Chemistry, Hege, Olaf Paetsch, Kai Pöthkow (2016). 3-D Fluid Dynamics, Gallery of Fluid Motion, Nov 37(16):1511-1520. Reconstruction, Visualization and Quantifica- 20-22, 2016, Portland, OR, USA. tion of Dislocations from Transmission Elec- Katharina Heye, Dennis Becker, Christian tron Microscopy Stereo-Pairs. In Microscopy Tobias Günther, Alexander Kuhn, Hans-Chris- Lütke-Eversloh, Vedat Durmaz, Thomas A. and Microanalysis 2016, July 24-28 Columbus, tian Hege, Holger Theisel (2016). MCFTLE: Ternes, Matthias Oetken, Jörg Oehlmann Ohio. Monte Carlo Rendering of Finite-Time Lya- (2016). Effects of carbamazepine and two punov Exponent Fields. Computer Graphics of its metabolites on the non-biting midge Klaus Jäger, Martin Hammerschmidt, Grit Forum, 35(3):381-390. Chironomus riparius in a sediment full life Köppel, Sven Burger, Christiane Becker cycle toxicity test. Water Research, 98:19-27. (2016). On Accurate Simulations of Thin-Film Martin Hammerschmidt, Sven Herrmann, Jan Solar Cells With a Thick Glass Superstrate. In Pomplun, Sven Burger, Frank Schmidt (2016). C. Hoppe, P. Obermeier, S. Mehlhans, M. Light, Energy and the Environment 2016, OSA Model order reduction for the time-harmonic Alchikh, L. Seeber, F. Tief, K. Karsch, X. Chen, Technical Digest, page PM3B.5. Maxwell equation applied to complex nano- S. Boettcher, S. Diedrich, T. Conrad (2016). structures. Proc. SPIE 9742, Physics and Simu- Innovative Digital Tools and Surveillance Klaus Jäger, Carlo Barth, Martin Hammer- lation of Optoelectronic Devices XXIV, 9742M. Systems for the Timely Detection of Adverse schmidt, Sven Herrmann, Sven Burger, Frank Events at the Point of Care: A Proof-of-Con- Schmidt, Christiane Becker (2016). Simulations Martin Hammerschmidt, Sven Herrmann, Jan cept Study. Drug Safety, 39(10):977-988. of sinusoidal nanotextures for coupling light Pomplun, Sven Burger, Frank Schmidt (2016). into c-Si thin-film solar cells. Opt. Express, Reduced basis method for electromagnetic Heide Hoppmann (2016). An Extended Formu- 24:A569. scattering problem: a case study for FinFETs. lation for the Line Planning Problem. In Oper- Optical and Quantum Electronics, 48:250. ations Research Proceedings 2015, 11-17. Klaus Jäger, Grit Köppel, Carlo Barth, Mar- tin Hammerschmidt, Sven Herrmann, Sven Martin Hammerschmidt, Carlo Barth, Jan Jesco Humpola, Felipe Serrano (2016). Suffi- Burger, Frank Schmidt, Christiane Becker Pomplun, Sven Burger, Christiane Becker, cient pruning conditions for MINLP in gas net- (2016). Sinusoidal gratings for optimized light Frank Schmidt (2016). Reconstruction of pho- work design. EURO Journal on Computational management in c-Si thin-film solar cells. Proc. tonic crystal geometries using a reduced basis Optimization. SPIE, 9898:989808. method for nonlinear outputs. Proc. SPIE 9756, Photonic and Phononic Properties of Engi- Nikolay Ledentsov, Jr., Vitaly A. Shchukin, Marika Karbstein (2016). Integrated Line Plan- neered Nanostructures VI, 97561R. Nikolay N. Ledentsov, Jörg-R. Kropp, Sven ning and Passenger Routing: Connectivity Burger, Frank Schmidt (2016). Direct Evidence and Transfers. In Operations Research Pro- Carsten Hartmann, Christof Schütte, Wei of the Leaky Emission in Oxide-Confined Ver- ceedings 2014, 263-269. Zhang (2016). Model reduction algorithms for tical Cavity Lasers . IEEE J. Quant. Electron., optimal control and importance sampling of 52(3):2400207. Jens Kasten, Jan Reininghaus, Ingrid Hotz, diffusions. Nonlinearity, 29(8):2298-2326. Hans-Christian Hege, Bernd R. Noack, Guil- laume Daviller, Marek Morzyński (2016). Accel- eration feature points of unsteady shear flows. Archives of Mechanics, 68(1):55-80.

106 2016 Annual Report Stefan Klus, Peter Koltai, Christof Schütte Michael Krone, Barbora Kozlíková, Norbert Robert Lemanis, Dieter Korn, Stefan Zachow, (2016). On the numerical approximation of Lindow, Marc Baaden, Daniel Baum, Julius Erik Rybacki, René Hoffmann (2016). The the Perron-Frobenius and Koopman operator. Parulek, Hans-Christian Hege, Ivan Viola Evolution and Development of Cephalopod Journal of Computational Dynamics, 3(1):51-77. (2016). Visual Analysis of Biomolecular Cav- Chambers and Their Shape. PLOS ONE, 11(3). ities: State of the Art. Computer Graphics Stefan Klus, Christof Schütte (2016). Towards Forum, 35(3):527-551. Robert Lemanis, Stefan Zachow, René Hoff- tensor-based methods for the numerical mann (2016). Comparative cephalopod shell approximation of the Perron-Frobenius and Olaf Krzikalla, Florian Wende, Markus Höh- strength and the role of septum morphology Koopman operator. Journal of Computational nerbach (2016). Dynamic SIMD Vector Lane on stress distribution. PeerJ, 4:e2434. Dynamics. Scheduling. In High Performance Comput- ing, ISC High Performance 2016 Interna- Lars Lubkoll, Anton Schiela, Martin Weiser James C Knight, Philip J Tully, Bernhard A tional Workshops, ExaComm, E-MuCoCoS, (2016). An affine covariant composite step Kaplan, Anders Lansner, Steve Furber (2016). HPC-IODC, IXPUG, IWOPH, P^3MA, VHPC, method for optimization with PDEs as equality Large-scale simulations of plastic neural net- WOPSSS, volume 9945 of LNCS, pages 354- constraints. Optimization Methods and Soft- works on neuromorphic hardware. Frontiers in 365. ware. Neuroanatomy, 10:37. Hans Lamecker, Stefan Zachow (2016). Sta- Stephen J. Maher, John M. Murray (2016). The Peter Koltai, Giovanni Ciccotti, Christof tistical Shape Modeling of Musculoskeletal unrooted set covering connected subgraph Schütte (2016). On metastability and Markov Structures and Its Applications. In Computa- problem differentiating between HIV enve- state models for non-stationary molecular tional Radiology for Orthopaedic Interven- lope sequences. European Journal of Opera- dynamics. The Journal of Chemical Physics, tions, volume 23 of Lecture Notes in Compu- tional Research, 248(2):668-680. 145(174103). tational Vision and Biomechanics, pages 1-23, Springer. Stephen Maher, Matthias Miltenberger, Joao Barbora Kozlíková, Michael Krone, Martin Pedro Pedroso, Daniel Rehfeldt, Robert Falk, Norbert Lindow, Marc Baaden, Daniel Itamar D. Landau, Robert Egger, Vincent J. Schwarz, Felipe Serrano (2016). PySCIPOpt: Baum, Ivan Viola, Julius Parulek, Hans-Chris- Dercksen, Marcel Oberlaender, Haim Som- Mathematical Programming in Python with tian Hege (2016). Visualization of Biomolecular polinsky (2016). The Impact of Structural Het- the SCIP Optimization Suite. In Mathematical Structures: State of the Art Revisited. Com- erogeneity on Excitation-Inhibition Balance in Software – ICMS 2016, volume 9725 of Lecture puter Graphics Forum. Cortical Networks. Neuron, 92(5):1106-1121. Notes in Computer Science, pages 301-307.

Tobias Kramer, Christoph Kreisbeck, Chris- Alexander Lange (2016). Reconstruction of Anirban Mukhopadhyay, Arun Kumar, Suchen- tian Riha, Olivio Chiatti, Sven Buchholz, disease transmission rates: applications to dra Bhandarkar (2016). Joint Geometric Andreas Wieck, Dirk Reuter, Saskia Fischer measles, dengue, and influenza. Journal of Graph Embedding for Partial Shape Matching (2016). Thermal energy and charge currents Theoretical Biology, 400:138-153. in Images. IEEE Winter Conference on Appli- in multi-terminal nanorings. AIP Advances, cations of Computer Vision, 1-9. 6:065306. Alexander Lange, Julia Plöntzke, Stefan Schäfer, Susanna Röblitz (2016). Follicular Anirban Mukhopadhyay, Fatih Porikli, Suchen- Tobias Kramer, Matthias Noack (2016). On maturation in cows: mathematical models and dra Bhandarkar (2016). Detection and Charac- the origin of inner coma structures observed data. In 10. European Conference on Mathe- terization of Intrinsic Symmetry of 3-D Shapes. by Rosetta during a diurnal rotation of comet matical and Theoretical Biology. Proceedings of IEEE International Conference 67P/Churyumov-Gerasimenko. The Astro- on Pattern Recognition. physical Journal Letters, 823(1):L11. Markus Leitner, Ivana Ljubic, Markus Sinnl, Axel Werner (2016). ILP heuristics and a new Anirban Mukhopadhyay (2016). Total Variation Tobias Kramer, Mirta Rodriguez, Yaroslav exact method for bi-objective 0/1 ILPs: Appli- Random Forest: Fully automatic MRI segmen- Zelinskyi (2016). Modeling of Transient cation to FTTx-network design. Computers & tation in congenital heart disease. RAMBO Absorption Spectra in Exciton Charge-Trans- Operations Research. 2016, HVSMR 2016: Reconstruction, Segmen- fer Systems. Journal of Physical Chemistry B. tation, and Analysis of Medical Images, LNCS 10129:165-171.

Zuse Institute Berlin 107 ZIB Publications

Anirban Mukhopadhyay, Oscar Morillo, Stefan J. Reimers, M. Biczysko, D. Bruce, D.F. Coker, Robert Schmidtke, Guido Laubender, Thomas Zachow, Hans Lamecker (2016). Robust and T.J. Frankcombe, H. Hashimoto, J. Hauer, Steinke (2016). Big Data Analytics on Cray XC Accurate Appearance Models Based on Joint R. Jankowiak, Tobias Kramer, J. Linnanto, F. Series DataWarp using Hadoop, Spark and Dictionary Learning Data from the Osteoar- Mamedov, F. Müh, M. Rätsep, T. Renger, S. Flink. In CUG Proceedings. thritis Initiative. Lecture Notes in Computer Styring, J. Wan, Z. Wang, Z.-Y. Wang-Otomo, Science, Patch-Based Techniques in Medical Y.-X. Weng, C. Yang, J.-P. Zhang, A. Freiberg, Peter Schnauber, Alexander Thoma, Chris- Imaging. Patch-MI 2016, 9993:25-33. E. Krausz (2016). Challenges facing an under- toph V. Heine, Alexander Schlehahn, Liron standing of the nature of low-energy excited Gantz, Manuel Gschrey, Ronny Schmidt, Cas- Adam Nielsen (2016). The Monte Carlo Com- states in photosynthesis. BBA Bioenergetics, par Hopfmann, Benjamin Wohlfeil, Jan-Hin- putation Error of Transition Probabilities. Sta- 1857(9):1627-1640. drik Schulze, Andre Strittmatter, Tobias tistics & Probability Letters, 118:163-170. Heindel, Sven Rodt, Ulrike Woggon, David Beate Rusch, Markus Putnings (2016). Deep- Gershoni, Stephan Reitzenstein (2016). Bright P. Obermeier, S. Muehlhans, Ch. Hoppe, K. Green – Entwicklung eines rechtssicheren Single-Photon Sources Based on Anti-Re- Karsch, F. Tief, L. Seeber, X. Chen, T. Con- Workflows zur effizienten Umsetzung der flection Coated Deterministic Quantum Dot rad, S. Boettcher, S. Diedrich, B. Rath (2016). Open-Access-Komponente in den Allianz-Li- Microlenses. Technologies, 4(1):1. Enabling Precision Medicine With Digital zenzen für die Wissenschaft. In Klaus-Rainer Case Classification at the Point-of-Care. EBio- Brintzinger, Ulrich Hohoff, Ulrike Scholle, Stefan Schäfer, Julia Plöntzke, Susanna Medicine, 4:191-196. Thomas Stäcker, Helge Steenweg, Heidrun Röblitz (2016). Mathematical Modelling of Wiesenmüller, editors, o-bib. Das oﬀene Bib- Follicular Maturation in Cows and Women. In liotheksjournal / herausgegeben vom VDB, Jan Pomplun, Sven Burger, Lin Zschied- 49. Jahrestagung der Physiologie und Patho- 4/2016, volume 4/2016 of o-bib rich, Philipp Gutsche, Frank Schmidt (2016). , pages 110-118. logie der Fortpﬂanzung und gleichzeitig 41. Method for fast computation of angular light Veterinär-Humanmedizinische Gemeinschaft- scattering spectra from 2-D periodic arrays. Manish Sahu, Daniil Moerman, Philip Mewes, stagung 2016, Leipzig. Proc. SPIE, 9778:977839. Peter Mountney, Georg Rose (2016). Instru- ment State Recognition and Tracking for Borong Shao, Tim Conrad (2016). Epithelial Lisa Poulikakos, Philipp Gutsche, Kevin Effective Control of Robotized Laparoscopic Mesenchymal Transition Regulatory Net- International Journal of Mechanical McPeak, Sven Burger, Jens Niegemann, Chris- Systems. work-based Feature Selection in Lung Cancer Engineering and Robotics Research tian Hafner, David Norris (2016). The Optical , 5(1):33-38. Prognosis Prediction. Lecture Notes in Com- Chirality Flux as a Useful Far-Field Probe of puter Science (LNCS), 9656:1235-146. Chiral Near Fields. ACS Photonics, 3:1619. Stanley Schade, Martin Strehler (2016). The Maximum Flow Problem for Oriented Flows. Yuji Shinano, Tobias Achterberg, Timo Ber- In Marc Goerigk, Renato Werneck, editors, thold, Stefan Heinz, Thorsten Koch, Michael 16th Workshop on Algorithmic Approaches for Winkler (2016). Solving Open MIP Instances Transportation Modelling, Optimization, and with ParaSCIP on Supercomputers using up Systems (ATMOS 2016), volume 54 of OpenAc- to 80,000 Cores. In Proc. of 30th IEEE Inter- cess Series in Informatics (OASIcs), pages 1-13. national Parallel & Distributed Processing Symposium.

108 2016 Annual Report Victor Soltwisch, Anton Haase, Jan Wernecke, Jürgen Titschack, Hiske G. Fink, Daniel Baum, Stefanie Winkelmann, Christof Schütte (2016). Jürgen Probst, Max Schoengen, Sven Burger, Claudia Wienberg, Dierk Hebbeln, André The Spatiotemporal Master Equation: Approx- Michael Krumrey, Frank Scholze (2016). Cor- Freiwald (2016). Mediterranean cold-water imation of Reaction-Diffusion Dynamics via related diffuse x-ray scattering from period- corals – an important regional carbonate fac- Markov State Modeling. Journal of Chemical ically nanostructured surfaces. Phys. Rev. B, tory? The Depositional Record, 2(1):74-96. Physics, 145(21). 94:035419. Iliusi Vega, Christof Schütte, Tim Conrad Stefanie Winkelmann (2016). Markov Control Gunther Sprösser, Sebastian Schenker, (2016). Finding metastable stated in real-world with Rare State Observation: Average Opti- Andreas Pittner, Ralf Borndörfer, Michael time series with recurrence networks. Physica mality. Markov Processes and Related Fields. Rethmeier, Ya-Ju Chang, Matthias Finkbeiner A: Statistical Mechanics and its Applications, (accepted for publication on 2016-08-29) (2016). Sustainable Welding Process Selection 445:1-17. based on Weight Space Partitions. In Procedia Benjamin Wohlfeil, Georg Rademacher, Chris- CIRP, volume 40 of 13th Global Conference José Villatoro, Martin Zülke, Daniel Riebe, tos Stamatiadis, Karsten Voigt, Lars Zimmer- on Sustainable Manufacturing – Decoupling Toralf Beitz, Marcus Weber, Jens Riedel, mann, Klaus Petermann (2016). A Two Dimen- Growth from Resource Use, pages 127-132. Hans-Gerd Löhmannsröben (2016). IR-MALDI sional Fiber Grating Coupler on SOI for Mode ion mobility spectrometry: physical source Division Multiplexing . IEEE Photon. Technol. Andreas Spötter, Pooja Gupta, Manfred characterization and application as HPLC Lett., 28:1241. Mayer, Norbert Reinsch, Kaspar Bienefeld detector. International Journal for Ion Mobility (2016). Genome-wide association study of a Spectrometry, 197-297. Robert Zahn, Sarah Grotjohann, Heiko Ramm, Varroa-specific defense behavior in honey- Stefan Zachow, Matthias Pumberger, Michael bees (Apis mellifera). Journal of Heredity, 107 C. Weinhold, A. Lackorzynski, J. Bierbaum, Putzier, Carsten Perka, Stephan Tohtz (2016). (3):220-227. M. Küttler, M. Planeta, H. Härtig, A. Shiloh, Influence of pelvic tilt on functional acetabular E. Levy, T. Ben-Nun, A. Barak, T. Steinke, T. orientation. Technology and Health Care, 1-9. Alexander Tesch (2016). A Nearly Exact Prop- Schütt, J. Fajerski, A. Reinefeld, M. Lieber, W. agation Algorithm for Energetic Reasoning in E. Nagel (2016). FFMK: A Fast and Fault-toler- Thomas Zander, Marcel Dreischarf, Anne-Ka- O(n^2 log n). In International Conference on ant Microkernel-based System for Exascale trin Timm, Wolfgang Baumann, Hendrik Principles and Practice of Constraint Program- Computing. In SPPEXA Symposium 2016. Schmidt (2016). Impact of material and mor- ming (CP 2016), volume 22 of Lecture Notes in phological parameters on the mechanical Computer Science, pages 493-519. Florian Wende, Martijn Marsman, Thomas response of the lumbar spine – A finite ele- Steinke (2016). On Enhancing 3-D-FFT Perfor- ment sensitivity study. Journal of Biomechan- F. Tief, Ch. Hoppe, L. Seeber, P. Obermeier, X. mance in VASP. In CUG Proceedings. ics, 53:185-190. Chen, K. Karsch, S. Muehlhans, E. Adamou, T. Conrad, B. Schweiger, T. Adam, B. Rath (2016). Florian Wende, Matthias Noack, Thomas L. Krebek, von, A. Achazi, M. Solleder, M. An inception cohort study assessing the role Steinke, Michael Klemm, Georg Zitzlsberger, Weber, B. Paulus, C. Schalley (2016). Allosteric of bacterial co-infections in children with Chris J. Newburn (2016). Portable SIMD Per- and Chelate Cooperativity in Divalent influenza and ILI and a clinical decision model formance with OpenMP* 4.x Compiler Direc- Crown Ether–Ammonium Complexes with for stringent antibiotic use. Antiviral Therapy, tives. In Pierre-Francois Dutot, Denis Trys- Strong Binding Enhancements. Chem. Eur. J., 21:413-424. tram, editors, , volume Euro-Par 2016: Parallel 22(43):15475-15484. Processing: 22nd International Conference on Parallel and Distributed Computing of LNCS.

Zuse Institute Berlin 109 ZIB Publications

NOT PEER- REVIEWED (17)

Pascal-Nicolas Becker, Roland Bertelmann, Sebastian Götschel, Christiane Maierhofer, Beate Rusch, Roland Bertelmann, Heinz Klaus Ceynowa, Jürgen Christof, Thomas Jan Müller, Nick Rothbart, Martin Weiser Pampel, Pamela Aust, Kerstin Helbig, Anja Dierkes, Julia Alexandra Goltz, Matthias Groß, (2016). Quantitative Defect Reconstruction Doreen Müller, Steffi Conrad-Rempel, Signe Regina Heidrich, Tobias Höhnow, Michael Kas- in Active Thermography for Fiber-Reinforced Weihe, Thomas Dierkes, Rainer Kuhlen, Dag- sube, Thorsten Koch, Monika Kuberek, Lilian Composites. In Proceedings 19th World Con- mar Schobert, Michaela Voigt, Paul Klimpel, Landes, Heinz Pampel, Markus Putnings, ference on Non-Destructive Testing (WCNDT Christina Riesenweber, Andreas Kennecke, Beate Rusch, Hildegard Schäffler, Dagmar 2016). Mario Kowalak, Birgit Schlegel, Niels Tau- Schobert, Oliver Schwab, Jens Schwidder, bert, Andreas Hübner, Katja Mruck (2016). Konstanze Söllner, Tonka Stoyanova, Paul Helene Hahn (2016). Kooperativ in die digitale KOBV-Sonderedition zur International Open Vierkant (2016). Questionnaire for effective Zeit – wie öffentliche Kulturinstitutionen Cul- Access Week 2016 “Open in Action”. exchange of bibliographic metadata – current tural Commons fördern. digiS – Servicestelle status of publishing houses. Digitalisierung Berlin, editor. Beate Rusch, Rita Albrecht, Gabriele Meßmer, Peter Thiessen (2016). Katalogisierung in der Florian Bernard, Luis Salamanca, Johan Thun- Klaus Jäger, Carlo Barth, Martin Hammer- Datenwolke. Zeitschrift für Bibliothekswe- berg, Alexander Tack, Dennis Jentsch, Hans schmidt, Sven Herrmann, Sven Burger, Frank sen und Bibliographie, Jahrgang 63 (Heft Lamecker, Stefan Zachow, Frank Hertel, Jorge Schmidt, Christiane Becker (2016). Sinusoidal 5-6):258-264. Goncalves, Peter Gemmar (2016). Shape- Nanotextures for Enhanced Light Manage- aware Surface Reconstruction from Sparse ment in Thin-Film Solar Cells. In European Guillaume Sagnol, Hans-Christian Hege, Data. arXiv, 1602.08425v1. Society for Quantum Solar Energy Conver- Martin Weiser (2016). Using sparse kernels to sion, editor, 28th Workshop on Quantum Solar design computer experiments with tunable Timo Berthold, James Farmer, Stefan Heinz, Energy Conversion – (QUANTSOL). precision. In 22nd Intern. Conf. on Compu- Michael Perregaard (2016). Parallelization of tational Statistics – COMSTAT 2016, Oviedo, the FICO Xpress-Optimizer. In Mathemati- Han Cheng Lie, Tim Sullivan (2016). Cam- Spain, 23-26 August 2016, Proceedings ISBN cal Software – ICMS 2016, 5th International eron-Martin theorems for sequences of 978-90-73592-36-0, 397-408. Conference Berlin, Germany, July 11-14, 2016 Cauchy-distributed random variables. arXiv, Proceedings, 251-258. 1608.03784. Yuji Shinano, Timo Berthold, Stefan Heinz (2016). A First Implementation of ParaXpress: Hagen Chrapary, Yue Ren (2016). The Software Stefan Lohrum, Mathias Kratzer, Uwe Risch, Combining Internal and External Paralleliza- Portal swMATH: A State of the Art Report and Peter Thiessen (2016). Zum Stand des Projek- tion to Solve MIPs on Supercomputers. In Next Steps. In Mathematical Software – ICMS tes “Cloudbasierte Infrastruktur für Biblio- Mathematical Software – ICMS 2016, 5th 2016, 5th International Conference Berlin, Ger- theksdaten” (CIB). Zeitschrift für Bibliotheks- International Conference Berlin, Germany, many, July 11-14, 2016 Proceedings, 397-402. wesen und Bibliographie, Jahrgang 63 (Heft July 11-14, 2016 Proceedings, 308-316. 5-6):250-257. Carl Martin Grewe, Lisa Schreiber (2016). Digi- Alexander Tesch (2016). Kompakte MIP tale Bildarchive. Archivierung und Codierung Anja Müller, Beate Rusch (2016). Was in der Modelle für das Ressourcenbeschränkte Pro- der Gefühle. In +ultra gestaltung schafft wis- Zwischenzeit geschah – vier Jahre und 49 Digi- jektplanungsproblem. OR News, 58:19-21. sen, 285-290, Seemann Henschel. talisierungsprojekte später: Förderprogramm Digitalisierung und Servicestelle Digitali- Sebastian Götschel, Christian Höhne, San- sierung Berlin (digiS) 2012 bis 2016. In Föder- jeevareddy Kolkoori, Steffen Mitzscherling, ale Vielfalt – Globale Vernetzung. Strategien Jens Prager, Martin Weiser (2016). Ray Trac- der Bundesländer für das kulturelle Erbe in der ing Boundary Value Problems: Simulation and digitalen Welt. SAFT Reconstruction for Ultrasonic Testing. In Proceedings 19th World Conference on Non-Destructive Testing (WCNDT 2016).

110 2016 Annual Report ZIB BOOKS (2) DISSERTATIONS (8) REPORTS (73)

Martin Weiser (2016). Inside Finite Elements. Animesh Agarwal (2016). Path Integral Tech- Tobias Achterberg, Robert E. Bixby, Zong- De Gruyter. niques in Molecular Dynamics Simulations of hao Gu, Edward Rothberg, Dieter Weninger Open Boundary Systems, Freie Universität (2016). Presolve Reductions in Mixed Integer Martin Greuel, Thorsten Koch, Peter Paule, Berlin. Programming. ZIB-Report 16-44. Andrew Sommese, editors (2016). Mathe- matical Software – ICMS 2016, 5th Int. Conf. Tomasz Badowski (2016). Adaptive importance Kilian Amrhein, Tim Hasler, Marco Klindt, Elias Berlin, Germany, July 11-14, 2016, Proceedings. sampling via minimization of estimators of Oltmanns, Wolfgang Peters-Kottig (2016). Springer. cross-entropy, mean square and inefficiency Digitale Langzeitarchivierung: Mustervorlage constants, Freie Universität Berlin. für Übernahmevereinbarungen. ZIB-Report 16-27. Adman Daragmeh (2016). Model Order Reduction of Linear Control Systems: Com- Daniel Baum, Jürgen Titschack (2016). Cavity parison of Balance Truncation and Singular and Pore Segmentation in 3-D Images with Perturbation Approximation with Application Ambient Occlusion. ZIB-Report 16-17. to Optimal Control, Freie Universität Berlin. Isabel Beckenbach, Robert Scheidweiler Raheem Gul (2016). Mathematical Modeling (2016). Perfect f-Matchings and f-Factors in and Sensitivity Analysis of Lumped Parameter Hypergraphs – A Combinatorial Approach. Model of the Human Cardiovascular System, ZIB-Report 16-22. Freie Universität Berlin. Isabel Beckenbach, Leon Eifler, Konstantin Han Cheng Lie (2016). On a strongly convex Fackeldey, Ambros Gleixner, Andreas Grever, approximation of a stochastic optimal control Marcus Weber, Jakob Witzig (2016). Mixed-In- problem for importance sampling of metasta- teger Programming for Cycle Detection in ble diffusions, Freie Universität Berlin. Non-reversible Markov Processes. ZIB-Re- port 16-39. Adam Nielsen (2016). Computation Schemes for Transfer Operators. Freie Universität Ber- Pascal-Nicolas Becker, Roland Bertelmann, lin. Klaus Ceynowa, Jürgen Christof, Thomas Dierkes, Julia Alexandra Goltz, Matthias Groß, Nisara Sriwattanaworachai (2016). Spectral Regina Heidrich, Tobias Höhnow, Michael Kas- approach to metastability of non-reversible sube, Thorsten Koch, Monika Kuberek, Lilian complex processes, Freie Universität Berlin. Landes, Heinz Pampel, Markus Putnings, Beate Rusch, Hildegard Schäffler, Dagmar Illiusi Vega (2016).Reconstruction and anal- Schobert, Oliver Schwab, Jens Schwidder, ysis of the state space for the identification Konstanze Söllner, Tonka Stoyanova, Paul Vier- of dynamical states in real-world time-series, kant (2016). DeepGreen – Metadata Schema Freie Universität Berlin. for the exchange of publications between publishers and open-access repositories. Ver- sion 1.1. June 2016. ZIB-Report 16-32.

Zuse Institute Berlin 111 ZIB Publications

Timo Berthold, Gregor Hendel, Thorsten Fabio D’Andreagiovanni (2016). Revisiting Ambros M. Gleixner, Timo Berthold, Benjamin Koch (2016). The Three Phases of MIP Solving. wireless network jamming by SIR-based con- Müller, Stefan Weltge (2016). Three Enhance- ZIB-Report 16-78. siderations and Multiband Robust Optimiza- ments for Optimization-Based Bound Tight- tion. ZIB-Report 15-12. ening. ZIB-Report 15-16. Marco Blanco, Ralf Borndörfer, Nam Dung Hoang, Anton Kaier, Thomas Schlechte, Swen Fabio D’Andreagiovanni, Ambros Gleixner Robert Lion Gottwald, Stephen J. Maher, Yuji Schlobach (2016). The Shortest Path Problem (2016). Towards an accurate solution of wire- Shinano (2016). Distributed domain propaga- with Crossing Costs. ZIB-Report 16-70. less network design problems. ZIB-Report tion. ZIB-Report 16-71. 16-12. Ralf Borndörfer, Oytun Arslan, Ziena Elija- Andreas Griewank, Tom Streubel, Richard zyfer, Hakan Güler, Malte Renken, Güvenc Rainald Ehrig, Thomas Dierkes, Stefan Schäfer, Hasenfelder, Manuel Radons (2016). Piecewise Sahin, Thomas Schlechte (2016). Line Planning Susanna Röblitz, Enrico Tronci, Toni Mancini, linear secant approximation via Algorithmic on Path Networks with Application to the Ivano Salvo, Vadim Alimguzhin, Federico Mari, Piecewise Differentiation. ZIB-Report 16-54. Istanbul Metrobüs. ZIB-Report 16-38. Igor Melatti, Annalisa Massini, Tillmann H. C. Krüger, Marcel Egli, Fabian Ille, Brigitte Leen- Philipp Gutsche, Matthias Läuter, Frank Ralf Borndörfer, Heide Hoppmann, Marika ers (2016). An Integrative Approach for Model Schmidt (2016). Parameter-dependent Parallel Karbstein, Fabian Löbel (2016). The Modulo Driven Computation of Treatments in Repro- Block Sparse Arnoldi and Döhler Algorithms Network Simplex with Integrated Passenger ductive Medicine. ZIB-Report 16-04. on Distributed Systems. ZIB-Report 16-15. Routing. ZIB-Report 16-43. Frank Fischer, Boris Grimm, Torsten Klug, Sebastian Götschel, Christiane Maierhofer, Ralf Borndörfer, Boris Grimm, Markus Thomas Schlechte (2016). A Re-optimization Jan P. Müller, Nick Rothbart, Martin Weiser Reuther, Thomas Schlechte (2016). Optimiza- Approach for Train Dispatching. ZIB-Report (2016). Quantitative Defect Reconstruction tion of Handouts for Rolling Stock Rotations 16-49. in Active Thermography for Fiber-Reinforced Visualization. ZIB-Report ZR-16-73. Composites. ZIB-Report 16-13. Gerald Gamrath, Ambros Gleixner, Thorsten Christopher Brandt, Christoph von Tycowicz, Koch, Matthias Miltenberger, Dimitri Kniasew, Sebastian Götschel, Christian Höhne, San- Klaus Hildebrandt (2016). Geometric Flows of Dominik Schlögel, Alexander Martin, Dieter jeevareddy Kolkoori, Steffen Mitzscherling, Curves in Shape Space for Processing Motion Weninger (2016). Tackling Industrial-Scale Jens Prager, Martin Weiser (2016). Ray Trac- of Deformable Objects. ZIB-Report 16-29. Supply Chain Problems by Mixed-Integer Pro- ing Boundary Value Problems: Simulation and gramming. ZIB-Report 16-45. SAFT Reconstruction for Ultrasonic Testing. Kevin K. H. Cheung, Ambros Gleixner, Daniel ZIB-Report 16-14. E. Steffy (2016). Verifying Integer Program- Gerald Gamrath, Tobias Fischer, Tristan ming Results. ZIB-Report 16-58. Gally, Ambros M. Gleixner, Gregor Hendel, Tobias Günther, Alexander Kuhn, Hans-Chris- Thorsten Koch, Stephen J. Maher, Matthias tian Hege, Holger Theisel (2016). MCFTLE: Jon Cockayne, Chris Oates, Tim Sullivan, Miltenberger, Benjamin Müller, Marc E. Monte Carlo Rendering of Finite-Time Lya- Mark Girolami (2016). Probabilistic Meshless Pfetsch, Christian Puchert, Daniel Rehfeldt, punov Exponent Fields. ZIB-Report 16-21. Methods for Partial Differential Equations and Sebastian Schenker, Robert Schwarz, Felipe Bayesian Inverse Problems. ZIB-Report 16-31. Serrano, Yuji Shinano, Stefan Vigerske, Dieter Martin Hammerschmidt, Sven Herrmann, Jan Weninger, Michael Winkler, Jonas T. Witt, Pomplun, Sven Burger, Frank Schmidt (2016). Jakob Witzig (2016). The SCIP Optimization Model order reduction for the time-harmonic Marta Costa, James D. Manton, Aaron D. Suite 3.2. ZIB-Report 15-60. Ostrovsky, Steffen Prohaska, Gregory S. X. E. Maxwell equation applied to complex nano- Jefferis (2016). NBLAST: Rapid, sensitive com- structures. ZIB-Report 16-05. parison of neuronal structure and construc- Gerald Gamrath, Thorsten Koch, Stephen J. tion of neuron family databases. ZIB-Report Maher, Daniel Rehfeldt, Yuji Shinano (2016). Martin Hammerschmidt, Sven Herrmann, Sven 16-34. SCIP-Jack – A solver for STP and variants with Burger, Jan Pomplun, Frank Schmidt (2016). parallelization extensions. ZIB-Report 16-41. Reduced basis method for the optimization of nano-photonic devices. ZIB-Report 16-10.

112 2016 Annual Report Martin Hammerschmidt, Carlo Barth, Jan Peter Koltai, Giovanni Ciccotti, Schütte Chris- Matthias Noack, Florian Wende, Georg Zit- Pomplun, Sven Burger, Christiane Becker, tof (2016). On metastability and Markov state zlsberger, Michael Klemm, Thomas Steinke Frank Schmidt (2016). Reconstruction of pho- models for non-stationary molecular dynam- (2016). KART – A Runtime Compilation Library tonic crystal geometries using a reduced basis ics. ZIB-Report 16-11. for Improving HPC Application Performance. method for nonlinear outputs. ZIB-Report ZIB-Report 16-48. 16-06. Tobias Kramer, Matthias Noack (2016). On the origin of inner coma structures observed Jonad Pulaj (2016). Cutting Planes for Fami- Kai Hennig, Robert Schwarz (2016). Using by Rosetta during a diurnal rotation of comet lies Implying Frankl’s Conjecture. ZIB-Report Bilevel Optimization to find Severe Transport 67P/Churyumov-Gerasimenko.. ZIB-Report 16-51. Situations in Gas Transmission Networks. 16-26. ZIB-Report 16-68. Ted Ralphs, Yuji Shinano, Timo Berthold, Thor- Michael Krone, Barbora Kozlikova, Norbert sten Koch (2016). Parallel Solvers for Mixed Benjamin Hiller, René Saitenmacher, Tom Lindow, Marc Baaden, Daniel Baum, Julius Integer Linear Programming. ZIB-Report Walther (2016). Analysis of operating modes Parulek, Hans-Christian Hege, Ivan Viola 16-74. of complex compressor stations. ZIB-Report (2016). Visual Analysis of Biomolecular Cavi- 16-61. ties: State of the Art. ZIB-Report 16-42. Daniel Rehfeldt, Thorsten Koch (2016). Trans- formations for the Prize-Collecting Steiner Heide Hoppmann (2016). An Extended Formu- Alexander Kuhn, Wito Engelke, Markus Flat- Tree Problem and the Maximum-Weight Con- lation for the Line Planning Problem. ZIB-Re- ken, Hans-Christian Hege, Ingrid Hotz (2016). nected Subgraph Problem to SAP. ZIB-Report port 16-08. Topology-based Analysis for Multimodal 16-36. Atmospheric Data of Volcano Eruptions. Ilja Klebanov, Alexander Sikorski, Christof ZIB-Report 16-03. Daniel Rehfeldt, Thorsten Koch, Stephen Schütte, Susanna Röblitz (2016). Prior estima- Maher (2016). Reduction Techniques for the tion and Bayesian inference from large cohort Ralf Lenz, Robert Schwarz (2016). Optimal Prize-Collecting Steiner Tree Problem and data sets. ZIB-Report 16-09. Looping of Pipelines in Gas Networks. ZIB-Re- the Maximum-Weight Connected Subgraph port 16-67. Problem. ZIB-Report 16-47. Ilja Klebanov, Alexander Sikorski, Christof Schütte, Susanna Röblitz (2016). Empirical Han Cheng Lie, T. J. Sullivan (2016). Camer- Guillaume Sagnol, Christoph Barner, Ralf Bayes Methods, Reference Priors, Cross on-Martin theorems for sequences of Cau- Borndörfer, Mickaël Grima, Mathees Seeling, Entropy and the EM Algorithm. ZIB-Report chy-distributed random variables. ZIB-Report Claudia Spies, Klaus Wernecke (2016). Robust 16-56. 16-40. Allocation of Operating Rooms: a Cutting Plane Approach to handle Lognormal Case Ilja Klebanov, Alexander Sikorski, Christof Stephen Maher, Matthias Miltenberger, Joao Durations and Emergency Arrivals. ZIB-Re- Schütte, Susanna Röblitz (2016). Empirical Pedro Pedroso, Daniel Rehfeldt, Robert port 16-18. Bayes Methods for Prior Estimation in Sys- Schwarz, Felipe Serrano (2016). PySCIPOpt: tems Medicine. ZIB-Report 16-57. Mathematical Programming in Python with Guillaume Sagnol, Felix Balzer, Ralf Borndör- the SCIP Optimization Suite. ZIB-Report fer, Claudia Spies, Falk von Dincklage (2016). Thorsten Koch, Rolf Griebel, Konstanze Söll- 16-64. Makespan and Tardiness in Activity Networks ner, Jürgen Christof, Roland Bertelmann with Lognormal Activity Durations. ZIB-Re- (2016). DeepGreen – Entwicklung eines Adam Nielsen (2016). The Monte Carlo Com- port 16-23. rechtssicheren Workflows zur effizienten putation Error of Transition Probabilities. Umsetzung der Open-Access-Komponente ZIB-Report 16-37. Guillaume Sagnol, Ralf Borndörfer, Mickaël in den Allianz-Lizenzen für die Wissenschaft. Grima, Mathees Seeling, Claudia Spies (2016). ZIB-Report 15-58. Robust Allocation of Operating Rooms with Lognormal case Durations. ZIB-Report 16-16.

Zuse Institute Berlin 113 ZIB Publications

Guillaume Sagnol, Hans-Christian Hege, Tim Sullivan (2016). Well-posed Bayesian Christian Tobias Willenbockel, Christof Martin Weiser (2016). Using sparse kernels to inverse problems and heavy-tailed stable Schütte (2016). Variational Bayesian Inference design computer experiments with tunable Banach space priors. ZIB-Report 16-30. and Model Selection for the Stochastic Block precision. ZIB-Report 16-33. Model with Irrelevant Vertices. ZIB-Report Alexander Tesch (2016). A Nearly Exact Prop- 16-01. Adrian Sali (2016). Coupling of Monodomain agation Algorithm for Energetic Reasoning in and Eikonal Models for Cardiac Electrophys- O(n^2 log n). ZIB-Report 16-25. Stefanie Winkelmann (2016). Markov Control iology. ZIB-Report 16-50. with Rare State Observation: Average Opti- Alexander Tesch (2016). Exact Energetic Rea- mality. ZIB-Report 16-59. Sebastian Schenker, Ralf Borndörfer, Martin soning in O(n^2 log^2 n). ZIB-Report 16-46. Skutella (2016). A novel partitioning of the set Stefanie Winkelmann, Christof Schütte (2016). of non-dominated points. ZIB-Report 16-55. Alexander Tesch (2016). Improved Compact The spatiotemporal master equation: approx- Models for the Resource-Constrained Project imation of reaction-diffusion dynamics via Stephan Schwartz, Ralf Borndörfer, Gerald Scheduling Problem. ZIB-Report 16-76. Markov state modeling. ZIB-Report 16-60. Bartz (2016). The Graph Segmentation Prob- lem. ZIB-Report 16-53. Christoph von Tycowicz, Felix Ambellan, Anir- Jakob Witzig, Timo Berthold, Stefan Heinz ban Mukhopadhyay, Stefan Zachow (2016). A (2016). Experiments with Conflict Analysis Stephan Schwartz, Thomas Schlechte, Elmar Riemannian Statistical Shape Model using Dif- in Mixed Integer Programming. ZIB-Report Swarat (2016). Designing Inspector Rosters ferential Coordinates. ZIB-Report 16-69. 16-63. with Optimal Strategies. ZIB-Report 16-65. Stefan Vigerske, Ambros Gleixner (2016). Wei Zhang, Carsten Hartmann, Christof Claudia Stötzel, Rainald Ehrig, H. Marike T. SCIP: Global Optimization of Mixed-Integer Schütte (2016). Effective Dynamics Along Boer, Julia Plöntzke, Susanna Röblitz (2016). Nonlinear Programs in a Branch-and-Cut Given Reaction Coordinates, and Reaction Exploration of different wave patterns in a Framework. ZIB-Report 16-24. Rate Theory. ZIB-Report 16-35. model of the bovine estrous cycle by Fourier analysis. ZIB-Report 16-02. Martin Weiser, Sunayana Ghosh (2016). The- oretically optimal inexact SDC methods. ZIB-Report 16-52.

114 2016 Annual Report Zuse Institute Berlin 115 REFERENCES SHAPE-BASED BIGGER DATA, ANALYSIS – METHODS BETTER HEALTH AND RESULTS

[1] M. Sahu, A. Mukhopadhyay, A. Szengel, [1] Kendall DG. A Survey of the Statistical [9] Baum D, Mahlow K, Lamecker H, Zachow S. Zachow, Addressing multilabel imbalance Theory of Shape. Statistical Science 4:2 (1989), D, Müller J, Hege HC. “The Potential of Sur- problem of Surgical Tool Detection using pp. 87-99 face-based Geometric Morphometrics for CNN, International Journal on Computer Evolutionary Studies: an Example using Dwarf Assisted Radiology and Surgery (IJCARS) – [2] Small CG. The Statistical Theory of Shape. Snakes (eirenis)”. Digital Specimen 2014, Sept. Spl. Issue IPCAI, 2017. Springer, 2012 8-12, Berlin, Germany, 2014, pp. 1-2.

[3] Dryden IL, Mardia KV. Statistical Shape [10] Bernard F, Salamanca L, Thunberg J, Analysis: With Applications in R. John Wiley Tack A, Jentsch A, Lamecker H, Zachow S, & Sons, 2016 Hertel F, Goncalves J, Gemmar P. Shape- aware Surface Reconstruction from Sparse [4] Srivastava A, Klassen EP, Functional and 3-D Point-Clouds. Medical Image Analysis 38 Shape Data Analysis, Springer, 2016 (2017), pp. 77–89

[5] von Tycowicz C, Ambellan F, Mukhopad- [11] Van Kaick O, Zhang H, Hamarneh G, hyay A, Zachow S. “A Riemannian Statistical Cohen- Or D. A Survey on Shape Corre- Shape Model using Differential Coordinates”. spondence. Computer Graphics Forum, 30:6 ZIB Report 16-69, Zuse Institute Berlin, 2016 (2011), pp. 1681-1707.

[6] Günther A, Lamecker H, Weiser M. “Flexi- [12] Ehlke M, Ramm H, Lamecker H, Hege HC, ble Shape Matching with Finite Element Based Zachow, S. “Fast Generation of Virtual X-Ray LDDMM.” International Journal of Computer Images for Reconstruction of 3-D Anatomy.” Vision 105.2 (2013), pp. 128-143 IEEE Transactions on Visualization and Com- puter Graphics, 19:12 (2013), pp. 2673-2682. [7] Lamecker H. “Variational and Statistical Shape Modeling for 3-D Geometry Recon- [13] Ehlke M, Frenzel T, Ramm H, Shandiz struction”. Doctoral thesis, Freie Universität MA, Anglin C & Zachow S. “Towards Robust Berlin, Fachbereich Mathematik und Informa- Measurement of Pelvic Parameters from AP tik, 2008 Radiographs Using Articulated 3-D Models.” Int J CARS 10 (Suppl 1) (2015), pp. 25-26.

[8] Grewe CM, Zachow S. “Fully Automated and Highly Accurate Dense Correspondence [14] Lamecker H, Zachow S, “Statistical Shape for Facial Surfaces.” European Conference Modeling of Musculoskeletal Structures and on Computer Vision ECCV 2016, pp. 552-568. Its Applications”, in: Zheng, G., Computational Springer, 2016. Radiology for Orthopaedic Interventions, pp. 1-23, Springer, 2016.

116 2016 Annual Report RESEARCH CAMPUS MODAL: REPORT COMPUTING PERFORMANCE IN FROM THE GASLAB THE TRUTH DATA SCIENCE

[1] Ying Chen and Bo Li (2015): An Adaptive [1] Christopher Benzmüller and Brune Wolt- [1] Robert Schmidtke, Guido Laubender, Functional Autoregressive Forecast Model zenlogel Paleo. The inconsistency in Gödel's Thomas Steinke: Big Data Analytics on Cray to Predict Electricity Price Curves, Journal of ontological argument: A success story for AI XC Series DataWarp using Hadoop, Spark and Business and Economic Statistics in metaphysics. In Proceedings IJCAI, pages Flink. CUG Proceedings (2016) 936–942, 2016. [2] Gas (2010): Verordnung über den Zugang [2] Felix Hupfeld, Toni Cortes, Björn Kolbeck, von Gasversorgungsnetzen (Gasnetz- [2] Kevin K.H. Cheung, Ambros Gleixner, and Jan Stender, Erich Focht, Matthias Hess, zugangsverordnung – GasNZV), 2010. Version Daniel E. Steffy. Verifying integer program- Jesus Malo, Jonathan Marti, Eugenio Cesario: released Sep. 03, 2010. ming results. ZIB-Report 16-58, Zuse Institute The XtreemFS architecture—a case for object- Berlin, November 2016. based file systems in Grids (2008) [3] Nina Geißler, Uwe Gotzes, Benjamin Hiller, Jessica Rövekamp, and Thorsten Koch [3] William Cook, Thorsten Koch, Daniel [3] Alexander Alexandrov, Rico Bergmann, (2015): Regulatory rules for gas markets in E Steffy, and Kati Wolter. An exact rational Stephan Ewen, Johann-Christoph Freytag, Germany and Europe. In Evaluating Gas mixed-integer programming solver. In Inter- Fabian Hueske, Arvid Heise, Odej Kao, Mar- Network Capacities [Thorsten Koch, Benja- national Conference on Integer Programming cus Leich, Ulf Leser, Volker Markl, Felix Nau- min Hiller, Marc Pfetsch, and Lars Schewe, and Combinatorial Optimization, pages 104– mann · Mathias Peters · Astrid Rheinländer, editors]. MOS-SIAM Series on Optimization. 116. Springer, 2011. Matthias J. Sax, Sebastian Schelter, Mareike SIAM, 2015. Höger, Kostas Tzoumas, Daniel Warneke: The [4] Marijn JH Heule, Oliver Kullmann, and Vic- Stratosphere platform for big data analytics. tor W Marek. Solving and verifying the bool- VLDB Journal (2014) ean pythagorean triples problem via cube- and-conquer. arXiv preprint arXiv:1605.00723, 2016.

[5] George Gonthier. Formal Proof – the four- color theorem. Notices Amer. Math. Soc. 55 (2008), no. 11, 1383–1393.

[6] Jonad Pulaj, Annie Raymond, and Dirk Theis. New conjectures for union-closed families. ZIB-Report 15-57, Zuse Institute Berlin, December 2015.

[7] Jonad Pulaj. Cutting planes for Families Implying Frankl's Conjecture. ZIB-Report 16-51, Zuse Institute Berlin, Novermber 2016.

[8] Nathan Wetzler, Marijn JH Heule, and Warren A Hunt Jr. Drat-trim: Efficient checking and trimming using expressive clausal proofs. In International Conference on The- ory and Applications of Satisfiability Testing, pages 422–429. Springer, 2014.

Zuse Institute Berlin 117 SUPERCOMPUTING AT THE LIMIT

[1] www.lustre.org

[2] www.beegfs.com

[3] TURBOMOLE V6.6 (June 26, 2014), a development of University of Karlsruhe and Forschungszentrum Karlsruhe GmbH, 1989- 2007, TURBOMOLE GmbH, since 2007; available from http://turbomole.com

[4] www.ixpug.org

118 2016 Annual Report Zuse Institute Berlin 119 120 2016 Annual Report Zuse Institute Berlin 121 IMPRINT

Published by Konrad-Zuse-Zentrum für Informationstechnik Berlin Takustrasse 7 | 14195 Berlin | Germany Phone +49-30-841 85-0 Fax +49-30-841 85-125 E-mail [email protected] www.zib.de

Layout SCHIEBE PREIL BAYER werbung und kommunikation – Berlin

ISSN 0934-5892

122 2016 Annual Report