The newsletter of EPCC, the supercomputing centre at the University of Edinburgh news Issue 74 Autumn 2013
In this issue
Managing research data
HPC for business
Simulating soft materials
Energy efficiency in HPC
Intel Xeon Phi ARCHER A new UK service for academic research
ARCHER is a 1.56 Petaflop Cray XC30 supercomputer that will provide the next UK national HPC service for academic research
Also in this issue Dinosaurs! From the Directors Contents 3 PGAS programming 7th International Conference Autumn marks the start of the new ARCHER service: a 1.56 Petaflop Profile Cray XC30 supercomputer that will provide the next UK national Meet the people at EPCC HPC service for academic research. 4 New national HPC service We have been involved in running generation of exascale Introducing ARCHER national HPC services for over 20 supercomputers which will be years and ARCHER will continue many, many times more powerful Big Data our tradition of supporting science than ARCHER. Big Data activities 7 Data preservation and in the UK and in Europe. are also playing an increasingly infrastructure important part in our academic and Our engagement with industry work. supercomputing takes many forms 9 HPC for industry - from racing dinosaurs at science We have always prided ourselves Making business better festivals, to helping researchers get on the diversity of our activities. more from HPC by improving This issue of EPCC News 11 Simulation algorithms and creating new showcases just a fraction of them. Better synthesised sounds software tools. EPCC staff design You can find out more about our Improving soft matter design and deliver high-quality training, we work on our website and blog: equip scientists with the skills they www.epcc.ed.ac.uk/blog Support for science need to produce ground-breaking 13 Advancing molecular computer simulations, and we Alison Kennedy & Mark Parsons dynamics undertake research into the EPCC Executive Directors software, tools and technologies [email protected] Future of HPC that will make possible a new [email protected] 15 A roadmap to the next generation Numerical simulations NAIS’s new GPGPUs 16 Energy efficiency in HPC Adrian Jackson More efficient parallel and [email protected] cloud computing
EPCC has joined the OpenACC implementations on a range of Legacy code consortium. OpenACC is a different hardware. We are also 19 Parallelising commercial directives-based parallel porting a number of applications to codes programming model, in the vein of GPGPUs using OpenACC, including OpenMP, designed to enable C, CASTEP (DFT-based materials Exascale C++, and FORTRAN codes to modelling code), and COSA 20 effectively utilise accelerator (frequency domain CFD code). European research projects technology such as GPGPUs. The consortium brings together: Intel Xeon Phi The consortium works on the hardware vendors (Cray and 21 Our first impressions OpenACC standard, OpenACC NVIDIA); software providers (CAPS, tools, and OpenACC benchmarks Allinea, PGI, and Rogue Wave), and Training and education and example applications. research establishments (including 23 Georgia Tech, Oak Ridge and MSc in HPC; research EPCC has a strong engagement Sandia National Labs, and the software; DiRAC, Summer with OpenACC, including OpenACC Tokyo Institute of Technology). of HPC compiler developers amongst our staff, and we have created a set of Outreach OpenACC benchmarks to enable www.openacc-standard.org 26 www.castep.org Spreading the word about users to evaluate OpenACC supercomputers 29 MSc in HPC Contact us Study with us www.epcc.ed.ac.uk [email protected] +44 (0)131 650 5030 EPCC is a supercomputing centre based at The University of Edinburgh, which is a charitable body registered in Scotland with registration number SC005336.
2 PGAS2013 in Edinburgh
The 7th International Conference on PGAS Programming Models visited Edinburgh on the 3rd and 4th October, making its first ever appearance outside the United States!
The PGAS conference is the support for PGAS-type languages Michele Weiland premier forum to present and in current (and future) HPC systems [email protected] discuss ideas and research Professor Mitsuhisa Sato from the developments in the area of PGAS University of Tsukuba in Japan, who models, languages, compilers, took the opportunity to discuss runtimes, applications and tools, how PGAS may play a role in the PGAS architectures and hardware race to the exascale. features. The conference, which attracted More information, including The keynote talks were given by over 60 attendees from across the links to the papers and two highly-regarded experts in the globe, had a varied programme of proceedings, can be found on field: research papers, “hot” sessions the conference website: Dr Duncan Roweth, a senior where speakers introduced work in principal engineer at Cray, who progress, as well as a poster www.pgas2013.org.uk focussed his talk on hardware reception. Staff profile Applications Consultant Eilidh Troup talks about her work here at EPCC.
I work as an Applications Consultant on a project called SPRINT (Simple Parallel R INTerface), which allows users of the statistical language R to make use of multi-core and HPC machines without needing any parallel programming skills. We provide parallelised versions of can be used for measuring gene Eilidh Troup standard R functions which can just expression to diagnose and [email protected] be slotted in to replace the usual R understand diseases or to function and will then run on many sequence genomes, for example to processors behind the scenes. find out which microorganisms are present in a habitat. I particularly enjoy working on SPRINT as it is mostly used by I am also involved in EPCC’s public biologists. I studied genetics before outreach events and love the I became a programmer, and love enthusiasm of children pretending this opportunity to keep up to date to be part of a computer and with the latest biology technology. working together to sort coloured balls or numbers. Next Generation Sequencing can rapidly produce terabytes of data People are very interested in the that must then be analysed. This science we support at EPCC and amount of data needs a lot of the real hardware that makes a computational power to process supercomputer is always popular and EPCC is well placed to work on too. this. Next Generation Sequencing
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 3 ARCHER: On target for a bullseye
Autumn ushers in a new era for UK supercomputing with the start of the ARCHER (Advanced Research Computing High End Resource) service in Edinburgh. ARCHER is the next national HPC service for academic research and comprises a number of components: accommodation provided by the University of Edinburgh; hardware from Cray; systems support from EPCC and Daresbury; and user support from EPCC.
In autumn 2011, the Minister for The latest Intel Xeon Ivy Bridge At your service Science announced a new capital processors used in ARCHER provide investment in e-infrastructure, which the next generation of computational The Service Provision function for included £43m for ARCHER, the muscle, with best-in-class floating- ARCHER is provided by UoE HPCX next national HPC facility for point performance, memory Ltd. This includes responsibilities academic research. After a brief bandwidth and energy efficiency. Each such as systems administration, overlap, ARCHER will take over ARCHER node comprises two 12-core helpdesk and website provision, from HECToR as the UK’s primary 2.7 GHz Ivy Bridge multi-core and administrative support. The Academic research supercomputer. processors, at least 64 GB of DDR3- work is subcontracted to EPCC at HECToR has been in Edinburgh 1833 MHz main memory and all the University of Edinburgh (EPCC) since 2007. compue nodes are interconnected via and the STFC’s Daresbury an Aries Network Interface Card. Laboratory. What is ARCHER? ARCHER has 3008 such nodes, ie, Service Provision will be delivered The new Cray XC30 architecture is 72,192 cores, in only 16 cabinets by two sub-teams: the Operations the latest development in Cray’s providing a total of 1.56 Petaflops of and Systems Group led by Mr long history of MPP architectures, theoretical peak performance.Scratch Michael Brown, and the User which have been supporting storage is provided by 20 Cray Support and Liaison Team led by fundamental global scientific Sonexion Scalable Storage Units, Dr Alan Simpson. research for over two decades. giving 4.4PB of accessible space with sustained read-write bandwidth of Enabling a smooth transition for The Cray XC30 incorporates two over 100GB per second. users from the HECToR to ARCHER major upgrades to the fundamental services is one of our key aims. For components of any MPP ARCHER is also directly connected ARCHER, we will utilise SAFE supercomputer: the introduction of to the UK Research Data Facility, (Service Administration from EPCC) Cray’s newest network interconnect, easing the transition of large data for both the ARCHER Helpdesk and Aries; and the use of Intel’s Xeon sets between high-performance Project Administration & Reporting series of multi-core processors. scratch space and long-term functions. Each has enhanced capabilities archival storage and between successive HPC services. The ARCHER website provided by over previous architectures. Aries EPCC contains supporting incorporates the novel dragonfly Updates included in the newest documentation for the service and network topology that provides versions of the Cray Compilation will also showcase the research multi-tier all-to-all connectivity. This Environment provide full support for that uses the system. The new network allows all applications, generating highly optimised configuration of the ARCHER even those that perform all-to-all executables that fully exploit the service will evolve over time to stay style communications, the potential “Ivy Bridge” processors. Users will in line with users’ needs. Continual to scale to the full size of the system also have access to the latest Intel Service Improvement will be a key allowing to tackle problems that Composer Suite of compilers, and goal, and as such the service will might have been considered the industry standard GNU be delivered following the ITIL Best impossible on previous systems. Compiler Collection, all of which are Practice Framework.
4 fully integrated with the feature-rich commissioned by year end. Cray Programming Environment The facility was fit for purpose two suite that is familiar to existing months ahead of schedule and was HECToR users. delivered in excess of specification ACF: building for the while under budget. The Cray XC30 and associated storage systems future were delivered in September 2013. ARCHER’s Accommodation and The installation went very smoothly, Management function is provided with all power and cooling by the University of Edinburgh. connections made and the system ARCHER is housed at the powered up within a few days. University’s Advanced Computing Facility (ACF). The University has a Ready for business long-term commitment to ensure The acceptance tests of the the ACF is capable of hosting ARCHER hardware were top-end facilities and deliver successfully completed in late Get in touch! excellent levels of energy efficiency. October. Usage of ARCHER will We welcome ideas and In readiness for ARCHER, the ACF ramp up in mid-November, with opportunities for enhancing the was extended, with the addition of core research consortia online on CSE support. If you want to be 500m2 of Computer Room floor November 13th. Remaining grant involved or have any thoughts, space, and an additional 760m2 holders will begin to transfer from please do not hesitate to get plant room to contain the additional HECToR to ARCHER in December in touch via the ARCHER electrical and mechanical 2013. The HECToR Service will Helpdesk. infrastructure. This included a new cease in March 2014. high-efficiency 4MW-capacity Access cooling system and an upgrade to Computational Science The first EPSRC call for access the site’s private high-voltage & Engineering Support to ARCHER has already network that increases the capacity Computational Science and opened, and details can be to around 7MW. Engineering (CSE) support on found on both the EPSRC and The project to extend the facility ARCHER is provided by EPCC and ARCHER websites. commenced in May 2012, with the includes responsibility for helping The first call for eCSE projects building ready for the installation of users with porting, optimising and has opened, details can be plant in September (despite the developing their codes for found on the ARCHER website. wettest summer on record). The HV ARCHER, ensuring that the correct scientific software is installed on the switch-room was commissioned in EPSRC is the managing agent for the November 2012 and the full system to meet user requirements, HPC facility on behalf of all of the capability of the plant providing advice and best practices Research Councils. to enable users to exploit ARCHER
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 5 Images. Above: Aries Interconnect. Above right: ARCHER’s water cooling system. Below left: new high-voltage Switch Room.
resources, and training and technical discussion around HPC). Tom Edwards, Cray developing scientific software The Forum consists of a series of [email protected] development expertise in the UK monthly meetings conducted using research communtity. webinar technology with a wide Mike Brown, EPCC range of technical experts invited to [email protected] Our goal for the CSE support is to speak and attend, and a public be as open and inclusive as mailing list for technical discussion. Liz Sim, EPCC possible; allowing ARCHER users to [email protected] draw on the full wealth of expertise Consortium Contacts available in the UK HPC and We have established a set of Alan Simpson, EPCC computational science community. Consortium Contacts: HPC experts [email protected] We will use a mix of established, who will provide a direct link successful activities and innovative between the research communities ideas to realise this goal. using ARCHER and the service Embedded CSE programme itself. These Contacts will allow the research communities to use The Embedded CSE (eCSE) ARCHER more effectively, have a programme expands and refines the role in driving the development of successful HECToR dCSE the service to meet their needs, and programme to allow software have a simple way to provide development expertise to be placed feedback to the CSE support team in academic research groups where and the service in geenral. it can provide the most benefit and have the greatest impact. The first eCSE call has already opened (deadline: 14 January, 4pm). Details Training can be found on the ARCHER Training will be provided all over the website. UK through links with the HPC-SIG, ARCHER website The in-depth CSE support will be HPC Wales, and the STFC’s www.archer.ac.uk fully integrated into the SAFE Daresbury Laboratory. We are Helpdesk, providing a seamless consulting people around the UK Support service to users that gives direct about the training requirements of access to ARCHER expertise and a different research communities. The For any questions about the rapid response to any queries. lectures from the first course have ARCHER service, please been recorded and will be publically contact: [email protected]. Technical Forum available on the ARCHER website in The ARCHER Technical Forum is the near future. Cray open to all users (and external www.cray.com people who are interested in
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 6 Managing research data
Publishing your research used to be straightforward. You’d note your hypothesis, describe your methods, pop in a table of measurement results and a few graphs, then conclude with your analysis, and bingo. Another paper published, with everything necessary included for others to verify and build upon your work. But this model broke during the last couple of decades, and it is becoming more broken as research becomes increasingly – exponentially – digital.
So many modern data sets simply EPCC’s research data Rob Baxter do not fit in the paper. Long gone is archive [email protected] e-shot (http://iesve-news.co.uk/t/ the single table of measurements; WN3-1SWK1-154DKGTPAA/ analyses in data-driven science are These criteria are feeding in to the cr.aspx) that shows the offering. based on datasets of gigabytes, design of a long-term research data terabytes and – soon – petabytes. archive here at EPCC. We are This raises the question: should delighted to host and manage, on Design aspects of EPCC’s researchers publish these datasets, behalf of the Research Councils, the research data archive and if so, how? UK’s Research Data Facility (RDF), For science to remain verifiable and a 26 petabyte combined disk and • Allow remote access clients to reusable, the answer to the first part tape storage system. add/retrieve data via network services, including, but not must be yes. The answer to the HPC systems users typically want a limited to, web browsers second part has given rise to big, fast file system, and the current increasing efforts to create better GPFS deployment on the RDF gives • Allow flexible authentication, ways to manage research data. (For just that. But, as projects come to without requiring local Unix a review of the arguments, see the an end, for all the reasons noted login credentials Royal Society report of June 2012, above, a big file system is no longer • Use flexible authorisation Science as an Open Enterprise.) the right environment in which to controls defined by external One of the biggest challenges in archive data for the longer term. data sources effective research data So, over the next six months we’ll • Make providing metadata management is dealing with this put a long-term data archive service straightforward for depositors separation of data from its context. in place. The technology we’ll use If data is to be stored away from the to deliver this is currently open, but • Provide persistent identifiers pages of the publication, we must we hope to leverage results from for data deposits, based on ensure it is findable, persistent and projects like EUDAT and PERICLES the Handle system (the system sufficiently well described – the (see opposite). behind DOIs, already in use intelligent openness of Science as at EPCC through EUDAT an Open Enterprise. And, for credit With luck we’ll be able to report a and more broadly within the to be given where it’s due, research running service in the next issue of University). data need to be citable. EPCC News!
7 PERICLES: archiving digital data
Preserving art, records and other items has been a challenge throughout history, not just how to store them but how to help future generations to understand them. Even in the short time digital art and records have been around, this problem has become increasingly apparent and is exacerbated by technology’s rapid cycles.
The PERICLES project is attempting At the latest meeting in Alistair Grant to define and develop a framework Thessaloniki, the project scoped out [email protected] for managing how digital data is detailed scenarios about how a stored and kept relevant and long-term preservation system may accessible. A small challenge it is be used and what will be required. not. Images show Rafael Lozano-Hemmer’s “Surface Within the two case studies, groups Tension” (1992), which is in the Tate’s collection. Although the project has two case of project members went into depth studies to look at (Art & Media and about what happens to new Space Science Data), PERICLES material up to the point of ingest to does not intend to create solutions an archive and the process of just for these areas, or for this ingesting a new object into an moment in time. PERICLES is to archive. This meeting has been consider how to build a framework highly informative about how to that will last and adapt through approach the concept of a long- different types of change including lived digital preservation system. policy and technological changes.
EUDAT: towards a collaborative data infrastructure
The European Data project EUDAT is two years old this month. Over this time EUDAT has got to grips with the challenge of integrating five established research infrastructures into the beginnings of a true pan- European collaborative data infrastructure. And it really is getting there. EUDAT brings together some of with another four planned for the Rob Baxter Europe’s largest HPC centres with coming months. A Joint Metadata [email protected] five of its discipline-specific Catalogue is harvesting metadata “research infrastructures”, covering records from five disciplines (and linguistics, climate science, counting), and the Simple Store EUDAT common core seismology, biodiversity and service for smaller-scale data and technologies integrative medicine. Its goal is to individual researchers will be rolled iRODS, the Integrated Rule- fashion common data services out this autumn. Oriented Data System from – standardised ways of managing 1 With one year to go of the core RENCI data and metadata – to realise project, EUDAT has made economies of scale and to create a Global Handle System through remarkable steps from such a basis for the preservation, sharing the European Persistent Identifier complex starting point. We’ve had 2 and recombination of research data. Consortium to take an incremental approach to 3 EUDAT makes use of common core systems integration, but a core of GridFTP and Globus Online technologies and is federating and common data services are now CERN’s Invenio4 connecting them across Europe, managing data across 20 sites, with 5. creating “islands” of safely- more to come. Our final year will be CKAN metadata repository replicated and discoverable data one of stabilisation and 1. http://www.renci.org 2. EPIC consortium: http://pidconsortium.eu which will, over time, merge consolidation. EUDAT has laid Global Handle System, http://www.handle. together into a single resource. excellent foundations for a truly net There are currently seven such pan-European haven for today’s – 3. http://www.globusonline.org islands in operation, from Edinburgh and tomorrow’s – research data. 4. http://invenio-software.org to Bologna, Barcelona to Helsinki, 5. ckan.org
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 8 HPC makes better business
SUPERCOMPUTING SCOTLAND business innovation through high performance computing
Integrated Environmental Systems (IES) is the world’s leading provider of software and consultancy services focused on making the built environment more energy-efficient, so reducing overheads and CO2 emissions. We worked with IES to improve the performance of its SunCast software, which is used by architects and designers to analyse sun shadows, solar penetration and the effects of solar gains on the thermal performance and comfort of buildings.
SunCast processes each hour of a New business model Ronnie Galloway design day in series. In all there are [email protected] MPI-enabled SunCast has opened 448 separate calculation tasks (160 up a new business model for IES - diffuse and 288 direct) to be being able to sell SunCast through performed for any given model. the cloud on a pay-per-use basis. EPCC ported SunCast to run over This offering is open to IES end- Microsoft MPI, so allowing the users either as a managed service, parallel processing of tasks, one per where IES provides additional computer processor. When a reportage, or as a self-managed processor has completed a service. Both routes will provide calculation, it uses MPI to notify the additional revenue streams to IES. controlling processor of its results Using the MPI in IES Consultancy and that it is ready to be assigned has also increased the efficiency another task. – and therefore profitability – of the Reduced analysis times company’s own consultancy offering. At time of writing, it had Now MPI-compliant, SunCast can been used with 4 live projects with run on a supercomputer, creating an average analysis time of under huge time savings. In one extremely This work was carried out as part 12 hours. These particular projects complex analysis, the run-time was of Supercomputing Scotland, a were very large and complex and reduced from an estimated 30 days joint EPCC-Scottish Enterprise would previously have taken several to 24 hours. programme. weeks. Reduction in analysis times allows FORTISSIMO IES to keep ahead of the IES competition by delivering faster IES is also engaged with EPCC on www.iesve.com turnaround times for clients. the FORTISSIMO programme: see Supercomputing Scotland Reductions in analysis times also opposite. www.supercomputingscotland.org allow IES to perform more detailed analysis than previously.
9 EPCC offers faster MATLAB® programs and Simulink® models
EPCC’s Accelerator service provides instant, on-demand access to the full suite of MathWorks’ software products. MATLAB® and Simulink® users can Server (DCS), providing access to George Graham now gain significant performance even greater performance [email protected] advantages by running their improvements. computations and models on large Full pay-per-use access is provided memory, multi-core HPC platforms using on-demand, hourly access to without the need for costly DCS and HPC cycles. DCS product investment in new computing use is billed by MathWorks, whilst infrastructure. use of the HPC infrastructure is Find out more Our unique service lets you launch billed by EPCC. computations and simulations on To set up a secure Accelerator As an alternative, users can run EPCC’s facilities directly from your account, or to request a trial, visit: MATLAB® and Simulink® on desktop using MATLAB® Parallel EPCC’s platforms within the context www.epcc.ed.ac.uk/facilities/ Computing Toolbox (PCT). Parallel of their current perpetual or annual demand-computing up-scaling can be achieved through PCT and DCS licences. MATLAB® Distributed Computing
Fortissimo! Digital simulation and modelling for European industry
EPCC has worked with companies – large and small – since it was set up in 1990. Despite our best efforts, we know there are many companies across Europe who could benefit from HPC- enabled modelling and simulation but who don’t use it, either through a lack of knowledge or fear of its costs or complexity. In Scotland, our most recent • A core team of partners, mainly Mark Parsons programme of support for smaller HPC service providers, who will [email protected] enterprises, Supercomputing create and manage the Cloud of Scotland, has been running HPC resources successfully for almost two years. • An initial tranche of 20 The pan-European Fortissimo experiments, each involving a project will complement our existing company with a modelling and activities in this area. simulation challenge and some The PlanetHPC project made the experts case for greater European • Two further Open Calls for Commission investment in support experiments which will start at for modelling and simulation for Month 12 and 18 of the project. Europe’s companies. Building on Fortissimo this work, EPCC led the With total costs of €22 million and www.fortissimo-project.eu development of the Fortissimo funding from the European project as part of the European Commission of €16 million, this is a Commission’s Framework 7 major project for us. PlanetHPC Factories of the Future initiative. www.planethpc.eu The initial consortium consists of 45 Project structure partners and 20 experiments. We expect this to grow to around 90 Supercomputing Scotland www.supercomputingscotland.org Fortissimo is split into four equal partners and 50-60 experiments by parts: the end of the project.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 10 The effects of a timpani drum strike modelled over time. Image: Adrian Mouat & Stefan Bilbao. Next Generation sound synthesis
When you think about applications for high performance computing and large-scale simulations, you probably don’t think of music. But the Next Generation Sound Synthesis project (NESS) may change that. Until now, most digital sound The computational difficulty of James Perry synthesis has either used primitive these problems varies widely; from [email protected] abstract methods (such as additive simple linear 1-dimensional models synthesis and FM synthesis) or that can easily run in real-time on a used combinations of pre-recorded single processor, to 3D models of samples to generate music. These large spaces that are not feasible to methods are computationally cheap run at all on current hardware due but they have their limitations - to memory constraints. However, notably, they don’t always sound the large problems are very well The project focuses on six areas, realistic, they can be hard for suited to GPU acceleration as they covering a range of musical musicians to control, and they lack mostly involve performing the same instrument families: flexibility. A newer method, physical simple operations over and over • Brass instruments modelling synthesis, promises to again on many different data items overcome these limitations - but at - exactly what GPUs are good at. • Electromechanical instruments the cost of being much more The NESS project started in • Nonlinear plate and shell vibration computationally intensive. January 2012 and will run for a total • Modular synthesis environments In the NESS project, researchers of five years. Several acoustic from the Acoustics Group at the models, including plates, timpanis • Room acoustics modelling University of Edinburgh have and whole rooms, are under • Embeddings and spatialisation teamed up with EPCC to further development and more are planned. develop physical modelling An interface is also being developed synthesis, using HPC techniques to to allow visiting composers to make overcome the computational use of the models as easily as barriers. The goal is to generate the possible. highest quality synthetic sounds NESS is funded by the European possible, with GPU (graphics Research Council. processing unit) acceleration to help www.ness-music.eu keep run times manageable.
11 Design rules for new soft materials
Soft materials include colloids, pastes, emulsions, foams, surfactant solutions, powders and liquid crystals. Everyday examples are (respectively) paint, toothpaste, mayonnaise, shaving cream, shampoo, talcum powder and the mess that results when soap is left in water. Soft materials are also used in many industrial areas such as drug delivery and electronic displays. Improving existing products and designing new ones are the goals of active research.
EPCC works with Prof. Mike Cates Important simulation approaches Oliver Henrich at The University of Edinburgh to for soft materials, where relevant [email protected] investigate soft materials as part of structure is typically at the Kevin Stratford a programme funded by the UK “mesoscale”, include atomistic [email protected] Engineering and Physical Sciences molecular dynamics and coarse- Research Council. The intention of grained methods (which discard this work is to combine theoretical atomistic detail in return for larger Soft materials form many and experimental approaches, length and time scales). Irrespective alongside computer simulation, to of the exact type of simulation, the consumer products: establish scientific design principles computational effort required calls computer simulation aids that will allow the creation of a new for state-of-the-art HPC. This soft their design. generation of soft materials for use materials research generates close in future technologies. Prof. Cates collaboration with other EPCC and his group perform theoretical, activities such as the CRESTA computational and experimental project, and makes use of UK work on many different aspects of Research Council resources, as well soft materials; EPCC supports the as European ones such as PRACE. www.rsc.org/softmatter Volume 9 | Number 43 | 21 November 2013 | Pages 10209–10428 computational side of these Even long-established products are activities by providing expertise in continuously being updated or simulation and high performance replaced. Such reformulation can computing (HPC). make products healthier, safer, or Such simulations complement the more environmentally-friendly. The work of both theorists and process of developing new soft experimentalists, and can help to materials, or improving existing ISSN 1744-683X
PAPER Oliver Henrich et al. identify design principles from ones, usually involves a large Rheology of cubic blue phases 1744-683X(2013)9:43;1-# existing materials. This is important element of trial and error. A set of The group’s work appeared in the in areas where analytical progress is design principles, based on well- Soft Matter journal: Henrich et al. difficult or impossible, and where understood fundamental science, Rheology of cubic blue phases, practical approaches are technically could speed up that process. Soft Matter 9,10243-10256, 2013. awkward or prohibitively expensive.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 12 In 2011 a call for proposals for transatlantic research collaborations to address “Software Grand Challenges in the Chemical Sciences” was published by EPSRC and the US National Science Foundation. EPCC has risen to the occasion with roles in three of the four consortia currently funded to research and develop software solutions to these challenges. Here are two of them.
APES: Advanced Potential Energy Surfaces
The APES (Advanced Potential Energy Surfaces) project aims to incorporate novel potential energy surface models into a range of computational chemistry packages including Amber, DL_POLY, ONETEP, and Q-Chem. The choice of a suitable potential includes polarisable atomic Arno Proeme energy function is critical to multipoles derived directly from ab [email protected] performing meaningful molecular initio quantum mechanical electron modelling and molecular mechanics densities, and offers clear and Mario Antonioletti simulations. Most established force systematic improvements in [email protected] field models in computational accuracy that make it a prime chemistry software use non- candidate for use in future grand polarisable fixed-point-charge challenge applications. approximations as these are EPCC, together with the Software computationally cheap and give Sustainability Institute, will: reasonable results for equilibrium properties and for homogeneous • Provide a distributed memory systems. However, these models parallelisation of TINKER, the fall short when describing many- reference implementation of body effects, dynamical properties, AMOEBA, to take advantage of heterogeneous and out of large-scale compute resources equilibrium systems. This is a major • Test and validate the algorithms About APES limiting factor for the successful used in TINKER and promote APES will develop and promote application of computer simulations interoperability with other the use of polarisable force to a variety of Grand Challenge packages to promote uptake of fields based on AMOEBA problems in computational advanced polarisable force fields. (Atomic Multipole Optimized chemistry, biochemistry and Energetics for Biomolecular materials science. APES’s outputs will give researchers better tools for Applications), a prominent APES will develop and promote the understanding the structure and empirical polarisable force field use of polarisable force fields based function of molecules. By using model that allows atom-centred on AMOEBA (Atomic Multipole open development processes, a charges to vary depending on Optimized Energetics for community will be built around their environment. Biomolecular Applications), a packages that implement AMOEBA. The APES project started in prominent empirical polarisable This should make the future April 2013 and will run for three force field model that allows atom- development and adoption of this years. centred charges to vary depending force field self-sustaining. on their environment. AMOEBA
13 A free energy landscape of the Alanine-12 molecule mapped out in two diffusion coordinates determined without a priori knowledge of the system. Key stable and transition structures are labelled as shown. From “Discovering Mountain Passes via Torchlight: Methods for the Definition of Reaction Coordinates and Pathways in Complex Macromolecular Reactions” by Mary A. Rohrdanz, Wenwei Zheng, and Cecilia Clementi. Annual review of physical chemistry 64 (2013): 295-316.
ExTASY: Extensible Toolkit for Advanced Sampling and analYsis
The ExTASY project tackles managing thousands of coupled Iain Bethune parallel jobs, and orchestrating [email protected] the problem of understanding ‘big data’ movement in a the behaviour and function of heterogeneous environment. complex macromolecules • Developing novel analysis tools to such as proteins, DNA, and allow on-the-fly control of the other biomolecules through simulations to rigorously bias sampling with Molecular sampling towards the rare events. Locally-scaled diffusion Dynamics. • Providing a flexible and portable maps The key problem is that to preserve interface to couple existing MD Mapping out the transformations accuracy, MD must use a time-step programs with new algorithms for of biomolecules from one of only a few femtoseconds, ultra-large time steps integration. folded state to another is of key whereas many events of biological If we can achieve these three importance in understanding importance occur on the order of objectives together in a single their function. seconds to hours. Even with framework or toolkit - ExTASY The LSDMap program, state-of-the art simulation software, - then we will truly make a step high-performance computing and developed in collaboration with change in our ability to compute EPCC, allows these mappings to purpose-built hardware, only and understand the dynamics of milliseconds of MD are feasible be generated up to 100x faster these complex macromolecular than conventional methods. today. systems. LSDMap analysis is one of the The ExTASY project proposes a The ExTASY project consortium is three-pronged attack on the tools which will become part of led by Prof. Cecilia Clementi of Rice ExTASY. problem: University and totals seven partner • Support for high-performance institutions, including EPCC. http://sourceforge.net/projects/ high-throughput execution of lsdmap ensembles of MD calculations -
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 14 Next Generation Computing
What will next generation computing be like, and how should research, development and innovation be directed to achieve our vision of the future?
This is the question that EPCC and media for example is something few Mark Sawyer our partners eutema, Optimat Ltd could have foreseen fifteen years [email protected] and 451 Research are attempting to ago. Will there be corresponding answer under a contract with the phenomena in the next decade that European Commission to advise on will define the markets for next its Horizon2020 work programme. generation computing? Under the contract, we will make We are trying to predict the future recommendations for the future by looking at the past and by direction of research in the form of a consulting today’s experts. We have roadmap. interviewed leading industry and It has become increasingly difficult academic figures, held an online to make predictions about consultation, and run a workshop to computing because many analyse possible future scenarios. challenges that we see today (eg Together with examining existing the need for energy-efficient technology and market trends that computing, more heterogeneous The project will publish its we know are happening today, we computing and dealing with big findings in early 2014 and are beginning to see a picture of data) will almost certainly cause will hold a further workshop what next generation computing disruptive changes. There are also to present the results to may look like. non-technical issues to consider. researchers. The massive take-up of social
GPGPU hardware for Numerical Simulations
The NAIS project (Numerical Algorithms and Intelligent Software), which EPCC is a member of, has recently NAIS purchased 8 NVIDIA K20 GPGPUs and associated computer nodes for use by NAIS members and researchers. The GPGPUs complement similar housed in a 2U server chassis, Adrian Jackson, EPCC hardware that EPCC hosts for NAIS along with two 8-core Intel Xeon [email protected] (NVIDIA Tesla GPGPUs) and will processors and 128GB of memory. allow NAIS researchers to explore This provides a double precision issues of performance and peak performance per node of over programmability associated with 5.3 TFlop/s, with 16 processor About NAIS using such computing resources for cores and 7488 GPU cores NAIS is a collaboration between scientific simulation. available in each node. the universities of Edinburgh, Strathclyde and Heriot Watt. It We are currently installing these Whilst these GPGPU resources is funded by EPSRC and the GPGPUs in two compute nodes have been purchased for NAIS, they Scottish Funding Council (SFC). that will be attached to EPCC’s are a resource that can be used by These GPGPU resources are existing hydra cluster, enabling the computational simulation, funded directly by a grant from access to the compilers and mathematics, and HPC SFC to provide high performance libraries installed on that system communities in general. If you are computing resources for and facilitating scheduling of access interested accessing these systems, researchers. to these GPGPUs through the please contact us. common batch system used on www.nais.org.uk hydra. Four GPGPUs are each
15 Adept kicks off!
The Adept project is motivated by the desire to understand the energy consumption of parallel codes on various hardware platforms, from HPC to Embedded systems, and how programmers can optimise their codes for power efficiency as well as runtime, memory usage and I/O.
Led by EPCC, Adept will build on • What micro-, kernel- and Michele Weiland the HPC community’s skills in application-level benchmarks can [email protected] writing efficient parallel software, we develop that will allow us to and the Embedded computing measure power consumption of community’s skills in working within hardware components and to strict power budgets. evaluate power use of different programming models and parallel EPCC’s technical contributions will algorithms? be providing benchmark codes, real-life case studies based on • What hardware architectures, both scientific software, and provisioning representative of HPC and and instrumenting hardware. Embedded, are we going to measure power consumption on, Kick-off and how will we achieve the high Adept started in September and we granularity we require for our held our kick-off meeting in modelling tool? Edinburgh shortly after that. The • What information and data will we project partners travelled from need to extract from both the Sweden (Uppsala University and benchmarks and the hardware, Ericsson), Belgium (Ghent) as well and how will it feed into our as from just across town (Alpha performance and power model? For more information Data). The meeting was wrapped up after on Adept, including This first meeting discussed the a day and a half of in-depth announcements of general logistics of the project, but technical discussions and the upcoming events, see: its key purpose was to ensure general feeling was that we had www.adept-project.eu everyone involved could get up to made an excellent start to the speed quickly and focus on project. We will all get together Adept is partially funded by the technical discussions. again in Ghent in early December. In 7th Framework Programme Key issues the meantime, work will start in of European Commission. It earnest at the different sites and we started on the 1st September We took the opportunity to start hope to be able to report on first 2013 and is set to run for 3 addressing some of the key issues results in the near future – watch years. the project wants to tackle: this space!
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 16 ECO2Clouds: energy-efficiency in cloud computing
The EU-funded ECO2Clouds project is investigating how to make cloud computing more energy-efficient. The overall goal of the project is to couple the functional and economic advantages of cloud computing with measures that enable cloud providers and applications developers to be more aware of the impact their operational and design decisions have on the environment.
The project was developed as an tracking all ECO2Clouds relevant Dominic Sloan-Murphy extension of the BonFIRE cloud resources, gathering their [email protected] testing platform, however the associated eco-metrics and approach is general and could be recording them in a persistent applied to other cloud platforms. database. These metrics are then utilised to inform decisions made by The ECO Clouds utilities track the 2 the ECO Clouds Scheduler: the energy usage on a cloud 2 software component which uses infrastructure at three different optimisation techniques to produce layers: energy efficient application • Infrastructure layer: ie on the deployment configurations. underlying physical host computer The Monitoring Collector comprises and site where it resides three elements: the Monitor, the • Virtual Machine (VM) layer: ie the Collector and the accounting VMs that run on the host database. A relational database computer system is used with the schema allowing for the association and • Application/Experiment layer: the dependencies between cloud application running on the cloud resources at the different layers to VMs. be successfully captured. Generally, To store and make available these these are that an experiment can metrics to all ECO2Clouds modules, contain zero or more virtual the ECO2Clouds Accounting Service machines while a virtual machine was devised. A component of this can belong to only a single Service, the Monitoring Collector, experiment, a virtual machine can has been the recent focus of the only be on a single physical host EPCC team. machine at any one time but may migrate to a different host during its Monitoring Collector lifetime, a host can only exist at a The Monitoring Collector is the single site, and a site can contain multiple hosts. component of the ECO2Clouds Accounting Service tasked with
17 Monitoring collector block interaction diagram.
Message management related to that site and its physical host machines. Similarly, each The Monitor subscribes to the ECO2Clouds experiment contains BonFIRE Management Message an experiment aggregator Queue (MMQ) through which all responsible for the application and BonFIRE experiment notifications VM layer metrics. pass. BonFIRE is the cloud infrastructure currently used by The Collector first queries the accounting database for a list of ECO2Clouds. All messages from the queue are filtered by relevance, ie active resources. This list is then are they associated with employed to selectively query the appropriate aggregators, using an ECO2Clouds and are they an experiment or compute resource extension to the BonFIRE event? The Monitor extracts Application Programming Interface, About the project for all ECO Clouds relevant metrics. identifying information, such as the 2 This process is then repeated at an ECO Clouds is a collaboration experiment ID, from the filtered 2 messages and updates the established polling rate dependent between: on the desired age of the data to accounting database to enable • EPCC tracking of experiment resources make available to the Scheduler • ATOS (Spain) and statuses. and other parts of the accounting service. • University of Manchester The Collector is a concurrent (UK) component responsible for Through the Monitoring Collector, the ECO Clouds Scheduler periodically gathering metric values 2 • The University of Stuttgart’s for each of the resource layers. therefore has access to the High Performance Metrics are measured by Zabbix, necessary information to enable it Computing Centre HLRS the enterprise-class monitoring to inform cloud providers and (Germany) solution capable of capturing application developers of the • Politecnico di Milano (Italy) statistics for all resource types. For impact that particular configurations host and site metrics, each site and deployments will have on their • Inria (France). maintains a Zabbix infrastructure environment. This tool will therefore enable them to make “aggregator” charged with http://eco2clouds.eu aggregating all monitoring data environmentally aware decisions.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 18 Addressing parallelism in legacy code
Over the past few years, multicore parallelisation approach that Adrian Jackson, EPCC processors have become standard overcomes the limitations of the [email protected] for all forms of computer, from more usual source-code analysis mobile phones, to laptops, through tools by incorporating information Björn Franke, Institute for to supercomputers. Processor from actual runs of the application Computing Systems Architecture, manufacturers, unable to deliver with representative datasets. We School of Informatics cost-effective performance have demonstrated an ability to [email protected] improvements through increased deliver 96% of the performance of a clock speed, have resorted to manual parallelisation approach1, 2, coupling together multiple cores taking a fraction of the time to into a CPU. complete and with limited demands on the programmer’s understanding Whilst Multicore and Manycore of parallelisation. (such as GPGPUs) processors EPCC, working with the School of provide unprecedented parallelism Platform portability Informatics at Edinburgh, has been and speeds in individual computing developing a parallelisation tool devices, the processing power of DynaPar not only helps to address called DynaPar, designed to assist the individual cores in those the parallel-programming skill developers with parallelising serial processors has declined. shortage, but it also makes programs. applications more sustainable Parallelisation problem through platform portability. Using The collaboration, funded through conventional approaches, taking a the Numerical Algorithms and This presents a problem for the parallel application from one Intelligent Software (NAIS) project, wide range of existing applications computer architecture to another is builds on research undertaken in currently used by companies and a time-consuming and complicated Informatics to identify parallel individuals for their work and play. process. DynaPar effectively patterns in computer programs. Ensuring these applications can automates the porting process by realise the performance potential of using machine learning: a little modern hardware, and indeed that preparation to train the tool for a they don’t decline in performance new architecture allows one to due to the reduction in power of compress the porting task into a individual cores, requires the short, clearly defined process. 1. G.Tournavitis, Z.Wang, B.Franke and M.O’Boyle: investment of significant effort and Towards a Holistic Approach to Auto-Parallelization: skill in parallelising software for We are currently working with Integrating Profile-Driven Parallelism Detection and Machine-Learning Based Mapping, ACM SIGPLAN these architectures. companies in Edinburgh to evaluate Conference on Programming Language Design and the performance and useability of Implementation (PLDI’09), Dublin, Ireland, June 15, However, few software engineers the tool on existing C and C++ 2009 have the experience of parallel 2. G.Tournavitis and B.Franke: Semi-Automatic codes, and following this evaluation programming needed to convert Extraction and Exploitation of Hierarchical Pipeline period we will be looking to roll out Parallelism Using Profiling Information, Proceedings companies’ large legacy code-base a full product, probably integrated of the International Conference on Parallel to parallelism. This skill shortage Architectures and Compilation Techniques (PACT into modern development ‘10), Vienna, Austria, September 11-15, 2010. represents a significant problem, environments (such as Eclipse), to which DynaPar addresses. help companies in parallelising www.nais.org.uk DynaPar implements an assisted- programs.
19 Participants of the exascale projects’ meeting in Barcelona. Exascale research in Europe
CRESTA is one of three complementary exascale projects funded by the EC. CRESTA’s software focus sits comfortably with the DEEP and Mont-Blanc projects’ focus on developing new hardware. Together, these projects underpin Europe’s strategy for developing, producing and exploiting exascale platforms.
SC’13 activities areas of potential collaboration. Lorna Smith These included: testing CRESTA [email protected] Collaboration activities have software on the other projects’ significantly increased between hardware; tools training on each these projects over the past year. CRESTA (Collaborative other’s profiling and debugging Following a series of successful Research into Exascale tools; and testing novel joint birds-of-a-feather (BOF) Systemware, Tools and programming models such as sessions at SC and ISC, we expect OpenACC and SMPSs on each Applications) investigates our SC’13 BOF on ‘Building on the other’s applications. the software challenges of European Exascale Approach’ to stimulate interesting debate. MPI for exascale utilising future exascale resources. Coordinated by The three projects will share a booth Finally, it is worth highlighting on the exhibit floor at SC’13, with EPCC’s new exascale project, EPCC, this FP7 project has demos from all three projects. developed from work within now entered its final year. CRESTA will demonstrate its tools CRESTA. Our collaborators at KTH www.cresta-project.eu (TUD’s VAMPIR and MUST, Allinea in Sweden are leading an FP7 DDT and Allinea MAP) and its project on preparing MPI for the applications (such as ECMWF’s exascale. This will explore numerical weather prediction code innovative, potentially disruptive Find us at SC’13 IFS, and HemeLB, the computation concepts and algorithms for • Exascale MPI workshop Haemodynamics code from UCL for message passing. Its ‘Exascale Nov 22, 9.00–13.00 simulating blood flow). MPI’ workshop at SC’13 will be a good opportunity to learn more. • Building on the European We will also show a series of videos Exascale Approach highlighting the socio-economic This is an exciting time in European Nov 19, 12:15–13:15 impact of the different applications exascale research. In its final year, engaged in CRESTA. CRESTA will produce a series of • European Exascale Projects systemware software components, (Booth 374) Future collaboration enhanced versions of our co-design • EPCC, University of Edinburgh The three projects had a successful applications and new novel (Our own booth: 3932) meeting in Barcelona this summer, scientific examples. We will keep where we identified a number of key you posted on their arrival.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 20 Performance of CP2K for an ab initio MD run with 64 water molecules on the Xeon Phi, showing the effect of task placement. First impressions of the Intel® Xeon Phi™
Based on Intel’s Many Integrated which we have successfully ported Fiona Reid Core (MIC) architecture, each Xeon to the Phi and are currently [email protected] Phi card comprises 60 physical optimising with support from the cores with 4-way simultaneous PRACE project. One of our MSc Iain Bethune multi-threading (SMT), meaning that students, Jonathan Low, ported a [email protected] there are up to 240 virtual threads medical imaging application as part available to the user. Each core has of his dissertation project, and we a 512-bit wide SIMD vector- have several external users working processing unit. With a clock speed with codes in the field of of 1.053 GHz, this results in an Geoscience and Optics. aggregate peak double precision For the benefit of others who may floating-point performance of 1.01 be considering Xeon Phi, our key TFLOP/s - all contained inside a findings are below. Over the last few single PCI card package, and using less than 225 Watts! Programmability months at EPCC we If we consider that just 10 years The Xeon Phi cards can be have been evaluating ago, the National Service HPCx programmed using a number of the new Xeon Phi Phase 1 delivered 2.2 TFLOP/s different models including OpenMP, peak, occupied over 40 cabinets MPI, Intel Thread Build Blocks co-processor from and was the sole Capability (TBB), Intel Cilk+, and OpenCL. Computing resource for the entire Intel emphasises the ease-of-use Intel, which recently UK computational science compared with competitors such as powered the Chinese community, then it seems we have CUDA on the NVIDIA GPU platform. come a rather long way in 10 years! To cross-compile code for the MIC Tianhe-2 cluster to architecture, all the user needs to the number 1 spot on Xeon Phi at EPCC do is use the Intel C/C++/Fortran Two Intel Xeon Phi 5110P cards compiler with the –mmic flag and the Top500 list. were installed in May as an ensure that any libraries required by extension to our ‘Hydra’ cluster and the code are also compiled for Xeon were made available to staff and Phi. This makes porting existing students for testing and evaluation. parallel codes much easier than The majority of our initial having to re-write large amounts of investigations have involved the code in another language. CP2K materials science application,
21 Performance of FFT libraries on Xeon Phi. Problem sizes are typical to those used by CP2K.
Application scalability OpenMP and MPI/OpenMP versions of CP2K running in native mode on and low memory usage the Xeon Phi. It is very important to To get the best performance from place threads as close as possible the Xeon Phi, you need an to their parent process while application which can scale up to maintaining overall load balance of 240 cores. Unlike conventional tasks, particularly when running in processors, the Xeon Phi needs to mixed-mode. keep as many of its 240 virtual cores busy as possible. However, Performance tuning the Xeon Phi cards have relatively In addition to parallelism, serial small amounts of memory - 8GB code must also be tuned for the total, or around 34MB per thread. Xeon Phi in order to get maximum Most applications designed for performance. Arranging loops to memory-rich, multi-core CPUs will allow compiler vectorisation (the therefore run out of memory before MIC SIMD unit can perform 8 enough threads can be generated to double-precision FMA per cycle) or make full use of the Xeon Phi by utilising specially tuned libraries processors. New algorithms that (such as Intel’s MKL) for key minimise memory use or maximise computational kernels of your code sharing between threads may be is crucial. needed. The image above shows a Task placement comparison of the performance of the widely-used FFTW 3.3.3 library Even for an application with 240 versus the Intel MKL implementation low-memory threads, if the threads of FFTW3, which has been tuned to are poorly distributed across the specifically for the Xeon Phi. In our physical cores then very poor tests, MKL outperformed FFTW3 by performance can result as the cores up to 6x for 1D FFTs and 3x for 3D will transfer much more memory FFTs. For codes which spend a across the ring interconnect which considerable time doing Fourier couples the cores to main memory. Transforms, using the correct library The figure on the opposite page could have a significant impact on shows the performance of MPI, performance.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 22 MSc in High Performance Computing
Students and staff from all three MScs.
This year we welcome new students from 10 different countries to our MSc in High Performance Computing. The School of Physics & Astronomy company. By undertaking an has recently launched new MSc industry-based dissertation project, Crystal Lei programmes in Theoretical and students will have the opportunity [email protected] Mathematical Physics, so our to enhance their skills and students are now part of a larger employability by tackling a real- taught postgraduate community in world industry project, gaining the School. workplace experience, exploring potential career paths and building It is an exciting time for the MSc in relationships with local companies. HPC, as this year’s students will be 2012 dissertations In addition there will be an the first to have access to ARCHER, The class of 2012 recently received opportunity to win the Summer the new UK National their degree classifications. To see Industry Project Prize. Supercomputer hosted in Edinburgh their MSc dissertations go to: (see p4 for more details). Cluster Competition www.epcc.ed.ac.uk/msc/overview/ At the recent two-day induction We will again be offering a team of programme-structure/msc- event, our new students were students the opportunity to dissertations/. introduced to the University, to participate in the International EPCC, and most importantly to their Supercomputing Conference’s fellow students and teaching staff. Student Cluster Competition as part Not only do the students come from of their MSc dissertation project. a wide range of countries, they have Student teams from around the a diverse range of backgrounds; world build high-performance some have recently graduated from clusters and compete against each MSc in HPC undergraduate studies, some have other to achieve the maximum www.epcc.ed.ac.uk/msc been in employment, while others performance from a set of have been doing research work in benchmarks and applications. other or related fields. They have all Industrial projects now come together to learn about The MSc dissertation is considered www.msc-projects.ph.ed.ac.uk/ parallel programming and HPC by many students to be the technologies. culmination of their degree and we’re pleased to be able to offer ISC’14 Student Cluster Industrial projects such a wide range of project Competition opportunities that cater for the wide In addition to a wide range of http://hpcadvisorycouncil.com/ range of interests, backgrounds and academic dissertation projects, this events/2014/isc14-student- aspirations of our students. year our MSc students will have the cluster-competition/ opportunity to undertake their dissertation project with a local
23 Workshop for research
Software software engineers Sustainability Institute
The Software Sustainability Manifesto. The afternoon was spent Mark Woodbridge, Imperial Institute’s inaugural Workshop for in groups discussing and proposing College London Research Software Engineers was solutions to the issues affecting [email protected] held at the Oxford e-Research software engineers who support Centre in September. research. These are wide-ranging and won’t be immediately resolved It brought together a wide range of but there was a consensus that a interested parties to discuss the strong community, fostered by challenges facing the application of events such as the workshop, software to research, from funding should inform policy-making at the models to infrastructure provision. SSI. They in turn can encourage The day was split into two - with the funders and institutions to apply morning dedicated to sharing of metrics that more effectively value experiences and best practices the contribution of RSEs in the including keynotes from Mark future, hopefully leading to more Software Sustainability Hahnel who left academia to recognition and higher-quality Institute develop Figshare, and Professor Ian software and research. www.software.ac.uk Gent, author of The Recomputation
DiRAC driving test roll-out under way
The “driving test” developed by The Software Sustainability Institute, Software Carpentry and the DiRAC consortium is now being rolled out across DiRAC’s regional sites.
DiRAC is the UK’s integrated Training coordinators at DiRAC’s Mike Jackson supercomputing facility for sites in Durham, London, Leeds, [email protected] theoretical modelling and HPC- Edinburgh, Leicester, Exeter and based research in particle physics, Cambridge are collectively running astronomy and cosmology. through 70 post-doctoral research associates, 130 PhDs and 40 other The driving test is a basic software new users by the end of this year skills aptitude test which covers between 16th September and early useful, and essential, software December. The ongoing cohort is development skills including: estimated to be 30-40 a year. • the shell and automation The test is designed to encourage • version control researchers to undertake training in • testing essential software development • code review skills to both benefit their research • using public/private keypairs and to help ensure DiRAC’s DiRAC at EPCC resources are used as efficiently • secure shell. www.epcc.ed.ac.uk/facilities/dirac and effectively as possible.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 24 We know what you did last summer
Image shows polarizing microscope texture of a thin film of liquid crystal. Pic: redorbit.com
The PRACE Summer of HPC is an outreach initiative which allowed 24 students from across Europe to spend two months at a High Performance Computing centre working on a visualisation project to produce a demo.
The students came from ten Marko Misic (Serbia), Simone de Irina Nazarova European countries, and their Camillis (Italy), and Stamatia Tourna [email protected] projects ranged from modelling (Greece). As Stamatia said in her bioflow in coronary arteries to blog: “‘A Belgian, a Serbian, an visualizing plasma turbulence. They Italian and a Greek are at a bus all shared a keen interest in stop’ could be the beginning of a “These two months working at computer simulation as a scientific joke, but in this case it is the EPCC were wonderful; methodology, a desire to learn more beginning of our day!” Edinburgh really welcomed about it and share their knowledge Projects with other young scientists – to us. It was an amazing “spread some HPC magic”, as one The work the students did was far summer and I would definitely student, Vojtech Bardiovsky, put it. from a joke, however. These were suggest to anyone to give it a exciting times – Marko, who worked Training try. It could really be a on the CP2K code with Iain lifetime’s experience.” The Summer of HPC began with a Bethune, had to defeat the training week at EPCC in Edinburgh. supercomputing Hydra along the Stamatia Tourna Five full-time days of courses in way, and Antoine, together with MPI, Open MP and Scientific Nick Brown, built virtual dinosaurs Visualisation were taught by and then raced them. Stamatia, with Summer of HPC staff. In the the help of Nick Johnson, worked evenings the students found the on implementing Python to work time to explore Edinburgh. After with ScoreP’s tracing library, and their training was completed, they Simone, supervised by Oliver flew off to their host centres to Henrich and Kevin Stratford, was begin work on their projects. looking inside the liquid crystal display with ParaView. Read more on the blog... Four students were hosted at http://summerofhpc.prace-ri.eu EPCC: Antoine Dewilde (Belgium),
25 Supercomputing for the masses
For the second year running, EPCC attended both the British Science Festival – one of Europe’s largest celebrations of science, engineering and technology – and Bang Goes the Borders, a local event that targets families and is held in Melrose, Scotland.
We hoped to enthuse and inform Alongside this we have another Mario Antonioletti the general public, especially young demo showing how parallelism can [email protected] minds, about what supercomputers solve problems more quickly by are, what they are used for, and to sorting coloured balls into boxes. even let them have a go on Individuals and small groups would HECToR, the UK national compete against the clock to supercomputing service, using our complete the sorting task. If too Both science festivals dinosaur-racing demo (see p26). many are working at once they might get in the way – a were extremely busy, For the dino-racer, we used a model synchronisation problem. of Argentinosaurus, a large with over 500 people quadruped that walked what is now Having learned about parallelism we visiting our exhibits. Argentina some 96-94 million years showed some HPC hardware – a ago. Our demo allows three different Cray XT4, a Cray T3E and a Dino-racing proved to aspects of the dinosaur to be Connection Machine blade – be a big draw. More configured: the foot size, the leg comparing these to more familiar dinosaur models are in size and the body size. The model is modern desktop hardware. All the then passed on to GaitSym (running activities work independently of the pipeline, especially on HECToR) which precalculates its each other and people can bipeds like T. rex! motion. participate in the parts that interest them. Several modified dinosaurs are placed on a running track and the We will continue to visit schools and race begins! Excited children science events to show why cheered on their creature to victory supercomputers are important and You can read more about our or ignominy – a bad design risks to convince young people to choose outreach work on the EPCC blog: causing your dinosaur to fall over a career in science. www.epcc.ed.ac.uk/blog before reaching the finishing line.
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 26 Running with dinosaurs
We have developed a dino-racing demo based on the pioneering work by the Animal Simulation Lab at Manchester University
The Animal Simulation Laboratory Our end goal was for the public to Nick Brown at the University of Manchester easily configure their own dinosaur, [email protected] investigates animal locomotion by simulate its movements on HECToR creating computer simulations. It is and then race it against other perhaps best known for its work people’s creatures to see who could simulating dinosaur movements design the fastest one. This allows and, by building up accurate people to do raw science at our models based upon fossil evidence, outreach events - they actually the team have deduced the likely design these creatures and then movements and top speeds of use HPC to validate their dinosaur these prehistoric creatures. before watching it race other HECToR, the UK national similarly designed creatures. supercomputing service, was used Initially we developed a prototype, in conjunction with their GaitSym which connected to GaitSym simulator to do much of the hard- running on HECToR and allowed us core computation and to race existing dinosaurs that the palaeontologists have learned much team at Manchester had very from the results of this work. helpfully provided models for. By far Dinosaurs have a special appeal to the most detailed model was the many and so we thought we might Argentinosaurus and whilst our be able to use GaitSym and these early version was certainly work in models as the basis of an outreach progress, the visual impact of demonstration. This actually forms dinosaur skeletons plodding across an ideal illustration of HPC because the screen was still impressive. This simulation is increasingly becoming gave us confidence that, with some the third research methodology, further development, dinosaur complementing theory and racing could prove to be a experimentation. Deducing successful outreach activity. dinosaur movements provides a Our early work coincided with very clear example of where project calls for the Summer of HPC scientists need to use simulation to programme, which offered test their theories, because undergraduate and junior experimentation is simply not postgraduate university students possible.
27 the opportunity to spend two outreach demo, and it would be months of the summer at European sitting alongside our other exhibits. HPC centres (see p24). The How would it work outside of the programme was specifically after lab? Would it stand up to a full day Our Virtual visualisation projects. Further of heavy use? Would the network development of our virtual connection to HECToR be good Palaeontology demo palaeontology prototype seemed enough? And, most importantly, went down very well at like an ideal fit and Antoine from the would the general public engage the British Science Université libre de Bruxelles joined with it and get a clear idea of the us for the summer. importance of HPC? Festival, and it has since Antoine certainly had his work cut The doors opened at 10am, and it been well received at out as there was plenty to do - such wasn’t long before we had a race two more outreach as configuration of the dinosaurs, going between two dinosaurs. Not events. improvements to the graphics and just kids, but adults too enjoyed reporting real-time simulation configuring their own dinosaur to We have plenty of ideas usage. Lots of development was see how well it would perform. for further done over the eight-week period. Some people thought that a smaller By the time Antoine left, all of our developments. Some of one would go faster, and others that project goals had been met and a larger one would be best. Not all the best have come Virtual Palaeontology had creatures were stable but all at least from questions asked developed from a rough prototype managed to run a few metres to a polished application. about dinosaurs and before falling over and probably We found out plenty along the way, 60% of dinosaurs completed the HPC by the public such as configuring dinosaurs is race. Regardless of whether a during these events. trickier than you might first think! dinosaur completed the race or not, Not all configurations will work and we gave the designer a certificate, poorly designed creatures will fall with an image of their customised over, so half the challenge for creature and vital statistics such as participants is completing the race. its weight, height and top speed. You can read more about It was with some trepidation that we Throughout the day the demo was our dino-racing demo on the arrived in Newcastle for the British kept busy, and at times people were EPCC blog: Science Festival. This was to be the queuing up to try to create the www.epcc.ed.ac.uk/blog debut of our Virtual Palaeontology fastest creature!
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 28 Postgraduate Master’s Degree in High Performance Computing Scholarships available for 2014/15
This MSc is offered by EPCC, an institute at the University of Edinburgh.
EPCC is one of This MSc will equip Europe’s leading participants with the multidisciplinary skills and supercomputing centres knowledge to lead the and operates ARCHER, way in the field of High a 72,000-processor Performance Computing. Cray XC30 system. Through our strong links The University of with industry, we also offer Edinburgh is consistently ARCHER is the new our students the opportunity ranked among the top 50 UK academic High to undertake their Master’s universities in the world*. Performance Computer dissertation with one of a wide range of local *Times Higher World System. companies. University Ranking
www.epcc.ed.ac.uk/msc
The newsletter of EPCC, the supercomputing centre at the University of Edinburgh 29