ExaNest Logo

European Exascale System Interconnect and Storage

European Exascale System Interconnect and Storage

News / Press

ExaNeSt Project Successfully Builds Prototype for Exascale, 16 September 2019

12 Blade HPC Testbed of ExaNeSt in Liquid-Cooling Rack

Today the whole world is trying to push supercomputer technology to the exascale level, i.e. a billion billion calculations per second. In Europe, this exascale vision will be made possible with contributions by Consortia funded by the Horizon 2020 Programme of the EU in projects such as ExaNeSt, which built a first ‘straw man’ prototype, having worked on this since 2016.

The ExaNeSt Consortium consists of twelve partners across seven EU countries, each of them with expertise in a core technology needed for innovation to reach exascale. ExaNeSt has taken a sensible integrated approach to co-design the hardware and software that is needed to enable the prototype to run in real-life evaluations.

The prototype is now successfully built, demonstrating energy efficiency in a High-Performance Computing (HPC) testbed: the energy consumed for solving a given problem on this new platform is 3 to 10 times lower than what traditional HPC processors of the same time generation consume for solving the same problem. The substantial prototype has been validated through the execution of full HPC applications from materials science, climate forecasting, computational fluid dynamics, astrophysics, neuroscience, and a database.

ExaNeSt addressed key issues such as energy efficiency by using ARM cores, moving data across shorter distances, and avoiding unneeded copyings of data; it uses power-efficient and silent liquid cooling; the prototype has non-volatile memories integrated into the processor fabric and innovative fast interconnects, while we worked to avoid congestion.

ExaNeSt has contributed to a new generation of European Supercomputers, thus creating new business opportunities. The project’s advances in performance and efficiency will enable SME's in several sectors of the economy to utilize HPC and data analytics, with an attractive trade-off between usability and affordability. ExaNeSt has also educated numerous young engineers, contributing to close the large skills gap in academia and industry.

The energy-efficient solutions are expected to spread to standard off the shelf computing systems, assisting the efforts of the EU to reduce carbon dioxide emissions and reach environmental targets.

ExaNeSt collaborates with other European R&D projects and partners, including:

  • EuroServer, which developed the underlying efficient communication between ARM processors;
  • ExaNoDe, which focused on novel three-dimensional integration technologies and on systems software;
  • ECOSCALE, which develops programmable-hardware accelerators for specialized computations;
  • EuroEXA, which is the follow-on project of the above;
  • Kaleao Ltd., which enables and produces new generation computing platforms by converging compute, storage, and networking into efficient, extremely compact, and transparent server solutions.
  • Xilinx Inc., providing FPGA technology and flexible high speed communication.
  • Micron Inc., for advanced low power memory and storage technology.


PRESS RELEASE in PDF: ExaNeSt Project Successfully Builds Prototype for Exascale

VIDEO on Youtube: https://www.youtube.com/watch?v=oEBJiP0wmPQ

ExaNeSt in the News :

ExaNeSt Milestone-4 achieved: Six-Blade HPC Testbed up and running, 16 April 2019

The HPC Testbed Prototype that ExaNeSt is building has now been upgraded, it is up and running 24/7, and it is being used by the partners of the project for running our Applications on it and evaluating the System as well as the effects of the tuning of the Applications during the project.

This HPC Testbed consists now of 6 liquid-cooled Blades, which contain a total of 24 "QFDB" daughter boards.

This amounts to a total of 96 nodes (FPGAs) in the system, containing in total: 384 processor cores (64-bit A53's), 1.5 TeraBytes of DRAM memory, and 6 TeraBytes of Solid-State Disk (SSD) storage.

The Interconnection Topology is a 3-Dimensional Torus using the project's own switches and routing functions, and the network interfaces feature ExaNeSt-own remote DMA engines with 1024 channels each, multiple mailbox queues, and resilience features.

The intra-QFDB links transfer 16 Gigabits per second each, at a sub-microsecond latency.

Our MPI library now contains multiple features for high performance, and the BeeGFS parallel file system has been integrated.

Boot reliability and speed have been improved, and the management network uses 10 GigEthernet.

Overall operational reliability is improved, including the handling of page faults during RDMA operations and network timeouts and retransmissions in hardware and using coprocessors.

This HPC Testbed has now been in continuous operation for many days, successfully running multiple Applications, including the very demanding HPCG Benchmark on all 384 cores for many hours.

More Blades will be added next month.

The HPC Testbed Prototype that ExaNeSt is building has now been upgraded, it is up and running 24/7, and it is being used by the partners of the project for running our Applications on it and evaluating the System as well as the effects of the tuning of the Applications during the project.

This HPC Testbed consists now of 6 liquid-cooled Blades, which contain a total of 24 "QFDB" daughter boards.

This amounts to a total of 96 nodes (FPGAs) in the system, containing in total: 384 processor cores (64-bit A53's), 1.5 TeraBytes of DRAM memory, and 6 TeraBytes of Solid-State Disk (SSD) storage.

The Interconnection Topology is a 3-Dimensional Torus using the project's own switches and routing functions, and the network interfaces feature ExaNeSt-own remote DMA engines with 1024 channels each, multiple mailbox queues, and resilience features.

The intra-QFDB links transfer 16 Gigabits per second each, at a sub-microsecond latency.

Our MPI library now contains multiple features for high performance, and the BeeGFS parallel file system has been integrated.

Boot reliability and speed have been improved, and the management network uses 10 GigEthernet.

Overall operational reliability is improved, including the handling of page faults during RDMA operations and network timeouts and retransmissions in hardware and using coprocessors.

This HPC Testbed has now been in continuous operation for many days, successfully running multiple Applications, including the very demanding HPCG Benchmark on all 384 cores for many hours.

More Blades will be added next month.

Visit of Mr. Jean-Eric Paquet, EU Director General of Research and Innovation, to FORTH, during which the results of ExaNeSt have been presented to him, 28 March 2019

M. Katevenis, ExaNeSt Coordinator, explains the project to Mr. Paquet, while standing in front of the secondary, air-cooled prototype (used for firmware & systems software development);

Mr. Paquet admires an ExaNeSt QFDB board (4 nodes)

ExaNeSt in the News, 22 Jan 2019

THE NEXT PLATFORM: "HiPEAC: Shifting Focal Points in European HPC Research"

Important milestone of the ExaNeSt project: integration of the QFDB compute node on a mezzanine board, 20 September 2018

An important integration milestone of the ExaNeSt project has been reached, with the successful integration of the project's Quad-FPGA Daughter Board (developed by FORTH) on a mezzanine board (developed by Iceotope). This is a major step towards the realization of the project's liquid-cooled rack infrastructure.

All four QFDBs have already booted on top of the Mezzanine offering a high density configuration. All intra- and inter-Mezzanine high speed links got successfully tested, and 1Gb management Ethernet to all 4 QFDBs as well.

Remote connectivity to the UART and JTAG  is also available through an upgraded BMC (Baseboard Management Controller).

This is also a good news for the sister EuroEXA project, which builds on ExaNeSt technology.

Test setup at Iceotope (Shefield, UK), with one QFDB compute node and one mezzanine

Test setup at FORTH (Heraklion, Greece), with four QFDB compute nodes and one mezzanine

Release of HPC mini-applications by the ExaNoDe project, 09 February 2018

https://gitlab.version.fz-juelich.de/exanode

ExascaleHPC: the ExaNoDe, ExaNeSt, EcoScale, and EuroEXA projects, 23 January 2018

This Workshop presents the main results from the three FET-HPC projects, ExaNoDe, ExaNeSt, and EcoScale, which are at the end of their 2nd year, each; also, the newly-started follow-up FET-HPC project, EuroEXA, presents its plans for the next 3 years.

The Path to Exascale, scientific computing world, 13 November 2017

Robert Roe reports on the potentially far-reaching benefits of exascale computing for European research. Online Article: https://www.scientific-computing.com/feature/path-exascale

Presentation of the ExaNeSt project, ETP4HPC, ESD ROUND-TABLE, 18 May 2017

ExaNeSt presentation slides by Peter Hopton (Iceotope), presented at the ESD ROUND-TABLE on "Exploiting the Potential of European HPC Stakeholders in Extreme-Scale Demonstrators" during the European HPC Summit Week at Barcelona.

Interview on ExaNeSt's exascale potential, April 2017

Martin Kersten (CWI and MonetDB Solutions) at HiPEAC CSW, April 2017 on "How ExaNeSt is clearing the memory barriers to exascale".
Interview video: https://www.youtube.com/watch?v=Og_aGNJg5pk

Presentation on Cooling, HPC User Forum, April 2017:

Video from the presentation:  "Cooling the EU ExaScale Project ExaNeSt", by Peter Hopton (Iceotope), presented at the HPC User Forum, Santa Fe, April 2017:  https://www.youtube.com/watch?v=mXHdjGfMy10.

Presentation on Cooling, HPC User Forum, April 2017:

Video from the presentation:  "Cooling the EU ExaScale Project ExaNeSt", by Peter Hopton (Iceotope), presented at the HPC User Forum, Santa Fe, April 2017:  https://www.youtube.com/watch?v=mXHdjGfMy10.

ACM SIGMOD, 26 May 2016:

Professor Martin Kersten (CWI and MonetDB Solutions), and a member of the ExaNeSt Consortium, is the recipient of the 2016 SIGMOD Systems Award for the design and implementation of MonetDB, a pioneering main-memory database system based on a columnar data organization, which is also one of the key applications in ExaNeSt.

Call for Tenders, 28 April 2016:

QFDBrevA CfT:  Call for Tenders for selecting a Contractor based on Negotiations, for the Implementation of the Quad-FPGA Daughter Board (revision A) Test PCBs.

Press Release, 7 April 2016:

ExaNoDe selects underlying technology components: The ExaNoDe project is part of a wider group of EU funded projects as part of a strategic vision for economical, low-power approaches. Together with ExaNeSt, EcoSCALE and Mont-Blanc, which are also funded from the EU framework program for research and innovation Horizon 2020, ExaNoDe will use an ARM-based architecture as a major component of its compute nodes.

Press Release, 1 February 2016:

How to fit ten million computers into a single Supercomputer? – The ExaNeSt project paves the way:  European consortium becomes the trailblazer in the development of the most challenging architectures in next-generation computing. [PDF ]

ExaNeSt in the News:

ExaNeSt in the HiPEAC 2016 Conference, Prague, Czech Republic, 18-20 January 2016

ExaNeSt kick-off meeting, 10-11 December 2015:

All ExaNeSt partners have met in Heraklion last week to get our exciting project started. During these two days of intense discussions, we exchanged on a variety of technical matters:

  • Applications for the exascale era, both in HPC and Big-data contexts
  • Interconnect architecture, implementation and simulation
  • Design of exabyte storage systems
  • Implementation of our node and rack prototypes
  • Integration and optimization of the partners' applications onto our prototypes

We also discussed many management points:

  • Project schedule and collaboration axes
  • Day-to-day management
  • Reporting of resources, publications and dissemination activities
  • Utilization of the project website and collaborative software infrastructure

A joint INFN-INAF Press Release in Reasearch Italy;  an interview about ExaNest topics in the monthly INFN newsletter;  a Press Release (in Italian) in the INFN newsletter.

Partners joined for a family picture in front of the FORTH premises

You can also view many pictures of the event on our Twitter account.

The European Exascale Projects, including ExaNeSt, were presented and discussed at a Bird-of-a-Feather session at Supercomputing'15 (SC15), in Austin, TX USA, on 19 November 2015: see the main slides here, and the overall BoF and conclusions here.

FET-HPC Projects Meeting, 30 Sep 2015:

HPC User Forum, 25 Sep. 2015:

In a video from the Disruptive Technologies session at the 2015 HPC User Forum, Peter Hopton from Iceotope presents:
ExaNeSt technology: Targeting Exascale in 2018: https://www.youtube.com/watch?v=GM0hkOLOzkw
Also reported in  http://insidehpc.com/2015/09/exanest-technology-targeting-exascale-in-2018/