Main Content Region

High Energy Physics meets High Performance Computing

High Energy Physics meets High Performance Computing

by Dr Sinead Ryan, School of Mathematics, Trinity College Dublin

Quantum Chromodynamics (QCD) is the theory which describes the strong nuclear force which binds elementary particles called quarks and gluons together to make hadrons such as the protons and neutrons. This force has an unusual property: it is very weak when the quarks are close together growing stronger as the distance between quarks grows and then remaining constant even as the quarks are moved still further apart.

This is called asymptotic freedom.

The idea that QCD describes the strong force has been experimentally confirmed many years ago but perhaps surprisingly, no free single quarks have been observed. This is called confinement and it is believed to arise from complex dynamics caused by the strength of the forces generated by QCD. A quantitative understanding of this mechanism is one of the outstanding challenges in modern physics. Asymptotic freedom means that the usual theoretical tools will not work for QCD and a more robust approach is needed. Lattice QCD describes QCD in terms of its fundamental particles, the quarks and gluons while also making predictions of hadron properties. It is these predictions that are vital to understand the complete picture of high-energy physics and also to understand new physics discovered at experiments like the Large Hadron Collider (LHC) at CERN.

In lattice QCD space and time are discretised on a four-dimensional grid and hadronic physics is determined by numerical simulation. This is one of the Grand Challenge projects, requiring capability computing on a vast scale. In addition to being users, Lattice QCD physicists are at the forefront of supercomputer design: for example QCDOC (, a joint project with IBM was a precursor to BlueGene. Currently, the top three computers on the “Green 500” list (the most energy-efficient supercomputers) use the QPACE architecture, a custom built system designed by lattice physicists (in collaboration with industrial partners).

The School of Mathematics, TCD has a lattice QCD research group with three permanent academic staff, one post-doctoral fellow and four graduate students. Currently, the main research goals of the TCD group are in spectroscopy, understanding QCD at finite temperature and measuring the fundamental parameters of QCD and similar theories. The spectroscopy effort is making predictions for what will be seen in the next generation of high-precision experiments that will probe the confinement mechanism of QCD. Research in QCD at the high temperatures found in the early universe will shed light on the first moments after the big bang, as the universe cooled from a hot plasma of quarks and gluons into its current state, where these constituents are confined. To understand new data coming from the LHC at CERN, new theories with structural similarities to QCD are being developed and tested. All these activities make very substantial use of the computer systems of TCHPC.

The group is a node in a Marie Curie Initial Training Network (ITN), funded by the European Commission's Seventh Framework Programme (FP7). “STRONGnet” is the Strong Interaction Supercomputing Training Network and its purpose is to provide training and experience to researchers of any age or nationality by facilitating interactions and exchanges between research institutions. Currently, STRONGnet consists of ten university teams from seven different EU member states with the common scientific goal of solving QCD by numerical simulation on supercomputers (Lattice QCD) and developing the methods and tools to do this. Since these calculations require enormous supercomputing power, there are working groups dedicated to computer hardware development, improvement of numerical algorithms and the optimization of software. STRONGnet's associated partners (including Eurotech and IBM at Böblingen in Germany) will share their expertise and provide internships to improve the training of the early-stage researchers.

The TCD group has recently begun to use Graphics Processors (GPUs) as compute accelerators for lattice QCD research. GPUs are mass-produced, mostly for the computer gaming market and so are extremely cheap. Each chip hosts a very large number of simple processing cores. The latest generation has 448 of these cores, clustered into groups of sixteen. While the original design of these cores was to enable the GPU to very rapidly render three-dimensional scenes on a two-dimensional screen, the mathematical operations needed to do this quickly are similar to those found in many scientific codes. If a problem can be coded effectively to make use of all these cores simultaneously, then the GPU can deliver impressive computing power very cost-effectively. Measuring how quarks move through a gluon field is just such a calculation and a research group based in Boston have developed software that sustains more than 100 GFlops per GPU. This is substantially better performance that a modern CPU and can be done for a small fraction of the price of building a computer cluster with the same performance. The TCD group is using this software system and is collaborating with TCHPC to maintain a 4-GPU system both to explore the advantages of GPUs in scientific calculations as well as continuing active research projects.

The group collaborates with researchers at the Thomas Jefferson National Acce lerator Facility (JLab) and other institutions in the US giving us access to significant computing resources through the US DOE SciDAC (Scientif ic Discovery through Advanced Computing Program) programme. In addition the group has been awarded computing time through the DEISA (Distributed European Infrastructure for Supercomputing Applications) initiative and is currently running code at a number of European supercomputing nodes.