Using HPC to unlock big data's potential
As big data becomes increasingly strategic to organisations, conventional technologies are failing to fulfil their needs. Only high-performance computing...
High-performance computing (HPC) isn’t just a new technology sector – it’s fast becoming a vital part of a diverse range of industries.
The international high-performance computing (HPC) market isn’t just about competing for the fastest supercomputer or providing bigger, faster clusters to process big data. It’s a huge growth industry that’s behind a myriad of industries – from scientific applications such as running the data from the Large Hadron Collider or DNA research to sectors as diverse as aerospace, manufacturing and finance.
In this new competitive landscape, traditionally technology-rich economies from the US to China, the UK and European Union are fighting to make their mark and so far, it looks like China could be in the lead.
China leads the way
In supercomputers, China is emerging as a world leader, according to the global foundation monitoring HPC systems, TOP500.
“For six years in a row, China has held on to the number-one spot thanks to Tianhe-2, a supercomputer developed by China’s National University of Defense Technology. More significantly still, China nearly tripled the number of systems on the list in 2015, while the number of supercomputers in the United States fell to its lowest point since TOP500 rankings began in 1993,” wrote Lucy Hattersley in her article for Think Progress.
And Chinese firms, such as Lenovo, are capitalising on this cutting edge.
According to Adalio Sanchez, senior vice president of Lenovo’s Enterprise Systems Group: “Clients seek an IT partner with extensive skills and experience in designing solutions that can meet their unique needs. Our Lenovo team has a long heritage of industry expertise in HPC leading-edge technology. We are investing in innovation – both in research and development and in services and support – to help clients capitalise on emerging trends in HPC, including cloud and big data analytics.”
What’s the rest of the world doing with HPC?
A new report from the Information Technology and Innovation Foundation (ITIF) talks about the “vital importance” of HPC to US competitiveness: “Because HPC stands at the forefront of scientific discovery and commercial innovation, it is positioned at the frontier of competition – for nations and their enterprises alike.”
While in 2012, the European Union released a similar report entitled High-Performance Computing: Europe’s Place in the Global Race, which set out goals and action plans to help Europe to the top of the HPC market, and the EU has invested €700m in HPC in a public/private initiative.
However, Europe has a lot of ground to make up. In the same TOP500 report, Europe’s supercomputer total came out at less than 100, with 32 in Germany and 18 each in the UK and France, compared to 199 in the US and 109 in China.
“Moreover, Europe has not produced a substantial supercomputer vendor, and the European marketplace represents only one-quarter of global spending on HPC systems,” the ITIF report adds.
However, it’s not all bad news – Europe is punching above its weight in HPC software applications. “While 83 per cent of HPC application software used in Europe was created there, Europe’s share of HPC system vendors stands at less than five per cent,” the ITIF says.
An IDC analysis also found that Europe was focused more on HPC for science than industry, although the situation is improving.
“Today, there is a European [HPC] consciousness, a European-wide scheme for categorising HPC centres, more of the world’s top 50 supercomputers and improved access for industry of all sizes (including SMEs)”, the IDC says.
Here’s what the UK should focus on
Providing industrial access to supercomputing should be a top priority for the UK and Europe, not just to improve its competitiveness in the HPC market, but also to ensure the competitiveness of companies in a whole host of other industries.
However, Europe’s focus on HPC software and applications could be a smart move. According to the ITIF report, the US currently leads in HPC adoption, deployment and development, but China is threatening that dominance. China is expected to bring two new supercomputers online this year, capable of 100 petaflops of performance, but the US’s next two systems for the Department of Energy, which will be 150 petaflop supercomputers, aren’t due until 2017 or 2018.
It would be difficult for Europe to catch up in hardware as the US and Asia move further into exascale computing, so a focus on software could help ensure partnerships with hardware vendors. It’s great for European and UK firms to make their mark in software but it’s still important to the rest of their economies to have the right hardware.
The ITIF recommends that the US stay ahead in hardware manufacture because it believes it would be risky to depend on foreign tech vendors for the systems they need. Fostering good working relationships with foreign tech vendors, whether American or Asian, by staying ahead in HPC software would greatly reduce that risk for Europe.
The EU also needs to continue to ensure HPC deployment, as it is with ExaNeSt, a consortium of 12 HPC hardware and software developers and suppliers that is hoping to “build a straw man exascale HPC prototype in 2016 and a full prototype version by 2018 as a means to drive indigenous HPC technology development across the EU”.
Improved industrial access to HPC, strong partnerships with HPC hardware vendors and a focus on HPC deployment are all necessary not just to help Europe and the UK in this technology sector, but to ensure their other industries remain competitive.