Cloud-based high-performance computing stokes innovation – TechRepublic

Traditionally, high-performance computing (HPC) was the bailiwick of scientific research, but a new report from Forrester reveals that cloud-based HPC has introduced a fresh wave of HPC, spanning across industries.

High-performance computing is made up of “large clusters, comprising compute, storage, and bandwidth, configured to minimize the time to run massive batch jobs that require very high volumes of calculations and/or data throughput typical of scientific research, simulation, and AI,” according to Forrester.

Cloud-based HPC makes HPC accessible to more industries

HPC was once considered “outrageously expensive” and many businesses opted out of exploring the possibilities and focused their attention elsewhere. But the innovation of the cloud in 2007 arrived just in time to make HPC not only available to but invaluable to a much wider swath of industries; HPC was able to address complex simulations, and  massive data analytics. It subsequently spurred AI’s impressive traction across all industries. 

SEE: Virtual hiring tips for job seekers and recruiters (free PDF) (TechRepublic)

“The elasticity that cloud-based solutions afford can minimize costs, but regular workloads that run hot will quickly result in huge bills,” and to combat this, Lauren Nelson, Forrester vice president and research director, said: “Have policies in place that create protections for cost escalations. Run tests to ensure things are set up correctly. Ensure that your mitigating ‘failure rate upon provisioning’ to ward off cost escalation of failures for short term data set analysis. Many companies lean on MSPs focused on HPC for their first cloud-based HPC workloads to ensure efficiency and avoid common mistakes that may escalate cost at scale.”

HPC provides enterprises with accessible (affordable) supercomputing, and was associated with one-time, government-commissioned research projects. HPC’s “computational nodes conjoined with high volumes of storage and bandwidth to handle complex scientific engineering and AI workloads.”

DNA sequencing was a particularly valuable arena for cloud-based HPC used by medical startups. Today, the use of the public cloud for cloud-based HPC is essential for North American and European enterprise (36%) infrastructure tech, and 34% for private cloud users.

The maturity of the cloud coincided with cloud-based HPC and Forrester explored newly available HPC options, key best practices and industry-level use cases.

Cloud-based HPC is thriving in industries and media, entertainment, and leisure (35%); financial services (33%); and manufacturing (26%) are either using HPC or plan to soon.

The Forrester report credits HPC’s growth due to the demand for fast answers to the growth of complex problems, the expansion of cloud-based HPC, the necessity to address the demand of the  expansion of private cloud options, and lastly, for such use cases as AI, faster networks enable high-data-throughput. 

In addition to being initially used by biotechnology, HPC’s development has continued in traditional sectors such as genetics, government (defense, election campaigns, cybersecurity, and simulations), the military and academia, but it’s also expanded into industries that need HPC’s substantial computational resources to get answers at fast speeds:

Adopting best practices from peers

Forrester conducted a series of  interviews with users of cloud-based HPC  to “better understand” the current best practices  and it advised: Pick the best solution for each HPC workload, requirements vary by app; group apps by the data they use; understand app requirements before you migrate to the cloud; evaluate the use of containers and.or bare metal, watch costs by considering hybrid approaches; monitor was closely to prevent exorbitant bills, and avoid HPC cloud lock-in (even if you sacrifice productivity and access to new services, because of costs). 

To start now, industries need to follow the following key steps: Start with some research and relationship building, factor in IT for on-premises deployments, and work with your security peers on regulations and impact on operations.

“HPC can be done either in the cloud or on-prem,” Nelson emphasized. “Public cloud just makes it more accessible with less upfront investment. For those that invested, they are less likely to leverage a pay-per-drink model, and that is OK. Part of what they will need to investigate is if their HPC usage exceeds their existing investment, whether they should leverage cloud or continue to invest on prem. This may be dependent on scope, and connectivity to the systems already in place.”

Best of the Week Newsletter

Our editors highlight the TechRepublic articles, downloads, and galleries that you cannot miss to stay current on the latest IT news, innovations, and tips.
Fridays