The University provides more than half of the resources for the National Science Foundation’s national research cyberinfrastructure ecosystem — a system allowing researchers nationwide to utilize the National Center for Supercomputing Applications’ prowess.
This leadership can be attributed to Blue Waters — the former world leader in computing and producer of 13 quadrillion calculations a second — and its successors: Delta, Hydro and soon the most powerful, DeltaAI.
“We hope that any day now, we will be given the official approval to go into production (on DeltaAI),” said Brett Bode, DeltaAI assistant director. “We should be within a couple of weeks at the latest.”
Supercomputing, according to Delta Technical Director Greg Bauer, is a more outdated term for what the NCSA actually does. More accurately and generally put, it participates in research computing, providing services to students looking for places to run their advanced code.
“Students have always been the dominant user of computational resources for academic projects, which is what mostly we support,” Bode said.
Get The Daily Illini in your inbox!
The NCSA packs all of the University’s research computing projects in a 20,000-square-foot room. Projects like Hydro, Delta and DeltaAI take up about half this space, with a massive vacancy where Blue Waters once sat, opening up the rest of the room for future projects.
The NCSA was created by researchers seeking to harness extensive computing to benefit society. A physicist, curious to study the inner workings of the universe, proposed the idea of the NCSA to Hydro and former Blue Waters Director Bill Kramer.
Blue Waters
Kramer’s Blue Waters project won its funding in an NSF national competition in 2007.
With a budget of over $520 million, the computer was designed by manufacturer Cray to support thousands of researchers to perform advanced frontier science and engineering research that would have been unattainable through other resources.
“Blue Waters can be considered a universal instrument, the world’s highest-precision and largest microscope,” Kramer said.
The computer has been used to research atmospheric science and study problems such as climate change, environmental changes and advanced geography. In a notable 2015 study, former President Barack Obama set a national goal of resolution and decision-making to protect the Arctic against climate change in Alaska.
Blue Waters was used to create digital elevation models, or DEMs, of Alaska. DEMs are 3D topographic maps designed by the supercomputer that display the elevation of all features on the Earth’s surface.
The maps produced by this project, called ArcticDEM, helped scientists better track ice loss and continue other research in the Arctic. They were created with data provided by the National Geospatial-Intelligence Agency.
“What it took was getting sterile satellite data from NGIA, having that flow into the University of Minnesota and then the data coming down to Blue Waters, all over networks in large amounts of data, tens and thousands of petabytes,” Kramer said.
The ArcticDEM project transformed research in the Arctic. It aided scientists in researching atmospheric conditions and monitoring ice caps, helping with a broad range of decision-making in the Arctic. The project was finalized in the fall of 2017.
“It was a really amazing system, but computing systems don’t last forever,” said Bill Gropp, NCSA director .
Blue Waters ended operations on Jan. 1, 2022, with some projects carrying over to future programs.
Hydro, New Frontier Initiative
The Hydro system, developed under the New Frontier Initiative, is a computing cluster composed of 70 server nodes. Nodes are units of computers all integrated into networks. Hydro and the New Frontiers Initiative specialize in advancing research and development in national security and preparedness by reacting to the responses correlated to the University.
Hydro was constructed with parts from Blue Waters after it was decommissioned. The work done in Blue Waters was moved to a smaller environment with a smaller setting and fewer parts of the system. The storage system in Hydro is the same as Blue Waters but will be retiring in December 2024 for a replacement.
“Hydro was a self-funded evolution of Blue Waters so we could provide continuing processing for the people that were using Blue Waters,” Kramer said, who is also the executive director of the New Frontiers Initiative.
Delta, DeltaAI
A downgrade in performance and size would lead one to believe that Delta was like a quick rebound from a long-term relationship. But, in an industry where minimalism is king, Delta is surely royalty.
“Blue Waters was decommissioned, but the same amount of computing power was basically replaced by Delta at about a 1oth the size of the physical infrastructure, space and cost,” Bode said.
While considered a lesser track two compared to Blue Waters’ designation of track one, Delta’s lower barrier of entry creates more opportunities for students.
“We serve a lot of smaller-scale projects that we have something on the order of 700-800 projects on our system, and something like 3,000 users,” Bode said. “Which is quite a substantial lot more than a track one will serve.”
These users are typically students, mostly from the University but many from around the nation.
While the University’s computing power is significant, it’s more of a stepping stone for academics before it works on production-scale projects at companies.
These projects, however, are increasingly becoming focused on artificial intelligence. As a reaction, DeltaAI was proposed.
The new system is set to be three times the performance of Delta, twice the performance of Blue Waters and made up of only three racks. Blue Waters had 288 racks.
However, this does not put Illinois in competition with industry giants like OpenAI and Elon Musk’s Colossus, who have tens to hundreds of thousands of GPUs compared to the University’s couple hundred, but that’s nowhere near the goal.
“It’s really great to be able to make these systems available to researchers across the country,” Gropp said. “It’s always a lot of fun to see what people are doing with them. It’s exciting to be able to enable that work.”
A look into the future
Where the University and the state may compete is in quantum advancements. With help from the University’s Illinois Quantum Information Science and Technology Center, the Midwest has become a “quantum powerhouse.”
However, integration with NCSA’s own powerhouse is not straightforward.
“Quantum machines are not going to be able to do everything, so you are going to have this hybrid system,” Gropp said, referencing a dynamic system between classical computing and quantum technology. “We’re going to have to do a lot of experimentation and exploration.”
These machines offer valuable power but have niche applications.
“There are a lot of different areas benefiting from quantum computing, but it’s still limited,” Bauer said.
He described quantum as a space to offload more computationally intensive problems rather than a realistic resource for researchers. However, with the help of IQUIST, the NCSA will soon be home to a quantum and classical computer hybrid, whether novel or not.
“We’ll have something in two years, probably,” Bauer said. “It’s really more of a proof of concept system than it is anything that’s going to be useful for real-world problems.”
