Wrangler Supercomputer Speeds through Big Data
March 18, 2016 | University of Texas at AustinEstimated reading time: 9 minutes
Handling big data can sometimes feel like driving on an unpaved road for researchers with a need for speed and supercomputers.
"When you're in the world of data, there are rocks and bumps in the way, and a lot of things that you have to take care of," said Niall Gaffney, a former Hubble Space Telescope scientist who now heads the Data Intensive Computing group at the Texas Advanced Computing Center (TACC).
Gaffney led the effort to bring online a new kind of supercomputer, called Wrangler. Like the old Western cowboys who tamed wild horses, Wrangler tames beasts of big data, such as computing problems that involve analyzing thousands of files that need to be quickly opened, examined and cross-correlated.
Wrangler fills a gap in the supercomputing resources of XSEDE, the Extreme Science and Engineering Discovery Environment, supported by the National Science Foundation (NSF). XSEDE is a collection of advanced digital resources that scientists can easily use to share and analyze the massive datasets being produced in nearly every field of research today. In 2013, NSF awarded TACC and its academic partners Indiana University and the University of Chicago $11.2 million to build and operate Wrangler, a supercomputer to handle data-intensive high performance computing.
Wrangler was designed to work closely with the Stampede supercomputer, the 10th most powerful in the world according to the bi-annual Top500 list, and the flagship of TACC at The University of Texas at Austin (UT Austin). Stampede has computed over six million jobs for open science since it came online in 2013.
"We kept a lot of what was good with systems like Stampede," said Gaffney, "but added new things to it like a very large flash storage system, a very large distributed spinning disc storage system, and high speed network access. This allows people who have data problems that weren't being fulfilled by systems like Stampede and Lonestar to be able to do those in ways that they never could before."
Gaffney made the analogy that supercomputers like Stampede are like racing sports cars, with fantastic compute engines optimized for going fast on smooth, well-defined race-tracks. Wrangler, on the other hand, is built like a rally car to go fast on unpaved, bumpy roads with muddy gravel.
"If you take a Ferrari off-road you may want to change the way that the suspension is done," Gaffney said. "You want to change the way that the entire car is put together, even though it uses the same components, to build something suitable for people who have a different job."
At the heart of Wrangler lie 600 terabytes of flash memory shared via PCI interconnect across Wrangler's over 3,000 Haswell compute cores. "All parts of the system can access the same storage," Gaffney said. "They can work in parallel together on the data that are stored inside this high-speed storage system to get larger results they couldn't get otherwise."
This massive amount of flash storage comes from DSSD, a startup co-founded by Andy Bechtolsheim of Sun Microsystems fame and acquired in May of 2015 by EMC. Bechtolsheim's influence at TACC goes back to the 'Magnum' Infiniband network switch he led design on for the now-decommissioned Ranger supercomputer, the predecessor to Stampede.
What's new is that DSSD took a shortcut between the CPU and the data. "The connection from the brain of the computer goes directly to the storage system. There's no translation in between," Gaffney said. "It actually allows people to compute directly with some of the fastest storage that you can get your hands on, with no bottlenecks in between."
Page 1 of 3
Suggested Items
KIC’s Miles Moreau to Present Profiling Basics and Best Practices at SMTA Wisconsin Chapter PCBA Profile Workshop
01/25/2024 | KICKIC, a renowned pioneer in thermal process and temperature measurement solutions for electronics manufacturing, announces that Miles Moreau, General Manager, will be a featured speaker at the SMTA Wisconsin Chapter In-Person PCBA Profile Workshop.
The Drive Toward UHDI and Substrates
09/20/2023 | I-Connect007 Editorial TeamPanasonic’s Darren Hitchcock spoke with the I-Connect007 Editorial Team on the complexities of moving toward ultra HDI manufacturing. As we learn in this conversation, the number of shifting constraints relative to traditional PCB fabrication is quite large and can sometimes conflict with each other.
Standard Of Excellence: The Products of the Future
09/19/2023 | Anaya Vardya -- Column: Standard of ExcellenceIn my last column, I discussed cutting-edge innovations in printed circuit board technology, focusing on innovative trends in ultra HDI, embedded passives and components, green PCBs, and advanced substrate materials. This month, I’m following up with the products these new PCB technologies are destined for. Why do we need all these new technologies?
Experience ViTrox's State-of-the-Art Offerings at SMTA Guadalajara 2023 Presented by Sales Channel Partner—SMTo Engineering
09/18/2023 | ViTroxViTrox, which aims to be the world’s most trusted technology company, is excited to announce that our trusted Sales Channel Partner (SCP) in Mexico, SMTo Engineering, S.A. de C.V., will be participating in SMTA Guadalajara Expo & Tech Forum. They will be exhibiting in Booth #911 from the 25th to the 26th of October 2023, at the Expo Guadalajara in Jalisco, Mexico.
Intel Unveils Industry-Leading Glass Substrates to Meet Demand for More Powerful Compute
09/18/2023 | IntelIntel announced one of the industry’s first glass substrates for next-generation advanced packaging, planned for the latter part of this decade.