Tech & Trends

General Information
June 15, 2007 • Vol.29 Issue 24Page(s) 24 in print issue

Liquid Cooling In The Data Center

An Update Of Products & Technology Designed To Beat The Heat

The inexorable increase in microprocessor speed and complexity is the foundation upon which the panoply of computing and communication devices powering this information age is built. The nasty side effect of denser CPUs with higher clock speeds is the result of an electronic form of friction—electrical signals (however small) travel over wires (however microscopic) with resistive and capacitive loads—a process requiring energy that is ultimately dissipated as heat. The faster signals are switched, the more power (energy per unit of time) is required.

Perusing Intel’s (http://www.intel.com/) CPU data sheets during the past few years illustrates the problem. A now-ancient 75MHz Pentium didn’t use much more power than a night light, a mere 8W. A state-of-the-art, quad-core Core 2 Q6600 runs a blistering 105W, while many high-end Athlon X2’s top that at 125W. Compounding the problem is dense 1U or blade server packaging, often with two CPUs per server, putting more processors in each rack. The net result is that it’s not uncommon for server racks full of high-density systems to consume more than 15kW, with projections upwards of 24 to 48kW per rack—translating to more than 4kW per square foot—by 2010. Getting this amount of heat away from sensitive components before they incinerate is a significant challenge.

SprayCool’s In-Chassis Cooling Module (www.spraycool.com) has cold plates with piping for cold input and hot return.

Why Liquid?

Recollecting from high school physics, remember that materials have a wide variety of heat-absorption capabilities. Unfortunately, gases (read: air) are at the low end of the spectrum—thus, it’s difficult to transfer much heat through air without a lot of airflow. According to Herb Villa, IT product specialist at enclosure manufacturer Rittal (www.Processor.com/Rittal), “Water is 3,500 times more efficient at transferring and transporting heat than plain old air.” In addition, as Peter Koch, CTO at Knürr (http://www.knuerr.com/), notes, the heat transfer from a solid to liquid is magnitudes better than that to air. Thus, vendors have developed a variety of products that bring liquid’s heat transfer advantage closer to the heat source.

Product Categories

Patchen Noelke, director of marketing at SprayCool (http://www.spraycool.com/), describes the cooling market as a four-layer hierarchy. In-room cooling is the most basic level, encompassing traditional HVAC systems using forced air, while in-row products consist of self-contained cooling units placed adjacent to high-density servers. In-row units still rely on airflow through a rack for heat transfer; however, the cool air source and hot air return are in closer proximity to the heat source.

In-rack products, the next layer in Noelke’s hierarchy, build a cooling unit directly into the enclosure, often replacing a rack door, such as in IBM’s Cool Blue Rear Door Heat eXchanger (http://www.ibm.com/). This has the advantage of completely isolating a hot rack from the room environment and channeling cool air through the enclosure, making it suitable for both blades and 1U servers. The final category of cooling product, so-called in-chassis, or chip-level coolers, brings liquid’s heat removal properties directly to the heat culprit—the CPU. These units replace the traditional radiator-like metal heatsink on a microprocessor with a liquid-filled cold plate that circulates to an external heat exchanger.

Interesting New Products

A number of companies, from behemoths such as IBM to niche specialists such as SprayCool and Rittal, have developed innovative products to address the burgeoning need for heat removal in high-density racks. Villa of Rittal points to the company’s next-generation Liquid Cooling Package, the LCP+, which features a 30kW capacity, noting, “in addition to providing the greater load capacity, airflow and water flow control has been improved, as well as onboard and remote monitoring and control capabilities,” along with the ability to cool one or two cabinets per unit. Rittal OEMs its product to a number of server and storage vendors, including Dell (http://www.dell.com/), IBM, and NetApp (www.Processor.com/NetworkAppliance), and is used in HP’s (http://www.hp.com/) MCS (Modular Cooling System).
The CoolTherm is a similar in-rack product from Knürr, acquired last year by Emerson (www.Processor.com/Emerson-Liebert) and used in Liebert’s X-Treme Density line of enclosures, which comes in four models ranging in cooling capacity from 10 to 35kW per rack. Koch points out that Knürr introduced this product in 2002 and is on its third generation of design.

Using liquid cooling for the CPU is an old technique introduced years ago in supercomputers such as the Cray-1 and lately popular in high-end gaming PCs. In-chassis cooling systems, such as the SprayCool M Series and similar products such as Knürr’s CPU Cooling System or that used in Pyramid Computer’s Cluster Server (www.Processor.com/Pyramid-USA), marry a cold plate on the processor with a rack-mounted manifold and heat exchanger. According to Noelke, a highlight of SprayCool’s system is the use of leak-proof quick connects for all hoses and a special nontoxic, dielectric fluid that won’t damage electronics in the unlikely event of a spill. Heat exchangers on in-chassis systems are typically located at the bottom of the rack and connect to the computer room’s chilled water system.

While ostensibly targeted at large installations with high-density racks, Noelke feels that the SME market is actually a sweet spot because SMEs often have equipment located in office areas not designed as computer rooms and thus difficult to expand with auxiliary conventional forced-air cooling. He remarks that the efficiency and compactness of in-chassis systems can enable smaller companies “to double the number of servers in a convenient location.” Furthermore, he feels that in-chassis systems are the most efficient means of heat removal, noting that they can cool a 12kW rack with only 200W, saving up to 20% in energy costs.

Cooling Of The Future

While CPU manufacturers seem committed to improving the power efficiency of new designs, as witnessed by Intel’s proclamation that its latest Xeon chips usher in “a new era of power efficiency,” the increasing number of processor cores per chip will likely erode these gains, as will the continued push to high-density server packaging. Thus, the need for heat removal in crowded 1U or blade server racks will escalate and exceed the physical limits of conventional room-level, forced-air systems. Liquid-cooling solutions, whether tightly integrated with an equipment rack or embedded within the server, are the most promising solution for taking the heat off of such consolidated, high-density servers.



Knürr’s (www.knuerr.com) CoolTherm In-Rack Cooling Product includes fans that are mounted in rack doors with a closed circulation heat exchanger at the bottom. They connect to a building’s chilled water system so that dissipated heat is not added to the room’s thermal load.

by Kurt Marko