Keeping it Cool: Liquid Cooling Options for High-Density Data Centers

As technology advances and computing power continues to increase, the demand for high-capacity racks is also on the rise. With specific computing applications requiring more and more processing power, many organizations are looking for racks with requirements of 45 kW or more. However, traditional air cooling systems have limitations when it comes to cooling high-density racks, as air does not have the thermal transfer properties needed to efficiently provide adequate cooling. This can decrease power, decrease performance and reliability of specialized servers, and become less energy efficient as rack power increases.

To address these challenges, more and more organizations are exploring the feasibility of bringing liquid to the rack to increase the capacity and efficiency of data center cooling. Liquid cooling solutions take advantage of the higher thermal transfer properties of water or other liquids to support efficient and cost-effective cooling of high-density rack computers.

There are three main types of liquid cooling solutions: direct to chip, immersive cooling, and rear door heat exchanger (RDHx). In the direct to chip method, the cooled liquid is pumped directly over the processor (CPU/GPU). The warm liquid is then pumped into a chiller that will cool the liquid. This method is the most common today and effectively removes heat from computer systems. Fan speeds can be reduced, reducing the power required.

In immersive cooling, entire systems, unaltered, can be completely immersed in non-conductive fluid. The whole system is usually lowered into a container filled with different types of liquids. As this fluid surrounds the CPUs and GPUs, the heat is drawn away, the warm fluid rises and is cooled externally. Immersion cooling is the most effective cooling method to cool High Performance Servers.

In RDHx, back doors can also be used to cool the heat exhaust air from the servers before the air enters the data center. These doors can be active (with fans) and passive (no fans) and cool liquid which – after the hot air has transferred its hat to the liquid – must be cooled externally via a heat exchanger (e.g. Chiller). Although this method does not use liquid to cool the system, it allows data centers to use current racks and cool up to 200 kW per rack.

There are several advantages to using liquid cooling. First, it can increase computing power per rack. With traditional cooling methods, data centers can only reach racks around 25 kW, but liquid cooling (direct or indirect) can push the limits beyond 100 kW per rack. Second, liquid cooling is more energy efficient and has a lower total cost of ownership. Air cooling and conditioning require a significant amount of electricity to operate with liquid-based refrigeration. Water can be used to sufficiently cool servers. Lower energy bills combined with less maintenance and (potentially) higher reliability levels lead to lower operating costs. Third, highly consistent and accurate cooling is required to meet the needs of the fastest NVidia GPU, Intel CPU, or AMD chips running at their highest performance. With air cooling, hot spots and thermal shutdowns are possible at maximum load on the systems. Fourth, the use of liquid cooling creates a better working environment. The shrill scream of thousands of servers running at maximum performance is eliminated with liquid cooled solutions. You can hear your colleagues again and work in the data center for extended periods of time. Finally, the outgoing heated fluid from the servers can be diverted to the (district) heating system to be reused as a heat source, reducing waste and increasing energy efficiency.

In conclusion, for organizations dealing with the challenges of increasing rack densities, it may be time to recognize the limits of air cooling and consider using liquid cooling technologies to help meet energy and sustainability goals. For those deploying extremely high-density racks (30kW+), there may be no other choice. Liquid cooling solutions offer several advantages over traditional air cooling, including increased computing power per rack