IDC Watch | Embracing Liquid Cooling: The Era of Sustainable and Efficient Operations

Guide:

In this article, we will focus on the liquid cooling technology of data centers, think "cool" about the challenges and solutions faced by data centers in heat dissipation under the "hot" of computing power.

Air Cooling - Far from Adequate

Currently, the power requirements of data centers are increasing with the ever growing demand for more powerful CPU and GPU power consumption in artificial intelligence and high-performance computing systems, which also puts forward higher requirements for heat dissipation. This is an urgent problem for data centers to solve. The use of liquid cooling technology has become an inevitable choice for data centers. A mix of air and liquid cooling technologies will also be put on the agenda.

Energy consumption and heat dissipation are new challenges for data centers. According to the "Report on 'New Infrastructure' Development in China" released by CCID in 2020, the energy consumption of global data centers will account for the largest share of global energy consumption by 2025 - up to 33%. In that year, the growth rate of China's data center power consumption has exceeded 12% for eight consecutive years. It is said that in 2022, the total electricity consumed in data centers in China is about 270 billion KWH, accounting for 3.1% of the electricity consumption of the whole society. This is even more than the annual output of two Three Gorges Hydroelectric Power Stations. And about half of the energy consumption is used for "computing" of chips. Definitely, the larger computational load requires the greater electricity consummation for heat dissipation. Therefore, in the context of the current rapid development of artificial intelligence (such as LLM) and accelerated upgrading of computing power, there is an urgent need to improve heat dissipation efficiency in data centers.

Heat dissipation has become one of the top challenges in current development of intelligent computing. The server power consumption increases further especially in scenarios such as LLM training. The most typical example is, the statistics show that, compared to the traditional server with a power of about 400W, the maximum power consumption of the NVIDIA DGX A100 server used for ChatGPT training is up to 6.5kW, what's more, it is estimated that more than 1000 sets of 6.5kW AI supercomputing servers are needed to train a single GPT3.0, and the power consumption of AI inferencing is even several times that of training.

GPU generates a large amount of heat during high-load operation and requires efficient heat dissipation to maintain a stable operating state. With the rapid growth of power consumption, traditional air cooling can no longer satisfy the needs of data centers: the reason is very simple, for air cooling, the heat dissipation capacity of air as a heat transfer medium is limited, and the heat dissipation efficiency is relatively low, increasing the risk of overheat and failure. In addition, in high-density data centers, the close arrangement of equipment due to space limitation hinders air circulation, limiting the supply of cool air and the discharge of hot air.

Liquid Cooling - at Just the Right Time

Since a single air cooling cannot meet the heat dissipation needs of data centers, liquid cooling technology is favored by more and more enterprises because of its efficient cooling performance to better cope with heat dissipation in high-power environments. ODCC pointed out that as the large data centers continues to increase, the use of advanced cooling technology has become the key to reducing energy consumption in data centers. The liquid cooling technology with high refrigeration efficiency is currently the most mainstream technology in this field.

Compared with the traditional air cooling, the liquid cooling is more and more widely applied to satisfy the increasing needs of customers, and gradually become the main approach to data center heat dissipation. Today, A variety of solutions are designed for cooling system in data centers, such as Rear Door Heat Exchanger, Direct-To-Chip Cooling, Immersion Cooling, etc. The rear door heat exchanger is mainly used for heat exchange between the liquid arranged at the back door of rack and the air inside. The energy efficiency can be improved by precisely controlling the temperature near the heat source, thus effectively achieving local cooling with less dependence on the air conditioning system; direct-to-chip cooling refers to arranging the cold plate directly on the heating element (such as CPU, GPU), and cooling the equipment by heat exchange between liquid and components; the immersion liquid cooling design can provide more precise temperature control in the data center through the high thermal conductivity of liquid, namely, immersing servers and components in liquid.

Data centers have higher requirements for stable computing power and efficient energy use. Therefore, the multi-scenario application of liquid cooling technology is particularly critical for high-performance computing and artificial intelligence. To a certain extent, more efficient thermal management and heat dissipation can significantly improve the utilization of chip. The reason is that, a higher frequency of chip brings a faster data processing speed and a greater computing power, but it takes more electricity and generates more heat. Under the reliable guarantee of sufficient electricity supply and well heat dissipation, the chip can operate at a higher frequency, so that the utilization of the chip has been greatly improved.

It is worth mentioning that the excellent cooling effect of liquid cooling technology can effectively improve the efficiency and stability of server, but also enable data centers to arrange more servers in a limited space, thus improving the operation efficiency of data centers. The utilization of waste heat can also bring more economic benefits.

It can be seen that the increasing power consumption of data centers makes the adoption of liquid cooling technology urgent. Liquid cooling will become the mainstream for data centers in the future. IDC data show that in the first half of 2023, the market size of liquid-cooled server in China reached 660 million US dollars, with a year-on-year increase of 283.3%. IDC also expects that from 2022 to 2027, the compound annual growth rate (CAGR) of China's liquid-cooled server market will reach 54.7%, and the market size will reach 8.9 billion US dollars in 2027. China Mobile, China Telecom and China Unicom and related enterprises of the liquid cooling industry chain jointly proposed in the "Liquid Cooling Technology White Paper" released in June 2023 that, the three major operators will pilot the use of liquid cooling technology in 10% of their new data center projects in 2024; and more than 50% of data center projects will use liquid cooling technology by 2025. The importance of liquid cooling technology in data center cooling design is self-evident.

On-Demand Combination - Leveraging the Best of Each Strength

Admittedly, liquid cooling technology is not perfect: the construction and maintenance costs are higher; sufficient space and a suitable operating environment are required; a high level of expertise and technical support are required in design and installation of liquid cooling system. In order to achieve better heat dissipation and easier operation and maintenance with higher cost performance, various cooling technologies can be combined to customize an optimal solution. Enterprises can choose an appropriate cooling mode for each application scenario and demand to achieve the best heat dissipation effect.

A mix of air and liquid cooling technologies is a more common option at present. It is an efficient, flexible, energy-saving, and environmentally-friendly cooling design for data centers that fully utilizes the advantages of air cooling and liquid cooling technologies, improving the reliability and extensibility of data centers, reducing operating costs and environmental pollution, and adding a new cooling option to data centers.

A mix of air and liquid cooling can be applied on the demand of server by zoning cooling, with automatic temperature regulation by intelligent control system, thus achieving an efficient, reliable and energy-saving heat dissipation: liquid cooling is used for high-density servers to carry away the heat through liquid circulation; and the air cooling is used for low-density servers to achieve heat dissipation by air flow. The option can secure the efficiency of heat dissipation while reducing the cost and maintenance difficulty.

Three Factors Considered in Choosing a Solution Provider

The cooling design for data centers is also constantly improving with the continuous development of technology. With the introduction and popularization of liquid-cooling-designed data centers, as well as the increasing demand for green and environmental protection, efficient energy utilization, overall optimization and integrated management, the choice of cooling solution provider has become more complicated.

Enterprises can focus on three factors when choosing a solution provider: first, technical feasibility and adaptability - whether the cooling design can satisfy the demand of data center, whether it can cope with the intensity and power consumption needs of equipment, and whether the heat dissipation equipment conforms to the industry standards. Enterprises can obtain data information about product performance and quality from providers for overall consideration; second, experience and expertise of providers - does the provider have a good reputation in the field, are there successful cases and clients for reference; third, innovation and future extensibility - does the provider have advanced cooling technology and the ability to continuously develop and innovate to adapt to the changing demand of data center and the technological development trend.

The eligible provider can provide an energy-efficient and reliable cooling solution, so that the cooling system of data center can satisfy the demand of long-term operation.

As a company focused on data center development and operation, Chayora has been researching and launching cutting-edge technologies including the cooling design of data center for many years. The company caters to the growing demand for data center cooling through a mix of air and liquid cooling technologies, and has achieved a remarkable result with a more advantageous solution based on this.

Moore's Law for Everything is about to embody in AI technology. The emergence of GPT and LLM has led to an exponential increase in AI computation. Increasing chip power brings more challenges to data centers. Next, Chayora will continue to explore cutting-edge solutions to provide users with safe, stable, reliable and green data centers.