Data centers are the most energy-intensive Department of Defense (DoD) buildings. They consume more than 10% of all DoD electricity (40% for cooling) and produce 7.5 trillion British thermal units/year of unused heat. Direct-to-chip liquid cooling is a unique data center efficiency technology that brings high-performance liquid-cooling directly to the hottest elements inside each server (“hot-spot cooling”), with the potential to cut cooling energy by 60-80%, and to allow for reuse of the heat as on-site energy. It can also enable 2.5x data center consolidation with no additional infrastructure costs. The need for liquid cooling is being driven by several factors, including the increased demand for greater power density, coupled with higher IT performance for high-performance computing and some hyper-scale computing, and the overall industry focus on energy efficiency.

The purpose of this demonstration was to document performance of the equipment about energy savings, reliability, and life cycle cost that can be achieved in the real-world environment of a DoD data center. The data and insights gained in this demonstration can be used to create awareness and acceptance of the technology to facilitate future technology transfer across all DoD data centers.

Direct-to-Chip liquid cooling technology direct to consumer products are built from a series of standard building blocks. These building blocks include processor coolers (pump and cold plate units) for various processors, tubing, quick connectors, the Direct-to-Chip liquid cooling technology unit and their frames. Tubing connects the other building blocks and routes the coolant through the system. Direct-to-Chip liquid cooling gathers heat from the hottest components in a server and removes it from the data center in an all liquid path. This heat load bypasses the air conditioning system that normally cools the data center and is rejected into the outdoors more efficiently via this all liquid path. The hottest components are a server and its processors (central processing units and general-purpose computing on graphics processing units) followed by memory modules. 

This demonstration yielded the following results for each performance objective:

1. Cooling Energy Usage - The average percentage of the energy into these five racks of IBM servers that was removed by the Direct-to-Chip liquid cooling technology system was 62.1%
2. Server Energy Usage - The project team doesn't have this data. They assume no server power savings from the retrofit. Normally would expect power savings at server level of 5 to 10% as indicated in Phase 1 result of the demonstration.
3. Power Usage Effectiveness (PUE) and Energy Reuse Effectiveness (ERE) - There is no ERE calculation as this site is not engaging in any energy reuse. Some implementations (such as at National Renewable Energy Laboratory’s Energy Systems Integration Facility high computing center) reuse the warm return liquid for low-quality heat applications.
4. Data Center Peak Load - Based on the max computer room air conditioning power draw post-retrofit and coefficient of performance calculation, the project team saw a peak load reduction of 7.8% in total room load.
5.  Server Up-Time - This was not measured during this project due to insufficient data.
6. Capacity Consolidation - Due to the change in site, and hence the change in leadership of the data centers, consolation was not studied, or actualized.
7. Ease of Use - While the retrofitting process can be challenging, the project did prove itself worthwhile and energy efficient. The process of having the factory complete the retrofits proved to be easily installed without any complications. 
8. Lifecycle GHG Emissions - Due to electricity reduction of 1424 kWh/day, or 520,116 kWh/year. equates to 387,075 kgCO2/year, or 427 tons. Over a 20-year lifetime, this amounts to 8540 tons of CO2 emissions avoided. This is substantially lower than the estimate outlined in the statement of work, because far smaller server capacity was retrofitted at the new site than was originally planned.
9. Lifecycle System Economics - Due to the various changes and complications during project execution, a lifecycle cost analysis was not performed for this project. See the cost assessment for more details.
10. End-User Acceptance - The project team did not perform a Likert survey as it has been initially planned, due to the logistics around project execution. Instead, the project team had email, and verbal confirmation from host sites that once installed, user satisfaction is high.

At the launch of this project, the initial site of demonstration, Redstone, was selected based on their planned data center expansion. Unfortunately, while the Phase 1 technical program had been extremely successful at Redstone, Phases 2 and 3 were in a stand-still for more than 18 months, due to factors outside the control of the product provider, Redstone or ESTCP.  In particular, as part of the Army’s data center consolidation initiative, Redstone Arsenal’s data center was selected to be decommissioned.   

In collaboration with the Army CIO’s office and data center consolidation team, the project team met with dozens of new host sites and were ultimately decided to relocate the demonstration at the Army Research Labs (ARL), located at Aberdeen Proving Ground.