In every great story, there's a great struggle. Good versus evil. Light versus darkness. Heroes versus villains. The data center industry has its own remarkable story to tell, but rather than an epic battle of superheroes; it's the tug-of-war between data center demand and data center efficiency.
Demand for data center solutions has been and will continue to grow for the next half-decade and beyond, with some analysts projecting an average growth rate of 13.8% on the low end to 21% per year on the high side through 2024. Simultaneously, data center-based computing quintupled from 2018 to 2020, while the rate of energy consumption only increased six percent in the same period.
These figures underscore the continued challenges that data centers face in balancing their customers' evolving demands (or the global market in general) and their responsibility to both the bottom line and to environmental stewardship by reining in runaway energy use. In previous articles, I highlighted the main drivers of data center demand in the next five years and the opportunities for continuing to improve utilization efficiency in the same period.
Here are the three key lessons to keep in mind about the constant back-and-forth between data center demand and operational efficiency.
1. The laws of physics
A key component in the battle between growth and efficiency is science - physics, to be exact. Data centers will continue to wage an internal war, pitting the physics of linear hardware improvement over time against the non-linear, exponential trend of total connectivity and time users spend on networks.
According to Moore's Law, and the measurable increases in hardware efficiency in the last 50 years, further improvements are reasonably predictable. We know to expect roughly double the power and capabilities approximately every two years and can reasonably calculate our predicted efficiency gains and associated cost reductions based on those figures if total demand were linear too.
Unfortunately, only one component of demand - the human user - is predictable and linear. There are only so many people on Earth and only so many hours in a day they could be online, in an app, or otherwise connected to a network. Machine-to-machine (M2M) connectivity, however, will continue to grow exponentially as more connected devices come online and drive more than half of all global connections and traffic by 2023.
The upshot is that while we're making continued, predictable progress on the hardware efficiency front, the hockey stick growth we'll see in global connectivity and consumption will continue to outpace those efficiency gains we've earned.
2. Application weight vs. software efficiencies
As the data center world becomes increasingly more software-oriented through initiatives like software-defined data centers, operators are ramping up their efforts to engineer greater efficiencies into those solutions. Entire teams of developers and engineers are tasked with consolidating, streamlining, and accelerating software-defined data center operations for organizations eager to eke out additional gains in resource utilization.
However, the rapid growth in application complexity and weight will continue to make software engineering teams' jobs more difficult. User demands for more powerful, more sophisticated, and more integrated applications will continue to stress data center infrastructure. And despite the struggle, this is likely a battle that Team Efficiency can't win because the bias toward complexity will dominate so long as it's better for business.
3. Dualities of centralization
On the one hand, centralizing services in the data center is a natural progression for delivering more robust solutions faster and more cost-effectively. Migrating those workloads to the cloud enables faster scale, greater agility in daily operations, and generally higher resource utilization.
But the rapid expansion of support infrastructure in the data center - more on-ramps, more virtual machines, and all the space and power that come with it - will continue to push the boundaries of how efficient all those components can be. The reality is that service demand will almost certainly outpace efficiency efforts because every percentage point gained in utilization frees additional resources that will ultimately be given back to and consumed by new demand.
What it all means
In most stories about a great battle or struggle, there's a definite resolution. The good guy usually wins, evil is defeated, and everyone lives happily ever after. The data center story, however, won't have that kind of ending.
In the next five years, we can expect demand to continue outpacing gains in efficiency, and not by a small margin. Even though when you digitize things - digital PDF versus multi-page printouts, video conferencing versus in-person meetings - it's naturally more efficient, there will always be new demand ready to occupy the bandwidth and newly freed resources.
In the end, it's not about winning or losing; satisfying only demand or efficiency. Rather, the single best thing data center operators can do is aim to strike a balance, understand that the nature of linear versus non-linear growth doesn't lend itself to a win for Team Efficiency, and simply commit to making a concerted effort to use every watt of power in a facility as efficiently as possible.
