Download the White Paper Here

With the growing demand for greater computing and storage capacity in the data center, one of the most important and challenging problems IT organizations face is how to address escalating power requirements. Large data centers consume megawatts of power causing significant costs and contributing to growing global energy problems. Data centers are by nature a collection of individual server and storage elements connected together to deliver computing capacity for business application services. This white paper describes how various networks can be combined into a data center “fabric.” Connecting all of these elements together has a significant impact on overall power consumption and efficiency of the data center.

Every data center performs computation while creating massive amounts of electronic data that need to be archived. The key requirement is to achieve those tasks while consuming less power. There are two ways to address this problem: use elements that require less power to deliver the same capacity or performance; or use elements that deliver more capacity or performance while using the same amount of power. In an ideal world, one can achieve both goals by delivering more capacity while requiring less power.

The ratio between the required power and the delivered capacity can be referred to as “Power Efficiency.” Data center managers should seek the best power
efficiency (the lowest value) across all data center elements. In many cases, the efficiency of one element will impact the efficiency of another.

Chart included in downloadable pdf version of this White Paper available above.

The goal is to maximize the power efficiency of the application. This implies maximizing the performance of the application while minimizing the power of the
infrastructure (server, storage, and networking) supporting that application.

The most obvious approach to maximizing the power efficiency is to choose the most power-efficient server, storage, and switching elements. A frequently
overlooked but equally important dimension is to increase the overall efficiency or utilization of the data center fabric. Frequently, application performance is
bottlenecked by inefficient network connectivity causing an immediate impact to the overall power efficiency. All too often, data center managers overlook the
impact of the interconnect fabric on power savings.

The data center network’s affect on a data center’s overall efficiency can be maximized by implementing the following key elements:

1. Deploying Power Efficient Networks- minimize the power required for each network port in the fabric. The power efficiency of the network fabric
is measured in power per port or power per delivered bandwidth.

2. Consolidating Server I/O Over a Unified Fabric - unified fabrics consolidate multiple networks allowing fewer switches and network interfaces per
server to deliver the same services of LAN, storage and IPC communication with the least amount of power and cooling required.

3. Eliminating I/O Bottlenecks- using a high-performance network can affect application performance and utilization dramatically. It leads to a
significant increase in application performance with improvements of up to 300 percent.

4. Enhancing Server Virtualization - provide improved application mobility and data center virtualization by allowing more applications to fit within an
existing infrastructure without compromising application performance.

Chart included in downloadable pdf version of this White Paper available above.

The following sections discuss the contribution of the fabric to data center power efficiency within the four elements identified above.

Power Efficient Networks

When defining the power efficiency of an element within a network, the goal is to minimize the power each one of the network ports consumes. Since the main
function of a network is to deliver traffic, the power-efficiency metric should represent how much power is required to deliver a given amount of traffic. An
example of this metric is watts/Gbps (how many watts are required to deliver a gigabit of data in a single second).

Consolidate Server I/O

In traditional data centers, each server has multiple redundant connections to IP networks and storage networks. InfiniBand can be used to consolidate server I/O onto a single unified fabric architecture that provides multiple isolated virtual lanes, hardware partitioning and other capabilities. This leads to much better power efficiency – not to mention the significant cost and complexity savings – when taking into account the alternative of using multiple switches and adapters.

Chart included in downloadable pdf version of this White Paper available above.

Eliminating I/O Bottlenecks

The main reason for building a data center is to run applications. The more efficiently your applications run, the smaller the footprint of your data center. The
power efficiency of a data center is determined not only by how efficient the data center components are, but even more so by how many applications can be run or how many transactions can be delivered per second within the given data center.

Even if data center managers choose power efficient components, these components may deliver significantly slower application performance. To deliver the
required application performance, the size of the data center would have to be increased to deliver the same level of service.

Again, the effect of data center networks on application performance is frequently overlooked. The affect of using a high-performance data center fabric on
application performance can be examined to see where the bottleneck can be eliminated. InfiniBand-based unified fabrics not only consume far less power than
any alternative, as shown in the previous section, but also significantly increase the performance of the application – allowing fewer servers to deliver the same
work.

The reasons InfiniBand can deliver increased levels of application performance on identical server and storage resources is derived from the fact that CPU power
has grown exponentially in the last few years. The time and the amount of data processed by those CPUs has also grown significantly while the wire connecting all of the CPUs together, as well as the CPUs to the data, has stayed at the same – leading to many cases in which the application bottleneck is in the server I/O between servers as well as between servers and storage.

So how does the data center manager eliminate this discrepancy? By using a data center fabric that uses InfiniBand. InfiniBand provides 20-40 Gbps links to
each server. InfiniBand’s access to the fabric is offloaded by the InfiniBand adapter, which requires fewer CPU cycles to process the work, significantly lowers
application-to-application latencies, and provides faster storage I/O access.

Chart included in downloadable pdf version of this White Paper available above.

Improving Data Center Virtualization

In addition to power efficiency improvements and the indirect influence that networks and I/O can have on application power efficiencies, virtualization
technologies also provide additional power savings.

In essence, virtualization technologies allow the consolidation of multiple applications onto fewer systems and less infrastructure, which in turn saves on the
power consumed by the unneeded infrastructure. For example, instead of using four servers for a given application and utilizing only 15 percent of the server
capacity, you can now run all four applications in a virtual server environment on a single server that will run at 60 percent capacity. This one server may use more power than it did before, but you can shut down the other three servers, saving significant power.

Fabric and I/O virtualization are key elements that allow the solution to achieve full system virtualization, save on power and provide three additional key benefits:

1. A reduction of the number of LAN and SAN switches in a configuration down to a single switch (or two switches in the case of a redundant configuration), saving the power required to drive all the extra fabrics.

2. A reduction of the number of NICs and storage adapters down to one or two per server while simultaneously improving the performance of server
virtualization software and the power efficiency of both the I/O (network) and the application.

3. The ability to form dynamic I/O connectivity and network configurations, which allows the use of smaller and more power-efficient servers and enables the use of server virtualization in more applications that have more varied I/O requirements. This also creates an easier migration of servers from one virtual machine to another – again saving power.

Importance of a High-Performance Unified Fabric

A unified fabric requires more than simply adding more bandwidth. Since a unified fabric delivers the aggregate capabilities of IP networking, inter-server
communication and storage networks, it must provide the appropriate level of capabilities and performance to support all of those functions. As an example,
inter-server messaging requires specific low latency, low jitter, and scalability characteristics whereas storage needs reliability and bandwidth. The different lanes (sub-fabrics) need to be isolated from each other to ensure such unique characteristics are met without interference.

In order to satisfy the requirements of a unified fabric, technologies such as InfiniBand deliver a unique set of features that allow multiple layers of traffic to have
unique characteristics and to co-exist on the same wire. This includes features such as L2 congestion management; class-based traffic isolation; multi-path and fabric topologies; storage tunneling protocols; I/O virtualization; central fabric virtualization; and provisioning management.

Summary

Power efficiency is one of the most important and most difficult challenges for large enterprises and data centers. In order to truly address the power challenge,
data center architects need to look at every data center component and every new application environment through the lens of “power-efficiency” metrics.
Power-efficiency metrics measure the power it takes to deliver a certain capacity or performance for a given infrastructure and seeks to minimize that number. The most important metric is the “Application Power Efficiency,” which measures the power consumed by the overall resources used for an application against the application performance.

The chosen networking technologies and the choice to deploy a high-performance data center fabric has a significant impact on data center power efficiency – both directly and indirectly .The benefits of such a deployment include:

1. Requiring less power to drive more network capacity
2. Consolidating multiple network devices and adapters to a single fabric
3. Opening application bottlenecks and improving application performance using the same or less power
4. Improving utilization of existing infrastructure by enabling end-to-end system virtualization

Voltaire’s innovative switching and software solutions incorporate the above benefits and significantly improve the power efficiency of data centers. To see how
utilizing an InfiniBand-based unified fabric can translate into a greener data center, visit http://www.voltaire.com/UnifiedFabric and try-out the Efficiency Calculator.

Contact Voltaire to Learn More
1.800.865.8247

http://www.voltaire.com

®2008 Voltaire Inc. All rights reserved. Voltaire and the Voltaire logo are registered trademarks of Voltaire Inc. Grid Director is a trademark of Voltaire, inc. Other company, product, or service names are the property of their respective owners.