Top-of-Rack Switching - a Power Saver?

By David Gross

In just about any survey about energy efficiency, data center managers reply that it's their number one concern, a major priority, or give some other kind of indication that they are highly focused on saving energy.   Then they go out and buy power hungry, high density GigE line cards for their switches and routers.

Many technology surveys ignore the fact that buyers often say one thing and do another, but investors can't.   And when it comes to network equipment, the best ways to reduce watts per Gbps include buying an OC-768 line card, which costs over half a million dollars, or an InfiniBand switch, which typically requires a Clos topology, which is still uncommon outside of supercomputing.

As an example, Voltaire's 4036E offers 1.36 Tbps of InfiniBand switching with just 240 Watts, or just .18 Watts per Gbps.   Meanwhile, the 32-port GigE/1 port 10GigE uplink card for the Nexus 7000 consumes 385 Watts, or 9.17 Watts per Gbps, 50x more, and there are far more Nexus switches than Voltaire devices in most data centers.

There is often a trade-off between power consumption and features, because much of a high-speed router or switch's power is tied to TCAMs, packet processing, and the memory/forwarding requirements of large routing tables.   However, with few of these sophisticated features, Top-of-Rack switches offer tremendous bandwidth with limited power.  Force10's S4810 ToR switch, for example, requires just .44 Watts per Gigabit.   But its larger Exascale switch has a 10-port 10 GigE line card that needs over 3 Watts per Gig.    Configuration flexibility has a price.

While many data center managers are still resisting ToR switches, there is roughly a 40% Watt per Gbps improvement just from upgrading from GigE to 10GigE.    And while there is a significant cost penalty at 10GigE going from short reach 850nm optics to longer reach 1310nm and 1550nm optics, there is no power penalty.    Here again, the least capital efficient networking options and the most power efficient.  

While the IRR on buying more expensive ports to save power is often negative, there are at least a couple options developing to reduce Watts/Gbps that don't require an entirely new topology, or a multi-million dollar router.

IBM to Sell Force10's Top-of-Rack Switch

By David Gross

Lots of interesting things happening in the Top-of-Rack (ToR) switch market.   Last week, Extreme Networks (EXTR) showcased a demo of it ToR switch running a 40 Gigabit Ethernet link in a supercomputing network, while BLADE Network Technologies, soon to be part of IBM (IBM), announced a new ToR product which offers 40 Gigabit Ethernet ports.   Not to be forgotten, Force10 Networks announced that it will be selling its S60 ToR switch through IBM. 

Force10 has a well-established relationship with IBM, centered around its high-end E series switches that are used in data center cores and supercomputing clusters.   While some core router/switch vendors, notably Juniper (JNPR), have entered the ToR market by OEM'ing other suppliers' products, the S60 appears to be home grown, and runs Force10's FTOS software.  

Like other ToR switches, the S60 is designed around price/performance, not a wide selection of configuration options.   10 Gigabit Ethernet uplinks are available as an option, using SFP+ transceivers, and LAN PHY framing, which is essential not just for short reach but to keep costs down, as 10G WAN PHY ports remain more expensive than the 40 Gigabit Ethernet ports on BLADE's and Extreme Networks' ToR switches.   The reason the S60 does not feature 40 Gigabit ports is that it is more of an entry-level product than the others, with a send and receive throughout capacity of 136 Gbps, compared to 1.2 Tbps (Terabits) for the BLADE box.    And while ToR switches are not overloaded with software features, all three vendors support IPv6.

IBM's Acquisition of Blade Network Technologies a Smart Move

By David Gross

Rumors were flying last week that IBM (IBM) was about to buy Brocade (BRCD). Well, that still hasn't happened, but the large technology supplier is going to make a much more sensible deal - acquiring top-of-rack switch maker Blade Network Technologies.

Cisco's (CSCO) proprietary VLAN protocols have far less of an impact in the top-of-rack segment than they do elsewhere, which has made this small portion of the Ethernet market more friendly to alternate suppliers than the traditional corporate LAN market. Moreover, many vendors missed out on this segment because they were so focused on building massive carrier Ethernet boxes with overcooked operating systems, while Blade and Arista have kept things simple and straight forward by focusing on latency, not accommodating massive routing tables.

In addition to an existing sales relationship with IBM, Blade has an OEM arrangement with Juniper (JNPR), who was an investor in the company. Financial terms of the acquisition have not been disclosed.