100G Ethernet Stays Pricey as Speed Needs Soar

Virtualization, video and massive amounts of data are all driving enterprises and service providers toward 100-Gigabit Ethernet, but the cost of the fledgling technology remains prohibitively high and few products have been installed, industry observers said at the Ethernet Technology Summit.

The 100GE standard was ratified by the IEEE last year, but the technology is just beginning to creep into use. Analyst Michael Howard of Infonetics Research estimates that only a few hundred ports of 100GE have been delivered and most of those are being used by service providers in tests or trials.

However, with the growing amounts of data going in and out of servers in big data centers, some large enterprises also are running into a need for something faster than 10-Gigabit or 40-Gigabit Ethernet, Howard said at the Ethernet Technology Summit this week in Santa Clara, California. Virtualization allows servers to run at higher utilization rates, and blade server chassis pack more computing power into a rack. Connecting these systems to top-of-rack switches, and linking those to other switches at the end of a row, is starting to require multiple 10-gigabit links in some cases, he said.

The Mayo Clinic is already testing 100GE as a possible replacement for multiple 10-Gigabit Ethernet links in its data center, which are needed because its virtualized servers can drive so much traffic onto the clinic’s LAN. One reason is that Mayo doctors frequently consult with other physicians around the world and need to share medical imaging files such as CAT scans and MRIs, said Gary Delp, a systems analyst at Mayo.

Aggregated 10-Gigabit ports are still an inefficient way to create fast links within the data center, and 100GE should be more efficient, Delp said. He expects vendors to come out with aggregation technology that pushes traffic more efficiently, and whether users adopt that or 100GE will be a matter of economics, he said.

Some other large enterprises are in similar situations, according to Howard. Using four to eight aggregated links also typically takes up more space and power and generates more heat than a single high-speed connection does, Howard said. One solution administrators are beginning to use is 40-Gigabit Ethernet, which is somewhat less expensive and more readily available today, but the traffic curve points to a need for 100GE, he said.

Cost is one of the main barriers to adoption of 100-Gigabit Ethernet and is likely to remain so for the next few years, Howard said. Though per-port prices can vary based on specific deals, the cost of a 100GE port is still effectively six figures. Juniper Networks typically charges about ten times the cost of a 10-Gigabit Ethernet port, meaning 100GE can cost about US$150,000 per port, a company executive said on Tuesday. Brocade Communications announced a two-port module with the new technology for $194,995 last year. It can be ordered now and is expected to start shipping in the first half of this year, the company said Wednesday.

The 100GE equipment is pricey because the technology is so new, according to Howard. Early components are expensive as well as large, power-hungry and hot, and there are still several generations to go in downsizing these parts for more economical systems. It’s a fundamental problem of networking today, as the cost of equipment falls by about 15 percent per year but traffic increases by 45 percent or more, Howard said.

This isn’t the fault of the Ethernet equipment industry or the Institute of Electrical and Electronics Engineers, which developed the 100GE standard, Howard said. At these speeds, the problem comes down to physics. “It’s just that we’re up against limits,” he said. An IEEE study group is now looking at writing specifications for some of the components that go into 100GE equipment, with an eye toward creating a vendor ecosystem around interoperable products.

Analyst Nathan Brookwood of Insight 64 doesn’t think price will deter enterprises that have to use 100GE in their data centers.

“The people who can’t solve the problem any other way will cough up the bucks to solve it this way,” Brookwood said. Initially, those will probably be operators of massive computing clouds, such as Facebook and Google, he said. A Facebook network engineer said a year ago that the company already needed 100GE. The new technology may also go into high-performance computing environments, according to Brookwood.

Despite these projections, 100GE remains far from being a mainstream technology. Even 10-Gigabit Ethernet is just starting to be used for server interfaces, albeit at a growing rate. The global market for 10-Gigabit Ethernet server interfaces was nearly $100 million in the fourth quarter, up 57 percent from a year earlier, research firm Dell’Oro Group reported this week. Some types of products for this technology are just beginning to reach the market. At the Ethernet event, Broadcom showed off its first server adapter based on the 10GBase-T specification, which uses standard Ethernet cabling and can reach 100 meters. It is expected to ship in commercial volumes in the second quarter.

Because of pricing and other issues, most observers don’t expect 100GE to be widely used until 2013 in service providers and at least 2015 in data centers.

“100 Gigabits is a lot of bandwidth, and most environments haven’t even figured out how to consume 10 gigabits,” said Greg Scherer, vice president of server and storage strategy in Broadcom’s High Speed Controller Business Unit.

Stephen Lawson covers mobile, storage and networking technologies for The IDG News Service. Follow Stephen on Twitter at @sdlawsonmedia. Stephen’s e-mail address is [email protected]