Flexible Pluggable Front Port Technology

I am writing this blog article in continuation of my previous blog – “Future-proof FPGA platforms the longevity and upgradability”. In this blog entry, I discuss the available pluggable front port technology, complementing the flexible properties of the described future-proof FPGA-based SmartNICs.

Here are the properties of front port technology that define its flexibility:

1. Level of backward compatibility
2. Number of link types supported
3. Number of link rates supported
4. Available lane breakout functionality

The market is swamped with different IEEE/MSA defined front port formfactors, that currently support link speeds up to 400G. Only a reduced subset of these meet the physical outline limitations imposed by the PCIe standard. With focus on the high-end SmartNIC use case, this blog will focus on the PCIe compliant multilane based subset.

Level of backward compatibility

In this context, a front port module form-factor is considered backward compatible if it supports deployment of at least one previous generation of pluggable modules. The multilane pluggable form-factors are currently dominated by two families – the QSFPx family and the CFPx family, see figures 1 & 2.

From a distance, it seems like the two families provide the same functionality, but a closer inspection reveals significant differences in their offering, having justified their co-existence thus far. The CFPx family is targeted at telecom use cases where performance is more important than price and power consumption. Whereas the CFPx family has no backward compatible members, the QSFPx family is fully backward compatible, enabling deployment of all previous pluggable family members, in a given form-factor member. The latest addition to the QSFPx family is the QSFP-DD form-factor, that brilliantly introduces a doubling of the available number of lanes on the system side without sacrificing backward compatibility.

Number of link types supported

The number of link types supported by a given formfactor is primarily dictated by market demand and the power envelope of the form-factor. The link types supported by the QSFPx form-factor family is primarily defined by datacenter and enterprise applications. The SR link type is used massively in this segment, but LR and PSM volume is ramping due to the mega sizing going on in the datacenters. With a primary focus on the telecom use cases, the CFPx form-factor family members have historically provided the space needed to implement the current relevant link types, including the power consuming long haul distances.

The gap in the supported link types between the CFPx- and the QSFPx family increases typically for each addition of new CFPx family members. Subsequently over time, the gap decreases as the technology allows the QSFPx form-factor to catch up. This is a phenomenon that is usually driven by commercial market demand.

Number of link rates supported

Supplemental to the multi-link speed support enabled by form factor backward compatibility, deployment of dual Ethernet rate modules allows the SmartNIC to support several Ethernet link speeds without physically changing the front port pluggable module.  As an example, a QSFP28 port will enable deployment of a 100GBASE-SR4 dual-rate module in the following 4 modes:

1. 4x 10G ports, breakout
2. 4x 25G ports, breakout
3. 1x 40G port
4. 1x 100G port

The market is supporting the QSFPx form-factor family with several dual rate offerings, adding further flexibility to this form-factor.

Available lane breakout functionality

With the market being introduced to the 40GBASE link type family back in 2010, there was, for the first time, a standardized multi-lane based pluggable front port form-factor, the QSFP+. This form-factor deploys four 10G lanes on the system side in order to handle the 40G aggregate bandwidth on the media side. In a market environment that is constantly struggling to increase port density, it was quickly recognized that this form-factor provided the highest available 10G port density at that point in time, resulting in the availability of the first breakout connectivity solutions. Figure 3. shows a breakout solution mapping directly between an SR4 port and four SR ports (MPO/LC), in this case 100G to 25G but also supporting 40G to 10G.

Initially it was only possible to support multimode fiber infrastructure through breakout solutions, as the single mode link types had the four 10G lanes multiplexed one fiber on the media side. With this limitation, breakout solutions had a significant limitation with respect to native 10G ports. With the MSA standardized introduction of 40GBASE-PSM4 (Parallel Single Mode 4) pluggables this limitation is gone, leaving breakout front ports as a viable and mostly very attractive alternative to native 10G port form-factors. Figure 4. shows a breakout solution mapping directly between an LR4 port and four LR ports, in this case 100G to 25G but also supporting 40G to 10G.

The MPO <-> LC breakout cable type deployed in Figure 3 and Figure 4 is not applicable for use cases relying on existing structured LC connected cable infrastructure. Figure 5 shows the applicable breakout solution for this use case. A rack mount shelf holds one or more modules converting between the existing structured LC connectivity and the breakout MPO-based connectivity.

A third and newly introduced breakout solution is to use the QSFP to SFP adapter type as shown in Figure 6. The QSFP form-factor adapter simply maps a QSFPx port into an SFPx port obsoleting the need for the breakout solutions shown in figure 3, figure 4. & figure 5. Nothing comes without a cost, and in this case the cost for obsoletion of the breakout cabling is that 3/4^th of the available lanes in the front port becomes inaccessible.

Napatech deployment of currently available flexible front port technology

It seems obvious from the above discussions that the QSFP28 form-factor is the most flexible from what is currently available. The maximized flexibility of the QSFP28 form-factor matches “hand in glove” with the flexibility offered by the latest Napatech SmartNIC, the NT200A01.

The NT200A01 HW platform deploys the QSFP28 pluggable front port form-factor and is currently supported by the following feature sets:

1. 2 port 40G
2. 2 port 100G
3. 2 port 40G/100G, configurable

Potential future feature set release candidates include:

1. Up to 8 port 1G
2. Up to 8 port 10G
3. Up to 8 port 25G
4. Up to 4 port 50G, 2-lane “consortium” variant

Next generation Napatech front port technology

As a logical consequence of the presented arguments in this blog, the obvious front port form-factor pick for a next generation Napatech SmartNIC is the QSFP-DD. The QSFP-DD form-factor supports the currently available QSFP+ and QSFP28 modules as well as the new PAM4 based 50G, 200G & 400G port rates. Stay tuned for updates on QSFP-DD deployment.