System Architectures Using OIF CEI

System Architectures Using OIF CEI-56G Interfaces
Nathan Tracy
Technologist, TE Connectivity
Technical Committee Chair, OIF
Agenda
OIF History of Common Electrical Interface (CEI)
Identification of CEI-56Gb/s Requirements
Typical Architectures for 56Gb/s Applications
56Gb/s Technology and Architecture Considerations
Channel Improvements to Enable Architectures
Summary
1
Notice: This contribution has been created to assist the Optical Internetworking Forum (OIF). This document is offered to the
OIF solely as a basis for discussion and is not a binding proposal on the companies listed as resources above. Each
company in the source list, and the OIF, reserves the rights to at any time to add, amend, or withdraw statements contained
herein.
This Working Text represents work in progress by the OIF, and must not be construed as an official OIF Technical Report.
Nothing in this document is in any way binding on the OIF or any of its members. The document is offered as a basis for
discussion and communication, both within and without the OIF.
For additional information contact:
The Optical Internetworking Forum, 48377 Fremont Blvd,
Suite 117, Fremont, CA 94538
510-492-4040 phone info@oiforum.com
© 2015 Optical Internetworking Forum
2
OIF Common Electrical Interface (CEI)
Electrical Implementation Agreements

CEI IA (Common Electrical Interface) is a clause-based format
supporting publication of new clauses over time:





CEI-1.0: included CEI-6G-SR, CEI-6G-LR, and CEI-11G-SR clauses.
CEI-2.0: added CEI-11G-LR clause
CEI-3.0: added work from CEI-25G-LR, CEI-28G-SR
CEI-3.1: includes CEI-28G-MR and CEI-28G-VSR
CEI-11G and -28G specifications have been used as a basis
for specifications developed in IEEE 802.3, ANSI/INCITS T11,
and IBTA.
3G
6G
11G
25G & 28G
56G
SxI-5
2000
3
2001
CEI-1.0 CEI-2.0
2002
2003
2004
2005
CEI-3.0
2006
2007
2008
2009
2010
2011
2012
CEI-3.1
2013
2014
CEI-25G-LR CEI-28G-MR CEI-28G-SR CEI-28G-VSR
CEI-25G Application Space
4
Chip
Chip
Optics
Optics
Published in CEI 3.0:
LR: Backplane, passive
copper cable.
SR: Chip-to-chip, and
chip-to-module.
Published in CEI 3.1:
MR: Chip-to-chip, and
low loss backplane.
VSR: Chip-to-module
(fully retimed optics)
Chip-to-Optics
Chip
Chip
Chip-to-Chip (300 mm)
Chip
Chip
Low loss Backplane (500 mm)
Chip
Chip
Backplane (700 mm) or
Passive Copper Cable
CEI Application Space is Evolving

The “OIF Next Generation Interconnect Framework” white
paper lays out a roadmap for CEI-56G serial links.


2.5D and 3D applications are becoming increasingly relevant.
High function ASICs (such as switch chips) are driving
requirements for higher I/O density and lower interface power.

Emerging trends


5
Pin density is not increasing
fast enough for high density
ASICs.
Power reduction of 30%
from one generation to
next is not good enough.
CEI-56G Application Spaces
CEI-56G-USR
3D Stack
Optics
2.5D Chip-to-OE
Chip to Nearby OE
Chip
Ultra short reach
CEI-56G-XSR
Extra short reach
Chip
Pluggable
Optics

CEI-56G-VSR


Very short reach
Chip-to-Module

Chip
Chip
Chip-to-Chip & Midplane
CEI-56G-MR
Medium reach
CEI-56G-LR
Chip
Backplane or Passive Copper Cable
6
Chip
Long reach

USR: 2.5D/3D applications

1 cm, no connectors, no
packages
XSR: Chip to nearby optics
engine

5 cm, no connectors

5-10 dB loss @28 GHz
VSR: Chip-to-module

10 cm, 1 connector

10-20 dB loss @28 GHz
MR: Interfaces for chip to chip
and midrange backplane

50 cm, 1 connector

15-25 dB loss @14 GHz

20-50 dB loss @28 GHz
LR: Interface for chip to chip
over a backplane

100cm, 2 connectors

35dB at 14Ghz
Optical Line Card Evolution
Today’s 100G based optical system based on 28G
electrical interconnects
Switch Card
Backplane
Nx
25G
Nx 25G
ASIC
Retimer
(CEI-25G-LR,
100GBASE-KR4)
Line Card
CFP4/QSFP28 100G
Module
Nx 25G
Retimer
(CEI-28GSR/MR,
CAUI-4 c2c)
4x 25G
ASIC
Retimer
(CEI-28G-VSR,
CAUI-4 c2m)
(CEI-28GSR/MR)
35dB 100GBASE-KR4
backplane interface
15-20 dB
chip-to-chip interface
10-12 dB chip-to-module
interface
These 100G systems could eventually increase
density by migrating to 2x50G optical modules
leveraging the OIF’s 56G-VSR specification
7
Source: Semtech
4x
25G
Tx/Rx
Optics
200G/400G Optical Line Card
Possible long term 4x/8x 50G optical system
200G Module
Switch Card
Backplane
4x 56G
Line Card
4x 56G
Nx
56G
ASIC
Nx 56G
Retimer
(CEI-56G-LR-PAM4)
(CEI-56GMR-PAM4,
CDAUI-8 c2c)
Tx/Rx
Optics
Retimer
Nx 56G
Retimer
(CEI-56GMR-PAM4,
CDAUI-8 c2c)
ASIC
(CEI-56G-VSR-PAM4,
CDAUI-8 c2m)
8x 56G
8x 56G
Tx/Rx
Optics
Retimer
400G Module
35dB PAM4
backplane interface
8
15-20 dB PAM4
chip-to-chip interface
10dB PAM4
chip-to-module interface
Note: PAM4
examples
shown, could
also be NRZ
56G-VSR enables higher density 200G/400G optical line
card
Note that 56G PAM4 max interconnect losses reflect
those of 28G NRZ allowing for similar component
placement to today’s 100G line card!
Source: Semtech
56G PAM-4 IC Technology Selection
CMOS/SiGe implementation tradeoffs
Factor
CMOS
Analog
Performance
Digital
Complexity
Favors SiGe
pJ/bit
efficiency
Analog
ADC/DSP
Notes
SiGe analog performance
typically better (jitter, NF,
etc)
Functions such as FEC
challenging in SiGe
Favors CMOS
Cost
Equalization
Options
SiGe BiCMOS
Favors SiGe
NRZ likely to require more
complex equalization
Analog
PAM-4 lends itself to
DAC/ADC approach
though analog
approaches still feasible
Expect comparable
transceiver efficiencies
CMOS suitable for Line Card SerDes
Potentially SiGe or CMOS for Module Retimer depending
upon IC functionality & complexity
9
Source: Semtech
50G NRZ Progress
Test board
50G NRZ Eval
Board (Tx)
Cable Backplane Demo
Cables
Insertion Loss ~ 1.5dB at 25Ghz
Cable Backplane
•
46” total channel length
• 40” copper backplane (30
gauge)
• Two STRADA Whisper connectors
• Two break out cards (5” & 1”)
Insertion Loss ~ 23dB at 25Ghz
BER ~ 1e-12
10
Source: Credo
Cables
50G NRZ Eval
Board (Rx)
Insertion Loss ~ 1.5dB at 25Ghz
•
Serdes
Serdes
Serdes
Core
Logic
16nm
Serdes
Transitioning from 25Gb/s to 56Gb/s
Same number of IO
Core logic complexity scales to 4X
Serdes Serdes
Serdes
27m
m
Next Generation Switch Chip
- Doubling the capacity
27mm
Serdes
45mm
28nm to 16nm process node
But LR Serdes may hardly scale
Limitations in die area and power density
USR and XSR may enable higher density
architectures by providing a low power
interface
Ability to escape to a SerDes which can
support multiple applications (VSR, MR,
LR)
45mm
•
Package Ball View
11
Source: Alcatel Lucent
System OEM Perspective on 56Gb/s
•
Product flexibility – must drive worst case Channel I/O will
see
•
•
Transistor BW/Nyquist Rate ratio varies greatly between
process technologies; 50GHZ power/performance is effected
by process (CMOS, SiGe, etc.)
•
•
VSR turns into MR or even LR(Lite) -> PAM4
Can’t just look at channel characteristics, must be IC/Channel
optimized
No errors after FEC (in large systems even 10^-18 BER have
>1error/hr.; unacceptable)
Need Low BER before FEC
• Broadband effects (transient behavior vs. pure steady state)
• Eliminate as many broadband effects as possible
• Broadband effects stress adaptation and reduce FEC
efficiency.
•
12
Source: Juniper Networks
System Channels
ASIC used in various systems
13
Source: Juniper Networks
System MR Channel
Suck out rules out
56G NRZ
Traditional Backplane and Orthogonal
LR SERDES
POWER
IC
Full Power Serdes on IC
capable of 35 dB channel Loss
without need for retimers
STRADA
Whisper
STRADA
Whisper
daughter card Trace
*
IC
daughter card Trace
LR SERDES
POWER
Orthogonal Backplane
Backplane Trace
Power
Traditional PCB & STRADA 100%
Whisper Connector
(LR Serdes)
56 Gbps Reach (PAM4)
1m
Traditional Backplane
Source: TE Connectivity
14
Cable Backplanes:
Margin/Thermal/Reach
LR/SR SERDES
POWER
IC
*
Full Power Serdes on Retimer
capable of 35 dB channel
STRADA
Whisper
Cable
STRADA
Whisper
Cable
Power
56 Gbps
Reach (PAM4)
Traditional PCB & STRADA Whisper Connector
100%
(LR Serdes)
1m
STRADA Whisper
Cable Backplane Connector
100%
(LR Serdes)
3m
Traditional PCB & STRADA 150%
Whisper Connector With Retimers (VSR + AEC)
Source: TE Connectivity
15
daughter card Trace
IC
daughter card Trace
LR/SR SERDES
POWER
Cabled Backplane Shuffle
Cabled Backplane
1.5m
Insertion Loss
0
16.9dB
@12.89GHz
-20
30dB
@12.89GHz
-40
PCB BACKPLANE
• Board Material = Megtron6 HVLP
• Trace length = 30”
• Trace geometry = Stripline
• Trace width = 6 mils
• Differential trace spacing = 9 mils
• PCB thickness = 200 mils, 20 layers
• Counterbored vias, 1 – 6mil stub
• STRADA Whisper Vertical Header and
Right Angled Receptacle
-60
-80
CABLED BACKPLANE
-100
0
5
10
15
20
25
30
freq, GHz
DAUGHTER CARD
• Board Material = Megtron6 VLP
• Trace length = 5”
• Trace geometry = Stripline
• Trace width = 6 mils
• Differential trace spacing = 9 mils
• PCB thickness = 110mils, 14 layers
• Counterbored vias, 1 – 6mil stub
• Test Points = 2.4mm (included in data)
Source: TE Connectivity
16
35
40
• TE-Madison TurboTwin 40G Cable
• 30AWG, 1 meter [39.37”]
• STRADA Whisper Cabled Header and
Right Angled Receptacle
Optical Backplanes:
Density/Reach Extension
½ Power Serdes + E/O + O/E Optical Power.
100M Reach including 2 Optical connectors
MBO MXC Optical Backplane MXC MBO
Mid Board Optics Switch
Power
56 Gbps
Reach (PAM4)
Traditional PCB & STRADA Whisper Connector
100%
(LR Serdes)
1m
STRADA Whisper
Cable Backplane Connector
Traditional PCB & STRADA Whisper Connector With Retimers
100%
(LR Serdes)
3m
150%
(VSR + AEC)
1.5m
150%
(VSR + Optics)
100m
VCSEL Optical Backplane
17
Source: TE Connectivity
Optical Backplane System
Mid Board Optics for I/O
Power Traditional PCB w/ Direct Attach 100% Passive Cable
Reference
Traditional PCB w/ Pluggable VCSEL Optics
Traditional PCB w/ Power Retimers and Dirct Attach Passive Cable
Mid Board VCSEL Optics
Source: TE Connectivity
18
56 Gbps
Reach (PAM4)
3m
150%
100m
150%
5m
150%
100m
Summary
56G systems require a balancing of requirements:
•
•
•
connector/channel demands
semiconductor demands
equipment demands
New processing techniques can enable new
architectures
Higher performance channels can enable new
architectures
Integrated optics can enable new architectures
High performance systems based on 56Gb/s signaling
rates are being enabled by the OIF‘s CEI-56G projects
Come join the OIF!
www.oiforum.com
19