The next iteration of PCI Express will double the bandwidth per lane, but that shouldn’t surprise anyone. During Intel Developers Forum, we sat down with Richard Solomon, Vice President of PCI-SIG, and found the hidden gem in the next specification; it provides at least a 4x increase in power delivery at the slot, which might eliminate the need for auxiliary power cables with some GPUs.
The PCI-SIG consists of more than 730 paying members that provide input and a board of directors that help steer the process. The standards body is just “days away” from releasing version 0.7 of the PCIe Express 4.0 specification to members, according to Solomon. The final specification should come by the end of 2016, but the group will have an additional 0.9 step in between before completion at 1.0. Some companies, such as Mellanox, are shipping leading-edge PCIe products with the 0.7 spec, but will update when version 1.0 arrives. The group is already working on PCIe 5.0
http://media.bestofmicro.com/0/N/605255/original/AMD_Board.png
As we sat through our presentation with Salomon, I kept thinking back to the AMD Naples server motherboard the company displayed the previous night. AMD didn’t want to divulge too many details about the next generation server platform, but a picture is worth a thousand words. If you look closely, there is a single 20-pin power connector like any other modern motherboard. There are also four 8-pin and two 6-pin power connectors. AMD marked the PCIe power connectors as follows:
P0 ABCD PWR
P1 ABCD PWR
P0 EFGH PWR
P1 EFGH PWR
These connectors are in addition to the processor 4+4 pin connector marked P0 and P1 CORE PWR. Even with a 64-core platform, that much power for the board and processors would severely limit AMD’s chances of getting into data centers. We feel that AMD added the extra power for something else, like the PCIe edge connectors.
When we asked the PCI-SIG, we received the news that for the first time, PCIe will get a massive power increase at the connector. Solomon couldn’t recall the exact ceiling because member companies have proposed several options. Solomon stated that the minimum would be 300W, but the ceiling “may be 400 or 500W."
Even with the minimum 300 watts, there is more than enough power provided by the edge connector to run a GeForce GTX 1080 (reference design 180 watts) and Titan X (250 watts) without an external power source. PCI Express 3.0 provides a maximum of 75 watts thus far, though many slots support the default 25W. The rating for a x16 lane card comes from 3 amps at 3.3 volts and 5.5 amps at 12 volts, which is a rating that dates back to the introduction of PCI Express 1.0a in 2003.
Speeds and Feeds
| PCIe Version | Line Code | Transfer Rate | x1 Bandwidth | x4 | x8 | x16 |
|---|---|---|---|---|---|---|
| 1.0 | 8b/10b | 2.5 GT/s | 250 MB/s | 1 GB/s | 2 GB/s | 4 GB/s |
| 2.0 | 8b/10b | 5 GT/s | 500 MB/s | 2 GB/s | 4 GB/s | 8 GB/s |
| 3.0 | 128b/130b | 8 GT/s | 984.6 MB/s | 3.938 GB/s | 7.877 GB/s | 15.754 GB/s |
| 4.0 | 128b/130b | 16 GT/s | 1.969 GB/s | 7.877 GB/s | 15.754 GB/s | 31.508 GB/s |
PCI Express 4.0 Usage
PCI Express has moved well beyond video cards. Currently, nearly every IO device routes through PCIe to send signals back to the CPU. We expect to see rapid adoption of the new standard for storage and networking products.
NVMe SSDs using PCIe 3.0 x4 recently reached the usable limit of the interface, such as the Samsung SM961 that reads data at 3,100 MB/s. Enterprise NVMe SSDs using the add-in card form factor and bifurcation will ship later this year with PCIe 3.0 x16. The increase in throughput will allow manufacturers to reduce costs by delivering the same performance with half the number of PCIe lanes. Analyst firm Forward Insights predicts enterprise PCIe storage devices will surpass the number of SATA products shipped between 2017 and 2018.
At Computex 2016, Marvell told us to expect consumer networking products with 10-gigabit speeds in January (CES time frame). 10GbE will finally fall out of the enterprise sector and into the laps of us at home and provide roughly 1GB/s of data transfer speeds over commodity CAT 6 cabling. A single 10GbE connection will only require a single PCIe lane with the 4.0 specification.
Mobile, Internet of Things (IoT), and other battery-operated devices will benefit from low-power states and a new focus on burst performance. PCIe 4.0 adapts new L1 sub-states with half- and quarter-swing bursts, which use just 400 and 200-millivolt steps.
Connecting External Devices Through A Standardized PCIe Cable
During the last year, we’ve seen a number of companies build external graphics housings that route back to a small form factor PC via external cabling. The big fear is that a product from Company A will not work with a product from Company B. The PCI-SIG looks to end interoperability issues with the OCuLink standard.
Storage systems will also adopt the PCIe-over-cable OCuLink connection. The cable is more graceful than SATA Express and the reworked SAS cables that we currently use for U.2 devices.
