2.7 External I/O subsystems
The I/O capacity of the Power E1080 can be expanded by using extra external PCIe Expansion Drawers and external SAS Disk Expansion Drawers.
This section describes the PCIe Gen3 I/O Expansion Drawer (#EMX0) that can be attached to the Power E1080 server.
At the initial availability, the Power E1080 supports four PCIe Gen3 I/O drawer per system node, which yields a maximum of eight I/O drawers per 2-node Power E1080 server configuration. One I/O drawer supports two Fanout Modules that offer six PCIe Gen 3 adapter slots each. This configuration delivers an extra 48 PCIe Gen 3 slot capacity per system node and a maximum of 96 PCIe Gen 3 slots per 2-node server.
All eight slots in a system node can be used to cable the four I/O drawers. It is always recommended to balance the I/O Drawer connectivity across the available system nodes.
96 IBM Power E1080: Technical Overview and Introduction
With the availability of 3-node and 4-node Power System E1080 configurations in later GA period, the number of PCIe Gen 3 I/O drawers scale accordingly with a maximum of 16 per 4-node Power E1080. A maximum of 48 PCIe Gen 3 slots per system node and a maximum of 192 PCIe Gen 3 slots per 4-node Power E1080 server to be available at that date.
Each Fanout Module in the PCIe Gen 3 I/O Expansion Drawer (#EMX0) is attached by one optical cable adapter, which occupies one x16 PCIe Gen 4 slot of a system node.
The maximum number of supported I/O drawers and the total number of PCIe slots are listed in Table 2-24.
Table 2-24 Maximum number of supported I/O drawers and the total number of PCIe slots
2.7.1 PCIe Gen 4 cable adapter
The PCIe Gen3 I/O Expansion Drawer (#EMX0) is connected to the system node by using a PCIe Gen 4 I/O expansion card (#EJ24) that is placed in any of the PCIe slots with in the system node. One expansion card adapter connects to one PCIe Gen 3 6-slot Fanout Module (#EMXH) of the PCIe Gen3 I/O Expansion Drawer(#EMX0) using a CXP 16X Active Optical Cable (AOC) Pair. Therefore, the adapter is also named as PCIe Gen 4 CXP cable adapter.
Chapter 2. Architecture and technical overview 97
The PCIe Gen 4 cable adapter is shown in Figure 2-24.
Figure 2-24 PCIe Gen 4 cable adapter (#EJ24)
The PCIe Gen 4 cable adapter (#EJ24) is a half-height, half-length, low-profile PCIe Gen 4 x16 adapter. The adapter provides two ports for the attachment of CXP 16x AOC pair. One adapter supports the attachment of one PCIe Gen 3 6-slot fanout module in a EMX0 PCIe Gen3 I/O expansion drawer.
Although these cables are not redundant, the loss of one cable reduces the I/O bandwidth (that is, the number of lanes that are available to the I/O module) by 50%.
A total of eight PCIe Gen 4 cable adapters (#EJ24) are allowed per system node to connect to maximum of eight PCIe Gen 3 6-slot Fanout Modules (#EMXH). Each Fanout Module must go with one CXP 16x AOC cable pair with both ports wired to the same PCIe Gen 4 cable adapter from the system node.
The cable adapter can be placed in any slot in the system node. The following adapter placement sequence in the system node is recommended:
System node slot 1, 7, 3, 6, 2, 8, 4, 5
The maximum number of adapters that is supported is 8 per system node and 32 per system, having four system nodes.
Whenever PCIe Gen3 I/O Expansion Drawer(#EMX0) is used, it is recommended to reserve the slots in the system nodes for the PCIe Gen 4 cable adapter (#EJ24) adapters. Place the high-profile adapters within the I/O Expansion Drawers.
98 IBM Power E1080: Technical Overview and Introduction
Three different types of CXP 16x AOC cable pairs are available for the connection between PCIe Gen 4 cable adapter (#EJ24) and PCIe Gen 3 fanout module (#EMXH) of the I/O Expansion Drawer:
Ê 2.0M (#ECCR)
This cable is used when I/O drawer is in the same rack as the E1080 server.
Ê 10.0M (#ECCY)
This cable is used when the I/O drawer is in a different rack from the E1080 server.
Ê 20.0M (#ECCZ)
Ê Like the 10.0M (#ECCY) cable, these cables are used when I/O drawer is in a different rack from the E1080 server.
Note: One #ECCR, one #ECCY, or one #ECCZ includes two AOC cables.