2.3.1 Memory bandwidth
The Power10 single chip module (SCM) supports 16 open memory interface (OMI) links and one differential DIMM (DDIMM) is driven by each OMI link in turn. One OMI link represents a bundle of 8 lanes and is as such capable to transfer 8 byte with one transaction.
The Power E1080 offers four different DDIMM sizes: 32 GB, 64 GB, 128 GB, and 256 GB. The 32 GB and 64 GB DDIMMs run at a data rate of 3200 Mbps; the 128 GB and 256 GB DDIMMs at a data rate of 2933 Mbps. Table 2-6 lists the available DDIMM capacities and their related maximum theoretical bandwidth figures per OMI link and per Power10 SCM.
Table 2-6 Memory bandwidth of supported DDIMM sizes
a. DDIMM modules attached to one SCM must be all of the same size
The Active Memory Expansion (AME) feature is an option that can increase the effective memory capacity of the system. AME uses the on-chip nest accelerator (NX) unit to compress or decompress memory content. This separately priced hardware feature is enabled at partition (LPAR) level. The related LPAR profile parameter allowsit to configure the compression factor in accordance with memory capacity needs and performance trade-off consideration.
74 IBM Power E1080: Technical Overview and Introduction
2.3.2 Memory placement rules
In a Power E1080 single node system, each SCM must have eight of the 16 memory DDIMMs slots populated. Only increments of four DDIMMs are supported, all of which have the same capacity. These four DDIMMs must all be plugged into four of the eight remaining slots that are connected to an SCM.
DDIMMs cannot be installed in such a way that an odd number of DDIMM slots are populated behind an SCM. DDIMMs also cannot be installed in a configuration that results in 10 or 14 DDIMM slots being populated behind an SCM. The only valid number of DDIMMs plugged into slots connected to a specific SCM is 8, 12, or 16.
The Power E1080 system must adhere to the following DDIM plug rules:
Ê Minimum memory that is allowed is 1 TB (see 1.5.1, “Minimum configuration” on page 15).
Ê All DDIMMs under each SCM must have the same capacity.
Ê DDIMMs must be plugged in groups of four (quads).
Ê Each DDIMM quad must all be the same size and type (identical to each other) and must be the same with other DDIMM quads under each SCM, but can be different from other DDIMM quads from other SCMs.
For example, in a one system node configuration:
– First SCM can be connected to 8×32 GB DDIMMS
– Second SCM can be connected to 8×64 GB DDIMMS – Third SCM can be connected to 8×128 GB DDIMMS – Forth SCM can be connected to 8×256 GB DDIMMS
Overall system performance improves when all the quads match each other. Physical memory features from a previous generation of servers cannot be migrated to Power E1080.
For the best possible performance, it is generally recommended that memory be installed in all memory slots and evenly across all system node drawers and all SCMs in the system.
Balancing memory across the installed system planar cards enables memory access in a consistent manner and typically results in better performance for your configuration.
You must account for any plans for future memory upgrades when you decide which memory feature size to use at the time of the initial system order.
Plug sequence for minimum memory
A Power E1080 drawer requires a minimum of 32 DDIMMs. Two quads of DDIMMs per SCM are required.
Spreading the DDIMMs evenly between the two memory controllers provides maximum performance. The DDIMMs within each quad must be the same size and type and all the DDIMMs under each SCM have to be same capacity.
A DDIMM quad can be different from the DDIMM quads plugged elsewhere.
Adhere to the following order to plug the 32 DDIMMs and use Figure 2-13 on page 73 as reference:
Ê C20, C21, C22, C23, C24, C25, C26, C27 (slots connected to SCM P1)
Ê C28, C29, C30, C31, C32, C33, C34, C35 (slots connected to SCM P3)
Ê C36, C37, C38, C39, C40, C41, C42, C43 (slots connected to SCM P2)
Ê C44, C45, C46, C47, C48, C49, C50, C51 (slots connected to SCM P0)
Chapter 2. Architecture and technical overview 75
Plug Sequence for the Remaining DDIMMs
For a Power E1080 drawer, the next quad of DDIMMs sequence and rules are described in this section.
For the next quad of DDIMMs to be installed in slots that are connected to one of the four pSCMs, consider the following rules:
Ê The DDIMMs within each quad must be the same size and type.
Ê All DDIMMs under each SCM must have the same capacity.
Ê The DDIMM quad can be different from the DDIMM quads plugged elsewhere, including the other quad connected to the same SCM.
Ê Plug the remaining quad of DDIMMs in any of the remaining open quad of DIMM slots and all DDIMMs under each SCM must have the same capacity
Adhere to the following order and use Figure 2-13 on page 73 as reference:
Ê C52, C53, C54, C55 (slots connected to SCM P1) or
Ê C60, C61, C62, C63 (slots connected to SCM P3) or
Ê C68, C69, C70, C71 (slots connected to SCM P2) or
Ê C76, C77, C78, C79 (slots connected to SCM P0) or
Ê C56, C57, C58, C59 (slots connected to SCM P1) or
Ê C64, C65, C66, C67 (slots connected to SCM P3) or
Ê C72, C73, C74, C75 (slots connected to SCM P2) or
Ê C80, C81, C82, C83 (slots connected to SCM P0)
Note: In a multi-system node configuration, a drawer can be fully populated. The other drawers can be partially populated or have all the drawers symmetrically populated.