IBM 88743BU - System x3950 E, Руководство пользователя

Страница: 85 / 426

Chapter 2. Product positioning
67
To realize the full benefit of NUMA systems, such as the x3950 M2, it is very
important that operating systems have NUMA support. A NUMA-aware operating
system must have the ability to schedule the use of system resource pools on
each NUMA node. This must be done so that any request for processor, memory,
and I/O resources for any given application process (which can spawn multiple
threads) be serviced from one NUMA node ideally, or from as few NUMA nodes
as possible to minimize inefficient multinode resource allocations.
The x3950 M2 multinode complex implements NUMA by connecting the
scalability ports of each node together. These ports are directly connected to the
Hurricane memory controller and allow high speed communication between
processors located in different nodes. The ports act like hardware extensions to
the CPU local buses. They direct read and write cycles to the appropriate
memory or I/O resources, and maintain cache coherency between the
processors.
In such multinode configurations, the physical memory in each node is combined
to form a single coherent physical address space. For any given region of
physical memory in the resulting system, some processors are closer to physical
memory than other processors. Conversely, for any processor, some memory is
considered local and other memory is remote.
The term NUMA is not completely correct because memory and I/O resources
can be accessed in a non-uniform manner. PCIe and USB devices may be
associated with nodes. The exceptions to this situation are existing I/O devices,
such as DVD-ROM drives, which are disabled because the classic PC
architecture precludes multiple copies of these existing items.
The key to this type of memory configuration is to limit the number of processors
that directly access a piece of memory, thereby improving performance because
of the much shorter queue of requests. The objective of the operating system is
to ensure that memory requests be fulfilled by local memory when possible.
However, an application running on CPUs in node 1 might still have to access
memory physically located in node 2 (a remote access). This access incurs
longer latency because the travel time to access remote memory on another
expansion module is clearly greater. Many people think this is the problem with
NUMA. But this focus on latency misses the actual problem NUMA is attempting
to solve: shorten memory request queues.
The performance implications of such a configuration are significant. It is
essential that the operating system recognize which processors and ranges of
memory are local and which are remote.
So, to reduce unnecessary remote access, the x3950 M2 maintains a table of
data in the firmware called the Static Resource Allocation Table (SRAT). The