Silicon Graphics SGI Altix 3700 Bx2, User Manual

Page: 76 / 214

52 007-4377-002
3: System Overview
Distributed Shared I/O
Like DSM, I/O devices are distributed among the compute nodes (each compute node
has two I/O ports) and are accessible by all compute nodes through the NUMAlink
interconnect fabric.
ccNUMA Architecture
As the name implies, the cache-coherent non-uniform memory access (ccNUMA)
architecture has two parts,
cache coherency and nonuniform memory access , which are
discussed in the sections that follow.
Cache Coherency
The Altix 3700 Bx2 server series use caches to reduce memory latency. Although data
exists in local or remote memory, copies of the data can exist in various processor caches
throughout the system. Cache coherency keeps the cached copies consistent.
To keep the copies consistent, the ccNUMA architecture uses directory-based coherence
protocol. In directory-based coherence protocol, each block of memory (128 bytes) has an
entry in a table that is referred to as a directory. Like the blocks of memory that they
represent, the directories are distributed among the compute nodes. A block of memory
is also referred to as a cache line.
Each directory entry indicates the state of the memory block that it represents. For
example, when the block is not cached, it is in an unowned state. When only one
processor has a copy of the memory block, it is in an exclusive state. And when more than
one processor has a copy of the block, it is in a shared state; a bit vector indicates which
caches contain a copy.
When a processor modifies a block of data, the processors that have the same block of
data in their caches must be notified of the modification. The Altix 3700 Bx2 server series
use an invalidation method to maintain cache coherence. The invalidation method
purges all unmodified copies of the block of data, and the processor that wants to modify
the block receives exclusive ownership of the block.