To enable attribute caching as well, use the
attrcache_timeout
mount option for
loadbalance
mount points.
This allows attribute-only file system operations to use local attribute data instead of sending the request to the
DVS server. This is useful in loadbalance mode because with a read-only file system, attributes are not likely to
change.
Figure 34. Cray DVS Loadbalance Mode
Cray System
FS Server
Disk FS
Application
DVS Client
DVS Server
FS Client
Application
DVS Client
Application
DVS Client
Application
DVS Client
FS Client
DVS Server
6.1.5.5 About DVS Stripe Parallel Mode
Stripe parallel mode builds upon cluster parallel mode to provide an extra level of parallelized I/O forwarding for
clustered file systems. Each DVS server can serve all files, and DVS servers are automatically chosen based on
the file inode and offsets of data within the file relative to the DVS block size value. For example, in the figure
below, DVS is mounted to
/foo
on the DVS client, and the I/O for three different blocks (or segments) of data
within file
bar
—
seg1
,
seg2
, and
seg3
—is handled by three different DVS servers, thus distributing the load at a
more granular level than that achieved by cluster parallel mode. Note that while file I/O is distributed at the block
level, file metadata operations are distributed as in cluster parallel mode: the metadata operations of a given file
are always handled by the same DVS server. Stripe parallel mode provides the opportunity for greater aggregate
I/O bandwidth when forwarding I/O from a coherent cluster file system. All I/O from all clients involving the same
file routes each block of file data to the same server to prevent file system coherency thrash. GPFS has been
tested extensively using this mode.
ATTENTION: NFS cannot be used in stripe parallel mode because NFS implements close-to-open cache
consistency; therefore striping data across the NFS clients could compromise data integrity.
The following example mount entry contains the mount options essential for stripe parallel mode: more than one
nodename
and
maxnodes
> 1.
mount -o nodename=server1:server2:server3, maxnodes=3
DVS stripe parallel mode does not adhere to POSIX read/write atomicity rules. See
for information about how to achieve POSIX read/write atomicity with this mode.
Cray DVS
S3016
153
