Red Hat GFS 6.0 - Скачать руководство пользователя страница 100

Страница: 100 / 192

Chapter 8. Using Clustering and Locking Systems

For optimal performance,

lock_gulmd

servers should be run on dedicated nodes; however, they can

also be run on nodes using GFS. All nodes, including those only running

lock_gulmd

, must be listed

in the

nodes.ccs

configuration file (

nodes.ccs:nodes

8.2.2. Number of LOCK_GULM Servers

You can use just one

lock_gulmd

server; however, if it fails, the entire cluster that depends on it

must be reset. For that reason, you can run multiple instances of the

lock_gulmd

server daemon

on multiple nodes for redundancy. The redundant servers allow the cluster to continue running if the

master

lock_gulmd

server fails.

Over half of the

lock_gulmd

servers on the nodes listed in the

cluster.ccs

file

(

cluster.ccs:cluster/lock_gulm/servers

) must be operating to process locking requests

from GFS nodes. That quorum requirement is necessary to prevent split groups of servers from

forming independent clusters — which would lead to file system corruption.
For example, if there are three

lock_gulmd

servers listed in the

cluster.ccs

configuration file,

two of those three

lock_gulmd

servers (a quorum) must be running for the cluster to operate.

lock_gulmd

server can rejoin existing servers if it fails and is restarted.

When running redundant lock_gulmd servers, the minimum number of nodes required is three; the

maximum number of nodes is five.

8.2.3. Starting LOCK_GULM Servers

If no

lock_gulmd

servers are running in the cluster, take caution before restarting them — you

must verify that no GFS nodes are hung from a previous instance of the cluster. If there are hung

GFS nodes, reset them before starting

lock_gulmd

servers. Resetting the hung GFS nodes before

starting

lock_gulmd

servers prevents file system corruption. Also, be sure that all nodes running

lock_gulmd

can communicate over the network; that is, there is no network partition.

The

lock_gulmd

server is started with no command line options.

8.2.4. Fencing and LOCK_GULM

Cluster state is managed in the

lock_gulmd

server. When GFS nodes or server nodes fail, the

lock_gulmd

server initiates a fence operation for each failed node and waits for the fence to complete

before proceeding with recovery.
The master

lock_gulmd

server fences failed nodes by calling the

fence_node

command with the

name of the failed node. That command looks up fencing configuration in CCS to carry out the fence

operation.
When using RLM, you need to use a fencing method that shuts down and reboots a node. With RLM

you

cannot

use any method that does not reboot the node.

8.2.5. Shutting Down a LOCK_GULM Server

Before shutting down a node running a LOCK_GULM server,

lock_gulmd

should be terminated

using the

gulm_tool

command. If

lock_gulmd

is not properly stopped, the LOCK_GULM server

may be fenced by the remaining LOCK_GULM servers.

Содержание GFS 6.0 -

Страница 1: ...Red Hat GFS 6 0 Administrator s Guide...

Страница 2: ...ailable at http www opencontent org openpub Distribution of substantively modified versions of this document is prohibited without the explicit permission of the copyright holder Distribution of the w...

Страница 3: ...3 4 Cluster Configuration Management 6 1 4 GFS Software Subsystems 7 1 5 Before Configuring GFS 9 2 System Requirements 11 2 1 Platform Requirements 11 2 2 TCP IP Network 11 2 3 Fibre Channel Storage...

Страница 4: ...age 30 5 8 2 Example procedure 31 5 9 Erasing a Pool Volume 31 5 9 1 Usage 32 5 9 2 Example 32 5 9 3 Comments 32 5 10 Renaming a Pool Volume 32 5 10 1 Usage 32 5 10 2 Example 33 5 11 Changing a Pool V...

Страница 5: ...5 8 2 1 Selection of LOCK_GULM Servers 85 8 2 2 Number of LOCK_GULM Servers 86 8 2 3 Starting LOCK_GULM Servers 86 8 2 4 Fencing and LOCK_GULM 86 8 2 5 Shutting Down a LOCK_GULM Server 86 8 3 LOCK_NOL...

Страница 6: ...itch 116 10 2 2 WTI Network Power Switch 117 10 2 3 Brocade FC Switch 117 10 2 4 Vixel FC Switch 117 10 2 5 HP RILOE Card 118 10 2 6 GNBD 118 10 2 7 Manual 118 11 Using GNBD 121 11 1 GNBD Driver and C...

Страница 7: ...OCK_GULM SLM Embedded 148 C 3 1 Key Characteristics 148 C 3 2 Kernel Modules Loaded 150 C 3 3 Setup Process 150 C 4 LOCK_GULM SLM External 153 C 4 1 Key Characteristics 154 C 4 2 Kernel Modules Loaded...

Страница 8: ......

Страница 9: ...presented this way include the following command Linux commands and other operating system commands when used are represented this way This style should indicate to you that you can type the word or p...

Страница 10: ...n menu If you click on the word on the GUI screen the rest of the menu should appear For example Under File on a GNOME terminal the New Tab option allows you to open multiple shell prompts in the same...

Страница 11: ...ionally we use several different strategies to draw your attention to certain pieces of information In order of how critical the information is to your system these items are marked as a note tip impo...

Страница 12: ...n and the support documenta tion included in your subscription you must activate your subscription by registering with Red Hat Registration includes these simple steps Provide a Red Hat login Provide...

Страница 13: ...Enter the following command from the command line as the root user usr bin up2date register 5 Recommended References For additional references about related topics refer to the following table Topic...

Страница 14: ...Applications and High Availability Blueprints for High Availability Designing Resilient Distributed Systems by E Marcus and H Stern Wiley 2000 Provides a summary of best practices in high availabilit...

Страница 15: ...d Changed Features with the Initial Release of Red Hat GFS 6 0 File access control lists ACLs and extended file attributes in GFS file systems This release adds the capability of setting and getting f...

Страница 16: ...r Suite For GFS upgrade instructions refer to Appendix B Upgrading GFS 1 2 Performance Scalability and Economy You can deploy GFS in a variety of configurations to suit your needs for performance scal...

Страница 17: ...ients as block storage devices by GNBD servers From the perspective of a client application storage is accessed as if it were directly attached to the server in which the application is running Stored...

Страница 18: ...SAN 1 2 3 Economy and Performance Figure 1 3 shows how Linux client applications can take advantage of an existing Ethernet topology to gain shared access to all block storage devices Client data fil...

Страница 19: ...configuration management 1 3 1 Cluster Volume Management Cluster volume management provides simplified management of volumes and the ability to dynam ically extend file system capacity without interru...

Страница 20: ...s for more information on cluster management Fencing is the ability to isolate or fence off a cluster node when that node loses its heartbeat no tification with the rest of the cluster nodes Fencing e...

Страница 21: ...re ccs_servd CCS server daemon that distributes CCS data from a single server to ccsd daemons when a shared device is not used for storing CCS data Fence fence_node Command used by lock_gulmd when a f...

Страница 22: ...LOCK_GULM lock_gulm o Kernel module that is installed on GFS nodes using the LOCK_GULM lock module lock_gulmd Server daemon that runs on each node and communicates with all nodes in GFS cluster gulm_...

Страница 23: ...ame later in this book The cluster name can be 1 to 16 characters long For example this book uses a cluster name alpha in some example configuration procedures Number of file systems Determine how man...

Страница 24: ...ook for the CCS files fence ccs and nodes ccs Refer to Section 6 6 Creating the fence ccs File and Section 6 7 Creating the nodes ccs File for more information To help determine the type of fencing me...

Страница 25: ...e Architecture RAM Red Hat Enterprise Linux AS ES or WS Version 3 Update 2 or later ia64 x86 64 x86 SMP supported 256 MB minimum Table 2 1 Platform Requirements 2 2 TCP IP Network All GFS nodes must b...

Страница 26: ...array or JBOD Note Make sure that the devices can operate reliably when heavily accessed simultaneously from multiple initiators Note Make sure that your GFS configuration does not exceed the number...

Страница 27: ...he key characteristics of your GFS configuration refer to Section 1 5 Before Configuring GFS and have completed the following tasks Installed prerequisite software Specified a persistent major number...

Страница 28: ...ith the HP RILOE card refer to HP RILOE Card on page 147 Verify that the utility is installed on each of those nodes by looking for usr sbin stunnel The Stunnel utility is available via up2date 3 1 2...

Страница 29: ...modules RPM is for an SMP or hugemem kernel GFS modules smp 6 0 2 20 1 i686 rpm The GFS tools RPM is common to all kernels 2 At each node install the GFS tools RPM using the rpm Uvh command For examp...

Страница 30: ...ULM If you are using LOCK_NOLOCK refer to Appendix C Basic GFS Examples for information about which GFS kernel modules you should load To load the GFS kernel modules follow these steps 1 Run depmod a...

Страница 31: ...Using Red Hat GFS with Red Hat Cluster Suite 4 1 Prerequisite Tasks Before setting up the GFS software make sure that you have noted the key characteristics of your GFS configuration refer to Section...

Страница 32: ...e for a New Volume b Create a pool for each file system Refer to Section 5 5 Creating a Pool Volume Command usage pool_tool c ConfigFile 2 Create a CCS pool a Create pool configuration file Refer to S...

Страница 33: ...age ccsd d CCADevice Note You can use GFS init d scripts included with GFS to automate starting and stopping the Cluster Configuration System For more information about GFS init d scripts refer to Cha...

Страница 34: ...mount t gfs o acl BlockDevice MountPoint The o acl mount option allows manipulating file ACLs If a file system is mounted without the o acl mount option users are allowed to view ACLs with getfacl bu...

Страница 35: ...t presents physical storage devices such as disks or RAID ar rays as logical volumes to GFS cluster nodes Pool can aggregate storage devices either by concate nating the underlying storage or by strip...

Страница 36: ...umber and printing a pool configuration act on existing pools and require one or more pool names to be specified on the command line Flag Function Section Page Reference c Create a pool Section 5 5 Cr...

Страница 37: ...olume Table 5 3 pool_assemble Command Functions Flag Option D Enable debugging output h Help Show usage information q Be quiet Do not display output from the command V Display command version informat...

Страница 38: ...ommand version information then exit v Verbose operation Table 5 6 pool_info Command Options 5 2 4 pool_mp The pool_mp command is for managing multipathing on running pools refer to Table 5 7 and Tabl...

Страница 39: ...ool label or an unknown label on them Note The pool_tool s command does not detect ondisk labels other than those mentioned in the pre ceding paragraph 5 3 1 Usage pool_tool s 5 3 2 Example In this ex...

Страница 40: ...extension poolname cfg For example the pool configuration file for a pool named pool0 would be defined in configuration file pool0 cfg Before creating a configuration file you can check to see what d...

Страница 41: ...he device is to be added id is the device identifier Number the devices in order beginning with 0 device specifies the device node to be used for example dev sda1 Table 5 9 Pool Configuration File Key...

Страница 42: ...at defines the pool 5 5 2 Example In this example the pool0 cfg file describes the new pool pool0 created by the command pool_tool c pool0 cfg 5 5 3 Comments Multiple pools can be created with one poo...

Страница 43: ...ames are specified all pools visible to the system are activated Deactivating a Pool Volume pool_assemble r PoolName PoolName Specifies the pool to deactivate More than one name can be listed If no po...

Страница 44: ...Usage pool_tool p PoolName PoolName Specifies the pool name s for which to display information If no pool names are specified all active pools are displayed 5 7 2 Example In this example the pool_tool...

Страница 45: ...ce 0 1 dev sdc1 pooldevice 0 2 dev sdd1 pooldevice 0 3 dev sde1 2 Edit the new file pool0 new cfg by adding one or more subpools that contain the devices or partitions as indicated in this example poo...

Страница 46: ...pool names are specified all pools are erased 5 9 2 Example This example erases all disk labels for pool0 pool_tool e pool0 5 9 3 Comments The O override flag bypasses the confirmation step 5 10 Renam...

Страница 47: ...ber of a pool 5 11 1 Usage pool_tool m Number PoolName Number Specifies the new minor number to be used PoolName Specifies the name of the pool to be changed The minor number must have a value between...

Страница 48: ...displays the following basic information about the named pool s the pool name the minor number the device node alias the capacity whether or not the pool is being used and the multipathing type Using...

Страница 49: ...l_info command with the c option clears statistics from the named pools 5 13 1 Usage Display the Number of Reads and Writes pool_info s PoolName PoolName Specifies the pool name for which to display i...

Страница 50: ...d robin stripe value PoolName Specifies the pool on which to adjust multipathing If no pool names are specified this action is attempted on all active pools Reintegrate Failed Paths pool_mp r PoolName...

Страница 51: ...GFS with Red Hat Cluster you can create CCS files with GFS Druid For information about configuring and using GFS with Red Hat Cluster Suite refer to Appendix A Using Red Hat GFS with Red Hat Cluster S...

Страница 52: ...y For each node having multiple paths to FC storage include in the fencing method section a fencing device for each path to FC storage GFS supports dual power supply fencing with the APC MasterSwitch...

Страница 53: ...10 Using the Fencing System 6 5 Creating the cluster ccs File Creating the cluster ccs file consists of specifying the following parameters Cluster name Each node that runs LOCK_GULM server Optional...

Страница 54: ...lt value of Seconds is 15 To ensure that nodes respond within the Seconds value the interval for heartbeats sent by all nodes is automatically set to two thirds of the Seconds parameter value The Seco...

Страница 55: ...evices in the fence ccs file APC MasterSwitch WTI NPS Network Power Switch Brocade FC Fibre Channel switch McData FC switch Vixel FC switch GNBD HP RILOE card xCAT Egenera BladeFrame system Manual War...

Страница 56: ...le 6 2 for a fence ccs file that specifies an APC MasterSwitch fencing device b For each WTI NPS fencing device specify the following parameters DeviceName the fencing agent agent as fence_wti IPAddre...

Страница 57: ...ers DeviceName the fencing agent agent as fence_xcat and RpowerBinaryPath Refer to Example 6 10 for a fence ccs file that specifies an xCAT fencing device i For each Egenera BladeFrame fencing device...

Страница 58: ...ile Structure fence_devices fence_wti fence_devices DeviceName agent fence_brocade ipaddr IPAddress login LoginName passwd LoginPassword DeviceName Figure 6 4 File Structure fence_devices fence_brocad...

Страница 59: ...ices fence_vixel fence_devices DeviceName agent fence_gnbd server ServerName server ServerName DeviceName Figure 6 7 File Structure fence_devices fence_gnbd without GNBD Multipath fence_devices Device...

Страница 60: ...fence_devices fence_rib fence_devices DeviceName agent fence_xcat rpower RpowerBinaryPath DeviceName Figure 6 10 File Structure fence_devices fence_xcat fence_devices DeviceName agent fence_egenera c...

Страница 61: ...n to a power switch an FC switch or a RILOE card multipath Selects GNBD multipath style fencing CAUTION When multipath style fencing is used if the gnbd_servd process of a GNBD server node cannot be c...

Страница 62: ...Named apc1 and apc2 fence_devices wti1 agent fence_wti ipaddr 10 0 3 3 passwd password wti2 agent fence_wti ipaddr 10 0 3 4 passwd password Example 6 3 WTI NPS Fencing Devices Named wti1 and wti2 fen...

Страница 63: ...Fencing Devices Named mdfc1 and mdfc2 fence_devices vixel1 agent fence_vixel ipaddr 10 0 3 3 passwd password vixel2 agent fence_vixel ipaddr 10 0 3 4 passwd password Example 6 6 Vixel FC Switch Fenci...

Страница 64: ...number of retries to 5 with retrys 5 and sets the wait time between retries to 3 with wait_time 3 Example 6 8 GNBD Fencing Device Named gnbdmp with GNBD Multipath fence_devices riloe1 agent fence_rib...

Страница 65: ...ode must specify at least two fencing devices one fencing device for each power supply Similarly if a node has multiple paths to FC storage then the fencing method for the node must specify one fencin...

Страница 66: ...ode needs recovery Cluster operation is halted during the intervention b If you are configuring a node for cascaded fencing use the file format in Figure 6 22 Refer to Table 6 3 for syntax description...

Страница 67: ...d PortNumber Refer to Example 6 16 for a nodes ccs file that specifies Brocade FC switch fencing Refer to Example 6 17 for a nodes ccs file that specifies McData FC switch fencing Refer to Example 6 1...

Страница 68: ...es IFNAME IPAddress fence MethodName DeviceName port PortNumber switch SwitchNumber NodeName File format for node identification same format for all nodes File format for APC MasterSwitch fencing meth...

Страница 69: ...fencing method for node with dual power supplies Fencing a node with dual power supplies requires that both power supplies be powered off before rebooting the node To accomplish that the nodes ccs fi...

Страница 70: ...NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName port PortNumber NodeName File format for node identification same format for all nodes File format for WTI NPS fencing meth...

Страница 71: ...ame port PortNumber DeviceName port PortNumber NodeName File format for node identification same format for all nodes File format for Brocade McData or Vixel FC Switch fencing method Additional fencin...

Страница 72: ...es NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName ipaddr IPAddress NodeName File format for node identification same format for all nodes File format for GNBD fencing met...

Страница 73: ...eName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName localport PortNumber NodeName File format for node identification same format for all nodes File format for HP RILOE fencing m...

Страница 74: ...des NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName nodename NodelistName NodeName File format for node identification same format for all nodes File format for xCAT fenci...

Страница 75: ...e ip_interfaces IFNAME IPAddress fence MethodName DeviceName lpan LPANName pserver PserverName NodeName File format for node identification same format for all nodes File format for Egenera BladeFrame...

Страница 76: ...odeName File format for node identification same format for all nodes File format for manual fencing method Figure 6 21 File Format nodes ccs Manual Fencing Method Warning Manual fencing should not be...

Страница 77: ...odeName ip_interfaces IFName IPAddress fence MethodName DeviceName Device specific parameter s MethodName DeviceName Device specific parameter s MethodName DeviceName Device specific parameter s NodeN...

Страница 78: ...le only with Red Hat GFS 6 0 for Red Hat Enterprise Linux 3 Update 5 and later For the fence section If GNBD fencing The IP address of this node the node to be fenced If manual fencing IP address of t...

Страница 79: ...verify the IP address associated with the hostname you can issue a ping command to the hostname PortNumber For power and FC switches The port number on the switch to which this node is connected For H...

Страница 80: ...nce power apc1 Fencing device for power supply 1 port 6 switch 1 option off Power down power supply 1 apc2 Fencing device for power supply 2 port 7 switch 2 option off Power down power supply 2 apc1 F...

Страница 81: ...i0 10 0 0 1 fence power wti1 port 1 n02 Example 6 15 Node Defined for WTI NPS Fencing nodes n01 ip_interfaces hsi0 10 0 0 1 fence san silkworm1 port 3 silkworm2 Additional fencing device for additiona...

Страница 82: ...fc2 Additional fencing device for additional port 4 path to FC storage n02 Example 6 17 Node Defined for McData FC Switch Fencing nodes n01 ip_interfaces hsi0 10 0 0 1 fence san vixel1 port 3 vixel2 A...

Страница 83: ...Files 69 nodes n01 ip_interfaces hsi0 10 0 0 1 fence server gnbd ipaddr 10 0 1 1 n02 Example 6 19 Node Defined for GNBD Fencing nodes n01 ip_interfaces hsi0 10 0 0 1 fence riloe riloe1 localport 2345...

Страница 84: ...er Configuration System Files nodes n01 ip_interfaces hsi0 10 0 0 1 fence blade center xcat nodename blade 01 n02 ip_interfaces hsi0 10 0 0 2 fence blade center xcat nodename blade 02 n03 Example 6 21...

Страница 85: ...enter egenera lpan opsgroup pserver ops 1 n02 ip_interfaces hsi0 10 0 0 2 fence blade center egenera lpan opsgroup pserver ops 2 n03 Example 6 22 Nodes Defined for Egenera BladeFrame Fencing nodes n01...

Страница 86: ..._interfaces eth0 10 0 1 21 fence san Fencing with Brocade FC switch brocade1 port 1 power Fencing with APC MasterSwitch apc port 1 switch 1 n02 This example shows a node that can be fenced using a Bro...

Страница 87: ...wer APC MasterSwitch fencing device apc1 port 6 switch 2 n02 Example 6 25 GNBD Server Node Defined for APC Fencing Single Power Supply nodes n01 ip_interfaces wizzy 10 0 0 1 Must be an IP address not...

Страница 88: ...74 Chapter 6 Creating the Cluster Configuration System Files...

Страница 89: ...are some considerations you must take into account for the location of CCS files Refer to Section 11 2 Considerations for Using GNBD Multipath 7 1 Creating a CCS Archive A CCS archive is a collection...

Страница 90: ...on the size of the device it may take several seconds to create a CCA device for the first time due to initialization of the device The ccs_tool command uses the Linux raw device interface to update...

Страница 91: ...odes 7 3 Using Other CCS Administrative Options The following sections detail other administrative functions that can be performed by the ccs_tool command 7 3 1 Extracting Files from a CCS Archive Whe...

Страница 92: ...to a CCS Archive The ccs_tool diff command can be used to compare a directory of CCS configuration files with the configuration files in a CCS archive The output from the ccs_tool diff command is disp...

Страница 93: ...ration files in root alpha new 3 Create a new CCS archive on the CCA device by using the O override flag to forcibly over write the existing CCS archive ccs_tool O create root alpha new dev pool alpha...

Страница 94: ...er supplied variable that specifies the name of the cluster The CCA file must be located on the node that runs ccs_servd 7 5 1 1 1 Usage ccs_tool create Directory CCAFile Directory The relative path t...

Страница 95: ...e network and requires two parameters on the ccsd command line the IP address and optional port number of the node running ccs_servd and the name of the cluster 7 5 1 3 1 Usage ccsd s IPAddress PortNu...

Страница 96: ...ing a CCS Archive and Section 7 2 Starting CCS in the Cluster The advantages are that there is no server point of failure and that updates to the CCS archive happen atomically However not every cluste...

Страница 97: ...Chapter 7 Using the Cluster Configuration System 83 Note When you update a CCS archive update the shared device archive first then update the local archives Be sure to keep the archives synchronized...

Страница 98: ...84 Chapter 7 Using the Cluster Configuration System...

Страница 99: ...GFS to be used as a local file system 8 2 LOCK_GULM RLM and SLM are both implemented by the LOCK_GULM system LOCK_GULM is based on a central server daemon that manages lock and cluster state for all G...

Страница 100: ...m number of nodes required is three the maximum number of nodes is five 8 2 3 Starting LOCK_GULM Servers If no lock_gulmd servers are running in the cluster take caution before restarting them you mus...

Страница 101: ...the IP address or hostname of the node running the instance of lock_gulmd to be terminated 8 3 LOCK_NOLOCK The LOCK_NOLOCK system allows GFS to be used as a local file system on a single node The kern...

Страница 102: ...88 Chapter 8 Using Clustering and Locking Systems...

Страница 103: ...on 9 14 Shutting Down a GFS Cluster Section 9 15 Starting a GFS Cluster 9 1 Making a File System Making a GFS file system is one of the final tasks in the process of configuring and setting up a GFS c...

Страница 104: ...be created by the gfs_mkfs command One journal is re quired for each node that mounts the file system More journals can be specified to allow for easier future expansion BlockDevice Usually specifies...

Страница 105: ...he standard GFS locking module LOCK_NOLOCK May be used when GFS is acting as a local file system one node only O Prevents the gfs_mkfs command from asking for confirmation before writing the file syst...

Страница 106: ...le system resides MountPoint Specifies the directory where the GFS file system should be mounted 9 2 2 Example In this example the GFS file system on the pool0 block device is mounted on the gfs1 dire...

Страница 107: ...ion should not be used when GFS file systems are shared Forces GFS to treat the file system as a multihost file system By default using LOCK_NOLOCK automatically turns on the localcaching and localflo...

Страница 108: ...Other Linux quota facilities cannot be used with GFS 9 4 1 Setting Quotas Two quota settings are available for each user ID UID or group ID GID a hard limit and a warn limit A hard limit is the amoun...

Страница 109: ...fs_quota warn g 21 l 50 k f gfs 9 4 2 Displaying Quota Limits and Usage Quota limits and current usage can be displayed for a specific user or group using the gfs_quota get command The entire contents...

Страница 110: ...rd limit set for the user or group This value is zero if no limit has been set Value The actual amount of disk space used by the user or group 9 4 2 3 Comments When displaying quota information the gf...

Страница 111: ...ges to the quota_quantum parameter are not persistent across unmounts You can use the gfs_quota sync command to synchronize the quota information from a node to the on disk quota file between the auto...

Страница 112: ...with a value of 1 instead of 0 as the final command line parameter Even when GFS is not enforcing quotas it still keeps track of the file system usage for all users and groups so that quota usage info...

Страница 113: ...tune gfs quota_account 1 gfs_quota init f gfs 9 5 Growing a File System The gfs_grow command is used to expand a GFS file system after the device where the file system resides has been expanded Runnin...

Страница 114: ...Pool Volume for additional information After running the gfs_grow command run a df command to check that the new space is now avail able in the file system 9 5 3 Examples In this example the file sys...

Страница 115: ...un on a mounted file system but it only needs to be run on one node in the cluster All the other nodes sense that the expansion has occurred To verify that the changes were successful you can use the...

Страница 116: ...fies the size of the new journals in MBytes Default journal size is 128 MBytes The minimum size is 32 MBytes To add journals of different sizes to the file system the gfs_jadd command must be run for...

Страница 117: ...r written to must also be 512 byte aligned One of the following methods can be used to enable direct I O on a file O_DIRECT GFS file attribute GFS directory attribute 9 7 1 O_DIRECT If an application...

Страница 118: ...erit_directio flag is set 9 7 3 2 Example In this example the command sets the inherit_directio flag on the directory named gfs1 data gfs_tool setflag inherit_directio gfs1 data 9 8 Data Journaling Or...

Страница 119: ...ory Specifies the directory where the flag is set or cleared File Specifies the zero length file where the flag is set or cleared 9 8 2 Examples This example shows setting the inherit_jdata flag on a...

Страница 120: ...ce MountPoint o noatime BlockDevice Specifies the block device where the GFS file system resides MountPoint Specifies the directory where the GFS file system should be mounted 9 9 1 2 Example In this...

Страница 121: ...ntPoint atime_quantum Seconds MountPoint Specifies the directory where the GFS file system is mounted Seconds Specifies the update period in seconds 9 9 2 2 Examples In this example all GFS tunable pa...

Страница 122: ...tool command Typical usage of the gfs_tool command is described here 9 11 1 Usage Displaying Statistics gfs_tool counters MountPoint The counters action flag displays statistics about a file system If...

Страница 123: ...ng the gfs_fsck command The gfs_fsck command must only be run on a file system that is unmounted from all nodes Note On nodes running Red Hat GFS 6 0 for Red Hat Enterprise Linux 3 Update 5 and later...

Страница 124: ...he link depends on the node or user following the link CDPN variables can be used in any path name not just with symbolic links However the CDPN variable name cannot be combined with other characters...

Страница 125: ...duced by the following command entry echo uname m _ uname s uid This variable resolves to a real file or directory named with the user ID string produced by the following command entry echo id u gid T...

Страница 126: ...wn a LOCK_GULM Server for more information 3 Kill the CCS daemon on all nodes 4 Deactivate all pools on all nodes Refer to Section 5 6 Activating Deactivating a Pool Volume for more information Note Y...

Страница 127: ...le 2 At each node start the CCS daemon specifying the CCA device on at the command line Refer to Section 7 2 Starting CCS in the Cluster for more information Command usage ccsd d CCADevice 3 Start the...

Страница 128: ...114 Chapter 9 Managing GFS...

Страница 129: ...f the nodes in the cluster Eventually that condition may bring the entire cluster to a halt as other nodes require access to those parts of the file system If a node fails it cannot be permitted to re...

Страница 130: ...orized Red Hat support representative if there is a device you wish to use for fencing that is not described in the following sections The following sections describe the fencing methods available wit...

Страница 131: ...itch and disables the specific port associated with the node Nodes with multiple FC paths can have each path disabled Refer to Section 6 6 Creating the fence ccs File and Section 6 7 Creating the node...

Страница 132: ...al hardware The fence_gnbd fencing agent instructs all GNBD servers to disallow all I O from a fenced node When a fenced node is reset and re imports the GNBD devices the GNBD servers again allow the...

Страница 133: ...istrator must run the command fence_ack_manual to indicate to the system that the failed node has been reset Recovery of the reset node will then proceed Refer to Section 6 6 Creating the fence ccs Fi...

Страница 134: ...120 Chapter 10 Using the Fencing System...

Страница 135: ...le clients forms a shared storage configuration through which GFS can be used The GNBD driver is implemented through the following client and server kernel modules gnbd o Implements the GNBD device dr...

Страница 136: ...re stored locally or on FC attached storage c Enable caching Reads from the exported GNBD and takes advantage of the Linux page cache By default the gnbd_export command does not enable caching Caution...

Страница 137: ...BD server by hostname or IP address from which to import GNBDs All GNBDs exported from the server are imported on the client running this command 11 1 2 2 Example This example imports all GNBDs from t...

Страница 138: ...side on local storage or FC attached storage Because lock servers need to start before GNBD servers can start a lock server cannot access CCS files through a GNBD If a lock server is running on a GFS...

Страница 139: ...he following fencing devices APC MasterSwitch fence_apc fence agent WTI NPS fence_wti fence agent Brocade FC switch fence_brocade fence agent McData FC switch fence_mcdata fence agent Vixel FC switch...

Страница 140: ...d to mount a GFS file system The GNBD server node must not import gnbd_import command other GNBD devices to run the file system 2 The GNBD server must export all the GNBDs in uncached mode and it must...

Страница 141: ...ovides the following init d scripts that are installed automatically when GFS is installed pool ccsd lock_gulmd gfs The scripts automatically start and stop GFS modules during startup and shutdown of...

Страница 142: ...s all the devices and loads any pool that it finds To specify the pools on which to operate the variable POOLS must be set in etc sysconfig gfs You can define multiple pools by separating the pool nam...

Страница 143: ...e that includes the GFS file system trin1 gfs dev pool trin1 gfs gfs gfs defaults 0 0 If you do not want a GFS file system to automatically mount on startup add noauto to the options in the ect fstab...

Страница 144: ...130 Chapter 12 Using GFS init d Scripts...

Страница 145: ...llowing at the command line redhat config gfscluster gulm bridge This is a fence method available for Red Hat Cluster nodes if and only if the Red Hat GFS RPM is installed on the node that the Cluster...

Страница 146: ...des for configuring an initial instance of Red Hat GFS by writing the three Red Hat GFS configuration files cluster ccs nodes ccs and fence ccs The GFS Setup Druid requires an etc cluster xml file whe...

Страница 147: ...our deployment you may want to consider using multiple Red Hat Cluster Manager clusters A 3 3 Upgrading Red Hat GFS 5 2 1 to Red Hat GFS 6 0 To upgrade Red Hat GFS 5 2 1 to Red Hat GFS 6 0 follow the...

Страница 148: ...134 Appendix A Using Red Hat GFS with Red Hat Cluster Suite...

Страница 149: ...wing actions a Unmount GFS file systems from all nodes b Stop lock servers c Stop ccsd on all nodes d Deactivate pools e Unload kernel modules 2 Install new software This step consists of the followin...

Страница 150: ...described in Step 5 Command usage ccs_tool create Directory Device Reference Section 7 1 Creating a CCS Archive Example ccs_tool create root alpha dev pool alpha_cca 7 Start ccsd on all nodes This in...

Страница 151: ...ULM servers This section provides the following information about the example Section C 1 1 Key Characteristics Section C 1 2 Kernel Modules Loaded Section C 1 3 Setup Process C 1 1 Key Characteristic...

Страница 152: ...device driver is loaded and that it loaded without errors The small partition dev sda1 is used to store the cluster configuration information The two re maining partitions dev sda2 sdb1 are used for t...

Страница 153: ...ing poolname alpha_cca subpools 1 subpool 0 0 1 pooldevice 0 0 dev sda1 3 Use the pool_tool command to create all the pools as follows n01 pool_tool c pool_gfs01 cf pool_gfs02 cf alpha_cca cf Pool lab...

Страница 154: ...ip_interfaces eth0 10 0 1 2 fence power apc port 2 n03 ip_interfaces eth0 10 0 1 3 fence power apc port 3 Note If your cluster is running Red Hat GFS 6 0 for Red Hat Enterprise Linux 3 Update 5 and la...

Страница 155: ...is rebooted The CCA device must be specified when starting ccsd n01 ccsd d dev pool alpha_cca n02 ccsd d dev pool alpha_cca n03 ccsd d dev pool alpha_cca 8 At each node start the LOCK_GULM server n01...

Страница 156: ...ning LOCK_GULM only This section provides the following information about the example Section C 2 1 Key Characteristics Section C 2 2 Kernel Modules Loaded Section C 2 3 Setup Process C 2 1 Key Charac...

Страница 157: ...ary to load an appropriate device driver If the shared storage devices are not visible on each node confirm that the device driver is loaded and that it loaded without errors The small partition dev s...

Страница 158: ...the pool that will be used for CCS data The pool does not need to be very large The name of the pool will be alpha_cca The name of the cluster alpha followed by _cca The file should look like the foll...

Страница 159: ...assembled pool_gfs02 assembled 5 Create CCS files a Create a directory called root alpha on node n01 as follows n01 mkdir root alpha n01 cd root alpha b Create the cluster ccs file This file contains...

Страница 160: ...les n03 ip_interfaces eth0 10 0 1 3 fence power apc port 3 lck01 ip_interfaces eth0 10 0 1 4 fence power apc port 4 lck02 ip_interfaces eth0 10 0 1 5 fence power apc port 5 lck03 ip_interfaces eth0 10...

Страница 161: ...gent fence_apc ipaddr 10 0 1 10 login apc passwd apc 6 Create the CCS Archive on the CCA Device Note This step only needs to be done once and from a single node It should not be performed every time t...

Страница 162: ...Protocol lock_gulm Lock Table alpha gfs02 Syncing All Done 10 Mount the GFS file systems on all the nodes Mount points gfs01 and gfs02 are used on each node n01 mount t gfs dev pool pool_gfs01 gfs01...

Страница 163: ...r to Table C 10 for storage device information Number of file systems 2 File system names gfs01 and gfs02 File system mounting Each GFS node mounts the two file systems Cluster name alpha Host Name IP...

Страница 164: ...example consists of the following steps 1 Create pool configurations for the two file systems Create pool configuration files for each file system s pool pool_gfs01 for the first file system and pool_...

Страница 165: ...assembled pool_gfs01 assembled pool_gfs02 assembled 5 Create CCS files a Create a directory called root alpha on node n01 as follows n01 mkdir root alpha n01 cd root alpha b Create the cluster ccs fil...

Страница 166: ...for syntax description of the usedev parameter d Create the fence ccs file This file contains information required for the fencing method s used by the GFS cluster The file should look like the follow...

Страница 167: ...ool pool_gfs01 Device dev pool pool_gfs01 Blocksize 4096 Filesystem Size 1963216 Journals 3 Resource Groups 30 Locking Protocol lock_gulm Lock Table alpha gfs01 Syncing All Done n01 gfs_mkfs p lock_gu...

Страница 168: ...3 Refer to Table C 12 for node information Number of lock server nodes 1 The lock server is run on one of the GFS nodes embedded Refer to Table C 13 for node information Locking protocol LOCK_GULM Th...

Страница 169: ...mmand cat proc partitions Depending on the hardware configuration of the GFS nodes the names of the devices may be different on each node If the output of the cat proc partitions command shows only en...

Страница 170: ...nodes Note This step must be performed every time a node is rebooted If it is not the pool devices will not be accessible Activate the pools using the pool_assemble a command for each node as follows...

Страница 171: ...ess and node specific I O fencing parameters The file should look like the following nodes n01 ip_interfaces eth0 10 0 1 1 fence power apc port 1 n02 ip_interfaces eth0 10 0 1 2 fence power apc port 2...

Страница 172: ...CCS Archive on the CCA Device Note This step only needs to be done once and from a single node It should not be performed every time the cluster is restarted Use the ccs_tool command to create the arc...

Страница 173: ...mount t gfs dev pool pool_gfs01 gfs01 n02 mount t gfs dev pool pool_gfs02 gfs02 n03 mount t gfs dev pool pool_gfs01 gfs01 n03 mount t gfs dev pool pool_gfs02 gfs02 C 5 LOCK_GULM SLM External and GNBD...

Страница 174: ...transport layer for the storage devices Refer to Table C 19 for storage device information Number of file systems 2 File system names gfs01 and gfs02 File system mounting Each GFS node mounts the two...

Страница 175: ...n at each node in your GFS cluster by running the follow ing command cat proc partitions Depending on the hardware configuration of the GFS nodes the names of the devices may be different on each node...

Страница 176: ...files should look like the following poolname pool_gfs01 subpools 1 subpool 0 0 1 pooldevice 0 0 dev gnbd gfs01 poolname pool_gfs02 subpools 1 subpool 0 0 1 pooldevice 0 0 dev gnbd gfs02 4 Create a p...

Страница 177: ...ssembled lcksrv pool_assemble a Activate pools alpha_cca assembled pool_gfs01 assembled pool_gfs02 assembled 7 Create CCS files a Create a directory called root alpha on node n01 as follows n01 mkdir...

Страница 178: ...Update 5 and later you can use the optional usedev parameter to explicitly specify an IP address rather than relying on an IP address from libresolv For more information about the optional usedev par...

Страница 179: ...CA device must be specified when starting ccsd n01 ccsd d dev pool alpha_cca n02 ccsd d dev pool alpha_cca n03 ccsd d dev pool alpha_cca lcksrv ccsd d dev pool alpha_cca 10 At each node start the LOCK...

Страница 180: ...ng two GFS file systems Only a single node is required because the file system will not be mounted in cluster mode This section provides the following information about the example Section C 6 1 Key C...

Страница 181: ...on each node If the output of the cat proc partitions command shows only entire disk devices for example dev sda instead of dev sda1 then the storage devices have not been partitioned To partition a...

Страница 182: ...y needs to be done once It should not be performed every time the cluster is restarted Use the ccs_tool command to create the archive from the CCS configuration files n01 ccs_tool create root alpha ro...

Страница 183: ...pool pool_gfs02 Blocksize 4096 Filesystem Size 1963416 Journals 1 Resource Groups 30 Locking Protocol lock_nolock Lock Table Syncing All Done 7 Mount the GFS file systems on the nodes Mount points gfs...

Страница 184: ...170 Appendix C Basic GFS Examples...

Страница 185: ...and server 79 local CCA files 82 changing CCS configuration files 79 combining CCS methods 82 creating a CCS archive 75 starting CCS in the cluster 76 using 75 using clustering and locking systems 85...

Страница 186: ...rectory attribute 104 file attribute 103 O_DIRECT 103 growing 99 making 89 mounting 91 quota management 94 disabling enabling quota accounting 98 disabling enabling quota enforcement 98 displaying quo...

Страница 187: ...tware 13 installation tasks 14 GFS RPM installation 15 loading GFS kernel modules 15 prerequisite tasks 13 clock synchronization software 14 perl Net Telnet module 13 persistent major number utility 1...

Страница 188: ...troduction prerequisite tasks configuration initial 17 installing system software 13 clock synchronization software 14 perl Net Telnet module 13 persistent major number 14 stunnel utility 14 Q quota m...

Страница 189: ...on examples 160 lock server node information examples 143 154 160 mount options 93 nodes css variables 64 platform requirements 11 pool configuration file keyword and variable de scriptions 26 pool_as...

Страница 190: ......

Страница 191: ...d Hat Enterprise Linux Installation Guide for x86 Itanium AMD64 and Intel Extended Memory 64 Technology Intel EM64T Karsten Wade Primary Writer Maintainer of the Red Hat SELinux Application Developmen...

Страница 192: ...8 Nadine Richter German translations Audrey Simons French translations Francesco Valente Italian translations Sarah Wang Simplified Chinese translations Ben Hung Pin Wu Traditional Chinese translation...

Результаты поиска

Содержание GFS 6.0 -

Отзывы:

Похожие инструкции для GFS 6.0 -

Бренды по названию

Популярные бренды

Red Hat GFS 6.0 -, Руководство администратора

Результаты поиска

Содержание GFS 6.0 -

Отзывы:

Похожие инструкции для GFS 6.0 -

Бренды по названию

Популярные бренды