Red Hat GFS 5.2.1 - Administrator'S Manual Download Page 1

Page: 1 / 192

Red Hat GFS 5.2.1

Administrator’s Guide

Summary of Contents for GFS 5.2.1 -

Page 1: ...Red Hat GFS 5 2 1 Administrator s Guide...

Page 2: ...rk of Linus Torvalds Motif and UNIX are registered trademarks of The Open Group XFree86 is a trademark of The XFree86 Project Inc and is pending registration Intel and Pentium are registered trademark...

Page 3: ...ing GFS 8 2 System Requirements 11 2 1 Platform Requirements 11 2 2 TCP IP Network 11 2 3 Fibre Channel Storage Network 11 2 4 Fibre Channel Storage Devices 12 2 5 Network Power Switches 12 2 6 Consol...

Page 4: ...2 Example procedure 32 5 9 Erasing a Pool Volume 33 5 9 1 Usage 33 5 9 2 Example 33 5 9 3 Comments 33 5 10 Renaming a Pool Volume 33 5 10 1 Usage 34 5 10 2 Example 34 5 11 Changing a Pool Volume Minor...

Page 5: ...LM 79 8 2 1 Selection of LOCK_GULM Servers 79 8 2 2 Number of LOCK_GULM Servers 79 8 2 3 Starting LOCK_GULM Servers 80 8 2 4 Fencing and LOCK_GULM 80 8 2 5 Shutting Down a LOCK_GULM Server 80 8 3 LOCK...

Page 6: ...How the Fencing System Works 107 10 2 Fencing Methods 107 10 2 1 APC MasterSwitch 108 10 2 2 WTI Network Power Switch 108 10 2 3 Brocade FC Switch 109 10 2 4 Vixel FC Switch 109 10 2 5 HP RILOE Card 1...

Page 7: ...2 B 1 3 Setup Process 133 B 2 LOCK_GULM RLM External 137 B 2 1 Key Characteristics 137 B 2 2 Prerequisites 139 B 2 3 Setup Process 140 B 3 LOCK_GULM SLM Embedded 145 B 3 1 Key Characteristics 145 B 3...

Page 8: ......

Page 9: ...presented this way include the following command Linux commands and other operating system commands when used are represented this way This style should indicate to you that you can type the word or p...

Page 10: ...n menu If you click on the word on the GUI screen the rest of the menu should appear For example Under File on a GNOME terminal the New Tab option allows you to open multiple shell prompts in the same...

Page 11: ...Additionally we use several different strategies to draw your attention to certain pieces of informa tion In order of how critical the information is to your system these items are marked as note tip...

Page 12: ...oving the documentation try to be as specific as possible If you have found an error please include the section number and some of the surrounding text so we can find it easily 4 Sign Up for Support I...

Page 13: ...d IP SAN Technology Building SANs with Brocade Fabric Switches by C Beauchamp J Judd and B Keo Syngress 2001 Best practices for building Fibre Channel SANs based on the Brocade family of switches incl...

Page 14: ...vi Introduction...

Page 15: ...es The GNBD servers in turn present mul tiple storage paths to GFS nodes via redundant GNBDs With GNBD multipath if a GNBD server node becomes unavailable another GNBD server node can provide GFS node...

Page 16: ...Note The deployment examples in this chapter reflect basic configurations your needs might require a combination of configurations shown in the examples Also the examples show GNBD Global Net work Blo...

Page 17: ...s running Stored data is actually on the SAN Storage devices and data can be equally shared by network client applications File locking and sharing functions are handled by GFS for each network client...

Page 18: ...ts GNBD servers Disk A Mirrored disk pairs for data availability GFS Applications 1 Disk C Disk B 1 Disk A Disk C 1 Disk B Shared Files Figure 1 3 GFS and GNBD with Direct Attached Storage 1 3 GFS Fun...

Page 19: ...very Cluster management functions in GFS monitor node status through heartbeat signals to determine cluster membership Also cluster management keeps track of which nodes are using each GFS file system...

Page 20: ...sd ccsd CCS daemon that runs on all cluster nodes and provides configuration file data to cluster software ccs_servd CCS server daemon that distributes CCS data from a single server to ccsd daemons wh...

Page 21: ...GFS file systems gulm_tool Command that configures and debugs the lock_gulmd server LOCL_NOLOCK lock_nolock o Kernel module installed on a node using GFS as a local file system GFS gfs o Kernel module...

Page 22: ...ster name can be 1 to 16 characters long For example this book uses a cluster name alpha in some example configuration procedures Number of file systems Determine how many GFS file systems to create i...

Page 23: ...ccs and nodes ccs Refer to Section 6 7 Creating the fence ccs File and Section 6 8 Creating the nodes ccs File for more information To help determine the type of fencing methods available with GFS re...

Page 24: ...10 Chapter 1 GFS Overview...

Page 25: ...x86 64 x86 SMP supported 256 MB minimum RHEL2 1 AS x86 ia64 SMP supported 256 MB minimum SLES8 x86 SMP supported 256 MB minimum SuSE9 x86 64 SMP supported 256 MB minimum Table 2 1 Platform Requirement...

Page 26: ...that the devices can operate reliably when heavily accessed simultaneously from multiple initiators Note Make sure that your GFS configuration does not exceed the number of nodes an array or JBOD sup...

Page 27: ...3 1 1 Net Telnet Perl Module The Net Telnet Perl module is used by several fencing agents and should be installed on all GFS nodes The Net Telnet Perl module should be installed before installing GFS...

Page 28: ...em software perform the following steps 1 Install a Linux kernel Note This step is required only if you are using Red Hat Enterprise Linux Version 3 Update 1 or earlier or if you are using Red Hat Lin...

Page 29: ...ke sure that the boot loader is configured to load the new kernel For example check the GRUB menu lst file 5 Reboot each node into the new kernel 6 At each node verify that the node is running the the...

Page 30: ...copy or download it to each GFS node Make sure to select the GFS software that matches the kernel selected in Section 3 2 1 Installing a Linux Kernel 2 At each node issue the following rpm command rp...

Page 31: ...e modprobe commands to load the GFS kernel modules all dependent modules will be loaded automatically Alternatively you can load the GFS kernel modules by following the steps in Table 3 4 The steps in...

Page 32: ...gulm o module 5 insmod gfs Loads the gfs o module 6 lsmod Verifies that all GFS kernel modules are loaded This shows a listing of currently loaded modules It should display all the modules loaded in t...

Page 33: ...our GFS configuration refer to Section 1 5 Before Configuring GFS and that you have installed the kernel and the GFS software into each GFS node refer to Chapter 3 Installing System Software 4 2 Initi...

Page 34: ...At each node activate pools Refer to Section 5 6 Activating Deactivating a Pool Volume Command usage pool_assemble 4 2 2 Setting Up and Starting the Cluster Configuration System To set up and start t...

Page 35: ...ers Command usage lock_gulmd 4 2 4 Setting Up and Mounting File Systems To set up and mount file systems follow these steps 1 Create GFS file systems on pools created in Step 1 of Section 4 2 2 Settin...

Page 36: ...22 Chapter 4 Initial Configuration...

Page 37: ...t presents physical storage devices such as disks or RAID ar rays as logical volumes to GFS cluster nodes Pool can aggregate storage devices either by concate nating the underlying storage or by strip...

Page 38: ...configuration act on existing pools and require one or more pool names to be specified on the command line Flag Function Section Page Reference c Create a pool Section 5 5 Creating a Pool Volume e Er...

Page 39: ...D Enable debugging output h Help Show usage information q Be quiet Do not display output from the command V Display command version information then exit v Verbose operation Table 5 4 pool_assemble Co...

Page 40: ...5 6 pool_info Command Options 5 2 4 pool_mp The pool_mp command is for managing multipathing on running pools refer to Table 5 7 and Table 5 8 One or more pool names can be specified on the command l...

Page 41: ...ls other than those mentioned in the pre ceding paragraph 5 3 1 Usage pool_tool s 5 3 2 Example In this example the response to the command displays information about one GFS file system other file sy...

Page 42: ...mber subpools number subpool id stripe devices type pooldevice subpool id device Figure 5 1 File Structure Pool Configuration File File Line and Keyword Variable Description poolname name The name of...

Page 43: ...3 dev sde1 This example creates a 4 disk pool named pool1 that has a dynamic minor number composed of a striped subpool and a concatenated subpool poolname pool1 minor 0 subpools 2 striped subpool sub...

Page 44: ...semble command activates or deactivates pools on a node Note The pool_assemble command must be run on every node that accesses shared pools also it must be run each time a node reboots The pool_assemb...

Page 45: ...run on every GFS node The pool_assemble command should be put in the node s system startup scripts so that pools are activated each time the node boots 5 7 Displaying Pool Configuration Information Us...

Page 46: ...onfiguration Information for information on creating a config uration file for an existing pool volume 5 8 1 Usage pool_tool g ConfigFile ConfigFile Specifies the file describing the extended pool Not...

Page 47: ..._data subpool 1 0 1 gfs_data Added pooldevice 0 0 dev sdb1 pooldevice 0 1 dev sdc1 pooldevice 0 2 dev sdd1 pooldevice 0 3 dev sde1 pooldevice 1 0 dev sdf1 Added 4 Run the pool_tool command with the gr...

Page 48: ...ged Note In releases before GFS 5 2 the r flag had a different usage Note You must deactivate a pool before renaming it You can deactivate a pool with the pool_assemble r PoolName command 5 10 2 Examp...

Page 49: ...st reload the pools on each node in the cluster by issuing a pool_assemble r PoolName command followed by a pool_assemble a PoolName command 5 12 Displaying Pool Volume Information The pool_info comma...

Page 50: ...sed to display pool read write information and to clear statistics from pools Using the pool_info command with the s option displays the number of reads and writes for the named pool s since the last...

Page 51: ...you can reintegrate failed paths Note Pool multipathing must be enabled in the license file to use this feature 5 14 1 Usage Change the Type of Multipathing pool_mp m none failover n PoolName none fai...

Page 52: ...or all pools to none pool_mp m none This example adjusts the multipathing for pool0 to failover pool_mp m failover pool0 This example adjusts the multipathing for pool0 to round robin with a stripe si...

Page 53: ...ing the license ccs File Section 6 6 Creating the cluster ccs File Section 6 7 Creating the fence ccs File Section 6 8 Creating the nodes ccs File 6 1 Prerequisite Tasks Before creating the CCS files...

Page 54: ...h node having multiple paths to FC storage include in the fencing method section a fencing device for each path to FC storage GFS supports dual power supply fencing with the APC MasterSwitch only it s...

Page 55: ...cing device using the fence_notify_gnbd fencing agent to indicate to the GFS nodes that a GNBD device is not available if it is fenced For more information about creating the fence ccs and nodes ccs f...

Page 56: ...Creating the cluster ccs file consists of specifying the following parameters Cluster name Each node that runs LOCK_GULM server Optional parameters To create the cluster ccs file follow these steps 1...

Page 57: ...are checked by the server in seconds Two thirds of this time is the rate at which the heartbeats are sent The default values is 15 Number Optional For allowed_misses How many consecutive heartbeats ca...

Page 58: ...efer to Table 6 2 for syntax description APC MasterSwitch Refer to Figure 6 2 WTI NPS Network Power Switch Refer to Figure 6 3 Brocade FC Fibre Channel switch Refer to Figure 6 4 Vixel FC switch Refer...

Page 59: ...BD multipath Note For GFS clusters using GNBD multipath do not configure the GNBD server nodes to be fenced with the GNBD fencing agent fence_gnbd f For Fence Notify GNBD used with GNBD multipath spec...

Page 60: ...ure 6 3 File Structure fence_devices fence_wti fence_devices DeviceName agent fence_brocade ipaddr IPAddress login LoginName passwd LoginPassword DeviceName Figure 6 4 File Structure fence_devices fen...

Page 61: ...nce_gnbd without GNBD Multipath fence_devices DeviceName agent fence_gnbd server ServerName server ServerName option multipath retrys Number wait_time Seconds DeviceName Figure 6 7 File Structure fenc...

Page 62: ...nodes ccs file also contains DeviceName parameters each mapping to a DeviceName in the fence ccs file HostName The host name of a RILOE card on the network to which stunnel connections can be made IP...

Page 63: ...r entry is for the retrys entry and is only valid when used with multipath style fencing Refer to the multipath entry in this table The default value of Number is 3 Seconds The length of time in secon...

Page 64: ...fence_devices silkworm1 agent fence_brocade ipaddr 10 0 3 3 login admin passwd password silkworm2 agent fence_brocade ipaddr 10 0 3 4 login admin passwd password Example 6 5 Brocade FC Switch Fencing...

Page 65: ...it_time 3 Wait time between retries set to 3 This example shows a fencing device named gnbdmp with two servers nodea and nodeb Because GNBD Multipath is employed an additional configuration entry unde...

Page 66: ...r dual power supplies or multiple paths to FC storage If a node has dual power supplies then the fencing method for the node must specify at least two fencing devices one fencing device for each power...

Page 67: ...Manual Refer to Figure 6 17 b If you are configuring a node for cascaded fencing use the file format in Figure 6 18 Refer to Table 6 3 for syntax description Note Figure 6 18 and Figure 6 19 do not sh...

Page 68: ...cing specify MethodName DeviceName and IPAddress Refer to Example 6 17 for a nodes ccs file that specifies GNBD fencing e If using HP RILOE fencing specify MethodName DeviceName and PortNumber Refer t...

Page 69: ...ddress fence MethodName DeviceName port PortNumber switch SwitchNumber NodeName File format for node identification same format for all nodes File format for APC MasterSwitch fencing method for node w...

Page 70: ...fencing method for node with dual power supplies Fencing a node with dual power supplies requires that both power supplies be powered off before rebooting the node To accomplish that the nodes ccs fi...

Page 71: ...NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName port PortNumber NodeName File format for node identification same format for all nodes File format for WTI NPS fencing meth...

Page 72: ...DeviceName port PortNumber DeviceName port PortNumber NodeName File format for node identification same format for all nodes File format for Brocade or Vixel FC Switch fencing method Additional fencin...

Page 73: ...es NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName ipaddr IPAddress NodeName File format for node identification same format for all nodes File format for GNBD fencing met...

Page 74: ...eName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName localport PortNumber NodeName File format for node identification same format for all nodes File format for HP RILOE fencing m...

Page 75: ...des NodeName NodeName ip_interfaces IFNAME IPAddress fence MethodName DeviceName ipaddr IPAddress NodeName File format for node identification same format for all nodes File format for manual fencing...

Page 76: ...odeName ip_interfaces IFName IPAddress fence MethodName DeviceName Device specific parameter s MethodName DeviceName Device specific parameter s MethodName DeviceName Device specific parameter s NodeN...

Page 77: ...evice to use with a node Use a valid fencing device name specified by a DeviceName parameter in the fence ccs file fence ccs fence_devices DeviceName IFName The interface name of the IP address specif...

Page 78: ...tiple networks To determine the hostname of a node you can use the uname n command at the node To verify the IP address associated with the hostname you can issue a ping command to the hostname PortNu...

Page 79: ...r power supply 2 port 7 switch 2 option off Power down power supply 2 apc1 Fencing device for power supply 1 port 6 switch 1 option on Power up power supply 1 apc2 Fencing device for power supply 2 po...

Page 80: ...lkworm2 Additional fencing device for additional port 4 path to FC storage n02 Example 6 15 Node Defined for Brocade FC Switch Fencing nodes n01 ip_interfaces hsi0 10 0 0 1 fence san vixel1 port 3 vix...

Page 81: ...Files 67 nodes n01 ip_interfaces hsi0 10 0 0 1 fence server gnbd ipaddr 10 0 1 1 n02 Example 6 17 Node Defined for GNBD Fencing nodes n01 ip_interfaces hsi0 10 0 0 1 fence riloe riloe1 localport 2345...

Page 82: ...encing with Brocade FC switch brocade1 port 1 power Fencing with APC MasterSwitch apc port 1 switch 1 n02 This example shows a node that can be fenced using a Brocade FC switch or an APC MasterSwitch...

Page 83: ...10 0 0 1 fence power APC MasterSwitch fencing device apc1 port 6 switch 2 notify_gnbd Fence Notify GNBD fencing device ipaddr 10 0 0 1 using fence_notify_gnbd fencing agent n02 Example 6 21 GNBD Serv...

Page 84: ...70 Chapter 6 Creating the Cluster Configuration System Files...

Page 85: ...ath 7 1 Creating a CCS Archive A CCS archive is a collection of CCS configuration files that can be accessed by the cluster The ccs_tool command is used to create a CCS archive from a directory contai...

Page 86: ...due to initialization of the device The ccs_tool command uses the Linux raw device interface to update and read a CCA device directly bypassing operating system caches Caching effects could otherwise...

Page 87: ...ct com mand creates a new directory specified on the command line and recreates the CCS files in the direc tory The CCS archive remains unaffected by this command 7 3 1 1 Usage ccs_tool extract CCADev...

Page 88: ...ation files in directory root alpha are compared with the con figuration files in CCA device dev pool alpha_cca ccs_tool diff dev pool alpha_cca root alpha 7 4 Changing CCS Configuration Files Based o...

Page 89: ...easonable to run a CCS server on the same node as the LOCK_GULM server The CCS server does not have failover capabilities The CCS server is called ccs_servd it can be run on any node in or out of the...

Page 90: ...cca 7 5 1 2 Starting the CCS Server There are two parts to starting CCS in the cluster when using a CCS server The first is starting ccs_servd and the second is starting ccsd on all the cluster nodes...

Page 91: ...ult port number IPAddress PortNumber ClusterName Specifies the name of the cluster The CCS server uses this to pick the correct CCA file that is named for the cluster 7 5 1 3 2 Example This example st...

Page 92: ...ternal lock servers that do not have access to the shared storage In that case the client server methodology Section 7 5 1 CCA File and Server could be employed but that approach introduces a server p...

Page 93: ...for all GFS LOCK_GULM file systems in the cluster In the case of RLM multiple servers can be run redundantly on multiple nodes If the master server fails another hot standby server takes over The LOC...

Page 94: ...e system corruption Also be sure that all nodes running lock_gulmd can communicate over the network that is there is no network partition The lock_gulmd server is started with no command line options...

Page 95: ...m allows GFS to be used as a local file system on a single node The kernel module for a GFS LOCK_NOLOCK node is lock_nolock o The lock protocol as spec ified when creating a GFS LOCK_NOLOCK file syste...

Page 96: ...82 Chapter 8 Using Clustering and Locking Systems...

Page 97: ...3 Context Dependent Path Names Section 9 14 Shutting Down a GFS Cluster Section 9 15 Restarting a GFS Cluster 9 1 Making a File System Making a GFS file system is one of the final tasks in the process...

Page 98: ...Specifies the number of journals to be created by the gfs_mkfs command One journal is re quired for each node that mounts the file system More journals can be specified to allow for easier future exp...

Page 99: ...ing as a local file system one node only O Prevents the gfs_mkfs command from asking for confirmation before writing the file system q Quiet Do not display anything r MegaBytes Specifies the size of t...

Page 100: ...In addition to using GFS specific options de scribed in this section you can use other standard mount command options for example r For information about other Linux mount command options see the Linu...

Page 101: ...le system can be unmounted the same way as any Linux file system Note The umount command is a Linux system command Information about this command can be found in the Linux umount command man pages 9 3...

Page 102: ...imit gfs_quota limit u User l Size f MountPoint gfs_quota limit g Group l Size f MountPoint Setting Quotas Warn Limit gfs_quota warn u User l Size f MountPoint gfs_quota warn g Group l Size f MountPoi...

Page 103: ...a specific user It can be either a user name from the password file or the UID number Group A group ID to display information about a specific group It can be either a group name from the group file o...

Page 104: ...3 Synchronizing Quotas GFS stores all quota information in its own internal file on disk A GFS node does not update this quota file for every file system write rather it updates the quota file once ev...

Page 105: ...s to one hour 3600 seconds for file system gfs on a single node gfs_tool settune gfs quota_quantum 3600 9 4 4 Disabling Enabling Quota Enforcement Enforcement of quotas can be disabled for a file syst...

Page 106: ...et This accounting incurs some overhead that is unnecessary if quotas are not used This quota accounting can be completely disabled by setting the quota_account tunable parameter to 0 This must be don...

Page 107: ...e space that has been added The gfs_grow command can only be run on a mounted file system but only needs to be run on one node in the cluster All the other nodes sense that the expansion has occurred...

Page 108: ...and version information then exit v Turn up the verbosity of messages Table 9 3 GFS specific Options Available While Expanding A File System 9 6 Adding Journals to a File System The gfs_jadd command i...

Page 109: ...Grow ing a Pool Volume for additional information After running the gfs_jadd command run a gfs_jadd command with the T and v flags enabled to check that the new journals have been added to the file s...

Page 110: ...this flag Using the v flag with the T flag turns up the verbosity level to display more information q Quiet Turns down the verbosity level V Display command version information then exit v Turn up th...

Page 111: ...7 2 1 Usage Set Direct I O Attribute Flag gfs_tool setflag directio File Clear Direct I O Attribute Flag gfs_tool clearflag directio File File Specifies the file where the directio flag is assigned 9...

Page 112: ...e file data is written to the journal in addition to the metadata An fsync returns as soon as the data is written to the journal which can be substantially faster than the time it takes to write the f...

Page 113: ...tus was changed mtime The last time the file or directory data was modified atime The last time the file or directory data was accessed If atime updates are enabled as they are by default on GFS and o...

Page 114: ...of sync by a significant fraction of the tunable parameter atime_quantum then atime updates are written more frequently Increasing the frequency of atime updates may cause performance degradation in c...

Page 115: ...ite activity to a file system can be suspended for a time by using the gfs_tool command s freeze action The unfreeze action returns the file system to its ordinary state That feature allows hardware b...

Page 116: ...file system to which the action applies File Specifies the file from which to get information The gfs_tool command provides additional action flags options not listed in this section For more informat...

Page 117: ...reated that point to variable destination files or directories The variables are resolved to real files or directories each time an application follows the link The resolved value of the link depends...

Page 118: ...cho uname s sys This variable resolves to a real file or directory named with the combined machine type and OS release strings produced by the following command entry echo uname m _ uname s uid This v...

Page 119: ...Activating Deactivating a Pool Volume for more information 9 15 Restarting a GFS Cluster When restarting a GFS cluster perform the following steps Tip You may want to automate some of the steps in the...

Page 120: ...r node start lock_gulmd Refer to Section 8 2 3 Starting LOCK_GULM Servers for more information Command usage lock_gulmd 4 At each node mount the GFS file systems Refer to Section 9 2 Mounting a File S...

Page 121: ...f the nodes in the cluster Eventually that condition may bring the entire cluster to a halt as other nodes require access to those parts of the file system If a node fails it cannot be permitted to re...

Page 122: ...pc logs into the device and reboots the specific port for the failed node The fence_apc fencing agent supports nodes with dual power supplies plugged into an APC Master Switch Support for nodes with d...

Page 123: ...should be avoided during the cluster operation A fencing operation trying to use the switch will be blocked until it can log in 10 2 4 Vixel FC Switch A node connected to a Vixel FC Fibre Channel swi...

Page 124: ...type of fencing Note If you are using GNBD multipath you must not specify the GNBD fencing agent fence_gnbd as a fencing device for GNBD server nodes nodes that export GNBDs to GFS nodes 10 2 7 Fence...

Page 125: ...rator must manually reset the node specified in the message After the node is reset the administrator must run the com mand fence_ack_manual to indicate to the system that the failed node has been res...

Page 126: ...112 Chapter 10 Using the Fencing System...

Page 127: ...must run with caching disabled Data corruption occurs if the GNBDs are run with caching enabled Refer to Section 11 2 1 Exporting a GNBD from a Server for more information about using the gnbd_export...

Page 128: ...ly removes the nodes from the network and notifies the GFS nodes that the GNBD server node has been fenced To physically remove a GNBD server node you can use any of the following fencing devices APC...

Page 129: ...been identified to be exported the gnbd_export command is used to export it Caution When configured for GNBD multipath a GNBD server a server that is exporting a GNBD ignores Linux page caching Cachi...

Page 130: ...on is required for automatic reconnection In order for a GNBD client to automatically reconnect to a GNBD server when Pool the GFS volume man ager tries to reintegrate paths the GNBD server must be at...

Page 131: ...Chapter 11 Using GNBD 117 11 2 2 2 Example This example imports all GNBDs from the server named nodeA gnbd_import i nodeA...

Page 132: ...118 Chapter 11 Using GNBD...

Page 133: ...License Type Database License Type Enterprise GFS Read Access X X X X X Write Access X X X X CDPN X X X Direct I O X X X Pool Volume Manager Mulitpathing X X Cluster Locking SLM X X X X RLM X X Fencin...

Page 134: ...placing a License File Changing the Lock Manager 12 3 1 Replacing a License File Not Changing the Lock Manager If you are replacing your license file and not changing the type of lock manager follow t...

Page 135: ...ecreate the archive using the ccs_tool create command 2 My license file is no longer valid How do I obtain a copy of my license Contact Red Hat Support for a new license 3 I have misplaced my license...

Page 136: ...been created ccs_tool can be run with the list action in order to list the files in an archive Again license ccs should be in the list of files If the CCS system is up and running the ccs_read list co...

Page 137: ...FS 5 2 x incorporates significant changes that relate to configuration information The single con figuration file used in GFS 5 1 x with the gfs_conf command has been replaced in the GFS 5 2 x Cluster...

Page 138: ...ation Differences A 1 3 GFS LockProto If the LOCK_GULM locking method was used with GFS 5 1 x the LockProto field in the GFS superblock does not need to be changed If the LOCK_DMEP locking method was...

Page 139: ...exists but will be obsolete after GFS 5 2 x Table A 5 summarizes the command difference for growing a pool in GFS 5 1 x and GFS 5 2 x Difference GFS 5 1 x GFS 5 2 x Command for growing a pool pool_gro...

Page 140: ...s d Unloading kernel modules Reference Chapter 3 Installing System Software 3 Install new software This step consists of the following actions a Installing kernel RPMs or compile a new kernel supporte...

Page 141: ...GFS 5 2 x cluster Example In the following example gfs_conf creates a directory named alpha containing the CCS files cluster ccs nodes ccs and fence ccs corresponding to the GFS 5 1 x CIDEV dev pool p...

Page 142: ...device on which the GFS file system resides ClusterName The name of the cluster specified in cluster css cluster name FSName A unique name from 1 to 16 characters long chosen for this file system Ref...

Page 143: ...server is started on server nodes listed in cluster ccs cluster lock_gulm servers Command usage lock_gulmd Reference Section 8 2 3 Starting LOCK_GULM Servers Example lock_gulmd 13 Mount GFS file syst...

Page 144: ...130 Appendix A Upgrading GFS...

Page 145: ...CK_GULM servers This section provides the following information about the example Section B 1 1 Key Characteristics Section B 1 2 Prerequisites Section B 1 3 Setup Process B 1 1 Key Characteristics Th...

Page 146: ...ver is loaded and that it loaded without errors The small partition dev sda1 is used to store the cluster configuration information The two re maining partitions dev sda2 sdb1 are used for the GFS fil...

Page 147: ...ock_gulm o pool o Note The GFS kernel modules must be loaded every time a node is rebooted If they are not loaded GFS will not function You can confirm that the modules have been loaded by running the...

Page 148: ...alpha followed by _cca The file should look like the following poolname alpha_cca subpools 1 subpool 0 0 1 pooldevice 0 0 dev sda1 3 Use the pool_tool command to create all the pools as follows n01 p...

Page 149: ...re the LOCK_GULM server is run The file should look like the following cluster name alpha lock_gulm servers n01 n02 n03 c Create the nodes ccs file This file contains the name of each node its IP addr...

Page 150: ...ce and from a single node It should not be performed every time the cluster is restarted Use the ccs_tool command to create the archive from the CCS configuration files n01 ccs_tool create root alpha...

Page 151: ...ups 30 Locking Protocol lock_gulm Lock Table alpha gfs02 Syncing All Done 10 Mount the GFS file systems on all the nodes Mount points gfs01 and gfs02 are used on each node n01 mount t gfs dev pool poo...

Page 152: ...nformation Locking protocol LOCK_GULM The LOCK_GULM server is run on each lock server node Number of shared storage devices 2 Refer to Table B 7 for storage device information Number of file systems 2...

Page 153: ...ster by running the follow ing command cat proc partitions Depending on the hardware configuration of the GFS nodes the names of the devices may be different on each node If the output of the cat proc...

Page 154: ...example shows the modprobe command being used for loading the ccs o module modprobe ccs Note When using modprobe do not specify the o suffix as part of the module name If the response to the modprobe...

Page 155: ...pool_gfs02 cf Pool label written successfully from alpha_cca cf 4 Activate the pools on all nodes Note Perform this step each time a node is rebooted otherwise the pool devices will not be acces sibl...

Page 156: ...The file should look like the following cluster name alpha lock_gulm servers lck01 lck02 lck03 c Create the nodes ccs file This file contains the name of each node its IP address and node specific I O...

Page 157: ...0 0 1 10 login apc passwd apc e The GFS license file should have been downloaded with the GFS RPMs Rename the license file to license ccs and move it into the CCS directory n01 mv license ccs root alp...

Page 158: ...lck01 lck02 and lck03 were specified in the cluster ccs file earlier 9 Create the GFS file systems Create the first file system on pool_gfs01 and the second on pool_gfs02 The names of the two file sy...

Page 159: ...B 3 1 Key Characteristics Section B 3 2 Prerequisites Section B 3 3 Setup Process B 3 1 Key Characteristics This example configuration has the following key characteristics Fencing device An APC Maste...

Page 160: ...ing partitions dev sda2 sdb1 are used for the GFS file systems You can display the storage device information at each node in your GFS cluster by running the follow ing command cat proc partitions Dep...

Page 161: ...at you use the modprobe command using the following syntax modprobe ModuleName The following example shows the modprobe command being used for loading the ccs o module modprobe ccs Note When using mod...

Page 162: ...gfs02 cf Pool label written successfully from alpha_cca cf 4 Activate the pools on all nodes Note This step must be performed every time a node is rebooted If it is not the pool devices will not be ac...

Page 163: ...0 1 2 fence power apc port 2 n03 ip_interfaces eth0 10 0 1 3 fence power apc port 3 d Create the fence ccs file This file contains information required for the fencing method s used by the GFS cluster...

Page 164: ...cca n03 ccsd d dev pool alpha_cca 8 Start the LOCK_GULM server on GFS node n01 n01 lock_gulmd Note The lock server nodes n01 was specified in the cluster ccs file earlier 9 Create the GFS file systems...

Page 165: ...acteristics Section B 4 2 Prerequisites Section B 4 3 Setup Process B 4 1 Key Characteristics This example configuration has the following key characteristics Fencing device An APC MasterSwitch single...

Page 166: ...ed and that it loaded without errors The small partition dev sda1 is used to store the cluster configuration information The two re maining partitions dev sda2 sdb1 are used for the GFS file systems Y...

Page 167: ...ock_gulm o pool o Note The GFS kernel modules must be loaded every time a node is rebooted If they are not loaded GFS will not function You can confirm that the modules have been loaded by running the...

Page 168: ...the following poolname alpha_cca subpools 1 subpool 0 0 1 pooldevice 0 0 dev sda1 3 Use the pool_tool command to create all the pools as follows n01 pool_tool c pool_gfs01 cf pool_gfs02 cf alpha_cca c...

Page 169: ...run The file should look like the following cluster name alpha lock_gulm servers lcksrv c Create the nodes ccs file This file contains the name of each node its IP address and node specific I O fenci...

Page 170: ...evice Note This step only needs to be done once and from a single node It should not be performed every time the cluster is restarted Use the ccs_tool command to create the archive from the CCS config...

Page 171: ...source Groups 30 Locking Protocol lock_gulm Lock Table alpha gfs02 Syncing All Done 10 Mount the GFS file systems on all the nodes Mount points gfs01 and gfs02 are used on each node n01 mount t gfs de...

Page 172: ...r node that is not mounting GFS but is dedicated as a LOCK_GULM server Number of shared storage devices 2 GNBD will be used as the transport layer for the storage devices Refer to Table B 19 for stora...

Page 173: ...systems You can display the storage device information at each node in your GFS cluster by running the follow ing command cat proc partitions Depending on the hardware configuration of the GFS nodes t...

Page 174: ...ote When using modprobe do not specify the o suffix as part of the module name If the response to the modprobe command displays a message about being unable to find a module run the depmod a command a...

Page 175: ...l_gfs01 subpools 1 subpool 0 0 1 pooldevice 0 0 dev gnbd gfs01 poolname pool_gfs02 subpools 1 subpool 0 0 1 pooldevice 0 0 dev gnbd gfs02 4 Create a pool configuration for the CCS data Create a pool c...

Page 176: ...sembled pool_gfs02 assembled lcksrv pool_assemble a Activate pools alpha_cca assembled pool_gfs01 assembled pool_gfs02 assembled 7 Create CCS files a Create a directory called root alpha on node n01 a...

Page 177: ...er apc port 5 d Create the fence ccs file This file contains information required for the fencing method s used by the GFS cluster The file should look like the following fence_devices apc agent fence...

Page 178: ...ev pool alpha_cca lcksrv ccsd d dev pool alpha_cca 10 At the lock server node lcksrv start the LOCK_GULM server lcksrv lock_gulmd Note The lock server node lcksrv was specified in the cluster ccs file...

Page 179: ...s required because the file system will not be mounted in cluster mode This section provides the following information about the example Section B 6 1 Key Characteristics Section B 6 2 Prerequisites S...

Page 180: ...mes of the devices may be different on each node If the output of the cat proc partitions command shows only entire disk devices for example dev sda instead of dev sda1 then the storage devices have n...

Page 181: ...he modprobe command displays a message about being unable to find a module run the depmod a command and try loading the module again B 6 3 Setup Process The setup process for this example consists of...

Page 182: ...This step only needs to be done once It should not be performed every time the cluster is restarted Use the ccs_tool command to create the archive from the CCS configuration files n01 ccs_tool create...

Page 183: ...pool pool_gfs02 Blocksize 4096 Filesystem Size 1963416 Journals 1 Resource Groups 30 Locking Protocol lock_nolock Lock Table Syncing All Done 7 Mount the GFS file systems on the nodes Mount points gfs...

Page 184: ...170 Appendix B Basic GFS Examples...

Page 185: ...74 combining CCS methods 78 creating a CCS archive 71 starting CCS in the cluster 72 using 71 using clustering and locking systems 79 fencing and LOCK_GULM 80 locking system overview 79 LOCK_GULM 79 L...

Page 186: ...file system adding journals 94 atime configuring updates 99 mounting with noatime 99 tuning atime quantum 100 context dependent path names CDPNs 103 data journaling 98 direct I O 96 directory attribu...

Page 187: ...g systems 21 installation tasks 14 GFS kernel modules loading 16 GFS RPM installation 16 Linux kernel installation 14 installing a Linux kernel table 14 installing system software 13 installation task...

Page 188: ...l_info command functions table 25 pool_info command options table 26 pool_mp command functions table 26 pool_mp command options table 26 pool_tool command functions table 24 pool_tool command options...

Page 189: ...nding file systems 94 gfs_mkfs command options 85 GNBD server node information examples 158 installing a Linux kernel 14 license installation differences upgrade 124 lock server node information examp...

Page 190: ......

Page 191: ...se Linux Introduction to Sys tem Administration Primary Writer Maintainer of the Release Notes Contributing Writer to the Red Hat Enterprise Linux Installation Guide for x86 Itanium AMD64 and Intel Ex...

Page 192: ...178...

Search results

Summary of Contents for GFS 5.2.1 -

Reviews:

Related manuals for GFS 5.2.1 -

Brands by name

Popular brands

Red Hat GFS 5.2.1 -, Administrator'S Manual

Search results

Summary of Contents for GFS 5.2.1 -

Reviews:

Related manuals for GFS 5.2.1 -

Brands by name

Popular brands