Novell LINUX ENTERPRISE 11 - HIGH AVAILABILITY Manual Download Page 8

Troubleshooting and Reference

Managing your own cluster requires you to perform a certain amount of troubleshoot-
ing. Learn about the most common problems and how to fix them. Find a compre-
hensive reference of the command line tools the High Availability Extension offers
for administering your own cluster. Also, find a list of the most important facts and
figures about cluster resources and resource agents.

Many chapters in this manual contain links to additional documentation resources.
These include additional documentation that is available on the system as well as doc-
umentation available on the Internet.

For an overview of the documentation available for your product and the latest docu-
mentation updates, refer to

http://www.novell.com/documentation

.

1 Feedback

Several feedback channels are available:

• To report bugs for a product component or to submit enhancement requests, please

use

https://bugzilla.novell.com/

. If you are new to Bugzilla, you

might find the Bug Writing FAQs helpful, available from the Novell Bugzilla home
page.

• We want to hear your comments and suggestions about this manual and the other

documentation included with this product. Please use the User Comments feature
at the bottom of each page of the online documentation and enter your comments
there.

2 Documentation Conventions

The following typographical conventions are used in this manual:

/etc/passwd

: directory names and filenames

placeholder

: replace

placeholder

with the actual value

PATH

: the environment variable PATH

viii

High Availability Guide

Summary of Contents for LINUX ENTERPRISE 11 - HIGH AVAILABILITY

Page 1: ...SUSE Linux Enterprise High Availability Extension www novell com 11 February 18 2010 High Availability Guide...

Page 2: ...o Novell the Novell logo the N logo are registered trademarks of Novell Inc in the United States and other countries Linux is a registered trademark of Linus Torvalds All other third party trademarks...

Page 3: ...ture 11 1 5 What s New 14 2 Getting Started 19 2 1 Hardware Requirements 19 2 2 Software Requirements 20 2 3 Shared Disk System Requirements 20 2 4 Preparations 21 2 5 Overview Installing and Setting...

Page 4: ...rom Command Line 59 5 1 Command Line Tools 59 5 2 Debugging Your Configuration Changes 60 5 3 Creating Cluster Resources 60 5 4 Creating a STONITH Resource 65 5 5 Configuring Resource Constraints 66 5...

Page 5: ...11 1 Preparation and Backup 100 11 2 Update Installation 101 11 3 Data Conversion 101 11 4 For More Information 103 Part III Storage and Data Replication 105 12 Oracle Cluster File System 2 107 12 1...

Page 6: ...2 Debugging a HA Cluster 134 15 3 FAQs 136 15 4 Fore More Information 137 16 Cluster Management Tools 139 17 Cluster Resources 193 17 1 Supported Resource Agent Classes 193 17 2 OCF Return Codes 194 1...

Page 7: ...f the key features and benefits as well as modifications since the last release Learn which hardware and software requirements must be met and what preparations to take before executing the next steps...

Page 8: ...your product and the latest docu mentation updates refer to http www novell com documentation 1 Feedback Several feedback channels are available To report bugs for a product component or to submit enh...

Page 9: ...ems buttons This paragraph is only relevant for the specified architectures The arrows mark the beginning and the end of the text block This paragraph is only relevant for the specified architectures...

Page 10: ......

Page 11: ...Part I Installation and Setup...

Page 12: ......

Page 13: ...essaging and cluster resource management func tionality supporting failover failback and migration load balancing of individually managed cluster resources The High Availability Extension is available...

Page 14: ...also comes with a cluster aware file system Oracle Cluster File System OCFS2 and volume manager clustered Logical Volume Manager cLVM For replication of your data the High Availability Extension also...

Page 15: ...moved manually to troubleshoot hardware or balance the workload The High Availability Extension provides high availability from commodity components Lower costs are obtained through the consolidation...

Page 16: ...ontent for each Web site are stored on a shared disk subsystem connected to each of the servers in the cluster The following figure depicts how this setup might look Figure 1 1 Three Server Cluster Du...

Page 17: ...ers When Web Server 1 failed the High Availability Extension software Detected a failure and verified with STONITH that Web Server 1 was really dead Remounted the shared data directories that were for...

Page 18: ...ntenance on Web Server 1 or just to increase performance or accessibility of the Web sites 1 3 Cluster Configurations Cluster configurations with the High Availability Extension might or might not inc...

Page 19: ...vides the best performance you can also configure your cluster to use iSCSI iSCSI is an alternative to Fibre Channel that can be used to create a low cost Storage Area Network SAN The following figure...

Page 20: ...ude a shared disk subsystem it is also possible to create a cluster without a share disk subsystem The following figure shows how a cluster without a shared disk subsystem might look Figure 1 5 Typica...

Page 21: ...ides information on the architectural components and describes how those components interoperate 1 4 1 Architecture Layers The High Availability Extension has a layered architecture Figure 1 6 Archite...

Page 22: ...nd current status One CRM in the cluster is elected as the Designated Coordinator DC meaning that it has the master CIB All other CIBs in the cluster are a replicas of the master CIB Normal read and w...

Page 23: ...box cluster integration for their own software 1 4 2 Process Flow SUSE Linux Enterprise High Availability Extension uses Pacemaker as CRM The CRM is implemented as daemon crmd that has an instance on...

Page 24: ...n order to protect shared data or complete resource recovery For this Pacemaker comes with a fencing subsystem stonithd STONITH is an acronym for Shoot The Other Node In The Head and is usually implem...

Page 25: ...before committing it and thus changing the active cluster configuration atomically Reusing Rules Options and Sets of Operations Rules instance_attributes meta_attributes and sets of operations can be...

Page 26: ...Master resources are a special type of clone that can operate in one of two modes Container Tag for Attributes The attributes container tag has been removed Operation Field for Prerequisites The pre...

Page 27: ...t just unique for that tag References to Other Objects Some fields such as those in constraints that refer to resources are IDREFs This means that they must reference existing resources or objects in...

Page 28: ......

Page 29: ...your cluster 1 to 16 Linux servers with software as specified in Section 2 2 Software Require ments page 20 The servers do not require identical hardware memory disk space etc At least two TCP IP com...

Page 30: ...Storage Area Network or SAN is recommended for your cluster if you want data to be highly available If a shared disk subsystem is used ensure the following The shared disk system is properly set up a...

Page 31: ...Enterprise Server Administration Guide chapter Time Synchronization with NTP available at http www novell com documentation The cluster nodes will use the time server as their time synchronization sou...

Page 32: ...gure STONITH devices as resources For detailed information see Chapter 8 Fencing and STONITH page 81 You might also need to create file systems on a shared disk Storage Area Network SAN if they do not...

Page 33: ...re packages needed for High Availability clusters are not automati cally copied to the cluster nodes Install SUSE Linux Enterprise Server 11 and SUSE Linux Enterprise High Availability Extension 11 on...

Page 34: ...taddr is the network address to bind to To ease sharing configuration files across the cluster OpenAIS uses network interface netmask to mask only the address bits that are used for routing the networ...

Page 35: ...ble Security Authentication is activated HMAC SHA1 authentication is used for communication between the cluster nodes This authentication method requires a shared secret which is used to protect and a...

Page 36: ...manually each time this cluster server is booted To start OpenAIS manually use the rcopenais start command To start OpenAIS immediately click Start OpenAIS Now 6 If all options are set according to y...

Page 37: ...nodes to start OpenAIS rcopenais start 2 On one of the nodes check the cluster status with the following command crm_mon If all nodes are online the output should be similar to the following Last upda...

Page 38: ......

Page 39: ...Part II Configuration and Administration...

Page 40: ......

Page 41: ...other server based applications or services you want to make available to users at all times To create cluster resources either use the graphical user interface the Linux HA Management Client or the...

Page 42: ...Do this on every node you will connect to with the Linux HA Management Client To start the Linux HA Management Client enter crm_gui at the command line To connect to the cluster select Connection Log...

Page 43: ...g up resources or setting nodes to standby Additionally you can easily view edit import and export the XML structures of the CIB by selecting any of the Configuration subitems and selecting Show XML M...

Page 44: ...nux HA Management Client page 32 2 In the left pane select Resources and click Add Primitive 3 In the next dialog set the following parameters for the resource 3a Enter a unique ID for the resource 3b...

Page 45: ...refer to Procedure 4 2 Adding or Modi fying Meta and Instance Attributes page 36 5 If all parameters are set according to your wishes click Apply to finish the confi guration of that resource The conf...

Page 46: ...u can also set particular parameters such as Timeout for start or stop operations For more information refer to Section 4 7 Con figuring Resource Monitoring page 46 Procedure 4 2 Adding or Modifying M...

Page 47: ...that is generated from the parameters that you have defined for a specific resource or for all the resources Select Show XML Mode in the top right corner of the resource configuration dialog or in th...

Page 48: ...dding or Modifying Meta and Instance Attributes page 36 and Section 4 7 Configuring Resource Monitoring page 46 5 If all parameters are set according to your wishes click Apply to finish the confi gur...

Page 49: ...NITY INFINITY INFINITY INFINITY INFINITY When defining resource constraints you also specify a score for each constraint The score indicates the value you are assigning to this resource constraint Con...

Page 50: ...ifetime just click Add Lifetime 10 If all parameters are set according to your wishes click OK to finish the con figuration of the constraint The configuration dialog is closed and the main window sho...

Page 51: ...d run on the same node Negative values indicate the resources should not run on the same node Values of INFINITY change should to must The score will be combined with other factors to decide where to...

Page 52: ...ory otherwise it is only a suggestion or Symmetrical if true stop the resources in the reverse order Depending on the parameters and options you choose a short Description explains the effect of the o...

Page 53: ...s over to is chosen by the High Availability software If you want to choose which node a resource will fail over to you must do the following 1 Configure a location constraint for that resource as des...

Page 54: ...tially allow it to move back depending on the stickiness and constraint scores after one minute There are two exceptions to the migration threshold concept occurring when a resource either fails to st...

Page 55: ...to automatic failback except that the resource may be moved to a node that is not the one it was previously active on Value is greater than 0 The resource will prefer to remain in its current locatio...

Page 56: ...monitoring for it Resource monitoring consists of specifying a timeout and or start delay value and an interval The interval tells the CRM how often it should check the resource status Procedure 4 8...

Page 57: ...ds 8 In the Timeout field enter a value in seconds After the specified timeout pe riod the operation will be treated as failed The PE will decide what to do or execute what you specified in the On Fai...

Page 58: ...ons which may include stopping the resource to repair the failed state and restarting the resource locally or on another node The resource also may not be restarted at all depending on the configurati...

Page 59: ...ts in a specific order and runs together on the same server To simplify this configuration we support the concept of groups Groups have the following properties Starting and Stopping Resources Resourc...

Page 60: ...r as described in Section 4 1 Linux HA Management Client page 32 2 In the left pane select Resources and click Add Group 3 Enter a unique ID for the group 4 Below Options set the Initial state of reso...

Page 61: ...main window shows the newly created or modified group Figure 4 4 Linux HA Management Client Groups Example 4 1 Resource Group for a Web Server An example of a resource group would be a Web server that...

Page 62: ...HA Management Client and log in to the cluster as described in Section 4 1 Linux HA Management Client page 32 2 In the left pane switch to the Resources view and in the right pane select the group to...

Page 63: ...The group configuration dialog shows the newly added primitive 5 Add the next sub resources file system and Web server by clicking Add again 6 Set the respective parameters for each of the sub resour...

Page 64: ...provided it is supported by the resource s Resource Agent Clone resources may even be configured differently depending on which nodes they are hosted There are three types of resource clones Anonymous...

Page 65: ...tion dialog are set according to your wishes click Apply to finish the configuration of the clone 4 12 Migrating a Cluster Resource As mentioned in Section 4 5 Specifying Resource Failover Nodes page...

Page 66: ...e back to its original location or it may stay where it is depending on resource stickiness 5 In cases where the resource cannot be migrated if the resource s stickiness and constraint scores total mo...

Page 67: ...e from http clusterlabs org wiki Documentation 4 13 For More Information http clusterlabs org Home page of Pacemaker the cluster resource manager shipped with the High Availability Extension http linu...

Page 68: ......

Page 69: ...ty or the crm command line utility to create resources This chapter introduces the several crm utilities 5 1 Command Line Tools After the installation there are several tools used to administer a clus...

Page 70: ...interval 120m timeout 60s crm live configure ptest crm live configure commit 5 3 Creating Cluster Resources There are three types of RAs Resource Agents available with the cluster First there are lega...

Page 71: ...t html The configuration of those services is not standardized If you intend to use an LSB script with High Availability make sure that you understand how the relevant script is configured Often you c...

Page 72: ...dtd DTD Basically this XML format has three sections first several common descriptions second all the available parameters and last the available actions for this agent This output is meant to be mach...

Page 73: ...File System Resource The filesystem resource is configured as an OCF primitive resource It has the task of mounting and unmounting a device to a directory on start and stop requests In this case the...

Page 74: ...primitive drbd_r0 ocf heartbeat drbd params crm live configure ms drbd_resource drbd_r0 meta clone_max 2 clone_node_max 1 master_max 1 master_node_max 1 notify true crm live configure commit NFS Serv...

Page 75: ...ernal drac5 external hmchttp external ibmrsa external ibmrsa telnet external ipmi external kdumpcheck external rackpdu external riloe external sbd external ssh external vmware external xen0 external x...

Page 76: ...hich nodes a resource may be run in the crm shell with the command location Collocational constraints that tell the cluster which resources may or may not run together on a node colocation Ordering co...

Page 77: ...raints can be used to start or stop a service right before or after a different resource meets a special condition such as being started stopped or promoted to master Use the following commands in the...

Page 78: ...st inf drbd_resource promote filesystem_resource 5 6 Specifying Resource Failover Nodes To determine a resource failover use the meta attribute migration threshold For exam ple crm live configure loca...

Page 79: ...ing example configures an Apache resource and monitors it for every 30 minutes with the op keyword crm live configure primitive apache apache params op monitor interval 60s timeout 30s The same can be...

Page 80: ...ich implies a commit too crm live configure delete YOUR_ID 4 Commit the changes crm live configure commit 5 11 Configuring a Cluster Resource Group One of the most common elements of a cluster is a se...

Page 81: ...ymous clones are the simplest type These resources behave completely identically wherever they are running Because of this there can only be one copy of an anonymous clone active per machine Multi Sta...

Page 82: ...nfigure crm live configure primitive myRSC ocf myCorp myAppl operations foo op monitor interval 60 op monitor interval 61 role Master 3 Create the master slave resource crm live configure clone apache...

Page 83: ...hadow configurations you can test them one by one to see the effects of your changes The usual process looks like this 1 User starts the crm tool 2 You switch to the configure subcommand crm live conf...

Page 84: ...x ha org Homepage of High Availability Linux http www clusterlabs org mediawiki images 8 8d Crm_cli pdf Gives you an introduction to the CRM CLI tool http www clusterlabs org mediawiki images f fb Con...

Page 85: ...of at least two nodes 6 1 Configuring a Resource with the GUI Creating a sample cluster resource and migrating it to another server can help you test to ensure your cluster is functioning properly A...

Page 86: ...Add a new instance attribute and specify nic as Name and eth0 as Value then click OK The name and value are dependent on your hardware configuration and what you chose for the media configuration dur...

Page 87: ...es are known to High Availability when they may be controlled by RAs Resource Agents which are LSB scripts OCF scripts or legacy Heartbeat 1 resources All resources can be config ured with the crm com...

Page 88: ...ctions If the configuration was successful a new resource appears in crm_mon that is started on a random node of your cluster To migrate a resource to another node do the following Procedure 6 4 Migra...

Page 89: ...and is commonly done with a STONITH resource All STONITH resources reside in usr lib stonith plugins on each node WARNING SSH and STONITH Are Not Supported It is impossible to know how SSH might react...

Page 90: ...the resource coffee_machine add this resource to the directory usr lib ocf resource d kitchen To access this RA execute the command crm configure primitive coffee_1 ocf coffee_machine kitchen When imp...

Page 91: ...one node Every node must report every change that happens to a resource The cluster state is thus a collection of resource states and node states If for whatever reason a state of some node or resour...

Page 92: ...powering down a node in the cluster For an overview of the available options run stonith help or refer to the man page of stonith for more information 8 2 1 STONITH Devices To use node level fencing y...

Page 93: ...rposes They are usually more gentle on the hardware Once the cluster goes into production they must be replaced with real fencing devices The choice of the STONITH device depends mainly on your budget...

Page 94: ...ns the STONITH resources can be started stopped and monitored just like any other resource Starting or stopping STONITH resources means enabling and disabling STONITH in this case Starting and stoppin...

Page 95: ...only Do not use any of the Testing Configuration examples in real life cluster scenarios Example 8 1 Testing Configuration configure primitive st null stonith null params hostlist node1 node2 clone fe...

Page 96: ...require more attributes An IBM RSA lights out device might be configured like this configure primitive st ibmrsa 1 stonith external ibmrsa telnet params nodename node1 ipaddr 192 168 0 101 userid USER...

Page 97: ...erSwitch via telnet NOTE The APC MasterSwitch accepts only one telnet connection session a time When one session is active subsequent attempts to connect to the MasterSwitch will fail For more informa...

Page 98: ...onitor them regu larly yet sparingly Fencing devices are an indispensable part of an HA cluster but the less you need to utilize them the better Power management equipment is known to be rather fragil...

Page 99: ...rg SBD_Fencing for more details meatware meatware requires help from a human to operate Whenever invoked meatware logs a CRIT severity message which shows up on the node s console The operator then co...

Page 100: ...ty Linux Project http linux ha org ConfiguringStonithPlugins Information about STONITH plug ins on the home page of the The High Availabil ity Linux Project http linux ha org CIB Idioms Information ab...

Page 101: ...ncers may be interconnected by either high speed LAN or by geographically dispersed WAN The load balancers can dispatch requests to the different servers and make parallel services of the cluster appe...

Page 102: ...a real server to the client based on a schedule The scheduler is the part of the IPVS kernel code which decides which real server will get the next new connection User Space Controller ipvsadm Provide...

Page 103: ...vely They periodically heartbeat the I m alive message to each other through serial lines and or network in terfaces When the heartbeat daemon of the backup cannot hear the heartbeat message from the...

Page 104: ...9 3 For More Information To learn more about Linux Virtual Server refer to the project home page available at http www linuxvirtualserver org 94 High Availability Guide...

Page 105: ...havior is determined through the mode of the bonding device By default this is mode active backup which means that a different slave device will become active if the active slave fails When using Open...

Page 106: ...sses select Statically assigned IP Address and assign a basic IP address on the interface 4 Select the ethernet devices that needs to be be included into the bond by activating the check box in front...

Page 107: ...ork device s hardware address while in use 6 Make sure that the parameter miimon 100 is added to Bond Driver Options Without this parameter the data integrity is not checked regularly 7 Click on Next...

Page 108: ......

Page 109: ...the Heartbeat to the OpenAIS cluster stack The script parses the configuration stored in etc ha d ha cf and generates a new configuration file for the OpenAIS cluster stack Furthermore it adjusts the...

Page 110: ...view the Heartbeat configuration file etc ha d ha cf and check that all communication media support multicasting 3 Make sure the following files are equal on all nodes etc ha d ha cf and var lib heart...

Page 111: ...r detailed information see Section 3 1 Installing the High Availability Extension page 23 11 3 Data Conversion After having installed SUSE Linux Enterprise Server 11 and the High Availability Extensio...

Page 112: ...about the conversion stages refer to usr share doc packages pacemaker README hb2openais in your installed High Availabil ity Extension Procedure 11 4 Converting the Data After doing a test run and ch...

Page 113: ...AIS All file processing is done on the node where conversion runs and replicated to the other nodes 4 Follow the instructions on the screen After the conversion has been finished successfully start th...

Page 114: ......

Page 115: ...Part III Storage and Data Replication...

Page 116: ......

Page 117: ...k in particular OpenAIS and the DLM 12 1 Features and Benefits Since SUSE Linux Enterprise Server 10 OCFS2 can be used for example for the fol lowing storage solutions General applications and workloa...

Page 118: ...up functionality is available on all back end storage An image of the shared application files can be easily created which can help provide effective disaster recovery OCFS2 also provides the followin...

Page 119: ...parameters including the vol ume label number of node slots journal size for all node slots and volume size tunefs ocfs2 12 3 OCFS2 Packages The OCFS2 kernel module ocfs2 is installed automatically i...

Page 120: ...f your application volumes and data volumes have different requirements for mounting Make sure that the ocfs2 tools package is installed Use YaST or command line methods to install them if they are no...

Page 121: ...m 3a Start the crm shell and create a new scratch configuration crm cib new oracle glue 3b Configure Pacemaker to start the o2cb service on every node in the cluster configure primitive o2cb ocf ocfs2...

Page 122: ...e mkfs ocfs2 man page To create a new OCFS2 file system on dev sdb1 that supports up to 16 cluster nodes use mkfs ocfs2 N 16 dev sdb1 See the following table for recommended settings Description and R...

Page 123: ...slot number is appended to the local file For example journal 0000 belongs to whatever node is assigned to slot number 0 Set each volume s maximum number of node slots when you create it according to...

Page 124: ...to mount the file system The ocf resource Filesystem can be used for this task For more details see Procedure 12 1 Mounting the File system with the Cluster Manager page 114 On a successful mount the...

Page 125: ...source on nodes that also have a clone of the o2cb resource already running colocation fs with o2cb INFINITY fs clone o2cb clone order start fs after o2cb mandatory o2cb clone fs clone end 4 Upload th...

Page 126: ......

Page 127: ...must be available Internally cLVM uses the Distributed Lock Manager DLM component of the cluster stack to coordinate access to the LVM2 metadata As DLM in turn integrates with the other components of...

Page 128: ...r clvm inf dlm clone clvm clone Before proceeding confirm that these resources have started successfully in your cluster You may use crm_mon or the GUI to check the running services 3 Prepare the phys...

Page 129: ...shing these configuration steps the LVM2 configuration can be done just like on any standalone workstation 13 2 Configuring Eligible LVM2 Devices Explicitly When several devices seemingly share the sa...

Page 130: ...MPIO devices but rejects all other devices would look like this filter a dev drbd a dev by id dm uuid mpath r 4 Write the configuration file and copy it to all cluster nodes 13 3 For More Information...

Page 131: ...run both nodes as primary devices By default DRBD uses the TCP port 7788 for communications between DRBD nodes Make sure that your firewall does not prevent communication on this port You must set up...

Page 132: ...2 and the cluster resource name r0 It sets up node 1 as the primary node Make sure to modify the instructions to use your own node and file names 1 Start YaST and select the configuration module Misce...

Page 133: ...you may use dev sda6 1 for the second resource However there must be at least 128 MB space for each resource available on this disk All of these options are explained in the examples in the usr share...

Page 134: ...MPORTANT Always use the dev drbd n name in the command not the actual dev disk device name 14 3 Testing the DRBD Service If the install and configuration procedures worked as expected you are ready to...

Page 135: ...secondary r0 2d On node 2 promote the DRBD service to primary drbdadm primary r0 2e On node 2 check to see if node 2 is primary rcdrbd status 2f On node 2 create a mount point such as srv r0mount mkd...

Page 136: ...1 check to see if node 1 is primary rcdrbd status 5 To get the service to automatically start and fail over if the server has a problem you can set up DRBD as a high availability service with OpenAIS...

Page 137: ...ecute the calls Review the output to make sure you know the source and cause of any errors 3 If there are errors in the drbd conf file correct them before continuing 4 If the partitions and settings a...

Page 138: ...nnect to each other In this case the following message is written to var log messages Split Brain detected dropping connection To resolve this situation enter the following on the node which has data...

Page 139: ...The project home page http www drbd org http clusterlabs org wiki DRBD_HowTo_1 0 by the Linux Pacemaker Cluster Stack Project Distributed Replicated Block Device DRBD 129...

Page 140: ......

Page 141: ...Part IV Troubleshooting and Reference...

Page 142: ......

Page 143: ...aging a cluster are included in the High Availability installation pattern available with the High Availability Extension Check if High Availability Extension is installed as an add on to SUSE Linux E...

Page 144: ...o make sure the mcastport is not blocked by the firewall check the settings in etc sysconfig SuSEfirewall2 on each node Alternatively start the YaST firewall module on each cluster node After clicking...

Page 145: ...IPaddr Stopped Operations Node d230 aa migration threshold 1000000 5 probe rc 0 ok 37 stop rc 0 ok 38 start rc 0 ok 39 monitor interval 15000ms rc 0 ok Node d42 aa migration threshold 1000000 3 probe...

Page 146: ...n for a split brain condition where the cluster is partitioned I want to list my currently known resources Use the command crm_resource L to learn about your current resources I configured a resource...

Page 147: ...ng line Jan 12 09 58 55 clusternode2 lrmd 3487 info RA output o2cb 1 start stderr 2009 01 12_09 58 55 ERROR Could not load ocfs2_stackglue Jan 12 16 04 22 clusternode2 modprobe FATAL Module ocfs2_stac...

Page 148: ......

Page 149: ...configu ration Its output includes the number of nodes uname uuid status the resources configured in your cluster and the current status of each The output of crm_mon can be displayed at the console o...

Page 150: ...of a CIB in the running cluster edit it validate it using crm_verify then put the new configuration into effect using cibadmin See crm_verify 8 page 190 for a detailed introduction to this tool s usa...

Page 151: ...become a fully active member of the cluster again See crm_standby 8 page 187 for a detailed introduction to this tool s usage and command syntax Cluster Management Tools 141...

Page 152: ...xml filename t t flag whatever h hostname cibadmin update U Vrwlsmfbp i xml object id o xml object type X xml string x xml filename t t flag whatever h hostname cibadmin modify M Vrwlsmfbp i xml objec...

Page 153: ...a complete lack of ambiguity and can only deal with valid XML subtrees tags and elements for both input and output NOTE cibadmin should always be used in preference to editing the cib xml file by han...

Page 154: ...name monitor The tag name and all attributes must match in order for the element to be deleted erase E Erase the contents of the entire CIB ismaster m Print a message indicating whether or not the lo...

Page 155: ...a complete tag or an XML fragment Advanced Options host hostname h hostname Send command to specified host Applies to query and sync commands only local l Let a command take effect locally rarely use...

Page 156: ...lue 10 10 10 102 NOTE This does not change the resource name to match the new IP address To do that delete then re add the resource with a new ID tag To stop disable the IP address resource added prev...

Page 157: ...status section on disk See Also crm_resource 8 page 166 crmadmin 8 page 148 lrmadmin 8 heartbeat 8 Caveats Avoid working on the automatically maintained copy of the CIB on the local disk Whenever anyt...

Page 158: ...e by other tools such as crm_attribute and crm_resource Its remaining functionality is mostly related to testing and the status of the crmd process WARNING Some crmadmin options are geared towards tes...

Page 159: ...achieved by sending the USR1 signal to the crmd process debug_dec node d node Incrementally decrease the CRM daemon s debug level on the specified node This can also be achieved by sending the USR2 si...

Page 160: ...e location of the DC is only of significance to the crmd internally and is rarely useful to administrators except when deciding on which node to examine the logs nodes N Query the uname of all member...

Page 161: ...ster configuration options that are used in the CIB Options help Print a help message verbose V Turn on debug information NOTE Increase the level of verbosity by providing additional instances quiet Q...

Page 162: ...the CIB the attribute that is queried belongs Possible values are nodes status or crm_config Examples Query the value of the location attribute in the nodes section for the host myhost in the CIB crm_...

Page 163: ...t nodes U myhost s set n location v office Change the value of the location attribute in the nodes section for the myhost host crm_attribute t nodes U myhost n location v backoffice Files var lib hea...

Page 164: ...cluster configuration or saving changes so they can be applied at a later time using cibadmin Options help Print a help message original filename o filename Specify the original file against which to...

Page 165: ...by running cibadmin on the two cluster setups to compare cibadmin Q cib1 xml cibadmin Q cib2 xml 2 Determine whether to diff the entire files against each other or compare just a subset of the configu...

Page 166: ...See Also cibadmin 8 page 142 156 High Availability Guide...

Page 167: ...d at which the resource should failover to another node The failcount attribute is added to the resource and increased on resource monitoring failure The value of failcount multiplied by the value of...

Page 168: ...Identifies the id attribute attr value string v string Specify the value to use This option is ignored when used with G node node_uname U node_uname Specify the uname of the node to change resource id...

Page 169: ...s status section on disk Editing this file directly is strongly discouraged See Also crm_attribute 8 page 151 cibadmin 8 page 142 and the Linux High Availability FAQ Web site http www linux ha org v2...

Page 170: ...ource agents RAs use crm_master to promote a particular instance to master mode or to remove this preference from it By assigning a lifetime determine whether this setting should survive a reboot of t...

Page 171: ...te attr id string i string For advanced users only Identifies the id attribute attr value string v string Value to set This is ignored when used with G lifetime string l string Specify how long the pr...

Page 172: ...e uuid status the resources configured in your cluster and the current status of each The output of crm_mon can be displayed at the console or printed into an HTML file When provided with a cluster co...

Page 173: ...h output suitable for CGI daemonize d Run in the background as a daemon pid file filename p filename Specify the daemon s pid file Examples Display your cluster s status and get an updated listing eve...

Page 174: ...option allows you to constantly create HTML output that can be easily processed by other monitoring appli cations crm_mon d p filename h filename Display the cluster configuration laid out in an exis...

Page 175: ...crm_node V p e q Description Lists the members of a cluster Options V be verbose partition p print the members of this partition epoch e print the epoch this node joined the partition quorum q print...

Page 176: ...can modify the definition of configured resources start and stop resources and delete and migrate resources between nodes help Print the help message verbose V Turn on debug information NOTE Increase...

Page 177: ...FINITY NOTE This prevents the resource from running on this node until the constraint is removed with U Requires r Optional N f un migrate U Remove all constraints created by M Requires r delete D Del...

Page 178: ...eta list operations string O string List the active resource operations Optionally filtered by resource node or both Optional N r list all operations string o string List all resource operations Optio...

Page 179: ...currently 100 000 NOTE This prevents the resource from running on this node until the constraint is removed with U s string Advanced Use Only Specify the ID of the instance_attributes object to chang...

Page 180: ...e_stickiness and default_resource_stickiness may mean that it does not move back In such cases you should use M to move it back before running this command Delete a resource from the CRM crm_resource...

Page 181: ...for resources started outside the CRM crm_resource P H c001n02 Files var lib heartbeat crm cib xml the CIB minus status section on disk Editing this file directly is strongly discouraged See Also ciba...

Page 182: ...allowing changes to be previewed and tested for side effects Options verbose V turn on debug info additional instance increase verbosity which w indicate the active shadow copy display p display the...

Page 183: ...h a shadow configuration you need to create one first crm_shadow create empty YOUR_NAME It gives you an internal shell like the one from the crm tool Use help to get an overview of all internal comman...

Page 184: ...e non option argument syntax is equivalent to that found in inputrc but must be passed as a single argument bind C x C r re read init file break N break Exit from within a for while or until loop If N...

Page 185: ...on A action G globpat W wordlist P prefix compgen S suffix X filterpat F function C command WORD Display the possible completions depending on the options Intended to be used from within a shell funct...

Page 186: ...ugh the list with the popd command disown h ar JOBSPEC disown By default removes each JOBSPEC argument from the table of active jobs If the h option is given the job is not removed from the table but...

Page 187: ...s from the shared object FILENAME The d option will delete a builtin previously loaded with f If no non option names are given or the p option is supplied a list of builtins is printed The a option me...

Page 188: ...a list of all names that are exported in this shell is printed An argument of n says to remove the export property from subsequent NAMEs An argument of disables further option processing false false...

Page 189: ...ATHNAME is used as the full pathname of NAME and no path search is performed The r option causes the shell to forget all remembered locations The d option causes the shell to forget the remembered loc...

Page 190: ...s is printed If x is given COMMAND is run after all job specifications that appear in ARGS have been replaced with the process ID of that job s process group leader kill s sigspec n signum sigspec pid...

Page 191: ...aracters which are simply copied to standard output character escape sequences which are converted and copied to the standard output and format specifications each of which causes printing of the next...

Page 192: ...f these NAMEs may not be changed by subsequent assignment If the f option is given then functions corresponding to the NAMEs are so marked If no arguments are given or if p is given a list of all read...

Page 193: ...ME the u flag unsets each OPTNAME The q flag suppresses output the exit status indicates whether each OPTNAME is set or unset The o option restricts the OPTNAMEs to those defined for use with set o Wi...

Page 194: ...he command ARG is to be read and executed when the shell receives signal s SIGNAL_SPEC If ARG is absent and a single SIGNAL_SPEC is supplied or each specified signal is reset to its original value If...

Page 195: ...number is output If p is supplied and MODE is omitted the output is in a form that may be used as input If MODE begins with a digit it is interpreted as an octal number otherwise it is a symbolic mod...

Page 196: ...and report its termination status If N is not given all currently active child processes are waited for and the return code is zero N may be a process ID or a job specification if a job spec is given...

Page 197: ...mode can be useful for performing maintenance tasks such as kernel updates Remove the standby attribute from the node when it needs to become a fully active member of the cluster again By assigning a...

Page 198: ...or advanced users only Identifies the id attribute node node_uname u node_uname Specify the uname of the node to change lifetime string l string Determine how long this preference lasts Possible value...

Page 199: ...y v true l forever U node1 Have a node go to standby until the next reboot of this node crm_standby v true l reboot U node1 Files var lib heartbeat crm cib xml the CIB minus status section on disk Edi...

Page 200: ...oblems errors and warnings Errors must be fixed before Heartbeat can work properly However it is left up to the admin istrator to decide if the warnings should also be fixed crm_verify assists in crea...

Page 201: ...Examples Check the consistency of the configuration in the running cluster and produce verbose output crm_verify VL Check the consistency of the configuration in a given file and produce verbose outp...

Page 202: ......

Page 203: ...the standard that the resource agent conforms to Resource agents abstract the services they provide and present an accurate status to the cluster which allows the cluster to be non committal about th...

Page 204: ...scripts However the configuration is always done with en vironmental variables which allow them to accept and process parameters easily The OCF specification as it relates to resource agents can be fo...

Page 205: ...Stop the resource and prevent it from being started on any cluster node A non transient error occurred that will be common to all cluster nodes This means a bad configu ration was specified fatal Ass...

Page 206: ...e resource s configuration is invalid for example required parameters are missing OCF_ERR_CON FIGURED 6 N A The resource is not running The cluster will not attempt to stop a resource that returns thi...

Page 207: ...Stopped Started target role Is the cluster allowed to start and stop the resource Allowed values true false is managed How much does the resource prefer to stay where it is Defaults to the value of de...

Page 208: ...long to wait before declaring the action has failed timeout What conditions need to be satisfied before this action occurs Allowed values nothing quorum fencing The default requires depends on whether...

Page 209: ...ich determine how they behave and which instance of a service they control If your resource agent supports parameters you can add them with the crm_resource command as described in In the crm command...

Page 210: ......

Page 211: ...se agents The data that is available in this appendix is directly taken from the meta data invocation of the respective RA Find all these agents in usr lib ocf resource d heartbeat When configuring an...

Page 212: ...t anything Supported Parameters OCF_RESKEY_binfile Full path name of the binary to be executed The full name of the binary to be executed This is expected to keep running with the same pid and not jus...

Page 213: ...OCF_RESKEY_user User to run the command as User to run the command as OCF_RESKEY_monitor_hook Command to run in monitor operation Command to run in monitor operation HA OCF Agents 203...

Page 214: ...does not succeed within the start operation timeout the apache resource will end with an error status The monitor operation by default loads the server status page which depends on the mod_status mod...

Page 215: ...iguration file If you set this make sure that it succeeds only from the localhost 127 0 0 1 Otherwise it may happen that the cluster complains about the resource being active on multiple nodes OCF_RES...

Page 216: ...dibleAlarm start stop restart status monitor meta data validate all Description Resource script for AudibleAlarm It sets an audible alarm running by beeping at a set interval Supported Parameters OCF_...

Page 217: ...Agent It outputs current cluster status to the html Supported Parameters OCF_RESKEY_user The user we want to run crm_mon as The user we want to run crm_mon as OCF_RESKEY_update Update interval How fre...

Page 218: ...stop status monitor validate all meta data methods Description Resource script for db2 It manages a DB2 Universal Database instance as an HA re source Supported Parameters OCF_RESKEY_instance instance...

Page 219: ...script is a test resource for introducing delay Supported Parameters OCF_RESKEY_startdelay Start delay How long in seconds to delay on start operation OCF_RESKEY_stopdelay Stop delay How long in seco...

Page 220: ...or monitor meta data validate all Description Master Slave OCF Resource Agent for DRBD Supported Parameters OCF_RESKEY_drbd_resource drbd resource name The name of the drbd resource from the drbd conf...

Page 221: ...ot modify the default OCF_RESKEY_master_max Number of primaries Maximum number of active primaries Do not modify the default OCF_RESKEY_master_node_max Number of primaries per node Maximum number of p...

Page 222: ...m meta data validate all Description This is a Dummy Resource Agent It has no purpuse other than to keep track of whether its running or not Its sole purpose is for testing and to serve as a template...

Page 223: ...ly recommended to put eDir configuration files as per the eDir_config_file parameter on local storage of each node This is necessary for this RA to be able to handle situations where the shared storag...

Page 224: ...ning for the eDirectory instance OCF_RESKEY_eDir_jvm_initial_heap DHOST_INITIAL_HEAP value Value for the DHOST_INITIAL_HEAP java environment variable If unspecified java defaults will be used OCF_RESK...

Page 225: ...ge medium Supported Parameters OCF_RESKEY_device block device The name of block device for the filesystem or U L options for mount or NFS mount specification OCF_RESKEY_directory mount point The mount...

Page 226: ...art stop status monitor validate all meta data Description Resource script for ICP It Manages an ICP Vortex clustered host drive as an HA re source Supported Parameters OCF_RESKEY_driveid ICP cluster...

Page 227: ...on of IDS Or in other words the path without trailing where IDS was installed to If this parameter is unspecified the script will try to get the value from the shell environment OCF_RESKEY_informixser...

Page 228: ...this parameter is unspecified the script will use the sysmaster database as a de fault OCF_RESKEY_sqltestquery SQL test query to use for monitoring defaults to SELECT COUNT FROM systables SQL test que...

Page 229: ...tor meta data validate all Description This Linux specific resource manages IP alias IP addresses It can add or remove an IP alias In addition it can implement Cluster Alias IP functionality if invoke...

Page 230: ...loopback device to allow the local node to continue to service requests but no longer advertise it on the network OCF_RESKEY_mac Cluster IP MAC address Set the interface MAC address explicitly Current...

Page 231: ...OCF_RESKEY_arp_mac ARP MAC MAC address to send the ARP packets to For advanced users only HA OCF Agents 221...

Page 232: ...RP_NETMASK string IPaddr start stop monitor validate all meta data Description This script manages IP alias IP addresses It can add an IP alias or remove one Supported Parameters OCF_RESKEY_ip IPv4 ad...

Page 233: ...address is stopped simply move it to the loopback device to allow the local node to continue to service requests but no longer advertise it on the network OCF_RESKEY_local_stop_script Script called wh...

Page 234: ...SKEY_ipaddress string IPsrcaddr start stop stop monitor vali date all meta data Description Resource script for IPsrcaddr It manages the preferred source address modification Supported Parameters OCF_...

Page 235: ...addr string IPv6addr start stop status monitor validate all meta data Description This script manages IPv6 alias IPv6 addresses It can add or remove an IP6 alias Supported Parameters OCF_RESKEY_ipv6ad...

Page 236: ...SCSI portal address in the form ip_address hostname port OCF_RESKEY_target target The iSCSI target OCF_RESKEY_discovery_type discovery_type Discovery type Currently with open iscsi only the sendtarget...

Page 237: ...le OCF RA wrapper for ldirectord and uses the ldirectord interface to create the OCF compliant interface You win monitoring of ldirectord Be warned Asking ldirectord status is an expensive action Supp...

Page 238: ...validate all Description This is a resource agent for LinuxSCSI It manages the availability of a SCSI device from the point of view of the linux kernel It make Linux believe the device is absent and i...

Page 239: ...ring LVM start stop status monitor methods meta data validate all Description Resource script for LVM It manages a Linux Volume Manager volume LVM as an HA resource Supported Parameters OCF_RESKEY_vol...

Page 240: ...tart stop status monitor meta data validate all Description This is a resource agent for MailTo It sends email to a sysadmin whenever a takeover occurs Supported Parameters OCF_RESKEY_email Email addr...

Page 241: ...p status monitor validate all meta data Description Manages starting stopping and monitoring of RAID devices which are preconfigured in etc conf d HB ManageRAID Supported Parameters OCF_RESKEY_raidnam...

Page 242: ...alidate all meta data Description This OCF complaint resource agent manages OpenVZ VEs and thus requires a proper OpenVZ installation including a recent vzctl util Supported Parameters OCF_RESKEY_veid...

Page 243: ...in teger OCF_RESKEY_additional_parameters integer mysql start stop status monitor validate all meta data Description Resource script for MySQL It manages a MySQL Database instance as an HA resource Su...

Page 244: ...se table notation OCF_RESKEY_test_user MySQL test user MySQL test user OCF_RESKEY_test_passwd MySQL test user password MySQL test user password OCF_RESKEY_enable_creation Create the database if it doe...

Page 245: ...hipped with the Linux distro The nfsserver resource agent offloads the start stop monitor work to the init script because the procedure to start stop monitor nfsserver varies on different Linux distro...

Page 246: ...OCF_RESKEY_nfs_ip IP address The floating IP address used to access the the nfs service 236 High Availability Guide...

Page 247: ...ome home The Oracle home directory aka ORACLE_HOME If not specified then the SID along with its home should be listed in etc oratab OCF_RESKEY_user user The Oracle owner aka ORACLE_OWNER If not specif...

Page 248: ...to use but in that case we cannot guarantee that the instance will start In case IPC objects were already left around because for instance of someone mercilessly killing Oracle processes there is no w...

Page 249: ...ges an Oracle Listener instance as an HA resource Supported Parameters OCF_RESKEY_sid sid The Oracle SID aka ORACLE_SID Necessary for the monitor op i e to do tn sping SID OCF_RESKEY_home home The Ora...

Page 250: ...escalate string pgsql start stop status monitor meta data validate all methods Description Resource script for PostgreSQL It manages a PostgreSQL as an HA resource Supported Parameters OCF_RESKEY_pgct...

Page 251: ...ning OCF_RESKEY_pgport pgport Port where PosrgeSQL is listening OCF_RESKEY_pgdb pgdb Database that will be used for monitoring OCF_RESKEY_logfile logfile Path to PostgreSQL server log output file OCF_...

Page 252: ...records in the CIB the current number of ping nodes a node can connect to Supported Parameters OCF_RESKEY_pidfile PID file PID file OCF_RESKEY_user The user we want to run pingd as The user we want to...

Page 253: ...eeds to be specified OCF_RESKEY_multiplier Value multiplier The number by which to multiply the number of connected ping nodes OCF_RESKEY_host_list Host list The list of ping nodes to count Defaults t...

Page 254: ...ta validate all Description Resource script for portblock It is used to temporarily block ports using iptables Supported Parameters OCF_RESKEY_protocol protocol The used protocol to be blocked unblock...

Page 255: ...ipt name with full path The full path to the Pure FTPd startup script For example sbin pure config pl OCF_RESKEY_conffile Configuration file name with full path The Pure FTPd configuration file name w...

Page 256: ...SKEY_raidconf RAID config file The RAID configuration file e g etc raidtab or etc mdadm conf OCF_RESKEY_raiddev block device The block device to use OCF_RESKEY_homehost Homehost for mdadm The value fo...

Page 257: ...ote network Then here is how you would implement this scheme making use of the Route resource agent Configure an ipsec LSB resource Configure a cloned Route OCF resource Create an order constraint to...

Page 258: ...OCF_RESKEY_gateway Gateway IP address The gateway IP address to use for this route OCF_RESKEY_source Source IP address The source IP address to be configured for the route 248 High Availability Guide...

Page 259: ...name with full path The rsync daemon configuration file name with full path For example etc rsyncd conf OCF_RESKEY_bwlimit limit I O bandwidth KBytes per second This option allows you to specify a max...

Page 260: ...IT string OCF_RESKEY_POST_START_USEREXIT string OCF_RESKEY_PRE_STOP_USEREXIT string OCF_RESKEY_POST_STOP_USEREXIT string SAPDatabase start stop status monitor validate all meta data methods Descriptio...

Page 261: ...forced abort of the RDBMS and or executing recovery commands OCF_RESKEY_DIR_BOOTSTRAP path to j2ee bootstrap directory The fully qualified path where to find the J2EE instance bootstrap directory e g...

Page 262: ...o find a script or program which should be executed before this resource gets stopped OCF_RESKEY_POST_STOP_USEREXIT path to a post start script The fully qualified path where to find a script or progr...

Page 263: ...e start recover stop status monitor validate all meta data methods Description Resource script for SAP It manages a SAP Instance as an HA resource Supported Parameters OCF_RESKEY_InstanceName instance...

Page 264: ...uting cleanipc OCF_RESKEY_PRE_START_USEREXIT path to a pre start script The fully qualified path where to find a script or program which should be executed before this resource gets started OCF_RESKEY...

Page 265: ...indicates the own of the specified SCSI device This resource agent depends on the scsi_reserve from scsires package which is Linux specific Supported Parameters OCF_RESKEY_scsi_reserve scsi_reserve co...

Page 266: ...nic string SendArp start stop monitor meta data validate all Description This script send out gratuitous Arp for an IP address Supported Parameters OCF_RESKEY_ip IP address The IP address for sending...

Page 267: ...start stop status monitor validate all meta data methods Description Resource script for ServeRAID It enables disables shared ServeRAID merge groups Supported Parameters OCF_RESKEY_serveraid serveraid...

Page 268: ...SKEY_device block device Block device path that stores exclusive control data OCF_RESKEY_index index Location in block device where exclusive control data is stored 1 or more is spec ified Default is...

Page 269: ...earch Daemon Supported Parameters OCF_RESKEY_config Configuration file searchd configuration file OCF_RESKEY_searchd searchd binary searchd binary OCF_RESKEY_search search binary Search binary for fun...

Page 270: ...squid_exe Executable file This is a required parameter This parameter specifies squid s executable file OCF_RESKEY_squid_conf Configuration file This is a required parameter This parameter specifies a...

Page 271: ...x or v If x is included both of STDOUT and STDERR redirect to the logfile specified by debug_log and then the builtin shell option x is turned on It is similar with v OCF_RESKEY_debug_log A destinatio...

Page 272: ...s OCF_RESKEY_state string Stateful start stop monitor meta data validate all Description This is an example resource agent that implements two states Supported Parameters OCF_RESKEY_state State file L...

Page 273: ...ap 1999 cpu_info Intel R Celeron R CPU 2 40GHz cpu_speed 4771 02 cpu_cores 1 cpu_load 0 00 ram_total 513 ram_free 117 root_free 2 4 Sample Darwin output arch i386 os Darwin 8 6 2 cpu_info Intel Core D...

Page 274: ...tring OCF_RESKEY_catalina_opts string OCF_RESKEY_catalina_rotate_log string OCF_RESKEY_catalina_rotatetime integer tomcat start stop status monitor meta data validate all Description Resource script f...

Page 275: ...f the Java Home directory of the Java OCF_RESKEY_catalina_home Home directory of Tomcat Home directory of Tomcat OCF_RESKEY_catalina_pid A PID file name of Tomcat A PID file name of Tomcat OCF_RESKEY_...

Page 276: ...t manages an IP alias in a different subnet with quagga ripd It can add or remove an IP alias Supported Parameters OCF_RESKEY_ip The IP address in different subnet The IPv4 address in different subnet...

Page 277: ...bvirt configuration file for this virtual domain OCF_RESKEY_hypervisor Hypervisor URI Hypervisor URI to connect to See the libvirt documentation for details on supported URI formats The default is sys...

Page 278: ...rtual domain add this parameter with a list of scripts to monitor Note when monitor scripts are used the start and mi grate_from operations will complete only when all monitor scripts have completed s...

Page 279: ...le string WAS6 start stop status monitor validate all meta data methods Description Resource script for WAS6 It manages a Websphere Application Server WAS6 as an HA resource Supported Parameters OCF_R...

Page 280: ...op status monitor validate all meta data methods Description Resource script for WAS It manages a Websphere Application Server WAS as an HA resource Supported Parameters OCF_RESKEY_config configration...

Page 281: ...tart stop status monitor validate all meta data Description Resource script for WinPopup It sends WinPopups message to a sysadmin s workstation whenever a takeover occurs Supported Parameters OCF_RESK...

Page 282: ...mfile attribute If you use a simple assignment statement then you should be fine If there s some python acrobacy involved such as dynamically assigning names depending on other variables system will t...

Page 283: ...amic memory management This parameter enables dynamic adjustment of memory for start and stop actions used for Dom0 and the DomUs The default is to not adjust memory dynamically OCF_RESKEY_reserved_Do...

Page 284: ...e that the xinetd daemon itself must be running the system will not start it or stop it Important in case the services managed by the cluster are the only ones enabled you should specify the stayalive...

Page 285: ...Part V Appendix...

Page 286: ......

Page 287: ...you can do these things To protect your rights we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights These restrictions translate to certain re...

Page 288: ...do not apply to those sections when you distribute them as separate works But when you distribute the same sections as part of a whole which is a work based on the Program the distribution of the who...

Page 289: ...in or among countries not thus excluded In such case this License incorporates the limitation as if written in the body of this License 9 The Free Software Foundation may publish revised and or new ve...

Page 290: ...n a copyright disclaimer for the program if necessary Here is a sample alter the names Yoyodyne Inc hereby disclaims all copyright interest in the program Gnomovision which makes passes at compilers w...

Page 291: ...if used for any substantial amount of text A copy that is not Transparent is called Opaque Examples of suitable formats for Transparent copies include plain ASCII without markup Texinfo input format...

Page 292: ...Sections and required Cover Texts given in the Document s license notice H Include an unaltered copy of this License I Preserve the section Entitled History Preserve its Title and add to it an item s...

Page 293: ...pies of the Document then if the Document is less than one half of the entire aggregate the Document s Cover Texts may be placed on covers that bracket the Document within the aggregate or the electro...

Page 294: ...and Back Cover Texts replace the with Texts line with this with the Invariant Sections being LIST THEIR TITLES with the Front Cover Texts being LIST and with the Back Cover Texts being LIST If you hav...

Page 295: ...partition cannot be con firmed a split brain scenario develops see also split brain page 288 consensus cluster membership CCM The CCM determines which nodes make up the cluster and shares this inform...

Page 296: ...hermore fencing is distinguished between node and resource fencing Heartbeat resource agent Heartbeat resource agents were widely used with Heartbeat version 1 Their use is deprecated but still suppor...

Page 297: ...source agents are similar to LSB resource agents init scripts Any OCF resource agent must support start stop and status sometimes called monitor options Additionally they support a metadata option tha...

Page 298: ...ailure of the entire cluster split brain A scenario in which the cluster nodes are divided into two or more groups that do not know of each other either through a software or hardware failure STONITH...

Reviews: