Fujitsu ETERNUS Multipath Driver V2 Скачать руководство пользователя страница 2

Страница: 2 / 140

This page is intentionally left blank.

Содержание ETERNUS Multipath Driver V2

Страница 1: ...FUJITSU Storage ETERNUS Multipath Driver V2 User s Guide P2U3 0031 15ENZ0 for Linux ...

Страница 2: ...This page is intentionally left blank ...

Страница 3: ...Fifteenth Edition October 2014 Intended Audience This guide is intended for system administrators who are connecting the storage system to a Linux server Organization and Contents This guide is composed of the following 11 chapters and an appendix Chapter 1 What is Multipath Driver This chapter explains the features functions and connection patterns of the Multipath Driver Chapter 2 Multipath Driv...

Страница 4: ...ion for RHEL6 RHEL7 or SLES11 This chapter explains how to change a multipath configuration for a FC FCoE connection with RHEL6 RHEL7 or SLES11 Chapter 8 Changing the Multipath Configuration SAS Connection for RHEL6 RHEL7 or SLES11 This chapter explains how to change a multipath configuration for a SAS connection with RHEL6 RHEL7 or SLES11 Chapter 9 Changing the Multipath Configuration iSCSI Conne...

Страница 5: ...anagement tool that is provided with the storage system as standard ETERNUSmgr that is installed in some storage systems Red Hat Enterprise Linux series products are as described below SUSE Linux Enterprise Server series products are as described below Related Manuals Refer to the following related manuals for more information The storage system manual The manual provided with ETERNUS Web GUI GRmg...

Страница 6: ... control air traffic control mass transport control medical life support system and missile launch control in weapon systems Do not use this Product for High Safety Required Use without securing the sufficient safety level required If you wish to use this Product for High Safety Required Use please consult with our sales representative before such use Linux is a registered trademark of Linus Torva...

Страница 7: ...r levels of some messages that were NOTICE or lower error levels have been changed to WARNING or higher In addition the error levels of some messages that were WARNING or higher have been changed to NOTICE or lower Modification target Multipath Driver Modification details Modification effect This change affects when error levels are used for message monitoring Version Modification details V2 0L02 ...

Страница 8: ... id name V2 0L11 modification Modification outline Removed the function that inquires whether the udev function is used or not when using the mpdconfig s command Modification target mpdconfig command Modification details Modification effect Does not affect any other operation Version Modification details V2 0L03 or earlier Creates a by path name V2 0L10 Does not create a by path name Version Modif...

Страница 9: ...del 300 and 500 is automated Modification target Multipath Driver Modification details Modification effect When connecting to the ETERNUS8000 the ETERNUS6000 and the ETERNUS4000 model 300 and 500 Max Throttle does not have to be set with the mpdconfig m command Version Modification details V2 0L11 or earlier The default Max Throttle value of the ETERNUS2000 is 256 V2 0L12 The default Max Throttle ...

Страница 10: ...r Modification details Modification effect When performing message monitoring the newly added message may not be monitored as intended Note that this change does not affect the message monitoring function with Fujitsu middleware Version Modification details V2 0L12 or earlier Up to 20 devices can be specified for the recognition order V2 0L13 Up to 32 devices can be specified for the recognition o...

Страница 11: ...ion details Version Modification details V2 0L12 or earlier When an I O error occurs the following message is output in var log messages MPDH d NOTICE SCSI u u u u I O LUN degrade number of online paths left of this LUN d MPDH d WARNING I O LUN degraded no more online path of this LUN SCSI u u u u ERROR 0x05 abort SCSI u u u u ERROR 0x06 parity error SCSI u u u u ERROR 0x07 internal error SCSI u u...

Страница 12: ...OTICE SCSI u u u u I O LUN degrade MPDH d NOTICE SCSI u u u u I O LUN failed number of online paths left for this LUN d or MPDH d NOTICE SCSI u u u u I O rejection number of online paths left for this LUN d MPDH d WARNING I O LUN failed no more online path for this LUN or MPDH d WARNING I O rejection no more online path for this LUN SCSI u u u u ERROR 0x05 HBA abort SCSI u u u u ERROR 0x06 HBA par...

Страница 13: ...nded Note that this change does not affect the Fujitsu middleware that analyzes the iompadm info command result Version Modification details V2 0L12 or earlier The path details in the iompadm info command output when executing the iompadm info command after an I O error occurs are shown below V2 0L13 The path details in the iompadm info command output when executing the iompadm info command after ...

Страница 14: ... is not used any more are automatically deleted Modification target Multipath Driver Modification details Modification effect Initial RAM disk settings that are not needed are deleted The boot loader setting to boot up with this Initial RAM disk is also deleted Version Modification details V2 0L12 or earlier The following message is displayed when the I O by diagnosis function fails V2 0L13 The fo...

Страница 15: ... following conditions no messages are output even when an I O error occurs When the storage system responds the sense information below Sense key 0x04 ASC 0xf2 ASCQ 0x00 0xff Sense key 0x06 ASC 0xfb ASCQ 0x81 0x8f When the HBA responds DID_TRANSPORT_FAILFAST V2 0L20 In the following conditions messages are output when an I O error occurs When the storage system responds the sense information below...

Страница 16: ...edundancy is maintained regardless of whether the path is recognized or not V2 0L23 The following syslog messages are sequentially output during system startup Note that the following syslog message is output when path redundancy is maintained and all the paths are recognized MPDH d NOTICE s Device configuration succeeded MPDH d WARNING s Device configuration is without redundancy MPDH d NOTICE s ...

Страница 17: ...dded under the boot directory Version Modification details V2 0L22 or earlier No messages are output on the console if the Multipath Driver is not loaded when starting up the system V2 0L23 The following message is output on the console if the Multipath Driver is not loaded when starting up the system WARNING ETERNUS Multipath Driver is not loaded Version Modification details V2 0L23 or earlier If...

Страница 18: ... system startup Apart from the adding of the services listed above there is no other affect on servers Version Modification details V2 0L23 or earlier There are problems with the load order of the Multipath Driver modules V2 0L24 There are problems with the load order of the Multipath Driver modules The Multipath Driver modules are not loaded and the Multipath Driver modules that correspond to the...

Страница 19: ...odification target Multipath Driver Modification details Modification effect When connecting to the ETERNUS DX60 DX60 S2 DX80 DX80 S2 DX90 DX90 S2 Max Throttle does not have to be set with the mpdconfig m command Version Modification details V2 0L24 or earlier The offline status is not checked for a device when the iompadm info command is executed V2 0L25 The offline status is checked for a device...

Страница 20: ... ES v 4 for x86 Update 2 Red Hat Enterprise Linux AS v 4 for EM64T Update 2 Red Hat Enterprise Linux ES v 4 for EM64T Update 2 Red Hat Enterprise Linux AS v 4 for Itanium Update 2 Red Hat Enterprise Linux AS v 4 for x86 Update 3 Red Hat Enterprise Linux ES v 4 for x86 Update 3 Red Hat Enterprise Linux AS v 4 for EM64T Update 3 Red Hat Enterprise Linux ES v 4 for EM64T Update 3 SUSE Linux Enterpris...

Страница 21: ...erprise Linux AS 4 6 for EM64T Red Hat Enterprise Linux ES 4 6 for EM64T Red Hat Enterprise Linux AS 4 6 for Itanium Red Hat Enterprise Linux 5 1 for x86 Red Hat Enterprise Linux 5 1 for Intel64 Red Hat Enterprise Linux 5 1 for Intel Itanium SUSE Linux Enterprise Server 9 for x86 Service Pack 4 SUSE Linux Enterprise Server 9 for EM64T Service Pack 4 SUSE Linux Enterprise Server 9 for Itanium Proce...

Страница 22: ...ux 5 3 for Intel64 Red Hat Enterprise Linux 5 3 for Intel Itanium SUSE Linux Enterprise Server 10 Service Pack 2 for x86 SUSE Linux Enterprise Server 10 Service Pack 2 for EM64T SUSE Linux Enterprise Server 10 Service Pack 2 for Itanium Processor Family SAS connection of the following OSs is supported Red Hat Enterprise Linux AS 4 7 or later Red Hat Enterprise Linux ES 4 7 or later Red Hat Enterpr...

Страница 23: ... Red Hat Enterprise Linux 6 for x86 Red Hat Enterprise Linux 6 for Intel64 SUSE Linux Enterprise Server 11 Service Pack 1 for x86 SUSE Linux Enterprise Server 11 Service Pack 1 for EM64T SAS connection of the following OS is supported SUSE Linux Enterprise Server 10 Service Pack 3 or later V2 0L22 June 2011 The following OSs are supported Red Hat Enterprise Linux 6 1 for x86 Red Hat Enterprise Lin...

Страница 24: ...nterprise Linux 6 SUSE Linux Enterprise Server 10 Service Pack 1 or later SUSE Linux Enterprise Server 11 Service Pack 1 or later For details refer to ETERNUS Web GUI User s Guide ETERNUS DX80 S2 DX90 S2 ETERNUS DX410 S2 DX440 S2 ETERNUS DX8100 S2 DX8700 S2 tenth edition or later V2 0L26 October 2014 The following OSs are supported Red Hat Enterprise Linux 7 for Intel64 The Storage Cluster functio...

Страница 25: ...ge System Recognition Order Setting 35 2 2 Multipath Driver Related Log 37 Chapter 3 Multipath Driver Functions and Operation 38 3 1 Path Management 39 3 1 1 Path Status 39 3 1 2 Checks and Path Status Transitions 42 3 2 Load Balancing Failover 44 3 2 1 Load Balancing 44 3 2 2 Failover 46 3 3 Identifying Disks and Paths 49 3 3 1 Device Identification 49 3 3 2 Path Identification 49 3 4 Multipath S...

Страница 26: ...e Reduction 62 4 4 1 Reduction without Server Rebooting 62 4 4 2 Reduction with Server Rebooting 63 4 5 Path Addition and Reduction HBA Addition and Reduction 64 Chapter 5 Changing the Multipath Configuration SAS Connection for RHEL AS4 RHEL ES4 RHEL5 or SLES10 65 5 1 Storage System Addition 66 5 1 1 Addition without Server Rebooting 66 5 1 2 Addition with Server Rebooting 66 5 2 Storage System Re...

Страница 27: ...inux Enterprise Server 9 78 6 6 Path Reduction Cable Reduction 79 6 6 1 Reduction without Server Rebooting 79 6 6 2 Reduction with Server Rebooting 80 6 7 Path Addition and Reduction NIC Addition and Reduction 81 Chapter 7 Changing the Multipath Configuration FC FCoE Connection for RHEL6 RHEL7 or SLES11 82 7 1 Storage System Addition 83 7 1 1 Addition without Server Rebooting 83 7 1 2 Addition wit...

Страница 28: ...LU Addition 96 8 4 LU Reduction 96 8 5 Path Addition Cable Addition 96 8 6 Path Reduction Cable Reduction 96 8 6 1 Reduction without Server Rebooting 96 8 6 2 Reduction with Server Rebooting 97 8 7 Path Addition and Reduction HBA Addition and Reduction 97 Chapter 9 Changing the Multipath Configuration iSCSI Connection for RHEL6 RHEL7 or SLES11 98 9 1 Storage System Addition 99 9 2 Storage System R...

Страница 29: ...e Multipath Driver 105 10 1 Configuring a Storage System 105 10 2 Considerations for Linux Servers 108 10 3 Notes when Installing the Multipath Driver 111 10 4 Troubleshooting 113 10 5 Frequently Asked Questions 114 Chapter 11 Command Reference 115 11 1 iompadm Commands 115 11 2 mpdconfig Command 120 11 3 mpd_snap Command 124 Appendix A Syslog Messages 125 Index 137 ...

Страница 30: ...th CM assigned 44 Figure 3 3 4 path load balancing on a storage system with CM assigned 45 Figure 3 4 Multipath load balancing on a storage system with CM not assigned 45 Figure 3 5 2 path failover on a storage system with CM assigned 46 Figure 3 6 4 path failover active to active 47 Figure 3 7 4 path failover active to standby 47 Figure 3 8 Multipath failover a storage system with CM not assigned...

Страница 31: ...MITED P2U3 0031 15ENZ0 31 List of Tables Table 3 1 Path status values 39 Table 3 2 Online path sub states 40 Table 3 3 Device identification example 49 Table 3 4 Change unit specification 51 Table 11 1 iompadm command output elements 117 Table 11 2 iompadm command status detail information 117 ...

Страница 32: ...tipath Driver creates a redundant multipath configuration that switches the failed path over to a standby path thus allowing continuous operation not only for a disk failure but for a path failure as well The Multipath Driver connects a storage system and Linux server with functions to diagnose the path status and control commands Even when multiple HBAs are installed in the server and the configu...

Страница 33: ...r s HBAs or NICs and the storage system s CAs Multipath connections provide greater system control and improve reliability The following figure illustrates a multipath connection Multi path connection Figure 1 1 Sample multipath connection In a multipath connection all HBAs or NICs must be of the same type Fibre Channel Hubs cannot be used HA Port Devices for ETERNUS3000 model 50 and ETERNUS GR710...

Страница 34: ...ed Hat Enterprise Linux 5 Red Hat Enterprise Linux 6 Red Hat Enterprise Linux 7 SUSE Linux Enterprise Server 10 Service Pack 1 or later SUSE Linux Enterprise Server 11 Service Pack 1 or later Since OSs other than the ones listed above do not support the udev function upgrading to OSs that support the udev function is recommended Using the by id names has the following advantages The same device na...

Страница 35: ... to use by id names are provided in the FUJITSU Storage ETERNUS Multipath Driver V2 for Linux Installation Information found under Downloads at the following URL http www fujitsu com global products computing storage software eternus mpd 2 1 2 When Using FUJITSU Software PRIMECLUSTER Global Disk Services Storage System Recognition Order Setting When the following configurations are used the relati...

Страница 36: ...ction is used to set the recognition order and a storage system then becomes unrecognizable for some reason the storage systems that should be recognized after the failed storage system will not be recognized For example if the recognition order is set while three storage systems are connected and the second storage system fails the third storage system will not be recognized either even though it...

Страница 37: ... the on screen instructions Example End of procedure 2 2 Multipath Driver Related Log Multipath Driver creates the following log files in the var opt FJSVmpd directory opt FJSVmpd bin iompadm info opt FJSVmpd bin mpdconfig c File name Maximum size Description mpdstrc mpdstrc 1 mpdstrc 2 Approx 1MB I O error information log When the mpdstrc reaches its maximum size the file rotates in the order of ...

Страница 38: ... model For details refer to section 3 2 Load Balancing Failover page 44 Load balancing Load balancing load distribution is performed on all available paths improving data performance The Load Balancing function operates differently for each storage system model For details refer to section 3 2 Load Balancing Failover page 44 Auto path check Regular checks are performed on all paths even when they ...

Страница 39: ...formed between them active online is displayed by the iompadm info command standby Path is connected to the non assigned CM which is normally on standby If the assigned CM active path fails the standby path becomes the active path When the assigned CM path returns to active status this path returns to standby standby online is displayed by the iompadm info command warning I O errors have been dete...

Страница 40: ...will never be used for I O fail error information is displayed by the iompadm info command details of the error appears in error information fatal fail Result of the Auto Path Check is constantly switching back and forth between normal and abnormal so the appropriate checks cannot be performed Fatal fail paths are not checked After replacement of the failed parts the iompadm restart command must b...

Страница 41: ...f time or when the status frequently changes between active standby and warning it is determined that appropriate checks cannot be performed and the status changes to fatal fail in which the target is not checked 6 fatal fail active standby When a failed path is re enabled by the iompadm restart command the path status changes back to active standby When the status frequently changes between activ...

Страница 42: ...s detected during a normal I O process Check in warning Warning status paths are checked every minute When a failure is detected by the check 10 is added to the error counter When it is normal 10 is deducted If the error counter reaches 0 the status becomes active standby If it reaches 200 the status becomes fail Check in fail Fail status paths are checked every 10 minutes When a failure is detect...

Страница 43: ...changes to warning and the error counter is set by the error cause the minimum value of 50 in this example Retry after setting Path 2 active for the I O process 1 minutes warning 60 active Check error 10 added 10 minutes warning 150 active Check error 10 added 14 minutes warning 190 active Check error 10 added 15 minutes fail active Check error 10 added Error counter reaches 200 and the status cha...

Страница 44: ... according to the storage system type whether the storage system has assigned CM or not and number of paths For details on which storage systems have assigned CMs or not are provided in the FUJITSU Storage ETERNUS Multipath Driver V2 for Linux Installation Information found under Downloads at the following URL http www fujitsu com global products computing storage software eternus mpd 3 2 1 Load B...

Страница 45: ...s are connected to each of the four CMs for a total of eight paths Figure 3 3 4 path load balancing on a storage system with CM assigned Storage system with CM not assigned For a storage system that does not have an assigned CM there is no concept of standby path so all connected paths are active and load balancing is performed over all paths Figure 3 4 Multipath load balancing on a storage system...

Страница 46: ...inues executing the I O requests This can prevent the host application from shutting down unnecessarily Failover operates differently depending on the storage system type as described below Storage system with CM assigned 2 path connections When an active path becomes unavailable the standby path is activated to continue processing Figure 3 5 2 path failover on a storage system with CM assigned LU...

Страница 47: ...one active path available Figure 3 6 4 path failover active to active Example 2 If both active paths for a CM become unavailable a standby path is then switched to active state to continue processing When there are two or more standby paths load balancing is also performed Figure 3 7 4 path failover active to standby Multipath Driver LU0 I O request Storage system with CM assigned 4 path or more H...

Страница 48: ...with CM not assigned When an active path becomes unavailable it is disabled leaving the remaining paths to continue load balancing in a degraded state Figure 3 8 Multipath failover a storage system with CM not assigned LU0 Multipath Driver I O request LU1 HBA 3 HBA 0 HBA 1 CM 0 HBA 2 CM 1 LU0 1 Load balance Storage system with CM not assigned Linux server Active path Active path Active path Active...

Страница 49: ...ifier has been outlined Example The 3rd number is called the port number Details of the port numbers CAID of each storage system model and their connections are provided in the FUJITSU Storage ETERNUS Multipath Driver V2 for Linux Installation Information found under Downloads at the following URL http www fujitsu com global products computing storage software eternus mpd Disk ID segment Descripti...

Страница 50: ...system is displayed A disk_device_node represented by a special file dev sdb dev sdc etc indicates the connection point between a disk device and the storage system device The output of the command appears as follows path_status active standby stop or fail is displayed status_details online warning stop by command or an error message is displayed Output example iompadm info disk_device_node IOMP C...

Страница 51: ...ice node and port number can be checked from the output result of the iompadm info command This command changes an active standby or warning status path to stop status This command cannot change a fail or fatal fail status path to stop status Example Change unit specification Table 3 4 Change unit specification Details of the relationship between change units and storage systems are provided in th...

Страница 52: ...nge a warning status path to active standby change its status to stop status using the iompadm change command and then use the iompadm restart command Example Change unit specification Refer to 3 5 1 iompadm change Command page 51 3 6 Multipath Recovery Perform the following procedure to recover from a path failure in a multipath configuration by using the iompadm command Procedure 1 Use the iompa...

Страница 53: ...owing OSs refer to Chapter 7 Changing the Multipath Configuration FC FCoE Connection for RHEL6 RHEL7 or SLES11 page 82 Red Hat Enterprise Linux 6 Red Hat Enterprise Linux 7 SUSE Linux Enterprise Server 11 Service Pack 1 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may execu...

Страница 54: ...er For details on the settings refer to the storage system manual LU addition Add a LU in the storage system For details on adding LUs refer to the storage system manual 2 Incorporate the new hardware using the mpdconfig a command Example The iompadm rescan command can also be used to incorporate the new hardware Example 3 When a new storage system is added an enquiry is sent regarding the storage...

Страница 55: ...a Max Throttle value for the new storage system Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation method Example End of procedure opt FJSVmpd bin iompadm info When a LU is added it is initially added as a new SCSI disk device after the LUs that have already been incorporated However after the system is restarted the newly added LU is inc...

Страница 56: ...nding on the addition target Storage system addition Set up the storage system so that its LUs are accessible to the server and then connect it to the server For details on the settings refer to the storage system manual LU addition Add a LU in the storage system For details on adding LUs refer to the storage system manual 4 Restart the server Power on the server and boot it up 5 Use the iompadm i...

Страница 57: ... hardware Proceed as follows depending on the reduction target Storage system reduction Disconnect the target storage system from the server turn the storage system power off disconnect the cables etc LU reduction Perform LU reduction in the storage system so that the server cannot recognize the LU Refer to the storage system manual for LU reduction details 2 Delete the removed hardware from the M...

Страница 58: ...hardware has disappeared from view Example End of procedure 4 2 2 Reduction with Server Rebooting Use the following procedure to remove storage systems and LUs while the server is shut down Procedure 1 If the storage system recognition order has been set change the settings before removing the storage system When the storage system reduction reduces the number of storage systems connected to the s...

Страница 59: ... recognize the LU Refer to the storage system manual for LU reduction details 4 Restart the server Power on the server and boot it up 5 Use the iompadm info command to check that the removed hardware is displayed as fail Example 6 Delete the removed hardware from the Multipath Driver target list using the mpdconfig d command Example 7 If a storage system is removed and the recognition order functi...

Страница 60: ...stem so that its LUs are accessible via the cable and then connect the cable between the server and the storage system an HBA must already be installed in the server For details on the settings refer to the storage system manual 2 Use the iompadm info command to check that the new cable has been correctly recognized and incorporated Example 3 Set a Max Throttle value for the new path Execute the m...

Страница 61: ...le via the cable and then connect the cable between the server and the storage system an HBA must already be installed in the server For details on the settings refer to the storage system manual 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check that the new cable has been correctly recognized and incorporated Example 5 Set a Max Throttle value for the...

Страница 62: ...move the connected cable etc 2 Delete the removed cable from the Multipath Driver target list using the mpdconfig d command Example 3 Use the iompadm info command to check that the removed cable has disappeared from view Example 4 Reset the Max Throttle value if necessary When the number of paths connected to a given CA port in the storage system is reduced due to the path removal execute the mpdc...

Страница 63: ...r on the server and boot it up 4 Use the iompadm info command to check that the removed cable is displayed as fail Example 5 Delete the removed cable from the Multipath Driver target list using the mpdconfig d command Example 6 Use the iompadm info command to check the results Example 7 Reset the Max Throttle value if necessary When the number of paths connected to a given CA port in the storage s...

Страница 64: ... decrease paths by removing HBAs Adding and reducing HBAs without server rebooting PCI Hot Plug is only supported for PRIMEQUEST servers For details on this procedure refer to the PRIMEQUEST manual Adding and reducing HBAs with server rebooting is the same procedure as adding and reducing cables with server rebooting Add or reduce HBAs at the same time that cables are being added or reduced When a...

Страница 65: ...OSs refer to Chapter 8 Changing the Multipath Configuration SAS Connection for RHEL6 RHEL7 or SLES11 page 94 Red Hat Enterprise Linux 6 Red Hat Enterprise Linux 7 SUSE Linux Enterprise Server 11 Service Pack 1 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may execute the mpd...

Страница 66: ...iompadm info command to check the result When the new storage system is connected to the server it should be automatically recognized and incorporated in the list Example 3 Set its Max Throttle value Execute the mpdconfig m command to set a Max Throttle value for the new storage system Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation me...

Страница 67: ...cally removed and disappear from the list Example 3 If a storage system is removed and the recognition order function is being used execute the mpdconfig s command to reconfigure the recognition order Example However if the storage system reduction reduces the number of storage systems connected to the server to one or less execute the mpdconfig c command to clear the recognition order setting ins...

Страница 68: ...ded without shutting down the server Use the following procedure Procedure 1 Attach the hardware Set up the storage system so that its LUs are accessible via the cable and then connect the cable between the server and the storage system an HBA must already be installed in the server For details on the settings refer to the storage system manual 2 Use the iompadm info command to check the result Wh...

Страница 69: ... server Use the following procedure Procedure 1 Disconnect the surplus cables from the server 2 Use the iompadm info command to check the result When the cables are disconnected they should be automatically removed and disappear from the list Example 3 Reset the Max Throttle value if necessary When the number of paths connected to a given CA port in the storage system is reduced due to the path re...

Страница 70: ...TED P2U3 0031 15ENZ0 70 5 7 Path Addition and Reduction HBA Addition and Reduction This section describes how to increase paths by adding HBAs and decrease paths by removing HBAs Adding and reducing HBAs without server rebooting PCI Hot Plug is not supported Adding and reducing HBAs with server rebooting is the same procedure as adding and reducing cables with server rebooting Add or reduce HBAs a...

Страница 71: ...9 Changing the Multipath Configuration iSCSI Connection for RHEL6 RHEL7 or SLES11 page 98 Red Hat Enterprise Linux 6 Red Hat Enterprise Linux 7 SUSE Linux Enterprise Server 11 Service Pack 1 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may execute the mpdconfig command This...

Страница 72: ...r any other relevant manuals and Appendix 2 in the software information opt FJSVmpd manual english readme pdf for details about iSCSI connection 3 Use the iompadm info command to check the result The new storage system should be automatically recognized and incorporated in the list Example 4 Set its Max Throttle value Execute the mpdconfig m command to set a Max Throttle value for the new storage ...

Страница 73: ...SI initiator settings 3 Shut down the server 4 Attach the hardware Set up the storage system so that its LUs are accessible to the server and then connect it to the server For details on the settings refer to the storage system manual 5 Restart the server Power on the server and boot it up 6 Use the iompadm info command to check the result When the server is restarted the new storage system should...

Страница 74: ...from the Multipath Driver target list using the mpdconfig d command Example 3 If a storage system is removed and the recognition order function is being used execute the mpdconfig s command to reconfigure the recognition order Example However if the storage system reduction reduces the number of storage systems connected to the server to one or less execute the mpdconfig c command to clear the rec...

Страница 75: ... server to one or less execute the mpdconfig c command to clear the storage system recognition order Example When the storage system reduction reduces the number of storage systems connected to the server to more than one execute the mpdconfig s command to set the recognition order of the storage system that is to be removed to last place If not the storage systems with a lower recognition order m...

Страница 76: ...ing the mpdconfig d command Example 7 If the recognition order function is being used execute the mpdconfig s command to reconfigure the recognition order Example 8 Use the iompadm info command to check the result Example 9 Set the iSCSI service and delete the iSCSI target information of the removed storage system End of procedure 6 3 LU Addition Same as for FC FCoE connection Refer to 4 1 Storage...

Страница 77: ...d in the server For details on the settings refer to the storage system manual 2 Set the server iSCSI service Set iSCSI initiator iSCSI target information and log in to the iSCSI target Make sure that automatic login to the iSCSI target has been specified Refer to storage systems User s Guide Server Connection or any other relevant manuals and Appendix 2 in the software information opt FJSVmpd man...

Страница 78: ...ings 2 Shut down the server 3 Attach the hardware Set up the storage system so that its LUs are accessible via the cable and then connect the cable between the server and the storage system a NIC must already be installed in the server For details on the settings refer to the storage system manual 4 Restart the server Power on the server and boot it up 5 Use the iompadm info command to check that ...

Страница 79: ...cable is displayed as fail Example 3 Delete the removed cable from the Multipath Driver target list using the mpdconfig d command Example 4 Use the iompadm info command to check that the removed cable has disappeared from view Example 5 Reset the Max Throttle value if necessary When the number of paths connected to a given port in the storage system is reduced due to the path removal execute the m...

Страница 80: ...o command to check that the removed cable is displayed as fail Example 5 Delete the removed cable from the Multipath Driver target list using the mpdconfig d command Example 6 Use the iompadm info command to check the result Example 7 Reset the Max Throttle value if necessary When the number of paths connected to a given CA port in the storage system is reduced due to the path removal execute the ...

Страница 81: ...ug is only supported for PRIMEQUEST servers For SUSE Linux Enterprise Server 9 Adding and reducing NICs without server rebooting PCI Hot Plug is not supported For Red Hat Enterprise Linux 5 and SUSE Linux Enterprise Server 10 Adding and reducing NICs without server rebooting is the same procedure as adding and reducing cables without server rebooting Add or reduce NICs at the same time that cables...

Страница 82: ... RHEL5 SLES9 or SLES10 page 53 Red Hat Enterprise Linux AS v 4 Red Hat Enterprise Linux ES v 4 Red Hat Enterprise Linux 5 SUSE Linux Enterprise Server 9 Service Pack 2 or later SUSE Linux Enterprise Server 10 Service Pack 1 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may e...

Страница 83: ...nfo command to check that the added storage system has been correctly recognized and incorporated Example 3 Set the Max Throttle value for the new storage system Execute the mpdconfig m command to set a Max Throttle value for the new storage system Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation method Example End of procedure 7 1 2 Ad...

Страница 84: ...tion method Example End of procedure 7 2 Storage System Reduction 7 2 1 Reduction without Server Rebooting Using the mpdconfig command storage systems can be removed without shutting down the server Use the following procedure Procedure 1 Detach the storage system Disconnect the target storage system from the server turn the storage system power off disconnect the cables etc 2 Delete the removed s...

Страница 85: ...g procedure to remove storage systems while the server is shut down Procedure 1 Shut down the server 2 Detach the storage system Disconnect the target storage system from the server turn the storage system power off disconnect the cables etc 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check that the removed storage system has disappeared from view Exam...

Страница 86: ...fig a command Example The iompadm rescan command can also be used to incorporate the new LU Example 3 Use the iompadm info command to check that the added LU has been correctly recognized and incorporated Example End of procedure opt FJSVmpd bin mpdconfig a opt FJSVmpd bin iompadm rescan opt FJSVmpd bin iompadm info When a LU is added it is initially added as a new SCSI disk device after the LUs t...

Страница 87: ...ure 1 Shut down the server 2 Attach the LU Add a LU in the storage system For details on adding LUs refer to the storage system manual 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check that the new LU has been correctly recognized and incorporated Example End of procedure When adding LUs the device names sd may be changed Alter the settings for any app...

Страница 88: ...arget list using the mpdconfig d command Example 3 Use the iompadm info command to check that the removed LUs have disappeared from view Example End of procedure When removing the LUs without server rebooting completely halt all access to the LUs that are to be removed If access cannot be halted follow the procedures described in 7 4 2 Reduction with Server Rebooting page 89 For storage systems ot...

Страница 89: ... Refer to the storage system manual for LU reduction details 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check that the removed LUs are disappeared from view Example End of procedure For storage systems other than the ETERNUS DX S2 series ETERNUS DX S3 series or ETERNUS All Flash Array never attempt to remove LU0 If SCSI disks are accessed using sd dev...

Страница 90: ...le via the cable and then connect the cable between the server and the storage system an HBA must already be installed in the server For details on the settings refer to the storage system manual 2 Use the iompadm info command to check that the new cable has been correctly recognized and incorporated Example 3 Set a Max Throttle value for the new path Execute the mpdconfig m command to set a Max T...

Страница 91: ...y recognized and incorporated Example 5 Set a Max Throttle value for the new path Execute the mpdconfig m command to set a Max Throttle value for the new path Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation method Example End of procedure 7 6 Path Reduction Cable Reduction This section describes how to decrease the paths by removing ca...

Страница 92: ... Server Rebooting Use the following procedure to remove cables while the server is shut down Procedure 1 Shut down the server 2 Detach the hardware Disconnect the target cable from the server remove the connected cable etc 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check that the removed cable has disappeared from view Example 5 Delete the removed cab...

Страница 93: ...ng PCI Hot Plug is only supported for PRIMEQUEST servers For details on this procedure refer to the PRIMEQUEST manual To shut down the server after the addition of paths by adding HBAs change the status of the new paths to stop with the iompadm change command Adding and reducing HBAs with server rebooting is the same procedure as adding and reducing cables with server rebooting Add or reduce HBAs ...

Страница 94: ...se Linux AS 4 6 or later Red Hat Enterprise Linux ES 4 6 or later Red Hat Enterprise Linux 5 Update 3 or later SUSE Linux Enterprise Server 10 Service Pack 3 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may execute the mpdconfig command This chapter describes how to change ...

Страница 95: ...ced 2 Use the iompadm info command to check the result When the cables are removed the disconnected storage system should be automatically removed and disappear from the list Example 3 Delete the removed hardware from the Multipath Driver target list using the mpdconfig d command Example End of procedure 8 2 2 Reduction with Server Rebooting Same as for FC FCoE connection Refer to 7 2 2 Reduction ...

Страница 96: ... 8 5 Path Addition Cable Addition Same as for FC FCoE connection Refer to 7 5 Path Addition Cable Addition page 90 8 6 Path Reduction Cable Reduction This section describes how to decrease the paths by removing cables 8 6 1 Reduction without Server Rebooting Cables can be removed without shutting down the server Use the following procedure Procedure 1 Disconnect the surplus cables from the server ...

Страница 97: ... to set the Max Throttle value correctly Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation method End of procedure 8 6 2 Reduction with Server Rebooting Same as for FC FCoE connection Refer to 7 6 2 Reduction with Server Rebooting page 92 8 7 Path Addition and Reduction HBA Addition and Reduction This section describes how to increase pa...

Страница 98: ...onnection for RHEL5 SLES9 or SLES10 page 71 Red Hat Enterprise Linux 5 Update 4 or later SUSE Linux Enterprise Server 9 Service Pack 3 or later SUSE Linux Enterprise Server 10 Service Pack 2 or later The multipath configuration can be changed using the mpdconfig command The mpdconfig command is installed in the opt FJSVmpd bin directory Only a root level user may execute the mpdconfig command This...

Страница 99: ...en specified Refer to storage systems User s Guide Server Connection or any other relevant manuals and Appendix 2 in the software information opt FJSVmpd manual english readme pdf for details about iSCSI connection 3 Use the iompadm info command to check the result The new storage system should be automatically recognized and incorporated in the list Example 4 Set its Max Throttle value Execute th...

Страница 100: ...e removed from the server 2 Use the iompadm info command to check that the removed hardware has disappeared from view The storage system is automatically removed from view after logging out Example 3 Delete the removed hardware from the Multipath Driver target list using the mpdconfig d command Example 4 Set the iSCSI service and delete the iSCSI target information of the removed storage system En...

Страница 101: ...wer off disconnect the cables etc 3 Restart the server Power on the server and boot it up 4 Use the iompadm info command to check the result When the cables are disconnected they should be automatically removed and disappear from the list Example 5 Delete the removed hardware from the Multipath Driver target list using the mpdconfig d command Example 6 Set the iSCSI service and delete the iSCSI ta...

Страница 102: ...ettings refer to the storage system manual 2 Set the server iSCSI service Set iSCSI initiator iSCSI target information and log in to the iSCSI target Make sure that automatic login to the iSCSI target has been specified Refer to storage systems User s Guide Server Connection or any other relevant manuals and Appendix 2 in the software information opt FJSVmpd manual english readme pdf for details a...

Страница 103: ...e iSCSI target 2 Use the iompadm info command to check the result When the cables are disconnected they should be automatically removed and disappeared from the list Example 3 Delete the removed cable from the Multipath Driver target list using the mpdconfig d command Example 4 Reset the Max Throttle value if necessary When the number of paths connected to a given port in the storage system is red...

Страница 104: ...rt in the storage system is reduced due to the path removal execute the mpdconfig m command to set the Max Throttle value correctly Refer to the software information opt FJSVmpd manual english readme pdf for the Max Throttle value computation method 7 Set the iSCSI service and delete the iSCSI target information of the removed path End of procedure 9 7 Path Addition and Reduction NIC Addition and ...

Страница 105: ... Installation Information found under Downloads at the following URL http www fujitsu com global products computing storage software eternus mpd 10 1 Configuring a Storage System Zoning for a Fibre Channel switch When using a Fibre Channel switch to connect to the storage system make sure to perform the zoning setting for a Fibre Channel switch Conditions for path configuration Configure the multi...

Страница 106: ... in a multipath connection to the same LUNs used on the storage system In the following example the Multipath Driver will not operate properly if the two paths are mapped to different LUNs Set the same LUNs for both paths as shown on the left side of the following diagram Figure 10 1 Use identical LUNs CA CA LU0 Volume set LU0 CA CA LU0 Volume set LU1 Storage system Linux server Linux server Stora...

Страница 107: ...ng OSs the configured LUNs must start from LUN0 Red Hat Enterprise Linux AS v 4 or Red Hat Enterprise Linux ES v 4 Red Hat Enterprise Linux 5 SUSE Linux Enterprise Server 9 SUSE Linux Enterprise Server 10 However the first LUN_V need not be LUN_V0 LUNs also need not be contiguous after LUN0 Set the LUNs visible from the path to start from LUN0 as shown on the left side of the following diagram Fig...

Страница 108: ...cessary to allow the storage system to be recognized again Maximum number and size of disks The maximum number and size of disks that may be used depend solely on the OS specification There is no limitation inherent in the Multipath Driver Maximum number of storage systems recognizable Up to 256 storage systems including those via LUN Mapping Affinity Groups and zones can be recognized by a server...

Страница 109: ...ation found under Downloads at the following URL http www fujitsu com global products computing storage software eternus mpd When using Red Hat Enterprise Linux 7 If a kernel is updated make sure to execute the following command before rebooting the OS If the following message is displayed the Multipath Driver cannot operate with the updated kernel Change the configuration to a single path or upda...

Страница 110: ...installed comment out these messages and then execute the opt FJSVmpd system mpdsetup command Setting up etc fstab When using the OSs described below refer to etc fstab to confirm that the root file system and the partition that is used as a swap area are specified by LABEL If other specification methods are used change the setting method to LABEL Red Hat Enterprise Linux AS v 4 Red Hat Enterprise...

Страница 111: ...th name for udev function After installing the Multipath Driver the by path name that was created by the udev function as the storage system device name cannot be used When installing the Multipath Driver in an environment that uses the by path name as the storage system device change all the settings that need to use the by id name before installation When using Xen kernel The Multipath Driver su...

Страница 112: ...ckage should be installed depends on the environment that is used Use the following command to check which environment should be used default or pae is displayed indicating the environment that should be used Use the following command to check that the packages are installed correctly Install any of the packages listed above that are not installed Notes for an environment in which Java is installe...

Страница 113: ...are points to note that apply to occurring symptom If you cannot find any solutions contact us according to your support contract The following information is required to contact us according to your support contract Information related to the Linux server in use Model name HBA or NIC information HBA or NIC product ID OS Version Applied updates Kernel version at the time of startup Following infor...

Страница 114: ...the following OSs is used in the conditions described below OS Red Hat Enterprise Linux AS v 4 Red Hat Enterprise Linux ES v 4 Red Hat Enterprise Linux 5 SUSE Linux Enterprise Server 9 Service Pack 2 or later SUSE Linux Enterprise Server 10 Service Pack 1 or later Condition When replacing a storage system When changing the server to storage system connection When changing the server side LU recogn...

Страница 115: ...t allowed Also note that iompadm is mutually exclusive with the mpdconfig command Explanation This command shows the path status and controls the paths stops restarts in a multipath connection controlled by the Multipath Driver Subcommands This command shows the multipath status and recognized disks that are managed by the Multipath Driver even when the configuration is changed to a single path th...

Страница 116: ...ng the iompadm change command and then use the iompadm restart command When executed with no change_unit specified the operation is performed as if controller was specified Refer to 3 5 Disconnecting Reconnecting Paths when Replacing Hardware page 51 for details of change_unit Displays the Multipath Driver version number Displays the directions for the iompadm command Releases HBA of the specified...

Страница 117: ...te an error situation further info may be found in the syslog Item Detail Description 1 online Online status Normal status 2 warning Warning status After warning items 7 onward may appear 3 stop by command Path is stopped by the iompadm change command 4 forcible fail Fatal fail status To restore the online path status fix the root problem then use iompadm restart Diagnostic restoration is not poss...

Страница 118: ...orage system 20 s SK SC SSC Sense information was received from the storage system Sense information appears in s sense key appears in SK sense code appears in SC and sub sense code appears in SSC 21 recovered from fail status Status has been changed from fail to warning following diagnostic 22 I O rejection LU fail status Current LU status does not allow I O to be accepted 23 empty status LUN map...

Страница 119: ...mands were started at the same time Action Wait until one command finishes before entering the next invalid subcommand Cause An invalid subcommand has been entered Action Enter a valid subcommand invalid parameter Cause There is an error in the command argument Action Specify a valid value for the command argument fail to start path s s lun d Cause The path status cannot be changed to Online using...

Страница 120: ...er Optional arguments may be used to perform the following operations a installs added storage systems paths HBA and LUs d uninstalls a storage system path HBA or LU m initiates a dialog that allows Max Throttle values to be set for the recognized storage systems s initiates a dialog that allows the storage system recognition order to be set Up to 32 storage systems can be set c clears the storage...

Страница 121: ...e 1 40 30 START THROTTLE No PRODUCT SERIAL LUN_V BUS ID PORT CUR NEW ETERNUS_DXL 130011 0000 0 0000 03 04 0 0 40 30 1 0000 03 05 0 2 40 30 Input a path number or r Return menu r START NUMBER No PRODUCT SERIAL LUN_V OF PATH 0 ETERNUS_DXL 130011 0000 2 Input a number c Complete or q Quit c START THROTTLE No PRODUCT SERIAL LUN_V BUS ID PORT CUR NEW ETERNUS_DXL 130011 0000 0 0000 03 04 0 0 40 30 1 000...

Страница 122: ...trd images including ETERNUS multipath driver Please input a number or q for Quit 0 INITRD IMAGE FILE initrd 2 6 18 8 el5 mpd img START RECOGNITION ORDER PRODUCT SERIAL LUN_V CURRENT NOT FIXED NEW FIXED ETERNUS_DXL 130012 0000 ETERNUS_DXL 130011 0000 This program remakes the initrd image to apply the device recognition order Check the above configuration Is the configuration correct y yes n no y m...

Страница 123: ...g ETERNUS_DXL 130012 0000 ETERNUS_DXL 130011 0000 1 initrd 2 6 18 8 el5 mpd2 img This is a list of initrd images including ETERNUS multipath driver Please input a number or q for Quit 0 INITRD IMAGE FILE initrd 2 6 18 8 el5 mpd img START RECOGNITION ORDER PRODUCT SERIAL LUN_V CURRENT FIXED ETERNUS_DXL 130012 0000 ETERNUS_DXL 130011 0000 NEW NOT FIXED This program remakes the initrd image to apply ...

Страница 124: ... Error messages This section explains the main error messages and the resolution process If any error message other than following appears contact your Fujitsu engineer failed to open control device name s ret d Cause The control device cannot be opened s is the path name Action Check whether the system was booted using an initial RAM disk initrd with the Multipath Driver failed to file lock file ...

Страница 125: ...H D D is any number level Indicates the importance of the message WARNING warning NOTICE important note message Actual message If the error message continues to appear even after performing the suggested action contact a Fujitsu engineer MPDT WARNING I O rejection d d d d Cause The indicated paths for the indicated LUN have changed to a status that cannot accept I O LUN my not have been correctly ...

Страница 126: ...gic_code LUN_V_of_LU_number_0 example ETERNUS_DXL 130011 0000 0000 The meaning of the last four numeric characters is different from the disk device identifier meaning explained in 3 3 1 Device Identification page 49 Action Check the state of the storage system indicated by the s and its connection to the server remove the cause of any problem Then confirm that the connection status is recovered u...

Страница 127: ...l_number Device_identification_no magic_code path_id example ETERNUS_DXL 130011 0000 10 10 0 Refer to 3 3 2 Path Identification page 49 for details of the path id Action Check the connection status of path indicated by s and remove the cause of any problem Then confirm that the connection status is recovered using the iompadm info command If the connection status is not recovered restart the serve...

Страница 128: ...e server and the indicated storage system s in a configuration that is path redundant with respect to CFL and restart the server MPDH d WARNING s Operation of LUN addition failed Cause Adding a LUN fails because a path that cannot recognize the LUN to be added exists amongst the paths configuring the multipath d is a number and s is storage system information specifically storage_system_name seria...

Страница 129: ...nfo command respond as necessary to resolve the problem and recover the path using the iompadm restart command MPDH d WARNING SCSI u u u forcible path degrade Cause The status of the indicated path changed to fatal fail because the path was found to be unstable d and u are numbers Action Identify the failed path using the iompadm info command respond as necessary to resolve the problem and recover...

Страница 130: ... failed paths using the iompadm info command then resolve any HBA and other path component problems and use the iompadm restart command to recover the failed paths If the iompadm restart does not work reboot the server MPDH d WARNING I O rejection no more online path for this LUN Cause All the paths for the indicated LUN have changed to a status that cannot accept I O d is a number Action Identify...

Страница 131: ...ntification_no magic_code path_id example ETERNUS_DXL 130011 0000 10 10 0 Refer to 3 3 2 Path Identification page 49 for details of the path id Action Disconnect the cable from the port indicated by s and reconnect it to the proper port MPDH d WARNING SCSI u u u I O timeout was detected by the scsi timer Cause I O has timed out d and u are numbers Action Use the iompadm info command to identify th...

Страница 132: ...tion Contact a Fujitsu engineer SCSI u u u u ERROR 0x5 HBA abort Cause An issued I O has been canceled u are numbers Action Contact a Fujitsu engineer SCSI u u u u ERROR 0x6 HBA parity error Cause The indicated HBA has reported a parity error u are numbers Action Contact a Fujitsu engineer SCSI u u u u ERROR 0x7 HBA internal error Cause The indicated HBA has reported an internal error u are number...

Страница 133: ...stem to server connections and respond as necessary SCSI u u u u HBA ERROR 0x x Cause The indicated HBA has reported an error u and x are numbers Action Contact a Fujitsu engineer SCSI u u u u MSG ERROR 0x x Cause The indicated HBA has reported an error u and x are numbers Action Contact a Fujitsu engineer SCSI u u u u check condition too long Cause The indicated storage system has reported a chec...

Страница 134: ...0x x Cause Sense data has been received from the indicated storage system u are numbers SK SC SSC and FRU indicate the following SK sense key hexadecimal SC sense code hexadecimal SSC sub sense code hexadecimal FRU failure point hexadecimal Action Check the storage system and respond as necessary SCSI u u u u SK 0x04 SC 0x40 SSC 0x x FRU 0x x Diagnostic failure hexadecimal es_qual_code Cause Sense...

Страница 135: ...FRU indicate the following SK sense key hexadecimal SC sense code hexadecimal SSC sub sense code hexadecimal FRU failure point hexadecimal Action Check the storage system and respond as necessary SCSI u u u u SK 0x06 SC 0xf9 SSC 0xe0 FRU 0x x timeout unit attention Cause Sense data has been received from the indicated storage system u are numbers SK SC SSC and FRU indicate the following SK sense k...

Страница 136: ...m the indicated storage system u are numbers SK SC SSC FRU and Z indicate the following SK sense key hexadecimal SC sense code hexadecimal SSC sub sense code hexadecimal FRU failure point hexadecimal Z logical sum of SSC and 0x07 Action Check the storage system and respond as necessary SCSI u u u u incomplete scsi cdb 0x x Cause I O by the diagnosis function has failed u are numbers and x is the h...

Страница 137: ...tion and Reduction 64 70 93 97 I iompadm 50 51 115 iompadm change 51 117 iompadm info 50 iompadm restart 52 L Load Balancing 38 44 LU Addition 54 68 76 86 96 101 LU Reduction 57 68 76 88 96 101 M mapping 106 maximum number of disks 108 mpdconfig 120 mpd_snap 124 Multipath Driver 32 Multipath Driver Features 32 Multipath Driver Functions 38 Multipath Recovery 52 N NIC Addition and Reduction 81 104 ...

Страница 138: ...Index FUJITSU Storage ETERNUS Multipath Driver V2 User s Guide for Linux Copyright 2014 FUJITSU LIMITED P2U3 0031 15ENZ0 138 T Transition to Fatal Fail 43 U udev 34 W warning 39 ...

Страница 139: ...the utmost attention to detail However Fujitsu shall assume no responsibility for any operational problems as the result of errors omissions or the use of information in this manual Fujitsu assumes no liability for damages to third party copyrights or other rights arising from the use of any information in this manual The content of this manual may not be reproduced or distributed in part or in it...

Страница 140: ......

Результаты поиска

Содержание ETERNUS Multipath Driver V2

Отзывы:

Похожие инструкции для ETERNUS Multipath Driver V2

Бренды по названию

Популярные бренды

Fujitsu ETERNUS Multipath Driver V2, Руководство пользователя

Результаты поиска

Содержание ETERNUS Multipath Driver V2

Отзывы:

Похожие инструкции для ETERNUS Multipath Driver V2

Бренды по названию

Популярные бренды