manualshive.com logo in svg

Fujitsu Eternus DX80 S2, User Manual

Share
Download
Fujitsu Eternus DX80 S2 User Manual Download Page 132

Appendix A  Event Notification Message List
  A.2  Message List

ETERNUS DX80 S2/DX90 S2 Disk storage system User’s Guide -Operation-

Copyright 2013 FUJITSU LIMITED

P3AM-4842-12ENZ0

132

M E8090

xxx

7

E8090

xxx

FTRP#

xxx

 was all allocated

These messages are 
notified when "Thin 
Provisioning Pool 
Rate" is enabled for 
Error Severity Level in 
the event notification 
setting. 
They are not notified 
during maintenance 
operations.

There is no more  available capacity in the Flexible Tier Pool.

xxx

: FTRP number (hexadecimal)

M E928

xxyy

7

E928

xxyy

Key Management Server communication Error (Key Server ID 
#

xx

)(Key Group ID#

yy

)

This message is 
notified when "SED 
Network Error" is 
enabled for Error 
Severity Level in the 
event notification 
setting. 
This is not notified 
during maintenance 
operations.

A communication error with the key server occurred.

xx

: Key server ID

yy

: Key group ID

Event type 

code

SNMP Trap

Upper: Message

Remarks

Trap

No.

Trap

Maintenance

Info

Lower: Description

Summary of Contents for Eternus DX80 S2

Page 1: ...ETERNUS DX80 S2 DX90 S2 Disk storage system User s Guide Operation P3AM 4842 12ENZ0 ...

Page 2: ...This page is intentionally left blank ...

Page 3: ...orage system This manual is intended for use of ETERNUS DX Disk storage system in regions other than Japan Please carefully review the information outlined in this manual Twelfth Edition June 2013 Windows is a registered trademark of Microsoft Corporation in the United States and or other countries IBM AIX and Tivoli are trademarks of International Business Machines Corporation registered in many ...

Page 4: ...nd provides notes for setting up each function of the ETERNUS DX80 S2 DX90 S2 Disk storage system Chapter 4 Operations This chapter describes the control of the LAN for operation management of the ETERNUS DX80 S2 DX90 S2 Disk storage system and the monitoring the status of the ETERNUS DX80 S2 DX90 S2 Disk storage system Chapter 5 Maintaining Optional Products This chapter explains how to install o...

Page 5: ...y CAUTION This symbol indicates the possibility of minor or moderate personal injury as well as damage to the ETERNUS DX Disk storage system and or to other users and their property if the ETERNUS DX80 S2 DX90 S2 Disk storage system is not used properly This symbol indicates IMPORTANT information for the user to note when using the ETERNUS DX80 S2 DX90 S2 Disk storage system Electric Shock The tri...

Page 6: ...to the interface module that is normally used by the server to connect to ETERNUS DX Disk storage systems An FC card LAN card Network Interface Card NIC Converged Network Adapter CNA or SAS card may be used instead depending on the server and interface Trademark symbols such as and are omitted in this document Warning Level Indicator Warning Type Indicator Warning Details To avoid damaging the ETE...

Page 7: ... 2 Basic Operation 32 2 1 Powering On and Off 32 2 1 1 Switching On and Off the Main Line Switch on the Power Distribution Unit for Regions Other than EMEA I 32 2 1 2 Switching On and Off the PSU Switch on the Power Supply Unit 35 2 1 3 Powering On 36 2 1 4 Powering Off 38 2 2 Using the FUNCTION Button 39 Chapter 3 Setup 41 3 1 Thin Provisioning Setup 41 3 1 1 Registering the Thin Provisioning Fea...

Page 8: ...cts 64 5 1 Installing Drives 64 5 1 1 Installation Rules for Drives 65 5 1 2 Installable Drives 65 5 1 3 Drive Handling Instructions 66 5 1 4 Additional Drive Installation Procedure 67 5 2 Installing Drive Enclosures 72 5 2 1 Installation Rules for Drive Enclosures 72 5 2 2 Installable Drive Enclosures 72 5 2 3 Drive Enclosure Handling Instructions 72 5 2 4 Additional Drive Enclosure Installation ...

Page 9: ...d Service 95 Chapter 7 Troubleshooting 96 7 1 Check List 96 7 2 Required Information for Inquiries 102 Appendix A Event Notification Message List 103 A 1 Message Format 103 A 1 1 E mail 103 A 1 2 Syslog 105 A 1 3 SNMP 107 A 1 4 Host Sense 111 A 2 Message List 112 A 2 1 Common Terms 112 A 2 2 Error Messages 113 A 2 3 Warning Messages 133 A 2 4 Informational Messages 143 A 2 5 Test Messages 149 A 2 ...

Page 10: ...umbers 2 5 type drive enclosure 26 Figure 1 24 3 5 drive 26 Figure 1 25 Drive slot numbers 3 5 type drive enclosure 26 Figure 1 26 I O module 27 Figure 1 27 Power supply unit 28 Figure 1 28 Power distribution unit for DX80 S2 DX90 S2 AC200 240V 1U Max 2 enclosures connection 29 Figure 1 29 Power distribution unit for DX80 S2 DX90 S2 AC200 240V 2U Max 6 enclosures connection 30 Figure 1 30 Power di...

Page 11: ... each LED controller 18 Table 1 4 Status and meanings of each LED power supply unit 22 Table 1 5 Status and meanings of each LED in front of drive enclosure 25 Table 1 6 Status and meanings of each LED drive 26 Table 1 7 Status and meanings of each LED I O module 27 Table 1 8 Status and meanings of each LED power supply unit 28 Table 4 1 General status of ETERNUS Web GUI 61 Table 6 1 Time for volu...

Page 12: ...peration panel and drives are installed in the front of the controller enclosure Controllers and power supply units are installed in the rear 1 1 1 Front 2 5 type Figure 1 1 Front view of a 2 5 type controller enclosure 3 5 type Figure 1 2 Front view of a 3 5 type controller enclosure Operation panel 2 5 disk or 2 5 Solid State Drive SSD Flange cover Flange cover Operation panel Flange cover Flang...

Page 13: ...pe Figure 1 3 Rear view of a controller enclosure single controller type Part explanation Cover Remove this when installing an additional controller optional Dual controller type Figure 1 4 Rear view of a controller enclosure dual controller type Power supply unit PSU 0 Power supply unit PSU 1 Controller CM 0 Cover Power supply unit PSU 0 Power supply unit PSU 1 Controller CM 0 Controller CM 1 ...

Page 14: ...er enclosure Operation panel An operation panel has LEDs a Power switch and a FUNCTION button 2 5 type Figure 1 5 Operation panel 2 5 type controller enclosure 3 5 type Figure 1 6 Operation panel 3 5 type controller enclosure With a flange cover CACHE LED MAINT ID LED FAULT LED READY LED Power switch POWER LED Without a flange cover FUNCTION button With a flange cover Power switch CACHE LED MAINT ...

Page 15: ...uring maintenance to restore the factory default settings of the LAN ports LEDs The states of LEDs are listed below Table 1 1 Status and meanings of each LED operation panel controller enclosure LED name LED status ETERNUS DX Disk storage system status POWER green DC power is supplied to the controller enclosure READY green The ETERNUS DX Disk storage system is available for use FAULT amber The ET...

Page 16: ...ontroller enclosure 3 5 drives Figure 1 9 3 5 drive Figure 1 10 shows the slot number of each drive Figure 1 10 Drive slot numbers 3 5 type controller enclosure DRIVE READY LED DRIVE FAULT LED Slot 8 Slot 9 Slot 10 Slot 11 Slot 12 Slot 13 Slot 14 Slot 15 Slot 0 Slot 1 Slot 2 Slot 3 Slot 4 Slot 5 Slot 6 Slot 7 Slot 16 Slot 17 Slot 18 Slot 19 Slot 20 Slot 21 Slot 22 Slot 23 DRIVE READY LED DRIVE FAU...

Page 17: ...ntains a CPU cache memory System Capacitor Unit SCU host interfaces drive interface DI ports and LAN ports The controller controls all operations in the ETERNUS DX Disk storage system Figure 1 11 Controller LED name LED status Drive status DRIVE READY green The drive is in normal status blinks green DRIVE FAULT amber The drive is in error status In response to commands from ETERNUS CLI the locatio...

Page 18: ...s READY FAULT green The controller is in normal status amber The controller is performing the initial setup after the power is turned on The controller is in error status blinks amber MASTER green The controller is set as a Master CM UNIT ID blinks green As ordered via ETERNUS Web GUI or ETERNUS CLI the location of the controller is identified System Capacitor Unit SCU is charging CA FAULT amber T...

Page 19: ... s FC 8Gbit s iSCSI 10Gbit s iSCSI 1Gbit s FCoE and SAS FC iSCSI 10Gbit s FCoE for FC cable connection Figure 1 12 Host interface FC iSCSI 10Gbit s FCoE for FC cable connection Part explanation Host interface ports FC iSCSI 10Gbit s FCoE for FC cable connection from left to right Port 0 and Port 1 in CA 0 Port 0 and Port 1 in CA 1 These are the Dual LC connectors to connect FC cables Port 0 Port 1...

Page 20: ...eft to right Port 0 and Port 1 in CA 0 Port 0 and Port 1 in CA 1 These are the connectors to connect Copper Twinax cables iSCSI 1Gbit s Figure 1 14 Host interface iSCSI 1Gbit s Part explanation Host interface ports iSCSI from left to right Port 0 and Port 1 in CA 0 Port 0 and Port 1 in CA 1 These are the RJ 45 connectors to connect LAN cables Port 0 Port 1 Port 0 Port 1 iSCSI 10Gbit s FCoE Install...

Page 21: ...CU is charged from an external power source while the ETERNUS DX Disk storage system is running normally If a power outage is detected data in the cache memory is saved to the non volatile memory in the controller using the SCU The saved data is retained in the non volatile memory indefinitely 1 1 4 2 Power Supply Units The power supply unit transforms input AC power from a power socket to DC powe...

Page 22: ...ply unit status POWER PSU FAIL AC MISSING FAN FAIL blinks green off off off AC power is supplied to the power supply unit but the ETERNUS DX Disk storage system DC power is not turned on green off off off The power of the ETERNUS DX Disk storage system DC power is turned on and the power supply unit is operating normally off off off off AC power is not supplied to power sup ply units off amber amb...

Page 23: ...losure I O modules and power supply units are installed in the rear There are two types of drive enclosures a 2 5 type and a 3 5 type 1 2 1 Front 2 5 type Figure 1 17 Front view of a 2 5 type drive enclosure 3 5 type Figure 1 18 Front view of a 3 5 type drive enclosure LED Flange cover Flange cover 2 5 disk or 2 5 Solid State Drive SSD DE ID drive enclosure number display panel Flange cover 3 5 di...

Page 24: ...ional I O module optional Dual IOM type Figure 1 20 Rear view of a drive enclosure dual IOM type 1 2 3 Components Front This section describes LEDs a DE ID drive enclosure number display panel and drives of the front of the drive enclosure DE ID drive enclosure number display panel The DE ID drive enclosure number of the drive enclosure is displayed I O module IOM6 0 Power supply unit PSU 0 Power ...

Page 25: ...Ds are listed below Table 1 5 Status and meanings of each LED in front of drive enclosure Drives 2 5 drives Figure 1 22 2 5 drive LED name LED status Drive enclosure status LINE green AC power is supplied to the drive enclosure FAULT amber The drive enclosure is in error status POWER green The drive enclosure is operating normally 2 5 type 3 5 type LINE LED POWER LED FAULT LED LINE LED POWER LED F...

Page 26: ...tion LEDs The states of LEDs are listed below Table 1 6 Status and meanings of each LED drive LED name LED status Drive status DRIVE READY green The drive is in normal status blinks green DRIVE FAULT amber The drive is in error status In response to commands from ETERNUS CLI the location of the drive is identified Slot 8 Slot 9 Slot 10 Slot 11 Slot 12 Slot 13 Slot 14 Slot 15 Slot 0 Slot 1 Slot 2 S...

Page 27: ...s interact Figure 1 26 I O module Part explanation DI OUT port DI IN port These ports are connectors for QSFP cables LEDs The states of LEDs are listed below Table 1 7 Status and meanings of each LED I O module LED name LED status I O module status FAULT amber The I O module is in error status DI OUT LINKUP green The link between the DI OUT port and the destination port has been established DI IN ...

Page 28: ...h is used to turn on and off the AC power supply LEDs The states of LEDs are listed below Table 1 8 Status and meanings of each LED power supply unit LED name Power supply unit status POWER PSU FAIL AC MISSING FAN FAIL green off off off AC power is supplied to the power supply unit off off off off AC power is not supplied to the power supply unit off amber amber off or amber AC power is not suppli...

Page 29: ...utlets and two inlets Power distribution unit for DX80 S2 DX90 S2 AC200 240V 1U Max 2 enclosures connection Figure 1 28 Power distribution unit for DX80 S2 DX90 S2 AC200 240V 1U Max 2 enclosures connection Part explanation Outlet OUTPUT This is a socket IEC60320 C13 for outgoing power supply This socket is used to connect a power cord AC output cable Inlet INPUT This is a socket IEC60320 C14 for i...

Page 30: ...power distribution units that can connect up to eight enclosures Power distribution unit for DX80 S2 DX90 S2 AC200 240V 2U Max 6 enclosures connection There are 12 outlets Figure 1 29 Power distribution unit for DX80 S2 DX90 S2 AC200 240V 2U Max 6 enclosures connection Part explanation Outlet OUTPUT This is a socket IEC60320 C13 for outgoing power supply This socket is used to connect a power cord...

Page 31: ...r DX80 S2 DX90 S2 AC200 240V 2U Max 8 enclosures connection There are 16 outlets Figure 1 30 Power distribution unit for DX80 S2 DX90 S2 AC200 240V 2U Max 8 enclosures connection Part explanation Outlet OUTPUT This is a socket IEC60320 C13 for outgoing power supply This socket is used to connect a power cord AC output cable Main line switch This turns on and off the power distribution unit Outlets...

Page 32: ...and off the ETERNUS DX Disk storage system 2 1 1 Switching On and Off the Main Line Switch on the Power Distribution Unit for Regions Other than EMEA I This section explains how to switch the main line switch of the power distribution unit to ON and OFF To switch to ON Turn the main line switch of the power distribution unit to the ON position marked I For 1U power distribution unit Figure 2 1 ON ...

Page 33: ...bution unit to the OFF position marked O For 1U power distribution unit Figure 2 3 OFF position marked O of the main line switches on a 1U power distribution unit Main line switch If the ETERNUS DX Disk storage system must be turned off such as before any inspections of power supply devices are performed turn off the ETERNUS DX Disk storage system by using the procedure described in 2 1 4 Powering...

Page 34: ...42 12ENZ0 34 For 2U power distribution unit Figure 2 4 OFF position marked O of the main line switches on a 2U power distribution unit When turning the main line switch to ON marked I right after turning the main line switch to OFF marked O turn it back to ON marked I after the POWER LED of the power supply unit has turned off completely Main line switch ...

Page 35: ...urn the PSU switch of the power supply unit to the ON position marked Figure 2 5 ON position marked of the PSU switch on a power supply unit AC power is supplied to an enclosure For the controller enclosure the POWER LED on the power supply unit blinks green For the drive enclosure the POWER LED on the power supply unit and the LINE LED on the front emit green lights and the fan revolves at high s...

Page 36: ...it is off 2 1 3 Powering On This section explains how to turn on the ETERNUS DX Disk storage system The following methods can be used to turn the power on Via the Power switch Press the Power switch to turn on the ETERNUS DX Disk storage system Via the power synchronized unit Use the power synchronized unit to turn on the ETERNUS DX Disk storage system Via the Wake On LAN function Use the Wake On ...

Page 37: ...heck that the ETERNUS DX Disk storage system and the network devices that connect the ETERNUS DX Disk storage system and the server are all in READY status If the server is turned on while any of these devices are not in READY status the server may not be able to recognize the ETERNUS DX Disk storage system When turning on the power in conjunction with the power synchronized unit perform the appro...

Page 38: ...how to use the Power switch to turn off the ETERNUS DX Disk storage system For other procedures refer to the related manuals Controller enclosure Drive enclosure Drive enclosure POWER READY FAULT MAINT ID CACHE LINE FAULT POWER When turning off the ETERNUS DX Disk storage system the power shuts off after the data in the cache memory is written to the drives Therefore it may take one minute maximum...

Page 39: ...roller enclosure The following settings can be performed by using the FUNCTION button Switching the Master CM to the other controller for a dual controller type Restoring the factory default settings network environment settings and firewall settings of the LAN ports Press the Power switch only once If the Power switch is pressed again between the time of the READY LED turning off and the POWER LE...

Page 40: ...he MAINT ID LED is green 3 Use the FUNCTION button to change the following settings Switching the Master CM to the other controller for a dual controller type Use the pin to push down the FUNCTION button twice within three seconds The MASTER LED for the controller that is set as the Master CM is green Restoring the factory default settings of the LAN ports Use the pin to push down the FUNCTION but...

Page 41: ... Thin Provisioning function can be used more efficiently by using the Zero Reclamation function The Zero Reclamation function releases physical areas with all zero bits To use the Thin Provisioning function register the Thin Provisioning Feature License in the ETERNUS DX Disk storage system and set up the Thin Provisioning function 3 1 1 Registering the Thin Provisioning Feature License For the EM...

Page 42: ...d drives to expand the TPP capacity before insufficient TPP capacity is detected The Thin Provisioning function is unsuitable for using with OSes that write meta information to all of the LUNs when creating a file system It is recommended that backing up of a TPV is performed in file units If a complete backup is performed areas that are not allocated to the drives are also backed up as dummy data...

Page 43: ...to a different TPP for which the target TPV does not belong If RAID groups that cannot perform balancing such as RAID groups without free capacity exist in a TPP balancing of physical allocation capacity is performed among the other RAID groups in the TPP If this occurs the balancing level after the balancing process is complete may not be High When TPV balancing is performed an area for the work ...

Page 44: ...ng function the allocated physical capacity as well as the usage rate of the physical disks in the pool can be balanced To use the Flexible Tier auto storage layering function register the Thin Provisioning Feature License in the ETERNUS DX Disk storage system and enable the Flexible Tier function 3 2 1 Registering the Thin Provisioning Feature License For the EMEA I region After the Thin Provisio...

Page 45: ...ty of 0MB are created when the Flexible Tier function is enabled The maximum number of volumes that can be created in the ETERNUS DX Disk storage system is reduced by 32 To encrypt data specify encryption for FTRP when creating an FTRP or use Self Encrypting Drives SED to configure the FTSP If usage of an FTRP or an FTV exceeds the threshold a notification is sent For FTRPs a notification SNMP Tra...

Page 46: ...comes insufficient while FTRP balancing is being performed an error occurs and the balancing session ends abnormally Note that insufficient physical capacity cannot be replaced by other FTSPs When FTRP balancing is performed an area for the work volume the destination FTV which has the same capacity as the source FTV is secured in the FTRP to which the FTV belongs As a result the status of the FTR...

Page 47: ...be performed when the migration destination device has a larger volume capacity than the migration source device Volumes for which the volume type Standard Open TPV or SDV is available for the host affinity settings can be specified as migration source volumes Standard Open WSV TPV or FTV type volumes can be specified as migration destination volumes Storage Migration can be performed if the follo...

Page 48: ...olen or replaced for maintenance This function also reduces discarding costs because SEDs do not need to be physically destroyed Volume conversion encryption function This function encrypts data on a volume basis by using the firmware of the ETERNUS DX Disk storage system Encryption can be performed in volume units according to the importance of the data No additional hardware is required to perfo...

Page 49: ...estore the backed up data It is recommended that the copy source volume and the copy destination volume use the same encryption method for remote copy between encrypted volumes When copying encrypted volumes using Advanced Copy or copy operations via server transfer performance may not be as good as when copying unencrypted volumes SDPVs cannot be encrypted after they are created To create an encr...

Page 50: ... key server by using the ETERNUS DX Disk storage system enables the key to be automatically updated by obtaining a new authentication key from the key server before the validity of the key expires Access from the host server can be maintained even if the SED authentication key is changed during operation When linking with the key management server the ETERNUS DX Disk storage system obtains the SED...

Page 51: ...Storage Manager HSM for Windows Dynamic Storage Tiering DST function in Symantec Veritas Storage Foundation The restrictions and notes on using Eco mode are described below For details on the Eco mode procedure refer to ETERNUS Web GUI User s Guide or ETERNUS CLI User s Guide To set Eco mode schedule use ETERNUS Web GUI ETERNUS CLI ETERNUS SF Storage Cruiser or ETERNUS SF AdvancedCopy Manager here...

Page 52: ...g to the date and time that are set in the ETERUS DX Disk storage system To turn on and turn off the disk motors according to the schedule that is set use the Network Time Protocol NTP server in the date and time setting in ETERNUS Web GUI or ETERNUS CLI to set automatic adjustment of the date and time Up to eight 2 5 drives or four 3 5 drives can be activated at the same time in a single drive en...

Page 53: ...s of free space to an existing volume to expand its capacity This uses existing free area to efficiently expand the volume Expansion of RAID group capacity Logical Device Expansion Adds new drives to an existing RAID group to expand the RAID group capacity This is used to expand the existing RAID group capacity instead of adding new RAID group to add the volumes Migration of RAID groups RAID Migra...

Page 54: ... Standard Open or a WSV Also note that when the RAID Migration source and destination are both WSVs the number of concatena tions for existing RAID groups can be changed Capacity expansion cannot be performed at the same time as RAID migration when the RAID Migration source is a TPV or an FTV or when the RAID Migration destination is a TPP or an FTRP To expand the capacity use the capacity expansi...

Page 55: ...pacity is the same or larger than the old capacity Capacity expansion and changing the level of a RAID group can be performed at the same time When adding drives use drives that have the same capacity and speed as the drives that configure the target RAID group If drives of different capacities exist in a RAID group the smallest capacity becomes the standard for the RAID group after expansion and ...

Page 56: ...ps with different types such as capacity or speed of drives can be concat enated However RAID groups with the same RAID level and same drive type are recommended for con catenated volumes to ensure that performance is not affected LUN Concatenation cannot be performed when the drive type for the RAID group to which the concatenation source volume belongs and the drive type to which the concatenati...

Page 57: ...performance by concatenating multiple RAID groups with striping and using many drives at the same time This function is especially effective for systems that require write access performance By increasing the number of drives that configure a LUN process performance is improved by the distribution of I O access from the server to multiple drives This section describes the points to note when using...

Page 58: ...system operates LAN control controller Master CM Slave CM For a dual controller type the controller CM that is given the authority to manage the LAN is called the Master CM and the other CM is called the Slave CM When an error occurs in the Master CM or LAN the Master CM is switched automatically IP addresses of the LAN ports are not assigned to each controller These IP addresses are assigned to t...

Page 59: ...TERNUS DX Disk storage system The IP address of the Slave CM does not need to be set for normal operation Figure 4 2 LAN control when the IP address of the Slave CM is set Controller enclosure MNT CM 0 Master CM MNT CM 1 Slave CM Controller enclosure MNT RMT RMT RMT RMT CM 0 Slave CM MNT CM 1 Master CM IP address A ETERNUS DX Disk storage system ETERNUS DX Disk storage system The Master CM is swit...

Page 60: ...e ETERNUS DX Disk storage system in order to monitor the status of the ETERNUS DX Disk storage system 4 2 1 LED Status Check The status of the ETERNUS DX Disk storage system can be checked by the LEDs If the FAULT LED in front of the ETERNUS DX Disk storage system is amber contact your maintenance engineer CAUTION Impending trouble may be indicated by the warning message FAULT LEDs of the ETERNUS ...

Page 61: ...he ETERNUS DX Disk storage system is monitored periodically and the result is displayed as a general status image with character strings Table 4 1 General status of ETERNUS Web GUI Refer to ETERNUS Web GUI User s Guide for details General status ETERNUS DX Disk storage system status Normal Green The ETERNUS DX Disk storage system is in normal state Not Ready Red Not Ready is a status where an abno...

Page 62: ...t information refer to Appendix A Event Notification Message List page 103 SNMP Trap When an event occurs in the ETERNUS DX Disk storage system an SNMP Trap is sent to the SNMP Manager monitoring server For more details on event information notified by SNMP Trap refer to Appendix A Event Notification Message List page 103 Syslog When an event occurs in the ETERNUS DX Disk storage system the event ...

Page 63: ...ystem actions that are associated with these operations can be sent to the syslog server For the procedure on setting audit logs refer to User s Guide Installation For audit log information refer to Appendix B Audit Log List page 154 Figure 4 5 Audit logs SNMP manager Mail server Syslog server ETERNUS DX Disk storage system Remote support center REMCS AIS Connect Host sense Server host Syslog SNMP...

Page 64: ...cts Disks drive enclosures and Long wave SFP modules can be added by users Users can also replace Short wave SFP modules with Long wave SFP modules However make sure to read the manual that contains the safety precautions before performing any of these procedures 5 1 Installing Drives Optional drives can be installed while the ETERNUS DX Disk storage system is running without affecting the system ...

Page 65: ...ositions of 2 5 drive slots Figure 5 1 Positions of 2 5 drive slots 3 5 drives 3 5 drives can be installed in the front side of a 3 5 type controller enclosure or a 3 5 type drive enclosure Different types of drives can be installed in the same enclosure Drives can be installed in any slots of an enclosure Figure 5 2 shows the positions of 3 5 drive slots Figure 5 2 Positions of 3 5 drive slots 5 ...

Page 66: ...such as an air conditioned room the severe temperature change may result in condensation forming To avoid this allow the packed drive sufficient time in the warmer place one hour for each 15 C of temperature difference to adapt to the new temperature CAUTION Do Not Do not touch the surface of the board of the drives Do When handling drives make sure to wear a wrist strap or touch a metal part to d...

Page 67: ... DX Disk storage system are in normal status 2 Wear a wrist strap or touch a metal part to discharge the human body s natural static electricity CAUTION Do Not Do not lay the drives directly on a desk or similar hard surface Always use a rubber mat or other soft material to cushion the drives against physical shocks Do not stack drives or stand drives on end Do not apply external force to the inte...

Page 68: ... in which the drive is to be installed Push into the tab of the dummy drive unit and pull it straight out of the drive slot 4 Install the new drive 4 1 Press the catch on the drive lock lever to unlock it 4 2 Insert the drive all the way into in the slot making sure that it is firmly seated and keeping the lock lever open The removed dummy drive unit should be retained for future use Tab Dummy dri...

Page 69: ... used remove the wrist strap 7 In the Disks screen of Component on the ETERNUS Web GUI screen check that the added drives have been recognized 8 Set up RAID groups volumes hot spares and Host Affinity settings on the new drives as required End of procedure For 3 5 drives Procedure 1 In the Storage screen of Component on the ETERNUS Web GUI screen check that all components of the ETERNUS DX Disk st...

Page 70: ...lled Push into the tab of the dummy drive unit and pull it straight out of the drive slot 4 Install the new drive 4 1 Press the catch on the drive lock lever to unlock it 4 2 Insert the drive all the way into in the slot making sure that it is firmly seated and keeping the lock lever open The removed dummy drive unit should be retained for future use WARNING Do Hold the drive with both hands to pr...

Page 71: ...until the lock clicks shut 4 4 Check that the newly installed drive s DRIVE READY LED lights up green 5 Repeat Step 3 and Step 4 for each additional drive 6 If a wrist strap is used remove the wrist strap 7 In the Disks screen of Component on the ETERNUS Web GUI screen check that the added drives are recognized 8 Set up RAID groups volumes hot spares and Host Affinity settings on the new drives as...

Page 72: ...r In this case the maximum number of drives allowed determines the maximum number of drive enclosures that may be installed 5 2 2 Installable Drive Enclosures For installable drive enclosures refer to Product List 5 2 3 Drive Enclosure Handling Instructions About condensation CAUTION Do Not Do not uninstall or move a drive enclosure that is already installed Contact your sales representative or ma...

Page 73: ...e numbers of the installed enclosures in the Drive Enclosure screen CAUTION Do When handling drive enclosures make sure to wear a wrist strap or touch a metal part to discharge the human body s natural static electricity Failure to discharge static electricity may cause device failure CAUTION Do Not Do not handle the drive enclosure roughly or subject it to physical shocks when laying it down Do n...

Page 74: ...re screen 2 1 In the Storage screen of Component click Add Drive Enclosure in Action The Add Drive Enclosure screen appears 2 2 Check the DE ID drive enclosure number for the drive enclosure that is to be added in Target Drive Enclosure and click the Next button A screen to show how to install the drive enclosure appears 3 Wear a wrist strap or touch a metal part to discharge the human body s natu...

Page 75: ...s should be inserted in the 1st and 6th holes above the base line of the controller enclosure 2 Insert the pins of the left slide rail in the 1st and 3rd holes from the bottom of the 2U rack rail 3 Secure the left slide rail to the front pillar using the M5 centering screw and M5 pan head screw 4 For the right slide rail insert the pins in the holes of the rear pillar and secure the other side of ...

Page 76: ...r right side to fit the rack Loosen the M4 screws to adjust the length of the rack rails to match the distance between the front and rear rack pillars Leave the M4 screws slightly unscrewed as the rack rail must be attached to the rack before they can be completely tightened Drive enclosure 1st position M5 pan head screw 3rd position M5 centering screw 1st position M5 pan head screw 3rd position M...

Page 77: ...nd 5th holes on the rear rack pillars above the base line to fasten the rack rails 3 Tighten the M4 screws of the rack rails that were slightly unscrewed in Step 1 Make sure to attach the rack rails and rack pillars so that they fit exactly together without any space between them Front rack pillars Rear rack pillars Washers M5 screws M5 screws Washers M5 screws Plate M5 screws Plate Rack rail for ...

Page 78: ... 78 4 2 Install the drive enclosure in the rack 4 3 Fasten the drive enclosure in the rack For the EMEA I region Use two M5 screws to fasten it in the rack pillar CAUTION Do When installing or removing the drive enclosure to or from the rack make sure that the sides and the bottom of the drive enclosure are held by two or more people Failure to do so may cause injury M5 screw M5 screw ...

Page 79: ...her than EMEA I Use four M5 screws to fasten it in the rack pillar 4 4 Attach the DE_No labels to the positions shown below Attach the labels in the same installation order as the drive enclosures QSFP cable connection order The DE_No labels are supplied with the ETERNUS DX Disk storage system M5 screws M5 screws Right front side of the drive enclosure Upper rear side of the drive enclosure DE_No ...

Page 80: ...ver that is provided with the ETERNUS DX Disk storage system 4 6 For regions other than EMEA I attach the brackets to the rack rails for the left and right sides Use the thumb screws to attach the brackets at the specified locations on the rear side Right 3 5 type drive enclosure 2 5 type drive enclosure Left Attachment location of the bracket 2 5 type drive enclosure 3 5 type drive enclosure Brac...

Page 81: ...s when drive enclosure 1 is added For a single controller type For a dual controller type To help with QSFP cable management and prevent incorrect connections make sure to attach the destination labels that are supplied with the ETERNUS DX Disk storage system to the connectors of the QSFP cables When connecting QSFP cables check the connector orientation and then firmly push it all the way in When...

Page 82: ...on the enclosure below the added drive enclosure 2 Attach the destination labels to the connectors of the QSFP cables Controller number or Drive enclosure number IOM number is printed on the destination labels in the Connection origin DI port Connection destination DI port format The or symbol is also printed on the destination labels Carefully check the connection origin and connection destinatio...

Page 83: ... 1 in the enclosure that has the highest DE ID drive enclosure number to the DI IN port of IOM6 1 in the drive enclosure that is to be added Perform the same operation as Step 5 2 to connect the QSFP cable 6 Connect the power cord 6 1 Connect the power cord to the drive enclosure 6 2 Connect the power cord plugs to the inlets of the power supply units Do not connect any cable to the last DI OUT po...

Page 84: ...he power of the drive enclosure is turned on and the fan revolves at high speed for about 30 seconds 8 Check that the LINE LED and the POWER LED on the drive enclosure are lit up 9 In the Add Drive Enclosure screen click the Next button The added drive enclosures are recognized and the installation completion screen appears 10 Click the Done button The Add Drive Enclosure screen closes and the scr...

Page 85: ... 5 4 show the installation diagrams for host interfaces Install the host interfaces for CA 0 first and then CA 1 Host interfaces that have different interfaces can be installed in CA 0 and CA 1 For a dual controller type the configuration is the same for Controller 0 CM 0 and Controller 1 CM 1 For a single controller type Figure 5 3 Installation diagram for a host interface single controller type ...

Page 86: ...erface CA 1 Controller 0 CM 0 Controller 1 CM 1 CAUTION Do When moving a host interface from a cold place such as an un conditioned store house in winter to a warmer place such as an air conditioned room the severe temperature change may result in condensation forming To avoid this allow the packed host interface sufficient time in the warmer place one hour for each 15 C of temperature difference ...

Page 87: ... two Short wave SFP modules 16Gbit s in a host interface FC 16Gbit s can be replaced with Long wave SFP modules 16Gbit s Short wave SFP modules 16Gbit s and Long wave SFP modules 16Gbit s can be mixed in the same host interface FC 16Gbit s For the dual controller type the SFP module configurations for controller 0 CM 0 and controller 1 CM 1 must be the same CAUTION Do Not Do not handle the host in...

Page 88: ...ces such as an air conditioned room the severe temperature change may result in condensation forming To avoid this allow the packed SFP modules sufficient time in the warmer place one hour for each 15 C of temperature difference to adapt to the new temperature CAUTION Do When handling SFP modules make sure to wear a wrist strap or touch a metal part to discharge the human body s natural static ele...

Page 89: ... that is to be replaced by using the Internal Parts information in the Channel Adapter Detail screen Note that a Short wave SFP module 16Gbit s is displayed as 16G SFP MMF in the SFP Type field 3 Turn off the ETERNUS DX Disk storage system Refer to 2 1 4 Powering Off page 38 for details on how to turn off the ETERNUS DX Disk storage system 4 Wear a wrist strap or touch a metal part to discharge th...

Page 90: ...Short wave SFP module 16Gbit s lever 6 2 Pull the Short wave SFP module 16Gbit s straight out of the port cage 7 Install the Long wave SFP module 16Gbit s 7 1 Remove the connector cover that is attached to the Long wave SFP module 16Gbit s port and pull down the lever Make sure to store the removed Short wave SFP module 16Gbit s with the fixing lever and the connector cover attached Keep the remov...

Page 91: ... port cage 7 3 Push up the Long wave SFP module 16Gbit s lever until it clicks into place 8 Connect the FC cable that is disconnected in Step 5 Connect the FC cable connector to the FC port on the Long wave SFP module 16Gbit s 9 Repeat Step 5 through Step 8 to replace multiple SFP modules FC 16Gbit s CAUTION Do When connecting the FC cables position them so that they will not obstruct replacement ...

Page 92: ...etails on how to turn on the ETERNUS DX Disk storage system 12 Check that the LINK FAULT LED for the FC port on the replaced Long wave SFP module 16Gbit s turns off or lights up green 13 Start ETERNUS Web GUI to check the status of the newly added Long wave SFP module 16Gbit s On the Internal Parts information in the Channel Adapter Details screen of ETERNUS Web GUI check that the FC port status i...

Page 93: ...re is no server I O The time that is required depends on the type and configuration of the disk CAUTION Do We recommend that you backup important data regularly If the ETERNUS DX Disk storage system fails and the data has not been backed up it may be lost RAID level Number of disks Volume formatting time 1 2 5 SAS disks 3 5 SAS disks Nearline SAS disks RAID1 2 Approx 40 minutes 100GB Approx 35 min...

Page 94: ...me that is required depends on the type and configuration of the disk RAID level Number of disks Rebuild process time 1 2 5 SAS disks 3 5 SAS disks Nearline SAS disks RAID1 2 Approx 27 minutes 100GB Approx 20 minutes 100GB Approx 47 minutes 100GB RAID1 0 8 Approx 7 minutes 100GB Approx 5 minutes 100GB Approx 11 minutes 100GB RAID5 5 Approx 10 minutes 100GB Approx 10 minutes 100GB Approx 15 minutes...

Page 95: ...greement SLA Contact your sales representative for details if you prefer to extend the period 6 4 2 Related Service Remote support If a possible error that may occur is detected or an error occurs when remote support service is used the contents of events are automatically sent to the remote support center and notified to the user When performing maintenance service the maintenance engineer connec...

Page 96: ...vice When the license key cannot be registered When I O access is slow When the server does not recognize the volumes 7 1 Check List When the ETERNUS DX Disk storage system is turned off Check the following points If none of these resolve the problem leave the ETERNUS DX Disk storage system alone and contact your maintenance engineer Is the ETERNUS DX Disk storage system s power cord disconnected ...

Page 97: ...intenance engineer When the FAULT LED is on Check the general status and the system messages in ETERNUS Web GUI Perform the necessary actions to correct the problem If the problem cannot be resolved leave the ETERNUS DX Disk storage system alone and contact your maintenance engineer When the general status of ETERNUS Web GUI is Check whether one side of the QSFP cable is disconnected If so install...

Page 98: ... connected in the correct order or check that the IN and OUT side of the QSFP cable is connected correctly If not use the following procedure to reconnect the QSFP cables Procedure 1 Press the controller enclosure s Power switch for four seconds or more to turn off the ETERNUS DX Disk storage system and turn off the AC power 2 Reconnect the QSFP cable 3 Make sure that power is being supplied then ...

Page 99: ...ment settings for the network service invalid Is the IP address incorrect Are the settings for the ETERNUS DX Disk storage system LAN ports firewall invalid If the problem is not resolved follow the procedure below to restore the default values for the ETERNUS DX Disk storage system network environment settings and the firewall settings Procedure 1 Remove the flange cover of the controller enclosu...

Page 100: ... When I O access is slow Check the following points When the cause is not identified contact your maintenance engineer Check that the ambient temperature does not exceed the operating environment conditions If so Nearline SAS drive performance may be reduced Click the icon of each part on the Storage screen of Component of ETERNUS Web GUI to check if an abnormality is detected in ETERNUS DX Disk s...

Page 101: ...eues so that the I Os are issued for the appropriate number For details on the number of command queues refer to User s Guide Server Connection Check the status of the RAID group When rebuild copyback or redundant copy is being performed in a RAID group with a High rebuild priority the performance may be reduced until the operation is complete Check the cache parameter settings of the volumes If t...

Page 102: ...ese actions The devices that are connected to the ETERNUS DX Disk storage system The models of the server and network devices The power synchronized unit model The UPS model Log dump information for the ETERNUS DX Disk storage system When your maintenance engineer asks for the log dump information of the ETERNUS DX Disk storage system obtain the required log dump information using the log dump col...

Page 103: ...ple of event notification by e mail and its contents are shown below Output example of event notification The number under each underline indicates an item in the table below A description is provided for each item Subject Message This is ETxxxxx notification message Serial No 0000000000 Information 1 2 3 user comment EVENT 3 volume s created EVENT CODE I 01030010 SEVERITY Information REPORT NO 0 ...

Page 104: ...llowing letters is output as an event type P J M I R Event code is an 8 digit hexadecimal number according to the message For more details refer to A 2 Message List page 112 7 The Severity of the event is displayed One of the following items is displayed for the message Error indicates an error Warning indicates a warning Information indicates that information is provided Test indicates a test mai...

Page 105: ...21 138 81 Storage 00 4 I 01030020 5 volume s deleted No 38 Name VOL40 0 No 39 Name VOL40 1 No 40 Name VOL40 2 No 41 Name VOL40 3 No 42 Name VOL40 4 1 2 3 4 5 6 7 8 8 8 14 1 2011 01 26T11 26 08 09 00 10 21 138 81 Storage 00 4 I 01030020 5 volume s deleted No 38 Name VOL40 0 No 39 Name VOL40 1 No 40 Name VOL40 2 No 41 Name VOL40 3 No 42 Name VOL40 4 8 8 1 2 3 4 5 6 7 No Item Description 1 Priority P...

Page 106: ...0 23 and mm indicates minutes 00 59 A blank follows this item 3 Host name The IP address of the sender port MNT or RMT for the ETERNUS DX Disk storage system is output in IPv4 or IPv6 format IPv4 xxx xxx xxx xxx The IPv4 address is output based on the IETF STD13 RFC 1035 specification IPv6 xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx The IPv6 address is output based on the RFC 4291 specification A blan...

Page 107: ...3 Trap type There are two types of Traps Generic Trap standard Trap and Specific Trap extended Trap Generic Trap is a Trap that is defined by the SNMP protocol Specific Trap is a Trap that has been created specifically for the ETERNUS DX Disk storage system The ETERNUS DX Disk storage system supports the following types of Traps Generic Trap standard Trap 7 Event type and event code The message ev...

Page 108: ...sent to notify that the link status of a CA port is changed 22 fjdarye100ItemFaultToNormal This Trap is sent to notify that an event notified by Trap No 2 returns to normal status 25 fjdarye100ItemWarningToNormal This Trap is sent to notify that an event notified by Trap No 5 returns to normal status 26 fjdarye100SensorStatusChangeToNormal This Trap is sent to notify that an event notified by Trap...

Page 109: ...ryTrapItemId Information 2 fjdarySspMachineId Information 3 fjdaryTrapSensorInfo Information 4 fjdaryTrapMessage 7 fjdarye100 Maintenance Required Notification of events that require maintenance or preventive maintenance 7 Information 1 fjdarySspMachineId Information 2 fjdaryTrapMaintenanceInfo Information 3 fjdaryTrapMessage 9 fjdarye100 Information Information notification 9 Information 1 fjdary...

Page 110: ...aintenance Required Notification of events that require maintenance or preventive maintenance 1 3 6 1 4 1 211 4 1 1 126 1 100 0 7 Information 3 fjdarySspMachineId Information 4 fjdaryTrapMaintenanceInfo Information 5 fjdaryTrapMessage 9 fjdarye100 Information Information notification 1 3 6 1 4 1 211 4 1 1 126 1 100 0 9 Information 3 fjdarySspMachineId Information 4 fjdaryTrapMessage 10 fjdarye100 ...

Page 111: ...he SNMP Agent centiseconds The operation time is reset when the SNMP Agent starts or restarts snmpTrapOID 1 3 6 1 6 3 1 1 4 1 0 Indicates the Object ID that identifies the type of Trap fjdarySspMachineId 1 3 6 1 4 1 211 1 21 1 100 1 1 0 Indicates the ID of the ETERNUS DX Disk storage system fjdaryTrapMessage 1 3 6 1 4 1 211 1 21 1 100 11 0 Indicates the message event type code blank message fjdary...

Page 112: ...nclosure Cabinets for installing drives DI Drive Interface Back end interface modules in CMs BUD Bootup and Utility Device A device in a CM in which the system area is allocated IOM I O module Units that control the connection between CM and drives EXP Expander Expander chips in a CM that are used for SAS connection SCU System Capacitor Unit A device in a CM that supplies power to back up the cach...

Page 113: ...ng maintenance operations CM is disconnected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 pp Component number ss Serial number rr Version P 02SSMM0p 2 0200MM0p Controller Module m zz DMA PORT p Fault pppppppppppp ssssssssss rrrrr CM DMA port is disconnected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 p Port ...

Page 114: ...XP is disconnected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 v EXP number 0 pp Component number ss Serial number rr Version P 08SSMMvp 2 0800MMvp Controller Module m zz DI Port P Fault pppppppppppp ssssssssss rrrrr CM DI port is disconnected SS Component subtype MM CM module ID 10 11 v Device number 0 p Device port number 0 1 m CM number 0 1 zz Model...

Page 115: ...ppppp ssssssssss rrrrr CM memory is disconnected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 x Capacity of the cache memory 2 or 4 nn Slot number of the cache memory 0 pp Component number ss Serial number rr Version P 0CSSMM0v 2 0C00MM0v Controller Module m zz BUD v xxGB Fault pppppppppppp ssssssssss rrrrr CM BUD is disconnected SS Component subtype MM...

Page 116: ... Component subtype MM CA module ID 40 7F S CA slot number 0 1 V Device number 0 1 zz CA type m CM number 0 1 pp Component number ss Serial number rr Version P 11SSMM0p 2 1100MM0p CA Slot S V zz on CM m Port P Fault pppppppppppp ssssssssss rrrrr CA port is disconnected SS Component subtype MM CA module ID 40 7F p Device port number 0 1 S CA slot number 0 1 V Device number 0 1 zz CA type m CM number...

Page 117: ...rrrr oooooooo CA SFP is disconnected MM CA module ID 40 7F p Device port number 0 1 S CA slot number 0 1 zz CA type m CM number 0 1 P Port number 0 1 pp Component number ss Serial number rr Version oo Additional information P 1C00MM0p 2 1C00MM0p FC SFP CA Slot S zz on CM m Port P Fault pppppppppppppppp ssssssssssssssss rrrr oooooooo CA SFP is disconnected MM CA module ID 40 7F p Device port number...

Page 118: ...ssssssss rrrr oooooooo CA SFP is disconnected due to an SFP type that cannot be recognized MM CA module ID 40 7F p Device port number 0 1 S CA slot number 0 1 zz CA type m CM number 0 1 P Port number 0 1 pp Component number ss Serial number rr Version oo Additional information P 1E00MM0p 2 1E00MM0p iSCSI FCoE SFP Cp CA Slot S zz on CM m Port P Fault pppppppppppppppp ssssssssssssssss rrrr oooooooo ...

Page 119: ...copy of a 3 5 SSD is complete xxx RAID group number uu DE ID nn Drive slot number 0 11 decimal xxx Drive capacity GB TB cc Drive type RAID level pp Component number ss Serial number rr Version P 21830xxx 2 21830xxx SSD 2 5 DE uu Slot nn SAS xxxGB ccccccc Redundant Copy end pppppppppppp ssssssssssss rrrr Redundant copy of a 2 5 SSD is complete xxx RAID group number uu DE ID nn Drive slot number 0 2...

Page 120: ...d when Disk Error is enabled for Error Severity Level in the event notification setting They are not notified during maintenance operations HDD is disconnected SS Component subtype uu DE ID 00 for a controller enclosure N Disk slot number 0 b hexadecimal n Disk slot number 0 11 decimal xxx Disk capacity GB TB yy Disk speed cc Disk information RAID level or hot spare Spare pp Component number ss Se...

Page 121: ...onfiguration code P 81SSuuNN 2 8000uuNN HDD 2 5 DE uu Disk nn SAS xxxGB yykrpm ccccccc Fault pppppppppppp ssssssssssss rrrr tttttt qq HDD is disconnected SS Component subtype uu DE ID 00 for a controller enclosure NN Disk slot number 0 17 hexadecimal nn Disk slot number 0 23 decimal xxx Disk capacity GB TB yy Disk speed cc Disk information RAID level or hot spare Spare pp Component number ss Seria...

Page 122: ...rsion tt Data code qq Configuration code P 84SSuu0N 2 8000uu0N SSD 3 5 DE uu Disk n SAS xxxGB ccccccc Fault pppppppppppp ssssssssssss rrrr tttttt qq SSD is disconnected SS Component subtype uu DE ID 00 for a controller enclosure N Drive slot number 0 b hexadecimal n Drive slot number 0 11 decimal xxx Drive capacity GB TB cc Drive information RAID level or hot spare Spare pp Component number ss Ser...

Page 123: ...e P 85SSuuNN 2 8000uuNN SSD 2 5 DE uu Disk nn SAS xxxGB ccccccc Failed Usable pppppppppppp ssssssssssss rrrr tttttt qq SSD is disconnected SS Component subtype uu DE ID 00 for a controller enclosure NN Drive slot number 0 17 hexadecimal nn Drive slot number 0 23 decimal xxx Drive capacity GB TB cc Drive information RAID level or hot spare Spare pp Component number ss Serial number rr Version tt Da...

Page 124: ...figuration code P 88SSdd0N 2 8000uu0N SSD 3 5 DE uu Disk n SAS xxxGB ccccccc Fault pppppppppppp ssssssssssss rrrr tttttt qq SSD is disconnected SS Component subtype uu DE ID 00 for a controller enclosure N Drive slot number 0 b hexadecimal n Drive slot number 0 11 decimal xxx Drive capacity GB TB cc Drive information RAID level or hot spare Spare pp Component number ss Serial number rr Version tt ...

Page 125: ...ent number ss Serial number rr Version P 9ASSuunp 2 9A00uunp IOM6 QSFP DE uu IOM n Port p Fault pppppppppppp ssssssssss rrrrr IOM QSFP is disconnected SS Component subtype uu DE ID 00 for a controller enclosure n IOM number 0 1 p Port number 0 1 pp Component number ss Serial number rr Version P D2SSuu0n 2 D200uu0n Power Supply Unit DE uu PSU n Fault pppppppppppp ssssssssss rrrr PSU is disconnected...

Page 126: ... operations Disconnection of a REC path between cabinets is detected MM CA module ID PP 00 or 01 xx CM number yy CA slot number 0 1 zz Port number 0 1 M 13CE0001 7 13CE0001 Error session was detected Local xxxxx Remote xxxxx These messages are notified when Copy Session Error is enabled for Error Severity Level in the event notification set ting They are not notified during maintenance operations ...

Page 127: ...rations Rebuild to the hot spare or the replaced drive is complete but Bad Data data lost identifier exists xxx RAID group number hexadecimal M A0110xxx 7 A0110xxx RAID Group 0xxx start failure of Rebuild processing These messages are notified when Rebuild Copyback w o redundant is enabled for Error Severity Level in the event notification setting They are not notified during maintenance operation...

Page 128: ...RAID Group 0xxx normal end of Rebuild processing Rebuild to the replaced drive is complete xxx RAID group number hexadecimal M A1A10xxx 7 A1A10xxx RAID Group 0xxx abnormal end of Rebuild processing because of source disk problem These messages are notified when Rebuild Copyback w o redundant is enabled for Error Severity Level in the event notification setting They are not notified during maintena...

Page 129: ...430xxx 7 A4430xxx RAID Group 0xxx start procedure of Redundant Copy processing Redundant copy is being performed xxx RAID group number hexadecimal M A4A30xxx 7 A4A30xxx RAID Group 0xxx abnormal end of Redundant Copy processing because of source disk problem Redundant copy to the replaced drive is halted because an error is found in the copy source drive xxx RAID group number hexadecimal M A4B30xxx...

Page 130: ...ons are not applied M E2070002 7 E2070002 NOT READY 02 CM F W Version Error The device is in Not Ready status Firmware version for the CM is not correct M E2070004 7 E2070004 NOT READY 04 Restore Fail The device is in Not Ready status Backup is successfully complete but the restoration cannot be performed M E207000B 7 E207000B NOT READY 11 Power Off Fail Incomplete The device is in Not Ready statu...

Page 131: ...tore Fail from System Area The device is in Not Ready status TPV table cannot be restored from the system area M E207001B 7 E207001B NOT READY 27 TPV Table Restore Fail to BUD The device is in Not Ready status TPV table cannot be restored to BUD M E8010xxx 7 E8010xxx TPP xxx turned to CAUTION decrease of available capacity These messages are notified when Thin Provisioning Pool Rate is enabled for...

Page 132: ...notified during maintenance operations There is no more available capacity in the Flexible Tier Pool xxx FTRP number hexadecimal M E928xxyy 7 E928xxyy Key Management Server communication Error Key Server ID xx Key Group ID yy This message is notified when SED Network Error is enabled for Error Severity Level in the event notification setting This is not notified during maintenance operations A com...

Page 133: ...d during maintenance operations CM error is detected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 ww Factor code pp Component number ss Serial number rr Version J 02SSMM0p 5 0200MM0p 0 Controller Module m zz DMA PORT p Warning pppppppppppp ssssssssss rrrrr CM DMA port error is detected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model ...

Page 134: ...M EXP error is detected SS Component subtype MM CM module ID 10 11 v EXP number 0 m CM number 0 1 zz Model name DX80S2 or DX90S2 pp Component number ss Serial number rr Version J 08SSMMvp 5 0800MMvp 0 Controller Module m zz DI Port P Warning pppppppppppp ssssssssss rrrrr CM DI port error is detected SS Component subtype MM CM module ID 10 11 v Device number 0 p Device port number 0 1 m CM number 0...

Page 135: ...v BUD number 0 1 m CM number 0 1 zz Model name DX80S2 or DX90S2 xx Capacity of BUD 2 pp Component number ss Serial number rr Version J 0ESSMM00 5 0E00MM00 0 Controller Module m zz NAND Controller Warning pppppppppppp ssssssssss rrrrr CM NAND Controller error is detected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 pp Component number ss Serial number rr...

Page 136: ...ppppp ssssssssss rrrrr CA FPGA error is detected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 pp Component number ss Serial number rr Version J 22SSMM0p 5 2200MM0p 0 Controller Module m zz LAN PORT p Warning pppppppppppp ssssssssss rrrrr CM LAN port error is detected SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90...

Page 137: ...u0N 5 8000uu0N 0 HDD 3 5 DE uu Disk n SAS xxxGB yykrpm ccccccc Warning pppppppppppp ssssssssssss rrrr tttttt qq HDD error is detected SS Component subtype uu DE ID 00 for a controller enclosure N Disk slot number 0 b hexadecimal n Disk slot number 0 11 decimal xxx Disk capacity GB TB yy Disk speed cc Disk information RAID level or hot spare Spare pp Component number ss Serial number rr Version tt ...

Page 138: ...uration code J 81SSuuNN 5 8000uuNN 0 HDD 2 5 DE uu Disk nn SAS xxxGB yykrpm ccccccc SMART pppppppppppp ssssssssssss rrrr tttttt qq SMART is reported from the HDD SS Component subtype uu DE ID 00 for a controller enclosure NN Disk slot number 0 17 hexadecimal nn Disk slot number 0 23 decimal xxx Disk capacity GB TB yy Disk speed cc Disk information RAID level or hot spare Spare pp Component number ...

Page 139: ...Configuration code J 81SSuuNN 5 8000uuNN 0 HDD 2 5 DE uu Disk nn SAS xxxGB yykrpm ccccccc SlowDown pppppppppppp ssssssssssss rrrr tttttt qq Disk performance error in the HDD is detected SS Component subtype uu DE ID 00 for a controller enclosure NN Disk slot number 0 17 hexadecimal nn Disk slot number 0 23 decimal xxx Disk capacity GB TB yy Disk speed cc Disk information RAID level or hot spare Sp...

Page 140: ...ion code J 85SSuuNN 5 8000uuNN 0 SSD 2 5 DE uu Disk nn SAS xxxGB ccccccc SMART pppppppppppp ssssssssssss rrrr tttttt qq SMART is reported from the SSD SS Component subtype uu DE ID 00 for a controller enclosure NN Drive slot number 0 17 hexadecimal nn Drive slot number 0 23 decimal xxx Drive capacity GB TB cc Drive information RAID level or hot spare Spare pp Component number ss Serial number rr V...

Page 141: ...u IOM n Port p Warning pppppppppppp ssssssssss rrrrr DE port error is detected SS Component subtype uu DE ID 00 for a controller enclosure p Port number 0 1 n IOM number 0 1 pp Component number ss Serial number rr Version J 9ASSuunp 5 9100uunp 0 IOM6 QSFP DE uu IOM v Port p Warning pppppppppppp ssssssssss rrrrr DE QSFP port error is detected SS Component subtype uu DE ID 00 for a controller enclos...

Page 142: ...Slot nn Correctable Error pppppppppppppppppp ssssssssssss rrrrrr Many correctable errors occurred in the CM memory SS Component subtype MM CM module ID 10 11 m CM number 0 1 zz Model name DX80S2 or DX90S2 x Capacity of the cache memory 2 or 4 nn Slot number of the cache memory 0 5 pp Component number ss Serial number rr Version J D2SSuu0n 5 D200uu0n 0 Power Supply Unit DE uu PSU n Warning pppppppp...

Page 143: ...d Level1 a and reaches b a Threshold of Snap Data Pool b Usage rate of Snap Data Pool I 00000011 9 The usage rate of SDP encryption exceeded the threshold Level1 value a and currently is at b Detail information is not displayed Snap Data Pool encrypted usage rate exceeds the threshold Level1 a and reaches b a Threshold of Snap Data Pool b Usage rate of Snap Data Pool I 00000012 9 The usage rate of...

Page 144: ... information is not displayed Usage rate of the copy table exceeds the threshold a and reaches b a Threshold of copy table b Usage rate of copy table I 00000015 9 The trial license Local Copy will expire on YYYY MM DD These messages are notified when Trial copy license expired is enabled for Informational Level in the event notification setting Detail information is not displayed The trial license...

Page 145: ...vel in the event notification setting No a Name b Level c CM d Capacity e Disk_Count f d RAID groups are registered d Quantity a RAID group number b RAID group name c RAID level d Assigned CM e Capacity f Number of drives I 01010020 9 d RAID group s deleted No a Name b d RAID groups are deleted d Quantity a RAID group number b RAID group name I 01020010 9 d hot spare s assigned These messages are ...

Page 146: ... created d Quantity a Volume number b Volume name c Volume type d Capacity e RAID group number only for Standard Open volume SDV or SDPV f RAID group name only for Standard Open volume SDV or SDPV g TPP number only for Standard Open volume or TPV h TPP name only for Standard Open volume or TPV I 01030020 9 d volume s deleted No a Name b d volumes are deleted This message is not notified when the d...

Page 147: ...ssage is notified when FC CA Port Link Status Changed is enabled for Informational Level in the event notification setting Detail information is not displayed The link status of the CA port changed to a MM CA module ID 40 7F p Device port number 0 1 a Link status LinkUp or LinkDown b CM number 0 1 c CA number 0 1 d Port number 0 1 I 03000002 9 The number of login host exceed the acceptable login c...

Page 148: ...ule m zz Normal pppppppppppp ssssssssss rrrrr These messages are notified when Recov ery module is enabled for Informa tional Level in the event notification setting Normal status is restored for a CM error R 06SSMM0v 25 0600MM0v 0 Controller Module m zz EXP e Normal pppppppppppp ssssssssss rrrrr Normal status is restored for a CM EXP error R 90SSuu0n 25 9000uu0n 0 IOM6 DE uu IOM i Normal pppppppp...

Page 149: ...tification of the SNMP Trap is performed in ETERNUS Web GUI or ETERNUS CLI A 2 6 Sense Codes This section describes the sense codes that are notified to the server Event type code Message T FFFFFFFF Test e mail Message Trap No TrapItemId TrapMessage 2 0x80000000 SNMP Trap Test The Trap No and ItemId in the test Trap are the same as the notification that is sent when a drive in Slot 0 in CE is in F...

Page 150: ...disconnected because an error occurred during the Write check of the disk patrol 04 F1 90 IOM is disconnected These sense codes are notified when Parts Error is enabled for Error Severity Level in the event notification setting They are not notified during maintenance operations 04 F1 91 IOM Port is disconnected 04 F1 9A IOM QSFP is disconnected 04 F1 D2 DPSU is disconnected 04 F1 DA CE or DE exha...

Page 151: ... in the event notification setting This is not notified during maintenance operations 06 F2 80 Redundant copy is being performed This sense code is notified when Redundant is enabled for Error Severity Level in the event notification setting This is not notified during maintenance operations 06 F2 81 Rebuild to the drive has started These sense codes are notified when Rebuild Copyback w redundant ...

Page 152: ...ty Level in the event notification setting This is not notified during maintenance operations 06 F2 8C Copyback is halted 06 F2 8D Copyback is halted because an error is found in the copy source drive 06 F2 8D Redundant copy is halted because an error is found in the copy source drive This sense code is notified when Redundant is enabled for Error Severity Level in the event notification setting T...

Page 153: ...ng They are not notified during maintenance operations 06 FB 81 SMART is reported from the 2 5 HDD 06 FB 84 SMART is reported from the 3 5 SSD 06 FB 85 SMART is reported from the 2 5 SSD 06 FB 90 DE IOM error is detected These messages are notified when Parts Warning is enabled for Warning Level in the event notification setting They are not notified during maintenance operations 06 FB 91 DE port ...

Page 154: ...rity of the message is output in the following format Priority Priority value indicates the facility and severity of the message The Priority value is calculated by Facility value 8 Severity value Facility value for messages is always 13 Severity value for messages is always 6 110 is output as a priority value for an audit log For the RFC3164 format the HEADER part follows the priority part For th...

Page 155: ...tes minutes 00 59 A blank follows this item 3 Host name The IP address of the sender port MNT or RMT for the ETERNUS DX Disk storage system is output in IPv4 or IPv6 format IPv4 xxx xxx xxx xxx The IPv4 address is output based on the IETF STD13 RFC 1035 specification IPv6 xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx The IPv6 address is output based on the RFC 4291 specification A blank follows this ite...

Page 156: ...10 Storage system name A friendly name for the ETERNUS DX Disk storage system is output If a friendly name is not specified no values are displayed A comma and then a blank follow this item 11 CM No The CM number of the controller that detects the event is output A comma and then a blank follow this item 12 Sequence number The message serial number is displayed for each controller A comma and then...

Page 157: ...D13 RFC 1035 specification IPv6 xxxx xxxx xxxx xxxx xxxx xxxx xxxx xxxx The IPv6 address is output based on the RFC 4291 specification If the storage system that performs the target event cannot be specified no values are output A comma and then a blank follow this item 20 Overview of audit log A message that indicates the event overview is output A comma and then a blank follow this item Refer to...

Page 158: ...ID1 0 RAID5 RAID5 0 RAID6 d Drive DE ID Slot number example 0000 0001 0002 No a Name b Level c Disk d 02010005 Set RAID Tuning The RAID group tuning parameters are set a RAID group number b DCMF 1 10 c Rebuild priority Low Middle High d Tuning parameter setting Enable Disable e Throttle 10 20 30 40 50 60 70 80 90 100 f Ordered cut 0 65535 No a DCMF b Rebuild_Priority c Disk_Tuning d Throttle e Ord...

Page 159: ... Pool The Thin Provisioning Pool is deleted a Pool number No a 02030006 Expand Thin provisioning Pool The Thin Provisioning Pool capacity is expanded a Pool number b Drive DE ID Slot number example 0000 0001 0002 c Assigned CM 0 1 No a Disk b CM c 02030007 Format Thin provisioning Pool The Thin Provisioning Pool is formatted a Pool number b Mode All Unformatted No a Mode b 02040001 Set Eco mode Th...

Page 160: ...on priority FTSP number Auto No a Name b Protection c Attention d Priority e 02050003 Delete Volume The volume is deleted a Volume number No a 02050004 Format Volume The volume is formatted a Volume number No a 02050005 Expand Volume The volume capacity is expanded a Volume number that is to be expanded b RAID group number in which the expanded volume belongs c Additional capacity example 1000MB d...

Page 161: ... Limit No a POOL_No b FP c PL d MWC e PSDC f SDDC g SS h SDS i Cache_Limit j 0205000B Create ODX Buffer Volume The ODX Buffer Volume is created a Volume number b Volume name c Volume type Standard TPV FTV d Capacity MB example 136704MB e RAID group number f Pool number g FTRP number h Encryption by CM Enable Disable i Attention threshold 1 200 j Priority FTSP number Auto No a Name b Type c Capacit...

Page 162: ...e n REC Stack transfer mode Enable Disable o REC Consistency transfer mode Enable Disable p REC Through transfer mode Enable Disable Port a Affinity b Connect c Rate d LoopID e Frame_Size f Host_Response g Reset_Scope h Reserve_Cancel i Port_Name j Node_Name k REC_Line_No l REC_Transfer_Mode_Sync m REC_Transfer_Mode_Stack n REC_Transfer_Mode_Consistency o REC_Transfer_Mode_Through p 02060002 Set S...

Page 163: ...r CA v CHAP for RA Enable Disable w CHAP User for RA x Header Digest Enable Disable y Data Digest Enable Disable z Jumbo Frame Enable Disable A Transfer rate 1GbAuto 10GbAuto 1GbFull 100MbFull Auto B CmdSN Count 20 40 80 120 180 Unlimited C VLAN ID Enable Disable D VLAN ID value 0 4095 E MTU size F Limit Band Width 10 400 G REC Line No 0 127 H REC Sync transfer mode Enable Disable I REC Stack tran...

Page 164: ..._Group e 02060007 Delete FC FCoE Host The FC host or the FCoE host is deleted a Host name Name a 02060009 Set SAS Host The SAS host settings are changed a Host number b Host name c SAS address d Host response name e Host group name No a Name b Address c Host_Response d Host_Group e 0206000A Delete SAS Host The SAS host is deleted a Host name Name a 0206000C Set iSCSI Host The iSCSI host settings a...

Page 165: ...1 Set Host Affinity The host affinity is set or copied a Host group name b Port group name c LUN group name d Host name e Port number Host_Group a Port_Group b LUN_Group c Host d Port e 02060012 Release Host Affinity The host affinity setting is released a Host group name b Host name c Port group name d Port number Host_Group a Port_Group c Host b Port d 02060013 Set Mapping LUN mapping is set or ...

Page 166: ...ion 00 FF or r Additional Sense Code for sense information before conversion 00 FF or s Sense Key for sense information after conversion 0 F or t Sense Code for sense information after conversion 00 FF or u Additional Sense Code for sense information after conversion 00 FF or v TPGS Mode Enable Disable w TPG Referrals Enable Disable x iSCSI Reservation Range Port System Name a No b Status c Inquir...

Page 167: ...1 Set Port QoS The bandwidth limit for the port is set a Port number example 001 or All b Band width limit 0 15 c Event number 1 7 d Schedule number 0 2047 or None Port a Limit b Event_No c Schedule_No d 02060022 Set Host QoS The bandwidth limit for the host is set a Host name b Band width limit 0 15 c Event number 1 7 d Schedule number 0 2047 or None Host a Limit b Event_No c Schedule_No d 020600...

Page 168: ...t displayed 02070006 Set SSH Public Key The SSH public key is imported or the existing SSH public key is deleted a SSH public key Key a 02080001 Set Advanced Copy License The Advanced Copy license is set Detail information is not displayed 02080002 Delete Advanced Copy License The Advanced Copy license is deleted Detail information is not displayed 02080003 Set Advanced Copy policy The pool policy...

Page 169: ...5535 Box_ID a Time b 0208000E Set REC Multiplicity The REC multiplicity is set a Box ID for the remote device b REC Multiplicity 1 1024 Auto c Priority level 1 8 Auto d Copy schedule mode Session Balancing Dst RG Balancing Box_ID a Multiplicity b Priority_Level c Sch_Mode d 0208000F Set REC Buffer The REC Buffer is set a Group number REC Buffer number b Volume type Open c Box ID for the remote dev...

Page 170: ...ress h Slave IP Link Local Address i Master Connect IP Address j Slave Connect IP Address k Length of Subnet Prefix bit 3 128 l Gateway Address m Primary DNS IP Address n Secondary DNS IP Address o Wake on LAN Enable Disable p Allowed IP Address q Allowed IP Offset 1 16 Format a Port b Speed c Master d Slave e Netmask f Master_Local g Slave_Local h Master_Connect i Slave_Connect j Prefix_Length k ...

Page 171: ... are added changed or deleted a SNMP trap number 1 10 b SNMP manager number 1 10 c IP address d SNMP version v1 v2c v3 e Community name f User name g Port number example 162 No a Manager b IP c Version d Community e User f Port g 02090009 Export Enhanced MIB The MIB file is exported Detail information is not displayed 020A0001 Set Email Notification The email notification settings are changed a Em...

Page 172: ..._Warning t Disk_Warning u Temperature_Warning v Battery_Life_Warning w Recovery_Module x Temperature_Restoration y User_Logon_Logoff z Operate_Raid_Group A Add_Release_Hot_Spare B Operate_Volume C Power_Off_On_Cfl D Sdp_Policy_Level1 E Sdp_Policy_Level2 F Sdp_Policy_Level3 G Limit_Copy_Table H Expire_Trial_Copy_License I ODX J SED_Network_Recovered K Change_FC_CA_Port_Link_Status L Limit_Host_Logi...

Page 173: ... l Sec_Secret m Sec_Timeout n 020B0005 Set Syslog The syslog notification settings are changed a Log sending function for syslog server 1 RFC3164 RFC5424 Disable b IP address for syslog server 1 c Port that is used by syslog server 1 MNT RMT d Port number for syslog server 1 e Log sending function for syslog server 2 RFC3164 RFC5424 Disable f IP address for syslog server 2 g Port that is used by s...

Page 174: ... or None Name a Machine_Group_Name b Security_Level c Recovery_Mode d Period e Master_ID f Slave_ID g 020B000C Delete SED Key Group The key group is deleted 020B000D Change SED Key The SED authentication key is changed a Mode Reuse Delete New Mode a 020B000E Recover SED Key Group The key group that is in locked status can now be accessed 020C0001 Set Date The date time and time zone of the storage...

Page 175: ...able Disable c Highland Mode Enable Disable d Read Sequential Enable Disable e Write Sequential Enable Disable f Reject INQUIRY from Unauthorized Host Enable Disable g Critical Disk Mode Enable Disable h Disk Media Error Check Mode1 Mode2 i Read Compare Mode for Online Disks 0 255 j Read Compare Mode for Nearline Disks 0 255 k All LBAs Enable Disable l Thin Provisioning Allocation Mode TPP Balanci...

Page 176: ... d Interval e Event_Notification f 020C000E Set Debug Mode The debug mode is set a Trace level example 00 FF b Collection mode of the panic dump Nose and Tail Fortnight Disable Trace_Level a Panic b 020C000F Start Performance Performance information collection is started a Collection interval sec 30 60 90 120 150 180 210 240 270 300 Interval a 020C0010 Stop Performance Performance information coll...

Page 177: ...o b 020C0020 Clear Diagnosis The diagnosis is complete a Type Disk RAID Group b RAID group number Type a RAID_No b 020C0021 Set LED The LED of the target component is turned on blinked or turned off a Target component CE Disk b Drive DE ID Slot number example 0000 0001 0002 c LED ON OFF Target a Disk b LED c 020C0022 Set Event Parameter The LED display turning on or blinking settings when an event...

Page 178: ... Automatic Download Enable Disable c Download Schedule Same as Connection Period Once per Week d Download Date example Sunday 00 00 e Automatic Controller Firmware Update Enable Disable f Bandwidth Limit Enable Disable g Download speed Kbit s 100 999 Destination a Auto_DownLoad b Download_Schedule c Download_Date d Auto_Update e Bandwidth_Limit f Speed g 020D0007 Remote Download Controller Firmwar...

Page 179: ...on is not displayed 020E0002 Stop Maintenance The maintenance operation is stopped Detail information is not displayed 020E0003 Forced Degradation The component is forcibly disabled a Component example cm0 ca0 Component a 020E0004 Forced Online The component is forcibly enabled a Component example cm0 ca0 Component a 020E0005 Preventive Maintenance Preventive maintenance of the component is perfor...

Page 180: ...d a Product ID b Firmware version c Drive DE ID Slot number example 0000 0001 0002 d Apply Mode Offline Online Product a Revision b Disk c Application_Type d 020F0006 Delete Disk Firmware The disk firmware is deleted a Product ID Product a 02100001 Clear Log The log is deleted Detail information is not displayed 02100002 Clear Panic Dump The panic dump is deleted Detail information is not displaye...

Page 181: ...er b Flexible Tier Sub Pool name c Flexible Tier Pool number d Drive type Online Nearline SSD SED e RAID level RAID0 RAID1 RAID1 0 RAID5 RAID5 0 RAID6 f Drive DE ID Slot number example 0000 0001 0002 g Assigned CM 0 1 h Stripe Depth 64KB 128KB 256KB 512KB 1024KB No a Name b FTRP_No c Attribute d Level e Disk f CM g Stripe_Depth h 02110007 Set Flexible Tier Sub Pool The Flexible Tier Sub Pool setti...

Page 182: ...N hexadecimal number example 0x0000 0x8000000000000000 No a LUN b 02120003 Restart Storage Migration Storage Migration is restarted a Path group number b LUN hexadecimal number example 0x0000 0x8000000000000000 No a LUN b 02120004 Stop Storage Migration Storage Migration is stopped a Path group number b LUN hexadecimal number example 0x0000 0x8000000000000000 No a LUN b 02120005 Delete Storage Mig...

Page 183: ... Severity SEVERITY Details SUMMARY specific trap No Meaning variable bindings 1 for s x Item fault CRITICAL Item fault event from s id x message s 2 A component has failed or degraded s fjdarySspMachineId x fjdaryTrapItemId s fjdaryTrapMessage Item warning MAJOR Item warning event from s id x x message s 5 A component requiring preventive maintenance is detected s fjdarySspMachineId x fjdaryTrapIt...

Page 184: ...ault to normal from s id x message s 22 An event notified by Item fault returns to normal status s fjdarySspMachineId x fjdaryTrapItemId s fjdaryTrapMessage Item warning to normal 2 INFORMATIONAL Item warning to normal from s id x x message s 25 An event notified by Item warning returns to normal status s fjdarySspMachineId x fjdaryTrapItemId x fjdaryTrapWarningInfo s fjdaryTrapMessage Sensor stat...

Page 185: ...he utmost attention to detail However Fujitsu shall assume no responsibility for any operational problems as the result of errors omissions or the use of information in this manual Fujitsu assumes no liability for damages to third party copyrights or other rights arising from the use of any information in this manual The content of this manual may not be reproduced or distributed in part or in its...

Page 186: ......

Reviews:

No comments

Brands by name

Popular brands

Load more brands