Oracle Sun ZFS Service Manual Download Page 69

Page: 69 / 146

Lift the riser up to remove it from the storage controller.

Carefully remove the PCIe card from the riser board connector and clean the slot with

filtered, compressed air if necessary.

Seat the replacement PCIe card in the slot of the riser and connect the cables.

Align the riser, together with any attached PCIe cards, with the intended location on the
motherboard, and carefully insert it into its slot.

Slide the back of the riser into the motherboard rear panel stiffener.

Tighten the screw that secures the riser to the motherboard.

7x20

Chapter 2 • Hardware Maintenance

Summary of Contents for Sun ZFS

Page 1: ...Sun ZFS Storage 7x20 Appliance Customer Service Manual Part No E26399 December 2011 E26399 02 ...

Page 2: ...s costs or damages incurred due to your access to or use of third party content products or services Ce logiciel et la documentation qui l accompagne sont protégés par les lois sur la propriété intellectuelle Ils sont concédés sous licence et soumis à des restrictions d utilisation et de divulgation Sauf disposition de votre contrat de licence ou de la loi vous ne pouvez pas copier reproduire trad...

Page 3: ...aintenance 23 Introduction 23 7120 24 7120 Hardware Overview 24 Chassis Overview 25 Electrical Specifications 27 Internal Components 27 Standalone Controller Configurations 32 Attached Storage 34 7320 34 7320 Hardware Overview 34 Chassis Overview 34 7320 Replaceable Components 40 7320 Single and Cluster Controller Configurations 41 7420 43 3 ...

Page 4: ...ks 54 SAS2 80 Sun Disk Shelf Overview 80 Shelf 86 Disk Shelf Maintenance Procedures 86 Prerequisites 86 Safety Information 86 Electrostatic Discharge Precautions 87 Tasks 87 Faults 93 Hardware Faults 93 HBA Expansion pt 1 95 Expanding from 2 to 3 HBAs 95 HBA Expansion pt 2 100 Expanding from 3 to 4 HBAs 100 3 System Maintenance 105 System 105 Introduction 105 System Disks 105 Support Bundles 106 I...

Page 5: ...27 Received Properties 127 Received Properties Deferred Update 127 Slim ZIL 128 Introduction 128 Snapshot Deletion 128 Snapshot Deletion Deferred Update 128 Recursive Snapshots 128 Recursive Snapshots Deferred Update 128 Multi Replace 129 Multi Replace Deferred Update 129 RAIDZ Mirror 129 RAIDZ Mirror Deferred Update 129 Optional Child Dir 129 Introduction 129 ConfigurationBackup 130 Configuration...

Page 6: ...Introduction 136 BUI 138 CLI 138 Glossary 141 Index 145 Contents Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 6 ...

Page 7: ...is Book These notes are for users and system administrators who service and use the Sun ZFS Storage 7x20 Appliances Related Documentation Refer to the following documentation for installation instructions hardware overviews service procedures and software update notes Installation Guide Analytics Guide and Administration Guide http www oracle com technetwork documentation Access to Oracle Support ...

Page 8: ...8 ...

Page 9: ...o clients over a network and a rich set of data services that can be applied to the data stored on the system Platforms 7120 7320 7420 Legacy platforms are documented in the 7110 7210 7310 7410 J4400 J4500 sections of the Sun Storage 7000 Unified Storage System Service Manual Expansion Storage Sun Disk Shelf 1 C H A P T E R 1 9 ...

Page 10: ... configure your Sun ZFS Storage system using the built in collection of advanced data services including LICENSE NOTICE Remote Replication and Cloning may be evaluated free of charge but each feature requires that an independent license be purchased separately for use in production After the evaluation period these features must either be licensed or deactivated Oracle reserves the right to audit ...

Page 11: ...uble and triple parity and optional RAID 6 across disk shelves Active active clustering for high availability 7310 7320 7410 and 7420 Link aggregations and IP multipathing for network failure protection I O Multipathing between the controller and disk shelves Integrated software restart of all system software services Phone Home of telemetry for all software and hardware issues Lights out Manageme...

Page 12: ...epetitive tasks The following sections describe details of the CLI When navigating through the CLI there are two principles to be aware of Tab completion is used extensively if you are not sure what to type in any given context pressing the Tab key will provide you with possible options Throughout the documentation pressing Tab is presented as the word tab in bold italics Help is always available ...

Page 13: ...strations that enable you to browse through the appliance and attached disk shelf components The screenshot at the top of this page shows a disk highlighted in a Sun Storage 7110 showing both its physical location and details The buttons in the hardware view are icon description icon description Show a more detailed view of this component Toggle blinking of the locator LED for this component Leave...

Page 14: ...cator notes if there are any faulted components within the chassis and the name of the chassis The chassis name is initially set to the appliance name during installation To change the chassis name use the entry field on the Configuration Services System Identity screen At the top right of the system chassis is the control to light the locate LED reboot the appliance and power off the chassis A th...

Page 15: ...shelves If there are no cache disks present will be displayed Log Size and number of log disks in the system chassis This is only valid for standalone systems If there are no log devices present will be displayed Total Total size and count of all disks in the system Disk Shelves A list of disk shelves if supported is displayed at the bottom of the view The thumbnail to the left represents the fron...

Page 16: ... within the disk shelf are not currently configured as part of a storage pool complete path information will not be available though it displays two paths to the chassis Locate Toggle the locate LED for this disk shelf If the LED is currently on then this indicator will be flashing Chassis Detail To view the chassis details click on the icon or one of the alternative forms described above This vie...

Page 17: ...nent This is an internal identifier used to identify the component within faults and is intended for service personnel Active Problems For a faulted component links to active problems affecting the component Manufacturer Component manufacturer Model Component model Build Manufacturing build identifier This is used to identify a particular location or batch where the component was manufactured Part...

Page 18: ...g event Online Online the disk Reverses the above operation Infiniband Host Controller Adapters Infiniband Host Controller Adapters HCA report additional properties for the list of available ports Action Description State When active the active port icon is displayed Other valid port states down init and arm are denoted by the inactive port icon Mousing over the port icon will display the current ...

Page 19: ... 000 CPU 0 ok AMD Quad Core AMD Op cpu 001 CPU 1 ok AMD Quad Core AMD Op cpu 002 CPU 2 ok AMD Quad Core AMD Op cpu 003 CPU 3 ok AMD Quad Core AMD Op disk 000 HDD 0 ok STEC MACH8 IOPS disk 001 HDD 1 ok STEC MACH8 IOPS disk 002 HDD 2 absent disk 003 HDD 3 absent disk 004 HDD 4 absent disk 005 HDD 5 absent disk 006 HDD 6 ok HITACHI HTE5450SASUN500G disk 007 HDD 7 ok HITACHI HTE5450SASUN500G fan 000 F...

Page 20: ...cturer model Model offline writeable status indicator part Part present status indicator revision Revision serial Serial size Size speed Speed type combined with use use Type When viewing a disk that is active as a hot spare the detach command is also available Viewing CPU details For example the following shows details for component CPU 0 tarpon maintenance hardware select chassis 000 tarpon main...

Page 21: ...s physically located Optionally click the icon for that component if the component has it The LED on the component will begin to flash CLI To turn on the locate LED using the CLI run the following commands Go to the maintenance hardware context hostname maintenance hardware List the appliance components hostname maintenance hardware list NAME STATE MODEL SERIAL chassis 000 hostname ok Sun Storage ...

Page 22: ...k ST3500630NS ST3500630NS 9QG1AC27 disk 012 HDD 12 ok ST3500630NS ST3500630NS 9QG1AC41 disk 013 HDD 13 ok ST3500630NS ST3500630NS 9QG1ACQ5 disk 014 HDD 14 ok ST3500630NS ST3500630NS 9QG1ACKA disk 015 HDD 15 ok ST3500630NS ST3500630NS 9QG1AC5Y disk 016 HDD 16 ok ST3500630NS ST3500630NS 9QG1ACQ2 disk 017 HDD 17 ok ST3500630NS ST3500630NS 9QG1A76S disk 018 HDD 18 ok ST3500630NS ST3500630NS 9QG1ACDY d...

Page 23: ...are Maintenance list The Maintenance Hardware screen of the BUI provides visual representations of the physical system components allowing you to visually identify and locate hardware components and verify their status Software Updates can be applied in the System section of the interface as well as viewing Logs and current Problems Hardware identify hardware components and verify their status 712...

Page 24: ...w 7210 Overview 7310 Overview 7410 Overview 7x10 Maintenance Procedures J4400 J4500 Overview System view system disks manage support bundles Updates manage appliance software updates Configuration Backup backup and restore appliance configuration Problems view current problems Logs view appliance logs Workflows manage and execute workflows 7120 7120 Hardware Overview Use the information in this se...

Page 25: ...rvers storage admin fbsanboot 365291 html for details on FC SAN boot solutions using the Sun ZFS Storage 7120 Appliance The 7120 is a standalone controller that consists of an internal SAS 2 HBA providing disk shelf expansion write flash acceleration and 11 x 300GB 15K 600GB 15K 1TB 7 2K 2TB 7 2K or 3TB 7 2K hard drive storage The SAS 2 storage fabric supports greater number of targets greater ban...

Page 26: ...cated in slots 1 2 or 3 Note that 375 3606 01 HCA expansion cards are not supported in the 10Gb network configurations Figure Legend 1 Power Supply Unit 1 6 PCIe 4 11 Network Management port 2 Power Supply Unit 0 7 Boot HDD 1 12 Gbit Ethernet ports NET 0 1 2 3 3 PCIe 0 8 Boot HDD 0 13 USB 2 0 ports 0 1 4 PCIe 3 9 Rear Panel System Status LEDs 14 HD15 Video port 5 PCIe 1 10 Serial Management port T...

Page 27: ...power consumption of the appliance Connectors Two C13 connectors which work on 110 220v outlets Input Nominal frequencies 50 60Hz Nominal voltage range 100 120 200 240 VAC Maximum current AC RMS 13 8A 100 VAC AC operating range 90 264 VAC Output 3 3 VDC STBY 3 0A 12 VDC 86 7A Power dissipation Max power consumption 1235 3 W Max heat output 4212 BTU hr Volt Ampere rating 1261 VA 240 VAC 0 98P F Int...

Page 28: ...erlock kill switch The power supplies connect directly to the power distribution board Connector Break Out Board The Connector Break Out Board is FRU and serves as the interconnect between the power distribution board and the fan power boards storage drive backplane and I O board It also contains the top cover interlock kill switch Fan Power Boards The two fan power boards are FRUs and carry power...

Page 29: ...e 7120 system are shown in the following figure and described in the table Figure Legend 1 Top Cover 3 Hard Disk Drives 2 Right Control Panel Light Pipe Assembly 4 Left Control Panel Light Pipe Assembly 7120 Chapter 2 Hardware Maintenance 29 ...

Page 30: ...nd table Note that the rear boot drives are not depicted in this illustration Cable Connection 1 Storage Drive Data Cable Connection between the HBA PCI Express Card and the storage drive backplane 7120 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 30 ...

Page 31: ...DR3 DIMMs The 7120 CPU and memory should only be replaced by field service professionals All sockets must be occupied by either a filler or a DDR3 DIMM All DDR3 DIMMs must be identical DIMMs are pre installed in P0 slots D1 D2 D4 D5 D7 and D8 Power Distribution Fan Module and Disk Components The fan modules and LEDs are shown in the following illustration 7120 Chapter 2 Hardware Maintenance 31 ...

Page 32: ...The following table shows the configuration options for a 7120 controller All PCIe cards are low profile and must be fitted with low profile mounting brackets This table describes base configurations for the 7120 with Aura Logzilla Mktg Part Number Description Mfg Part Number TA7120 12TB S7120 1xCPU 24GB 12TB 597 0754 01 TA7120 24TB S7120 1xCPU 24GB 24TB 597 0755 01 This table describes base confi...

Page 33: ... Copper NIC Optional Recommended Front end 0 PCIe 375 3617 01 Intel Niantic DP Optical 10GE NIC Optional Recommended Front end 0 PCIe 371 4325 01 QLogic 8Gb DP FC HBA Optional FC Target or Initiator Backup 0 PCIe 375 3606 01 Mellanox MHJH29 XTC InfiniBand HCA Optional Recommended Front end 1 PCIe 375 3617 01 Intel Niantic DP Optical 10GE NIC Optional Recommended Front end 1 PCIe 375 3606 01 Mellan...

Page 34: ...ont end support tape backup Infiniband and dual port SAS HBA for storage expansion The CPUs are Intel Xeon 5600 series 2 40GHz 80W 4 core processors Standard memory configuration is 24GB 3 x 4GB DDR3 1333 DR DIMMs and may be upgraded to 48GB using 6x4GB DDR3 1333 DR DIMMs per CPU or to 72GB using 9x4GB DDR3 DR DIMMs per CPU The clustered configuration simply uses two servers and a cluster card in ...

Page 35: ... from the power supplies to the rest of the storage controller It is directly connected to the paddle board and to the motherboard through a bus bar and ribbon cable It also supports a top cover interlock kill switch Paddle board The paddle board is a FRU and serves as the interconnect between the power distribution board and the fan power boards hard drive backplane and I O board Fan Board This b...

Page 36: ...30 4431 01 Cable Fan data FRU internal F530 4417 01 FRU Cable Fan paddle FRU internal F530 3880 01 Cable Assembly Ethernet Shielded RJ45 RJ45 6m CRU external F530 3883 01 FRU 2M 4X Mini SAS Cable CRU external 7320 I O Components The I O components of the storage controller are shown in the following figure and identified in the table 7320 Sun ZFS Storage 7x20 Appliance Customer Service Manual Dece...

Page 37: ...iption FRU CRU F371 4898 01 DIMM 4GB DDR3 2RX4 13 FRU F371 4885 01 Intel E5620 2 40G FRU The storage controller motherboard has 18 slots in two groups that hold industry standard DDR3 DIMM memory cards All sockets must be occupied by either a filler or a DDR3 DIMM 7320 Power Distribution and Fan Module Components The Power Distribution Fan Module components of the storage controller are shown in t...

Page 38: ...he maximum rated power numbers for the power supply The numbers are not a rating of the actual power consumption of the appliance Connectors Two C13 connectors which work on 110 220v outlets Input Nominal frequencies 50 60Hz Nominal voltage range 100 120 200 240 VAC Maximum current AC RMS 9 0 amps Max AC operating range 90 264 VAC Output 3 3 VDC STBY 3 6A 12 VDC 62 3A 7320 Sun ZFS Storage 7x20 App...

Page 39: ...3 Power OK LED green 4 Power Button 5 Rear Power Supply 6 System Overtemperature LED 7 Top Fan The following graphic illustrates the 7320 front panel with drive locations described in the table Two mirrored hard disk drives HDDs that store the operating system reside in slots 0 and 1 Up to four solid state drives ReadZilla SSDs that store the read cache fill slots 2 through 5 in order Slots 6 and ...

Page 40: ... F542 0330 01 2 5 512GB ReadZilla SSD CRU F541 276 01 ASSY FAN Module CRU F541 4274 02 Fan Board 1U FRU 7320 PCIe Cards and Risers Following is the complete list of replaceable PCIe cards for the 7320 system Part Number Description FRU CRU F371 4325 01 8Gb FC HBA PCIe CRU F375 3609 02 PCA SAS 6GBS 8 Port PCIe CRU F375 3606 03 Dual Port x4 IB HCA PCIe CRU F375 3696 01 Dual Port CX2 4XQDR PCIe CRU F...

Page 41: ...escribes base configurations for the 7320 Mktg Part Number Description Mfg Part Number TA7320 24A S7320 2xCPU 24GB Single 597 1060 01 TA7320 24A HA S7320 2xCPU 24GB Cluster 597 1061 01 Following are the PCIe configuration options for a single controller All PCIe cards are low profile and must be fitted with low profile mounting brackets Slot Type Part Number Vendor Part Description Note 0 PCIe 375...

Page 42: ...cluster The following options are available for clustered storage controllers Note that both storage controllers in a cluster must be configured identically with regard to card configurations and all optional NIC HBA card configurations chosen for clustered storage controllers must be identical in both chassis Slot Type Part Number Vendor Part Description Note 0 PCIe 375 3617 01 Intel Niantic DP O...

Page 43: ...in of 1 to 4 Sun Disk Shelves Any combination of disk only and Logzilla capable shelves may be combined within the chain in any order The cabling configurations are unchanged Half populated shelf configurations are not supported See Also Disk Shelf Overview Disk Shelf Details Controller Details 7420 7420 Hardware Overview Use the information on this page as a preparation reference for servicing re...

Page 44: ...ension Measurement Height 13 3 cm 5 25 in Depth 70 6 cm 27 8 in Width 43 7 cm 17 19 in Weight 16 36 kg 96 lbs Front Panel Figure Legend 1 Locator LED and button white 7 Power Supply PS Service Required LED 13 Boot drive 1 required 2 Service Required LED amber 8 Over Temperature Warning LED 14 Solid state drive 2 optional 3 Power OK LED green 9 USB 2 0 Connectors 15 Solid state drive 3 optional 4 P...

Page 45: ... The following graphic shows the 7420 rear panel Base configuration HBAs are not depicted in this illustration Figure Legend 1 Power supply unit 0 status LEDs 8 Network NET 10 100 1000 ports NET0 NET3 2 Power supply unit 0 AC inlet 3 Power supply unit 1 status LEDs 10 PCIe slots 5 9 4 Power supply unit 1 AC inlet 11 Network management NET MGT port 5 System status LEDs Power green Attention amber L...

Page 46: ...terlock kill switch In the storage controller the power supplies connect to the power supply backplane which connects to the power distribution board Vertical PDB Card The vertical power distribution board or Paddle Card is a FRU and serves as the interconnect between the power distribution board and the fan power boards hard drive backplane and I O board Power Supply Backplane Card This board con...

Page 47: ...emory risers required by each CPU Four or eight 4GB or 8GB DDR3 DIMMs are installed on each riser accommodating up to 256GB of memory for two CPUs or up to 512GB for four CPUs Empty CPU sockets must have memory riser fillers installed for proper cooling The new 7420 controller has different CPU options and memory risers but is visually identical to the existing 7420 controller with 1 86GHz or 2 00...

Page 48: ...ght 16GB DDR3 DIMMs installed on each riser accommodating 512GB or 1TB of memory for 2 4Ghz CPUs Refer to the service label on the cover for DIMM placement information On every memory riser slots D0 D2 D4 and D6 must be populated optionally slots D1 D3 D5 and D7 may be populated as a group on all installed memory risers All DIMMs in the system must be identical 7420 Sun ZFS Storage 7x20 Appliance ...

Page 49: ...own in the following figure The following LEDs are lit when a fan module fault is detected Front and rear Service Action Required LEDs Fan Module Service Action Required TOP LED on the front of the server Fan Fault LED on or adjacent to the faulty fan module The system Overtemp LED might light if a fan fault causes an increase in system operating temperature 7420 Chapter 2 Hardware Maintenance 49 ...

Page 50: ...10 100 1000 Mbit sec These network interfaces must be configured before use 7420 Standalone and Cluster Controller Configurations The following tables show the configuration options for a single standalone 7420 controller or two clustered 7420 controllers All PCIe cards are low profile and must be fitted with low profile mounting brackets Standalone Base Options This table describes 7420 standalon...

Page 51: ... HBA 8Gb PCIe 594 5684 01 X4446A Z 4 port PCIe Quad GigE UTP 594 4024 01 X4237A 2 port 4X IB HCA PCIe 594 5862 02 X1109A Z 2 port 10Gig SFP NIC PCIe 594 6039 01 X2129A XCVRm 850NM 1 10GPS Short Reach SFP 594 6508 01 PCIe Options This table describes the supported single and clustered PCIe configuration option summary for 7420 Slot Type Sun Part Number Description Note 0 PCIe 371 4325 01 8Gb DP FC ...

Page 52: ...1 DP Optical 10GE NIC Optional Recommended Front end 6 PCIe 375 3481 01 QP Copper NIC Optional Recommended Front end 6 PCIe 371 4325 01 8Gb DP FC HBA Optional FC Target or Initiator Backup 6 PCIe 375 3606 01 InfiniBand HCA Optional Recommended Front end 6 PCIe 375 3617 01 DP Optical 10GE NIC Optional Recommended Front end 7 PCIe 375 3609 02 DP SAS 2 HBA Additional Optional Back end 7 PCIe 375 3481...

Page 53: ...ormation in the overview section for your model to become familiar with the replaceable parts of the system 7120 Overview 7320 Overview 7420 Overview Caution Read and follow the instructions in the Safety Information and Required Tools and Information sections Safety Information This section contains safety information that you must follow when servicing the storage system For your protection obse...

Page 54: ...area on the chassis Following this practice equalizes the electrical potentials between you and the storage controller Antistatic mat Place static sensitive components on an antistatic mat No 2 Phillips screwdriver Nonconducting No 1 flat blade screwdriver or equivalent Nonconducting stylus or pencil to power on the storage controller Chassis Serial Number To obtain support for your storage contro...

Page 55: ...k the drive in place The Sun ZFS Storage system software automatically detects and configures the new drive The device appears in the BUI Maintenance Hardware screen when you view details for the controller drive shelf or cage Fan Module Fan modules are hot swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities Caution Opera...

Page 56: ...click the associated locate icon on the Maintenance Hardware screen of the BUI or issue the set SYS LOCATE status on command at the service processor SP prompt The locate LED will flash on the controller chassis 2 Verify that no cables will be damaged or will interfere when the storage controller is extended from the rack 3 From the front of the storage controller release the two slide release lat...

Page 57: ...ply firm pressure to fully seat the fan module Verify that the Fan OK status indicator is lit and that the fault status indicator on the replaced fan module is dim Close the top cover door immediately after replacing the fan to maintain airflow in the storage controller 6 7420 To replace the fan module Identify the faulted fan module by locating the corresponding Service Required status indicator ...

Page 58: ...m 8 Push the release tabs on the side of each rail and slowly slide the storage controller into the rack Power Supply Storage controllers are equipped with redundant hot swappable power supplies If a power supply fails and you do not have a replacement leave the failed power supply installed to ensure proper air flow A faulted power supply is indicated by an amber colored status LED 1 Gain access ...

Page 59: ...20 or 7320 Release the latch then remove the power supply Components may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 7420 Grasp the power supply handle and press the release latch to remove the power supply 5 Align the replacement power supply with the empty power supply chassis bay 7x20 Chapter 2 Hardware Maintenance ...

Page 60: ... sensitive to static discharge which can cause the component to fail To avoid damage wear an antistatic wrist strap and use an antistatic mat when handling components You must shut down the appliance before beginning this task Note that there will be a loss of access to the storage unless the system is in a clustered configuration Shut down the appliance using one of the following methods Log in t...

Page 61: ...omponents may differ slightly between the 7120 and 7320 however service procedures for each are identical The illustration shows the 7320 To remove the top cover Unlatch the fan module door pull the two release tabs back rotate the fan door to the open position and hold it there Press the top cover release button and slide the top cover to the rear about a half inch 1 3 cm Lift up and remove the t...

Page 62: ...remove the top cover 7 To locate the DIMM you want to service press the Fault Remind Button on the storage controller The following illustration shows the Fault remind button on the 7120 7x20 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 62 ...

Page 63: ...DIMM by the Service Required status indicator Lift the memory riser straight up to remove it from the motherboard and place it on an antistatic mat 9 Rotate both DIMM slot ejectors outward as far as they will go and carefully lift the faulted DIMM straight up to remove it from the socket 7x20 Chapter 2 Hardware Maintenance 63 ...

Page 64: ...er service procedures for each are identical The illustration shows the 7320 To replace the cover Place the top cover on the chassis so that it hangs over the rear of the storage controller by about an inch 2 5 cm Slide the top cover forward until it seats Close the fan cover and engage the fan cover latches The cover must be completely closed for the storage controller to power on 13 7420 To repl...

Page 65: ...in place Place the top cover on the chassis so that it is forward of the rear of the storage controller by about an inch 2 5 cm Slide the top cover toward the rear of the chassis until it seats and press down on the cover with both hands until both latches engage 7x20 Chapter 2 Hardware Maintenance 65 ...

Page 66: ...lashing on the front panel about two minutes after the power cords are plugged in 17 Use a pen or other pointed object to press and release the recessed Power button on the storage controller front panel The Power OK status indicator next to the Power button lights and remains lit The Maintenance Hardware screen of the BUI provides status of the replacement on the Details page for DIMMs 7x20 Sun Z...

Page 67: ...d SSH or serial console into the service processor SP and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on the front panel To initiate emergency shutdown wherein all applications and files will be closed abruptly without saving press and hold the power button for at least four seconds until the Power OK status indicator on the front pan...

Page 68: ...the PCIe card Disconnect any data cables connected to the cards on the PCIe riser you want to replace Label the cables for proper connection later Loosen the two captive Phillips screws on the end of the rear panel crossbar and lift the crossbar up and back to remove it Loosen the captive retaining screw holding the front end of the riser and the Phillips screw on the end of the riser 7x20 Sun ZFS...

Page 69: ...eat the replacement PCIe card in the slot of the riser and connect the cables Align the riser together with any attached PCIe cards with the intended location on the motherboard and carefully insert it into its slot Slide the back of the riser into the motherboard rear panel stiffener Tighten the screw that secures the riser to the motherboard 7x20 Chapter 2 Hardware Maintenance 69 ...

Page 70: ...to an upright position Remove the retaining screw that holds the PCIe card to the chassis Carefully remove the PCIe card from the connector and clean the slot with filtered compressed air if necessary Install the replacement PCIe card into the PCIe card slot Install the retaining screw to hold the PCIe card to the chassis Return the crossbar to its closed and locked position 7x20 Sun ZFS Storage 7...

Page 71: ...fan cover latches The cover must be completely closed for the storage controller to power on 11 7420 To install the top cover Place the top cover on the chassis 1 so that it is forward of the rear of the storage controller by about an inch 2 5 cm Slide the top cover toward the rear of the chassis 2 until it seats Press down on the cover with both hands until both latches engage 12 Push the release...

Page 72: ...ustered configuration Shut down the appliance using one of the following methods Login to the BUI and click the power icon on the left side of the masthead SSH into the storage system and issue the maintenance system poweroff command SSH or serial console into the service processor and issue the stop SYS command Use a pen or non conducting pointed object to press and release the Power button on th...

Page 73: ...ove the top cover 6 7420 To remove the top cover Simultaneously lift both cover latches in an upward motion 7 Lift up and remove the top cover 8 Using a small non metallic screwdriver press the latch and remove the battery from the motherboard The 7420 battery is shown here 7x20 Chapter 2 Hardware Maintenance 73 ...

Page 74: ... it hangs over the rear of the storage controller by about an inch 2 5 cm then slide the top cover forward until it seats Close the fan cover and engage the fan cover latches The cover must be completely closed for the storage controller to power on 11 7420 To install the top cover Place the top cover on the chassis 1 so that it is forward of the rear of the storage controller by about an inch 2 5...

Page 75: ...anel The Power OK status indicator next to the Power button lights and remains lit 16 Connect data cables to the PCIe card routing them through the cable management arm 17 When the system has finished booting log in and set the time using the steps in the BUI Clock task HDD or SSD The installation for an hard disk drive HDD or a solid state drive SSD is the same Note SSDs are restricted to slots 3...

Page 76: ...age FanTray Fan trays are hot swappable and can be removed and installed while the storage controller is running without affecting other hardware capabilities An amber colored status indicator indicates a faulted fan module Caution Fan trays must not be removed from the cabinet for more than one minute Removal of a fan tray for more than one minute will result in the cabinet overheating and automa...

Page 77: ...us indicator Service Required status indicators and the Locator status indicator Locator button are dim Power Supply Storage controllers are equipped with redundant hot swappable power supplies If a power supply fails and you do not have a replacement leave the failed power supply installed to ensure proper air flow A faulted power supply is indicated by an amber colored status LED The power suppl...

Page 78: ...l indicator boards are located on the front and rear panels under the top disk cage Both indicator boards can be hot swapped while the system is running Both boards contain the following system level LEDs Locator white which helps you locate the storage system within the data center Fault amber which indicates a system level fault OK green which indicates that the storage system is operating norma...

Page 79: ...el OK 6 Rear panel RCC fault In addition to system level indicators the rear panel indicator board also includes LEDs for components on the front panel Rear Panel LED Indicators 1 System level locator 4 Front panel drive set fault 2 System level fault 5 Front panel power supply fault 3 System level OK 7x20 Chapter 2 Hardware Maintenance 79 ...

Page 80: ...interface connector 6 Push the board into the slot and ensure that the board is fully seated in the slot 7 Verify that the procedure was successful by logging in to the Sun ZFS Storage appliance software checking the system status and clicking the associated locator icon in the Cabinets section of the Maintenance Hardware BUI page SAS2 Sun Disk Shelf Overview The Sun Disk Shelf SAS 2 is a high ava...

Page 81: ... Storage 7000 series with a standard chip set to support a high density of target devices capable of attachment to 1024 targets With this new fabric you are encouraged to apply entire shelves to pools so you can get the benefits of No Single Point of Failure NSPF configurations and striping across the maximum possible number of devices Part Number Description FRU CRU F542 0194 01 SAS Interface Mod...

Page 82: ...wer input connector 4 Power supply status indicator 8 Right ejection arm and captive screw latch Drive Locations The following table shows the shelf drive locations Drive Locations SSD20 SSD21 SSD22 SSD23 HDD16 HDD17 HDD18 HDD19 HDD12 HDD13 HDD14 HDD15 HDD8 HDD9 HDD10 HDD11 SAS2 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 82 ...

Page 83: ...lt indicator 5 Disk fault indicator 8 SIM board fault indicator 3 System power indicator 6 Disk activity indicator 9 Power supply fault indicator Sun Disk Shelf Logzilla SSD Locations Up to four Logzilla SSDs are supported per disk shelf Logzilla SSDs should be populated in order of slots 20 21 22 and 23 Drive Assembly Exterior The following figure shows the drive assembly exterior for the Sun Dis...

Page 84: ... in the Legend SIMs are hot pluggable on the shelf Figure Legend 1 SIM board 1 3 SIM board 0 5 Network management port 2 Host or SIM Link In 4 SIM link out 6 For Sun service only The following figure shows the SIM board and Power Supply exterior indicator LEDs SIM 0 is on the left and SIM 1 is on the right SAS2 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 84 ...

Page 85: ... provide part numbers for the supported expansion storage shelves Sun Disk Shelf Mktg Part Number Description Mfg Part Number DS2 0BASE Sun Disk Shelf 24x3 5 SAS 2 597 0764 02 DS2 HD1T Sun Disk Shelf 1TB 7 2Krpm SAS 2 3 5 597 0823 01 DS2 HD2T Sun Disk Shelf 2TB 7 2Krpm SAS 2 3 5 597 0625 01 DS2 LOGZ Sun Disk Shelf 18GB Logzilla SSD 597 0626 01 DS2 LOGFILLER Sun Disk Shelf 24x3 5 Filler for Logzill...

Page 86: ... and become familiar with the replaceable parts of the system 7120 Overview 7420 Overview Follow the instructions in the Electrostatic Discharge Precautions section Legacy platform overviews are provided in the following sections 7110 7210 7310 7410 J4400 J4500 Disk Shelf Safety Information Follow all Sun cautions warnings and instructions marked on the equipment and described in Important Safety ...

Page 87: ...p static electricity during the removal and replacement of a CRU Tasks ShelfTasks Replacing a Drive The shelf drives are hot swappable and may be replaced without removing power from the shelf The replacement drive must be of the same capacity and type as the drive to be replaced To avoid possible data loss when removing non faulted drives label each drive with the number of the slot from which it...

Page 88: ...the drive body and pull it toward you to remove it from the chassis Ensure the new drive ejection lever is in the fully extended position Align the new drive with the open slot and slide the drive into the disk chassis 3 4 5 6 Shelf Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 88 ...

Page 89: ...ies can produce high energy hazard and should only be replaced by instructed individuals with authorized access to the equipment Failed power supplies are indicated by an amber LED on the rear panel see the Shelf Rear Panel illustration Go to the Maintenance Hardware screen of the BUI Double click the shelf and then click PSU to view details or to turn on the locator LED Locate the chassis and fai...

Page 90: ...tending from the back of the power supply With the ejection arms fully open slide the new power supply into the chassis slot until it contacts the chassis backplane and the ejection arms begin to swing closed 5 6 Shelf Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 90 ...

Page 91: ... the SIM boards at any time regardless of the state of the blue SIM OK indicator A faulted SIM board is indicated by an amber LED Locate the failed SIM at the back of the disk tray Label and disconnect the tray interface cables Use two hands to disconnect the SAS cable Grasp the metal body of the connector with one hand and firmly grasp and pull the tab gently toward the connector body with the ot...

Page 92: ...and push to the sides to release and partially eject the SIM from the chassis Grasp the middle of the SIM board and slide it out of the slot 5 6 Shelf Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 92 ...

Page 93: ...ly 60 seconds for the SIM board to complete its boot process at which time the Power LED should be solid green and the SIM locate LED should be off Faults Hardware Faults This section describes connecting to the controller Service Processor SP and configuration considerations for maximum serviceability In rare cases faults associated with uncorrectable CPU errors are not diagnosable or displayed i...

Page 94: ... controller For example nodeA uses nge0 and nodeB uses nge1 so that neither may be used as a cluster data interface In addition these interfaces must be locked to the controller using the Configuration Cluster option in the BUI In some cases this may require installation of an additional network interface card on each controller in a cluster configuration If access to the appliance data interfaces...

Page 95: ...sure you want to clear SYS MB P0 y n y See Also Cluster Configuration HBA Expansion pt 1 Expanding from 2 to 3 HBAs The figures below show steps to migrate a cluster from two to three HBAs NOTE Diagrams below are not representative of proper slot location for HBAs Cabling Diagrams fig 1 Cluster with two HBAs and chains of disk shelves HBA Expansion pt 1 Chapter 2 Hardware Maintenance 95 ...

Page 96: ... remove all cabling from the first controller Add and cable together the new chain of disk shelves then power on shelves HBA Expansion pt 1 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 96 ...

Page 97: ...rds and connect the controller to the second and third chain as shown fig 4 Reconnect the two original HBAs as before to the first two chains Power on the controller and wait for it to regain control of connected storage HBA Expansion pt 1 Chapter 2 Hardware Maintenance 97 ...

Page 98: ...d controller fig 6 Install new HBA according to the instructions on installing PCI cards and connect the controller to the second and third chain as shown HBA Expansion pt 1 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 98 ...

Page 99: ...the two original HBAs as before to the first two chains Power on the controller and wait for it to regain control of connected storage fig 8 Cluster with three HBAs HBA Expansion pt 1 Chapter 2 Hardware Maintenance 99 ...

Page 100: ... a cluster from three to four HBAs NOTE Diagrams below are not representative of proper slot location for HBAs Cabling Diagrams fig 1 Cluster with three HBAs and chains of disk shelves HBA Expansion pt 2 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 100 ...

Page 101: ...dd and cable together the new chain of disk shelves then power on shelves fig 3 Install new HBA according to the instructions on installing PCI cards and connect the controller to the third and fourth chain as shown HBA Expansion pt 2 Chapter 2 Hardware Maintenance 101 ...

Page 102: ...t three chains Power on the controller and wait for it to regain control of connected storage fig 5 Power down and remove all cabling from the second controller HBA Expansion pt 2 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 102 ...

Page 103: ...s and connect the controller to the third and fourth chain as shown fig 7 Reconnect the three original HBAs as before to the first three chains Power on the controller and wait for it to regain control of connected storage HBA Expansion pt 2 Chapter 2 Hardware Maintenance 103 ...

Page 104: ...fig 8 Cluster with four HBAs HBA Expansion pt 2 Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 104 ...

Page 105: ... upload a support bundle Repeat the initial setup with existing settings Reset the system to the factory defaults View pending disk firmware updates System Disks The system disks section shows the status of the system disks and their current usage The BUI displays this with a pie chart and the CLI as a text list For example tarpon maintenance system disks show Properties profile mirror root 1 14G ...

Page 106: ... the system will make another attempt After a support bundle has been successfully uploaded the support bundle and core files are automatically deleted from the system Managing Support Bundles Using the BUI To generate a support bundle click the icon next to Support Bundles on the Maintenance System screen You are presented with the randomly generated filename for the support bundle Provide this f...

Page 107: ...ting the ak prefix and the file type suffix To delete a support bundle use the destroy command To view details use the select and list commands loader maintenance system bundles select 9a4c3d7b 50c5 6eb9 c2a6 ec9808ae1cd8 loader maintenance system bundles 9a4c3d7b 50c5 6eb9 c2a6 ec9808ae1cd8 list Properties filename cores ak 9a4c3d7b 50c5 6eb9 c2a6 ec9808ae1cd8 tar gz status uploading step_progres...

Page 108: ...el with the ability to update a system s software after the system is installed Software updates are delivered as opaque binary downloads that contain some or all of Management and system software Firmware for internal components such as HBAs and network devices Firmware for disks and flash devices Firmware for external storage enclosure components In general the update release notes describe what...

Page 109: ...complete at that point See below regarding additional firmware upgrades that may take place following the reboot While the upgrade is in progress up until the reboot and following the reboot during any firmware upgrades it is non disruptive the controller continues to provide data services to clients If the system software fails during the upgrade it will reboot and continue running the software f...

Page 110: ...ify that the rest of the software is functional and a rollback is not required before applying the update If you elect to not apply deferred updates during an upgrade you can return to the updates view at any point to apply the update If deferred updates are available for the current software version they will appear as a list below the current set of available updates with an Apply button to appl...

Page 111: ...mponents other than hardware As of 2010Q3 4 when there are outstanding updates an informational or warning icon will appear next to the number of updates remaining Clicking the icon brings up the Firmware Updates dialog which lists the current remaining updates For each update we also show the current version of the component the time of the last attempted update as well as the reason why the last...

Page 112: ...operations should not be performed while firmware upgrades are in progress The rolling upgrade procedure documented below meets all of these best practices and addresses the per device class restrictions described below It should always be followed when performing upgrades in a clustered environment In both clustered and standalone environments these criteria will also be reevaluated upon any rebo...

Page 113: ...he update process the system will reboot automatically If you have the serial console open you will notice during this reboot that multiple GRUB menu entries are available ordered from the newest software at the top to the oldest software at the bottom The default menu entry will be the top the new software to which you just updated If you do nothing this entry will boot by default completing the ...

Page 114: ...the way to version V 2 thereby skipping V 1 then you not only are removing the system software settings and system software for V but also for V 1 That is after a rollback to V 2 it is as if updates V 1 and V never happened However the software upload images for V 1 and V will still be saved on the system and you can apply them again after the rollback if you wish by re executing the update If aft...

Page 115: ...with the Sun Storage clustering model if you are not familiar with the clustering concepts and terminology please read about clustering concepts in the System Administration Guide first To describe the rolling upgrade procedure this document will refer to the two clustered storage controllers as A and B where A is the controller that will be updated first and B is the controller that will be updat...

Page 116: ...are working correctly as seen on client systems If a serious problem is encountered roll back the software on controller A which will cause it to reboot controller B still running previous software will take over and when controller A recovers it will be running the previous version as well Important Controller firmware updates will not proceed while the two controllers are running different versi...

Page 117: ...pathologies that will interfere with the system software update and firmware upgrade processes If a problem is found the update process will be aborted and you will be directed to the Alert Log for any problems encountered System software updates will not be permitted until the problems have been corrected The system software Release Notes for the target update media contain a list of all update p...

Page 118: ...ain in the Unchecked state until the Check button is clicked During the health check operation an indicator will show its progress Updates Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 118 ...

Page 119: ...Once complete the System Ready state will change to Yes or No with a link to the Alert Log Updates Chapter 3 System Maintenance 119 ...

Page 120: ...ighlight conditions that would prevent successful upgrade No actual upgrade will be attempted and the checks performed are of static system state and non invasive Do you wish to continue Are you sure Y N Healthcheck running Healthcheck completed There are no issues at this time which would cause an upgrade to this media to be aborted Updating via the BUI Click the add icon next to Available Update...

Page 121: ... happen automatically after the media is done uploading Beginning an upgrade After the update is uploaded unpacked and verified it will appear as an update Click the information icon to view the Release Notes for the software update To begin the upgrade click on the apply icon As the upgrade progresses you will see the most recent message in the status field of the update To cancel the update at a...

Page 122: ...ve is actually a download To download the media onto the appliance via the CLI execute the download command in maintenance system updates dory maintenance system updates download dory maintenance system updates download uncommitted get url unset user unset password unset You must set the url property to be a valid URL for the download This may be either local to your network or over the internet T...

Page 123: ...update_zfs_upgrade Apply incompatible storage pool update clownfish maintenance system updates ak nas 2009 04 03 1 0 get version 2009 04 03 1 0 date 2009 4 3 08 45 01 status AKUP_WAITING update_zfs_upgrade deferred clownfish maintenance system updates ak nas 2009 04 03 1 0 set update_zfs_upgrade onreboot update_zfs_upgrade onreboot clownfish maintenance system updates ak nas 2009 04 03 1 0 After y...

Page 124: ...e media To remove update media use the destroy command specifying the update to be removed dory maintenance system updates destroy ak nas 2009 10 14 1 0 nd This will destroy the update ak nas 2009 10 14 1 0 nd Are you sure Y N y dory maintenance system updates Applying Deferred updates To see if there are any available deferred updates run the show command If deferred updates are available you can...

Page 125: ...ated with a mode such as 0664 or 0666 This all works as expected for data files but you might want to optionally include the execute bit from the file creation mode into the inherited ACL One example is an output file that is generated from tools such as cc or gcc If the inherited ACL doesn t include the execute bit then the output executable from the compiler won t be executable until you use chm...

Page 126: ...ate The COMSTAR framework relies on a ZFS pool upgrade for complete support of persistent group reservations PGRs Before this upgrade has been applied the number of reservations stored with each LUN is severely limited and may even be zero Applying this update is equivalent to upgrading the on disk ZFS pool to version 16 Triple Parity RAID Triple Parity RAID Deferred Update This update provides th...

Page 127: ...incoming replication updates for existing replicas will fail Replicas received under earlier releases will not be manageable via the BUI or CLI though they will occupy space in the storage pool Additionally the system will not send replication updates for actions configured on earlier releases After applying this update incoming replication updates for replicas originally received on earlier relea...

Page 128: ...ate has been applied new snapshot data will be stored in the old format and deleted using the old algorithm Note that any snapshots created before this update is applied will also be deleted using the old algorithm Applying this update is equivalent to upgrading the on disk ZFS pool version 26 Recursive Snapshots Recursive Snapshots Deferred Update This deferred update allows recursive snapshots t...

Page 129: ...erred Update This deferred update improves both latency and throughput on several important workloads These improvements rely on a ZFS pool upgrade provided by this update Applying this update is equivalent to upgrading the on disk ZFS pool to version 29 Optional Child Dir Introduction This deferred update improves list retrieval performance and replication deletion performance by improving datase...

Page 130: ...erface configuration User accounts roles and privileges preferences and encrypted passwords for local users not directory users Alerts and thresholds and their associated rules A configuration backup does not include User data shares and LUNs Your user data must be backed up separately using NDMP backup software snapshots and or remote replication User passwords for directory users These remain st...

Page 131: ...e browser at a new restored network address to continue For this reason it may be desirable to initiate a complex configuration restore from the service processor serial console using the CLI Un cluster restore and re cluster Configuration backups may be initiated for appliances that are joined in a cluster but a configuration restore may not be used while systems are actively clustered The cluste...

Page 132: ...appliance and therefore are not stored in the configuration backup If you have deployed a directory service such as LDAP or AD for administrative user access there are no copies of directory service password hashes for directory users stored in the configuration backup Only the user name user ID preferences and authorization settings for directory users are stored in the backup and then restored F...

Page 133: ...on click the add icon at the top of the saved configurations list and then use your web browser s file selection dialog to locate the previously exported configuration You should upload the single compressed archive file previously saved using the export function Managing Configuration Backups Using the CLI The following section outlines how various Configuration Backup tasks can be accomplished u...

Page 134: ... deletes a saved configuration host maintenance system configs destroy cb2f005f cf2b 608f 90db fc7a0503db2a Are you sure you want to delete the saved configuration new y host maintenance system configs Export a Saved Configuration The export command exports a saved configuration by means of executing an HTTP or FTP PUT operation against a remote HTTP or FTP server You can also use the export funct...

Page 135: ...sk of a different type cache log or data was inserted into a slot The newly inserted device must be of the same type Minor Fault Never 2009 08 21 16 40 37 The ZFS pool has experienced currently unrecoverable I O failures Major Error Never 2009 07 16 22 03 22 A memory module is experiencing excessive correctable errors affecting large numbers of pages Major Fault Never This information can also be ...

Page 136: ...f service personnel or as part of a documented Oracle repair procedure Related features A persistent log of all faults and defects is available under Logs as the Fault log Faults and defects are subcategories of Alerts Filter rules can be configured to cause the appliance to email administrators or perform other actions when faults are detected Logs Introduction Alerts This is the appliance alert ...

Page 137: ...ID Description Type 2009 9 5 17 42 35 9e46fc0b b1a4 4e69 f10f e7dbe80794fe The device HDD 6 has failed or could not be opened Major Fault 2009 9 3 19 20 15 d37cb5cd 88a8 6408 e82d c05576c52279External sensors indicate that a fan is no longer operating correctly Minor Fault 2009 8 21 16 40 48 c91c7b32 83ce 6da8 e51e a553964bbdbcThe ZFS pool has experienced currently unrecoverable I O failures Major...

Page 138: ...sion Annotation 2009 10 12 05 20 24 root deimos Disabled ftp service 2009 10 12 03 17 05 root deimos User logged in 2009 10 11 22 38 56 root deimos Browser session timed out 2009 10 11 21 13 35 root console Enabled ftp service Phone Home If Phone Home is used this log will show communication events with Oracle support The following are example phone home entry as it would appear in the BUI Time De...

Page 139: ...004 2009 10 15 01 42 01 root console Imported storage pool pool 0 entry 005 2009 10 15 17 56 30 root console User logged in entry 006 2009 10 15 17 56 53 root deimos sf fishworks com User logged in via CLI entry 007 2009 10 15 18 00 21 root deimos sf fishworks com User logged out of CLI entry 008 2009 10 15 18 14 47 root console Browser session timed out entry 009 2009 10 15 20 46 27 root deimos s...

Page 140: ...mmary Enabled datalink nge0 service annotation The annotation is the session annotation which can be enabled when configuring users Logs Sun ZFS Storage 7x20 Appliance Customer Service Manual December 2011 E26399 02 140 ...

Page 141: ...feature for graphing real time and historic performance statistics ARC Adaptive Replacement Cache BUI Browser User Interface CLI Command Line Interface Cluster Multiple heads connected to shared storage Controller See Storage Controller CPU Central Prcessing Unit CRU Customer Replaceable Component Dashboard appliance summary display of system health and activity Dataset the in memory and on disk d...

Page 142: ... System Interface Kiosk a restricted BUI mode where a user may only view one specific screen L2ARC Level 2 Adaptive Replacement Cache LDAP Lightweight Directory Access Protocol LED light emitting diode Logzilla write IOPS accelerator LUN Logical Unit Masthead top section of BUI screen Modal Dialog a new screen element for a specific function NFS Network File System NIC Network Interface Card NIS N...

Page 143: ...s Service appliance service software Share ZFS filesystem shared using data protocols SIM SAS Interface Module Snapshot an image of a share SSD Solid State Drive SSH Secure Shell Statistic a metric visible from Analytics Storage Controller the head node of the appliance Support Bundle auto generated files containing system configuration information and core files for use by remote support in debug...

Page 144: ...144 ...

Page 145: ... 35 36 37 38 39 40 41 42 43 53 D DIMM 47 48 49 Disk Shelf 80 81 83 84 F FRU 35 H Hardware Locating a failed component 21 L Logzilla 25 32 S Shelf Replacing a Drive 87 89 Replacing a Power Supply 89 91 Replacing a SIM Board 91 93 SIM 81 84 91 SSD 75 U Updates 111 112 145 ...

Page 146: ...146 ...

Search results

Summary of Contents for Sun ZFS

Reviews:

Related manuals for Sun ZFS

Brands by name

Popular brands

Oracle Sun ZFS, Service Manual

Search results

Summary of Contents for Sun ZFS

Reviews:

Related manuals for Sun ZFS

Brands by name

Popular brands