Quantum QXS G2 Series Скачать руководство пользователя страница 116

Страница: 116 / 246

104 QXS G2 Hardware Installation and Maintenance Guide

Connecting Two 2U Systems (2-Host Ports) for Replication (Multiple

Servers/Switches/Locations)

Figure 92

provides an illustration of connecting two 2U12/2U24 RAID chassis (storage systems) for

replication (multiple servers/switches/two locations).

Figure 92

Connecting Two 2U Systems (2-Host Ports) for Replication (Multiple

Servers/Switches/Locations)

Connecting Two 5U Systems for Replication (Multiple Servers/Switches)

Figure 93

provides an illustration of connecting two 5U84 RAID chassis (storage systems) for

replication (multiple servers/switches/two locations).

Figure 93

Connecting Two 5U Systems for Replication

(Multiple Servers/Switches)

NOTE:

The diagrams shown in

Figure 91

and

Figure 93

represent two branch offices cabled to

enable disaster recovery and backup. In case of failure at either the local site or the remote site, you

can fail over the application to the available site.

RAID Chassis

I/O Switch

To Host Servers

(Site A)

To Host Servers

(Site B)

Ethernet WAN

Содержание QXS G2 Series

Страница 1: ...QXS G2 Hardware Installation and Maintenance Guide 12 24 and 84 Drive Systems 6 68649 01 Rev A...

Страница 2: ...000 DXiV Series FlexTier Lattus the Q logo the Q Quantum logo Q Cloud Quantum and the Q brackets design the Quantum logo Quantum Be Certain and the Q brackets design Quantum Vision Scalar StorageCare...

Страница 3: ...Label Location 8 2U24 Drive System 10 2U24 Drive System Serial Number Label Location 11 5U84 Drive System 13 5U84 Drive System Serial Number Label Location 13 Chassis Variants 14 2U Chassis Core Produ...

Страница 4: ...8 Drive Carrier Module in 2U Chassis 49 DDIC in 5U chassis 51 Chassis Management 53 Management Interfaces 54 Installation 55 Installation Checklist 55 Planning for Installation 57 Preparing for Instal...

Страница 5: ...ng On Powering Off 105 Guidelines 106 Powering On the 2U12 and or 2U24 System 106 Powering Off the 2U12 and or 2U24 System 108 Powering On the 5U84 System 109 Powering Off the 5U84 System 111 Operator...

Страница 6: ...ide Connection Fault 139 Host Side Connection Troubleshooting Featuring CNC Ports 139 Isolating a Controller Module Expansion Port Connection Fault 141 Isolating Replication Faults 142 Diagnostic Step...

Страница 7: ...ing a 5U84 DDIC 192 Installing a 5U84 DDIC 194 Populating 5U84 Drawers 195 Preparation 195 Installation Guidelines 196 Replacing a 5U84 Bezel 196 Removing a 5U84 Bezel 197 Installing a 5U84 Bezel 199...

Страница 8: ...Equipment WEEE 223 USB Device Connection 225 Rear Panel USB Ports 225 Emulated Serial Port 226 Supported Host Applications 226 Command line Interface 226 Device Driver Special Operation Mode 227 Micro...

Страница 9: ...able Fibre channel FC SFP option supporting 8 16 Gb iSCSI SFP option supporting 10GbE Copper RJ 45 SFP option supporting 1GbE The QXS G2 systems are SBB compliant Storage Bridge Bay chassis These chas...

Страница 10: ...ou can use the CLI to set CNC ports to support a combination of host interface protocols When configuring a combination of host interface protocols host ports 0 and 1 are set to FC either both 16Gb s...

Страница 11: ...n not included in product documentation QXS G2 Release Notes Overview of hardware installation QXS G2 Quick Start Guide Product hardware installation and maintenance QXS G2 Hardware Installation and M...

Страница 12: ...shortcuts Table 2 Document conventions Convention Element Blue text Cross reference links and e mail addresses Blue underlined text Web site addresses Bold text Key names Text typed into a GUI element...

Страница 13: ...s Safe Handling of Equipment Always follow these safety cautions when handling the QXS G2 equipment CAUTION Use this equipment in a manner specified by the manufacturer failure to do this may cancel t...

Страница 14: ...of the QXS G2 Systems IMPORTANT All systems must have the following installed Two controllers in the RAID chassis or two IOMs in the expansion chassis Two PSUs in all chassis Five fans in the 5U84 ch...

Страница 15: ...o PSUs CAUTION Do not remove covers from the PSU there is a danger of electric shock inside Return the PSU to your supplier for repair When bifurcated power cords Y leads are used these cords must onl...

Страница 16: ...system It must provide over current protection for the chassis and must not be overloaded by the total number of chassis installed in the rack When addressing these concerns consideration should be gi...

Страница 17: ...Drive 4 Port FC or iSCSI QXS G2 424 24 Drive 4 Port FC or iSCSI QXS G2 484 84 Drive 4 Port FC or iSCSI Chassis Configurations The storage system supports three RAID chassis configurations 2U12 rack sp...

Страница 18: ...1 inch high 3 5 form factor drives in a vertical orientation within the drive drawer Two vertically stacked drawers each hold 42 drives If used 2 5 drives require 3 5 adapters The 5U84 RAID chassis s...

Страница 19: ...ontroller there is no need to transport failed controller cache to a replacement controller because the cache is duplicated between the controllers provided that volume cache is set to standard on all...

Страница 20: ...nt Figure 5 provides a rear view of the 2U12 drive system The 2U12 RAID chassis has dual controllers 4 port FC iSCSI model shown installed Figure 5 2U12 Drive System Rear 2U12 Drive System Serial Numb...

Страница 21: ...of the chassis The serial number label can be located on the left rear ear of the chassis or on the right rear ear of the chassis The primary location of the serial number label is on the left rear e...

Страница 22: ...Figure 7 provides the serial number label location on the right rear ear of the chassis Figure 7 Serial Number Label on Right rear Ear of Chassis 2U24 Drive System Figure 8 provides a front view of t...

Страница 23: ...system serial number label that is placed on a chassis NOTE Refer to Figure 10 and Figure 11 for the exact location of the system serial number label Do not confuse other QTM serial numbers on the cha...

Страница 24: ...he serial number label location on the left rear ear of the chassis Figure 10 Serial Number Label on Left rear Ear of Chassis Figure 11 provides the serial number label location on the right rear ear...

Страница 25: ...w of the 5U84 drive system The 5U84 RAID chassis has dual controllers 4 port FC iSCSI model shown installed Figure 13 5U84 Drive System Rear 5U84 Drive System Serial Number Label Location This section...

Страница 26: ...OM A Figure 14 5U84 System Serial Number Label Location Chassis Variants The 2U chassis can be configured as a RAID chassis or an expansion chassis The 5U chassis can be configured as a RAID chassis o...

Страница 27: ...rives 24 SFF 2 5 SAS 2 5 SATA 2 2 Expansion IOMs2 N A 1 5U84 RAID Chassis 12Gb s direct dock drives 84 LFF or SFF 2 5 and 3 5 SAS 3 5 SATA 2 2 Controllers 53 2 5U84 Expansion Chassis 12Gb s direct doc...

Страница 28: ...ont and rear panels 2U12 Drive Chassis Front View Integers on the drives indicate drive slot numbering sequence 0 11 Figure 15 provides a front view of the 2U12 drive chassis Figure 15 2U12 Drive Chas...

Страница 29: ...Drive 2U24 Drive RAID Chassis 2 FC iSCSI ports CNC FC iSCSI Controller Callout 1 Figure 18 is the Controller A location Callout 2 Figure 18 is the Controller B location Both controllers are shown in t...

Страница 30: ...optionally added storage RAID Chassis CNC Controllers and 4 FC iSCSI ports Figure 19 provides a rear view of the 2U12 drive or 2U24 drive RAID chassis with two CNC controllers 4 FC iSCSI ports Figure...

Страница 31: ...the 2U12 drive or 2U24 drive expansion chassis rear panel The expansion chassis attaches to the RAID chassis for additional storage Figure 21 2U12 2U24 Drive Expansion Chassis Rear View Expansion Cha...

Страница 32: ...provides and illustration of the PSU used in the RAID chassis and the expansion chassis They are identical in the 12 drive and 24 drive chassis Figure 23 shows an example of a PSU oriented for use in...

Страница 33: ...ne see Figure 25 on page 22 The chassis has a 19 inch rack mounting that enables it to be installed onto standard 19 inch racks and uses two EIA units of rack space 3 5 for a 2U chassis The midplane P...

Страница 34: ...sion chassis 5U Chassis Core Product The 5U chassis consists of a sheet metal chassis integrated PCBs and a module runner system The design concept is based on an chassis subsystem together with a set...

Страница 35: ...he chassis Drawer 1 is at the bottom of the chassis Figure 26 Front of 5U84 RAID or Expansion Chassis 5U Chassis Drive Slots View Figure 27 provides an illustration of the 5U84 RAID or expansion chass...

Страница 36: ...lustration of the 5U84 RAID chassis rear view with two CNC controllers FC iSCSI 4 ports installed NOTE The 5U84 RAID chassis supports attachment of expansion chassis for adding storage 2 1 3 4 0 42 2...

Страница 37: ...troller B location Both controllers are shown in the closed locked position Figure 29 provides a rear view of the CNC FC iSCSI controller used in the 5U84 drive systems You can configure the CNC host...

Страница 38: ...the 5U84 expansion chassis CRUs with expansion IOMs NOTE The 5U84 expansion chassis uses the same expansion IOMs as the 2U12 and 2U24 expansion chassis Figure 30 5U84 Expansion Chassis Rear View SAS...

Страница 39: ...d with RAID chassis for adding storage its middle HD mini SAS expansion port B is disabled by the firmware The Ethernet port on the expansion module is not used in RAID chassis and expansion configura...

Страница 40: ...idplane PCB module runner system and two drawers for drive modules NOTE Customers select a chassis type and drives separately Empty drive slots in the 5U chassis require drive blanks The 5U84 chassis...

Страница 41: ...e chassis A drawer is designed to support its own weight plus the weight of installed DDICs when fully opened Safety features The chassis safety features include To reduce the possibility of toppling...

Страница 42: ...nnects the Ops panel to the midplane The Ops panel is a passive component the midplane controls the panel and the RAID controllers or expansion IOMs control all the panel s functions 2U12 Chassis Ops...

Страница 43: ...elow and listed in the table See also 2U Chassis Ops Panel LEDs on page 118 Figure 37 2U Chassis Ops Panel Functions 1 Ops Panel 2 2U12 Chassis Note Drives are numbered from 0 11 12 drives 0 1 2 3 4 5...

Страница 44: ...CLI Unit Identification Display The UID is a dual seven segment display that can be used to provide feedback to the user Its primary function is to display a chassis unit identification number to ass...

Страница 45: ...ceable on site The Ops panel provides the functions shown in the illustration below and listed in the table The 5U ops panel provides the following Unit Identification Display UID Mute Input button Po...

Страница 46: ...lt LED amber LED illuminates when experiencing a system hardware fault It may be associated with a Fault LED on a PSU fan RAID controller or expansion IOM DDIC or drawer that helps the user to identif...

Страница 47: ...The 2U12 and 2U24 use the same type of PSUs AC DC power is provided by up to two auto ranging power PSUs with integrated axial cooling fans The RAID controllers and expansion IOMs control fan speed P...

Страница 48: ...s soon as possible after the removal of the defective PSU to avoid a thermal exception The replacement procedure should be completed within an absolute maximum of 2 minutes Operation of the chassis wi...

Страница 49: ...amber a module fault condition or failure has occurred Dual PSUs provide redundant power for the 5U system if one PSU fails the other keeps the system running while you replace the faulty module PSUs...

Страница 50: ...ent of a fan can be performed while the chassis is running but the procedure must be completed within two minutes of the removal of the defective module Verify that you have a replacement module on ha...

Страница 51: ...this way the SBB modules can discover the type and capabilities of the partner SBB module s and vice versa within the chassis A system alarm occurs when incompatible configurations are detected 12Gb...

Страница 52: ...s powered off 4 Fault Off The controller is operating normally Amber A fault has been detected or a service action is required Blinking amber Hardware controlled power up or a cache flush or restore e...

Страница 53: ...and QXS G2 424 Figure 46 CNC iSCSI Controller LEDs 9 Network Port Link Speed4 Off Link is up at 10 100base T negotiated speeds Amber Link is up and negotiated at 1000base T 10 Expansion Port Status Of...

Страница 54: ...emoval Blue The controller module is prepared for removal 6 Identity White The controller module is being identified 7 Cache Status Green Cache is dirty contains unwritten data and operation is normal...

Страница 55: ...nking green System is booting Off The controller module is not OK or is powered off 4 Fault Off The controller is operating normally Amber A fault has been detected or a service action is required Bli...

Страница 56: ...o Remove Off The controller is not prepared for removal Blue The controller module is prepared for removal 6 Identity White The controller module is being identified 7 Cache Status4 Green Cache is dir...

Страница 57: ...isks Later when the storage system is either idle or aging and continuing to receive new I O data the controller writes the data to the disks Read ahead caching The controller detects sequential data...

Страница 58: ...refresh mechanism is an important data protection feature essentially four copies of user data are preserved one in controller cache and one in CompactFlash of each controller The Cache Status LED ill...

Страница 59: ...a strobe 1 10 rate 1 Hz and the controller does not boot You may need to replace the controller module Yes at a strobe 1 10 rate 1 Hz and the controller boots The system is flushing data to CompactFl...

Страница 60: ...fferent drive modules for use in 2U and 5U chassis The drive modules used in 2U chassis are referred to as drive carrier modules whereas those used in 5U chassis are referred to as a Disk Drive in Car...

Страница 61: ...which provides thermal conduction radio frequency and electro magnetic induction protection and physically protects the drive The front cap also has an ergonomic handle which gives the following funct...

Страница 62: ...rives Integers on the drives indicate drive slot numbering sequence 0 23 Figure 51 provides a front view of the 2U24 drive chassis fully populated with drives Figure 51 2U24 Drive Chassis Front View N...

Страница 63: ...Blanks Drive blanks also known as dummy drive carrier modules are provided in 3 5 2U12 and 2 5 2U24 form factors They must be installed in empty disk drive slots to create a balanced air flow Figure...

Страница 64: ...5U84 chassis Figure 56 DDIC with Adapter and 2 5 Drive The DDIC features a latch button center of top view and a slide latch with arrow label left of latch button These features allow you to install...

Страница 65: ...led in the same row should have the same rotational speed Although DDICs holding 3 5 disks can be intermixed with DDICs holding 2 5 disks within the chassis each row should be populated with drives of...

Страница 66: ...dware installation you can access the controller module s web based management interface disk management utility GUI to configure monitor and manage the storage system See also Accessing the disk mana...

Страница 67: ...stall the system use only the power cords supplied or power cables that match the specification quoted in AC Power Cords on page 223 Table 7 outlines the steps required to install the chassis and init...

Страница 68: ...face protocol for qualified SFP options See Change CNC Port Mode on page 98 7 Install required host software See Host System Requirements on page 72 8 Connect hosts 2 See Host Connection on page 76 9...

Страница 69: ...Chassis Drive Slots View on page 23 5U RAID chassis rear panel see 5U84 RAID Chassis Rear View Two CNC Controllers on page 24 5U expansion chassis rear panel see 5U84 Expansion Chassis Rear View on pa...

Страница 70: ...for a total of 84 drives Expansion IOMs fans and PSUs are installed within the chassis CAUTION Lifting chassis A 2U chassis together with all its component parts is too heavy for one person to lift an...

Страница 71: ...for the correct software configurations Before installing the chassis verify the existence of the following Qualified cable options for host connection Depending upon the controller module FC or iSCS...

Страница 72: ...ck 2U Chassis 2U24 Chassis Shown The 2U chassis are supplied with the midplane PCB PSUs drives and RAID controllers and or expansion IOMs installed For information about plug in module replacement see...

Страница 73: ...s grip the straps securely by the loops and lift the chassis out of the box using appropriate lifting technique Place the chassis in a static protected area NOTE If your product model uses CNC ports f...

Страница 74: ...chassis weight and to make sure that multiple chassis may be installed without loss of space within the rack Use of other mounting hardware may cause some loss of rack space Contact your supplier to m...

Страница 75: ...it reaches the hard stops approximately 400 mm 15 75 and tighten the mounting screws callout 3 A screws in the front rail kit bracket e Return the chassis to the fully home position 5 Install the 2u c...

Страница 76: ...shortest length refer to the reference label on the inside of the rail 3 Locate seat the rail location pins callout 5 inside the front of the rack and extend the length of the rail assembly to enable...

Страница 77: ...anel CRUs and install all of the DDICs into the drawers accessed from the front panel per the following instructions Install five fans Installing a 5U84 System Fan on page 206 Install two PSUs Install...

Страница 78: ...their installation NOTE Reverse cabling straight through cabling for the RAID chassis and expansion is provided Reverse cabling is suggested because it provides redundancy alternate path to access ch...

Страница 79: ...hase additional cables depending upon number of RAID chassis expansion chassis and cabling method used You may need to order additional or longer cables when reverse cabling a fault tolerant configura...

Страница 80: ...ccess to other chassis The diagram at right above shows the same storage components connected to use straight through cabling Using this method if an expansion chassis fails the chassis that follow th...

Страница 81: ...des an illustration of connecting RAID chassis and expansion chassis in a straight through cabling configuration for 5U84 systems Figure 64 5U84 Straight Through Cabling NOTE Figure 63 and Figure 64 s...

Страница 82: ...an illustration of connecting a 2U RAID chassis and three 5U84 expansion chassis in a straight through cabling configuration Figure 66 2U RAID Chassis to 5U84 Expansion Chassis Straight Through Cabli...

Страница 83: ...hassis PSUs to an AC PDU Figure 68 Connecting 5U RAID chassis PSUs to AC PDU IMPORTANT When more than one PSU is fitted all power cords must be connected to at least two separate and independent power...

Страница 84: ...may require that multipathing is supported If fault tolerance is required then multipathing software may be required Host based multipath software should be used in any configuration where two logical...

Страница 85: ...wnership TIP See the topic about configuring system settings in the QXS G2 Disk Management Utility User Guide to initially configure the system or change system configuration settings such as Configur...

Страница 86: ...e Guide for command syntax and details about parameter settings relative to supported link speeds Fibre Channel ports are used for attachment to FC hosts directly or through a switch used for the FC t...

Страница 87: ...s using the Internet SCSI interface protocol for host port connection The QXS G2 312 and QXS G2 324 1Gb iSCSI controller module provides two host ports The QXS G2 412 QXS G2 424 and QXS G2 484 1Gb iSC...

Страница 88: ...l replication feature 10GbE iSCSI Host Connection To connect controller modules supporting 10GbE iSCSI host interface ports to a server HBA or switch using the controller s CNC ports select a qualifie...

Страница 89: ...controller B paths for host connection 2U12 2U24 4 Host Ports One Server One HBA Dual Path Figure 69 provides an illustration of connecting a 2U12 2U24 to one server one HBA with dual path Figure 69...

Страница 90: ...provides an illustration of connecting a 2U12 2U24 to two servers one HBA per server with dual path Figure 72 2U12 2U24 Two Servers One HBA Per Server Dual Path 2U12 2U24 2 Host Ports Two Servers One...

Страница 91: ...5U84 Two Servers One HBA Per Server Dual Path 2U12 2U24 Four Servers One HBA Per Server Dual Path Figure 75 provides an illustration of connecting a 2U12 2U24 to four servers one HBA per server with d...

Страница 92: ...hin the storage system Using switches a SAN shares a storage system among multiple servers reducing the number of storage systems required for a particular environment Using switches increases the num...

Страница 93: ...U24 to two servers and two switches Figure 78 2U12 2U24 Two Servers Two Switches 5U84 Two Servers Two Switches Figure 79 provides an illustration of connecting a 5U84 to two servers and two switches F...

Страница 94: ...abric Figure 80 2U12 2U24 Four Servers Multiple Switches SAN Fabric 5U84 Four Servers Multiple Switches SAN Fabric Figure 81 provides an illustration of connecting a 5U84 to four servers multiple swit...

Страница 95: ...net 3 Do not interconnect iSCSI and management Ethernet on the same network Figure 82 2U RAID Chassis to Management Network Figure 83 5U84 RAID Chassis to Management Network NOTE Connections to this d...

Страница 96: ...ormation Replacing an expansion I O module s NOTE Updating controller firmware with expansion I O modules active ensures that the controller firmware and expansion I O module s firmware are at a compa...

Страница 97: ...cation with a Dynamic Host Configuration Protocol DHCP server See the topic about configuring network ports in the QXS G2 Disk Management Utility User Guide This section provides the following informa...

Страница 98: ...ck on System Action System Settings NOTE When you change any setting in the System Settings panel the Apply and Close and the Apply buttons will become active To save your changes and continue changin...

Страница 99: ...nformation click on the Help icon which is the at the top right of the screen and the following screen appear NOTE Notice there are hyperlinks that will take you directly to help content corresponding...

Страница 100: ...tem time and date can be obtained from the NTP server This allows multiple storage devices hosts and log files to be synchronized Click on Manage Users To secure the storage system set a new password...

Страница 101: ...bundled license is bought at the initial purchase of the system the Licensed Snapshots Performance Tier Tiering and Replication Async will be enabled at the factory If a permanent license is not inst...

Страница 102: ...alues or use DHCP When setting static IP values you can use either IPv4 or IPv6 format In DHCP mode the system obtains values for the network port IP address subnet mask and gateway from a DHCP server...

Страница 103: ...ement services You can enable or disable management services to limit the ways in which users and host based management applications can access the storage system Network management services operate o...

Страница 104: ...n and information description values The name is shown in the browser title bar or tab The name location and contact are included in event notifications All four values are recorded in system debug lo...

Страница 105: ...the system to send notifications You can enable the system to send notifications to SNMP trap hosts and email addresses when events occur in the system You can also enable the managed logs feature wh...

Страница 106: ...P Addresses Using DHCP In DHCP mode network port IP address subnet mask and gateway values are obtained from a DHCP server if one is available If a DHCP server is unavailable current addressing is unc...

Страница 107: ...onfiguration See the QXS G2 Disk Management Utility User Guide for more information concerning the web based storage management application 1 From your network administrator obtain an IP address subne...

Страница 108: ...tained in step 1 for each network port first for controller A and then for controller B set network parameters ip address netmask netmask gateway gateway controller a b where address is the IP address...

Страница 109: ...CSI system or a system using a combination of FC and iSCSI SFPs do not restart the Management Controller or exit the terminal emulator session until configuring the CNC ports as described in Change CN...

Страница 110: ...1 of Setting Network Port IP Addresses Using CLI Port and Cable on page 94 Set CNC Port Mode to FC and iSCSI To set the CNC port mode for use with FC SFPs and iSCSI SFPs in combination run the followi...

Страница 111: ...are owned by controller A and others are owned by controller B then allow at least one port for replication on each controller module and possibly more than one port per controller module depending on...

Страница 112: ...qualified SFP options of the same type or they can use a combination of qualified SFP options supporting different interface protocols IMPORTANT QXS G2 312 QXS G2 324 QXS G2 412 QXS G2 424 and QXS G2...

Страница 113: ...Systems for Replication Multiple Servers Different Networks Figure 88 2U and Figure 90 on page 103 5U each show the rear panel of two chassis with I O and replication occurring on different physical n...

Страница 114: ...necting Two 2U Systems 4 Host Ports for Replication Multiple Servers Switches Connecting Two 2U Systems 2 Host Ports for Replication Multiple Servers Switches Figure 89 provides an illustration of con...

Страница 115: ...occurring on different networks The I O traffic occurs on the SAN and the replication traffic occurs on the Ethernet WAN Connecting Two 2U Systems 4 Host Ports for Replication Multiple Servers Switche...

Страница 116: ...ms for Replication Multiple Servers Switches Figure 93 provides an illustration of connecting two 5U84 RAID chassis storage systems for replication multiple servers switches two locations Figure 93 Co...

Страница 117: ...ment interfaces using your web browser to complete the system setup Powering On Powering Off CAUTION Do not operate the storage system until the ambient temperature is within the specified operating r...

Страница 118: ...left in a power on state for 30 seconds following resumption from standby before the chassis can be placed into standby again applies to 5U84 only Although the chassis supports standby the expansion...

Страница 119: ...the PSU1 power switch Expansion chassis powers on within 3 minutes While the expansion chassis power up their LEDs blink After the LEDs stop blinking if no LEDs on the front and back of the chassis a...

Страница 120: ...is complete and no faults have been detected The System Power LED on the 2U Ops panel should be lit green when the chassis power is activated See 2U Chassis Ops Panel LEDs on page 118 for details pert...

Страница 121: ...U1 Powering On the 5U84 System This section provides the following information Powering On the 5U84 Expansion Chassis Powering On the 5U84 RAID Chassis Powering On the 5U84 Expansion Chassis To power...

Страница 122: ...inking if no LEDs on the front and back of the chassis are amber the power on sequence is complete and no faults have been detected The Power on Standby LED on the 5U Ops panel should be lit green whe...

Страница 123: ...gement Utility Users Guide Use the CLI to shut down both controllers as described in the QXS G2 CLI Reference Guide 3 On the RAID chassis complete the following a Turn off 0 the PSU0 power switch and...

Страница 124: ...t DDICs Used in 5U Chassis Each disk has a single amber drive fault LED as shown in 5U Chassis DDIC LEDs on page 130 If the LED is illuminated a disk failure condition has occurred The DDIC should be...

Страница 125: ...torage system If you are licensed to use the optional replication feature you may also need to set up the storage systems for replication Refer to the following topics within the QXS G2 Disk Managemen...

Страница 126: ...114 QXS G2 Hardware Installation and Maintenance Guide...

Страница 127: ...troubleshooting procedures for configured systems using production data and I O For further troubleshooting help after setup and when data is present contact Quantum at http www quantum com Servicean...

Страница 128: ...carrier modules have been correctly installed 3 Check any visible SAS indicators RAID chassis expansion chassis and HBA 4 Check HBA BIOS for SAS target visibility 5 Verify that the operating system d...

Страница 129: ...Table 12 3 4 1 2 1 Fan Fail LED 2 AC Fail LED 3 PSU OK LED 4 DC Fail LED Table 12 PSU LEDs PSU OK Green Fan Fail Amber AC Fail Amber DC Fail Amber Status Off Off Off Off No AC power to either PSU Off...

Страница 130: ...RAID or expansion chassis Figure 99 2U Chassis Ops Panel Functions LED Status 1 Unit Identification Display Green Seven segment display chassis sequence 2 Identity Blue Power On 5s test state 3 Module...

Страница 131: ...ansion IOM and reinsert it If the above actions do not resolve the fault contact Quantum for assistance Table 13 Ops Panel LED States System Power Green Amber Module Fault Amber Identity Blue LED Disp...

Страница 132: ...n the amber LED will be Off if there is no drive present Off as the drive operates On if there is a drive fault Figure 100 2U Chassis Drive LEDs The drive module LED conditions are defined in Table 14...

Страница 133: ...uction Table 14 2U Drive LED States Drive LED Green Drive LED Amber Status 4 5 6 7 9 2 2 2 10 10 2 1 8 3 1 1 1 LED Description Definition 1 Host 4 8 16Gb FC1 Link Status Link Activity Off No link dete...

Страница 134: ...al The unwritten information can be log or debug data that remains in the cache so a Green cache status LED does not by itself indicate that any user data is at risk or that any action is necessary Of...

Страница 135: ...s not OK or is powered off 4 Fault Off The controller is operating normally Amber A fault has been detected or a service action is required Blinking amber Hardware controlled power up or a cache flush...

Страница 136: ...e the fault remove the controller and reinsert it If the above actions do not resolve the fault contact your supplier for assistance Controller replacement may be necessary 9 Network Port Link Speed5...

Страница 137: ...rmally Blinking green System is booting Off The expansion module is powered off 4 HD mini SAS connector LEDs A B C See Table 17 for SAS Port Activity Green LED states See Table 18 on page 126 for Acti...

Страница 138: ...an illustration of the 5U chassis PSU LEDs The RAID chassis and the expansion chassis uses the same PSU There are two PSUs in the RAID chassis and the expansion chassis Figure 104 PSU 5U84 CRU LEDs T...

Страница 139: ...Green Status On Off Off No AC power to either PSU On On Off PSU present but not supplying power or PSU alert state usually due to critical temperature Off Off On Main AC present switch on This PSU is...

Страница 140: ...in Replacing a 5U84 System Fan on page 205 Fan Fault Amber indicates the fan module has failed Follow procedure in Replacing a 5U84 System Fan on page 205 LED Status 1 Unit Identification Display Gree...

Страница 141: ...ler or expansion IOM Check the drawer LEDs for indication of a disk fault See also 5U Chassis Drawer LEDs on page 129 Logical Status Amber indicates a fault from something other than firmware usually...

Страница 142: ...he controller LEDs Table 22 5U Chassis Drawer LEDs LED Status Description Sideplane OK Power Good Green if the sideplane card is working and there are no power problems Drawer Fault Amber if a drawer...

Страница 143: ...1 if the mute button is not pressed after 2 minutes the system automatically transitions to state S2 or S3 Alarm states S1 through S4 return to S0 state upon cessation of the fault Critical Fault stat...

Страница 144: ...Alarm Conditions Status Severity Alarm PSU alert loss of DC power from a single PSU Fault loss of redundancy S1 PSU fan fail Fault loss of redundancy S1 SBB module detected PSU fault Fault S1 PSU rem...

Страница 145: ...observed to increase in speed then some restriction on airflow may be causing additional internal temperature rise Note This is not a fault condition The first stage in the thermal control process is...

Страница 146: ...ure is within the acceptable range See also QXS 2U 5U Environmental Requirements on page 217 2 Check the installation for any airflow restrictions at either the front or back of the chassis A minimum...

Страница 147: ...ntify the pertinent CRUs affected As noted in Accessing Disk Management Utility GUI on page 112 use the disk management utility GUI to configure and provision the system upon completing the hardware i...

Страница 148: ...ication View the chassis LEDs Use the Disk Management Utility GUI The disk management utility GUI uses health icons to show OK Degraded Fault or Unknown status for the system and its components The di...

Страница 149: ...located on the Ops panel on an chassis s left ear illuminates Check the LEDs on the back of the chassis to narrow the fault to a CRU connection or both The LEDs also help you identify the location of...

Страница 150: ...g the data path For example if a host side data error occurs it could be caused by any of the components in the data path controller module cable or data host If the Chassis Does Not Initialize It may...

Страница 151: ...cuitry Isolating a Host Side Connection Fault During normal operation when a controller module host port is connected to a data host the port s host link status link activity LED is green If there is...

Страница 152: ...link status link activity LED on Yes You now know that the SFP host cable and host side devices are functioning properly Return the cable to the original port If the link status LED remains off you ha...

Страница 153: ...ta yet to be written to the disk Blinking Cache data is being written to CompactFlash Flashing at 1 10 second on and 9 10 second off Cache is being refreshed by the supercapacitor Off Cache is clean n...

Страница 154: ...e Alternatively you can use SSH or telnet to access the IP address of the controller module and access the replication feature using the CLI Basic information for enabling the RAID chassis for replica...

Страница 155: ...s System functioning properly No action required No The replication feature is not licensed on each RAID chassis used for replication Verify licensing of the optional feature per system In the Home to...

Страница 156: ...o On RAID chassis equipped with iSCSI host interface ports replication set creation fails due to use of CHAP If using CHAP Challenge Handshake Authentication Protocol see the topics about configuring...

Страница 157: ...ath component Follow any Recommended Actions Click in the Volumes topic then click on a volume name in the volumes list Click the Replication Sets tab to display replications and associated metadata R...

Страница 158: ...s are using the current firmware release Periodically you should ensure that the firmware versions used in expansion IOMs are compatible Also see Updating Firmware on page 84 Customer Replaceable Unit...

Страница 159: ...the type of host attach for the 5U84 controllers in the RAID chassis Table 34 Notes 1 This model uses a qualified FC SFP option within the CNC ports used for host connection When in FC mode the SFPs...

Страница 160: ...n the 2U12 RAID chassis front and additional CRUs not shown 1 2 0 3 4 5 6 7 8 9 10 11 12 12 Table 35 2U12 RAID Chassis CRUs Front Item Chassis Component Description 0 11 Drives SFF or LFF 3 5 drive mo...

Страница 161: ...ndard HD mini SAS SFF 8644 to mini SAS SFF 8088 Cable kit Cable package USB Type B CLI USB Power Cord AC power cord compatible with AC PSU s Table 35 2U12 RAID Chassis CRUs Front Item Chassis Componen...

Страница 162: ...ns per controller NOTE Depending on customer requirements the controllers can be CNC FC or iSCSI for host connection 4 host port connections per controller Figure 111 2U12 RAID Chassis CRUS Rear with...

Страница 163: ...nt and additional CRUs not shown 5 Small form pluggable SFP connectors for CNC controllers 4 each per controller SFP transceiver 8 16Gb s FC 10GbE iSCSI 1Gb s iSCSI QXS G2 412 RAID chassis support FC...

Страница 164: ...412 QXS G2 424 and QXS G2 484 use the same identical IOMs 2U24 RAID Chassis Expansion Chassis CRUs This section provides the following information Chassis Chassis sheet metal chassis that is configur...

Страница 165: ...shown 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 24 23 0 Table 40 2U24 RAID Chassis CRUs Front Item Chassis Component Description 0 23 Drives 2 5 drive module SFF Dummy drive car...

Страница 166: ...e standard HD mini SAS SFF 8644 to mini SAS SFF 8088 Cable kit Cable package USB Type B CLI USB Power Cord AC power cord compatible with AC PSU s Table 40 2U24 RAID Chassis CRUs Front Item Chassis Com...

Страница 167: ...t ports NOTE Depending on customer requirements the controllers can be CNC FC or iSCSI for host connection 4 host ports per controller Figure 116 2U24 RAID Chassis CRUS Rear with Two CNC Controllers 4...

Страница 168: ...t Table 43 provides a list of CRUs in the 2U24 expansion chassis front and additional CRUs not shown 5 Small form pluggable SFP connectors for CNC controllers 4 each per controller SFP transceiver 8 1...

Страница 169: ...4 Expansion Chassis bezel is a CRU not shown and can be replaced See Removing a 2U Bezel on page 167 and Installing a 2U Bezel on page 167 for the 2U24 bezel replacement Additional CRUs Not Shown Chas...

Страница 170: ...nstalled Drawer 0 is at the top of the chassis Drawer 1 is at the bottom of the chassis NOTE The bezel for drawer 0 and drawer 1 are CRUs and can be replaced if required See Replacing a 5U84 Bezel on...

Страница 171: ...front of the drawer on the left side of the illustration The back of the drawer is on the right side of the illustration Figure 120 5U84 RAID or Expansion Chassis Drives CRUs 2 1 3 4 0 42 2 44 1 43 3...

Страница 172: ...t Description Disk Drive in Carrier DDIC DDICs must be installed into drawers after delivery 2 5 disk with 3 5 adapter disks of differing type speed and storage capacity SAS SSD 3 5 disk disks of diff...

Страница 173: ...f the 5U84 expansion chassis rear view with two expansion IOMs 2 SAS ports used installed Refer to Figure 122 for all the 5U84 expansion chassis CRUs with expansion IOMs NOTE The 5U84 expansion chassi...

Страница 174: ...312 and QXS G2 324 RAID chassis 2 host port systems Figure 123 Controller in RAID Chassis 2 Port FC iSCSI module CNC Controller for RAID Chassis 4 Host Ports Figure 124 provides an illustration of a...

Страница 175: ...lem Solving 163 Expansion IOM for Expansion Chassis Figure 125 provides an illustration of an expansion IOM used in the expansion chassis 3 Port SAS IOM Figure 125 Expansion IOM in Expansion Chassis 3...

Страница 176: ...164 QXS G2 Hardware Installation and Maintenance Guide...

Страница 177: ...a specific task Certain procedures refer to related documentation See Related Documentation on page xi for a list of these documents CRU replacement procedures are grouped by chassis form factor as f...

Страница 178: ...ar an ESD wrist strap connected by a ground cord to a grounded workstation or unpainted surface of the computer chassis Wrist straps are flexible straps with a minimum of 1 megohm 10 percent resistanc...

Страница 179: ...ight opening and insert the middle finger of each hand into the bottom of the respective opening with thumbs on the bezel s bottom 3 Gently pull the top of the bezel while applying slight inward press...

Страница 180: ...bezel will be attached Figure 127 shows a 2U12 chassis with ears exposed that is ready for bezel installation 2 Open the box containing the bezel and swing the lid back to reveal the bezel assembly pa...

Страница 181: ...ezel 5 Gently push fit the two ear cover areas of the bezel onto the ball studs to secure the bezel in place Replacing a 2U PSU This section provides procedures for replacing a failed PSU Illustration...

Страница 182: ...U by either the mains switch where present or by physically removing the power source in order to ensure your system has warning of imminent power shutdown A faulty PSU must be replaced by a fully ope...

Страница 183: ...andle and withdraw the PSU taking care to support the base of the module with both hands as you remove it from the chassis as shown in Figure 132 Figure 132 Removing a PSU from Chassis IMPORTANT Remov...

Страница 184: ...ooling fans are spinning with no fail states c Verify that Ops panel states show no amber module faults 7 If replacing two PSUs repeat steps 1 through 5 being mindful of the tips for step 1 and step 2...

Страница 185: ...ive module replacement within the drive slots as you view the chassis front panel Removing a 2U 3 5 LFF Drive Carrier Module 1 Refer to Using the Anti Tamper Locks on page 175 to unlock the anti tampe...

Страница 186: ...d outside the chassis 1 Release the drive carrier handle by depressing the latch in the handle Figure 135 3 5 LFF Drive Carrier Module in Open Position 2 Insert the drive carrier module into the chass...

Страница 187: ...5 LFF and 2 5 SFF drive modules provide anti tamper locks Refer to Figure 138 on page 176 when using this procedure 1 Carefully put the T10 lock key into the cutout in the handle The 3 5 LFF drive mo...

Страница 188: ...lly 1 Refer to Using the Anti Tamper Locks on page 175 to unlock the anti tamper lock on the drive module to allow removal of the drive module 2 Press the latch in the carrier handle downward to relea...

Страница 189: ...Drive Carrier Module 1 If the anti tamper lock is engaged unlock it per the instructions provided in Using the Anti Tamper Locks on page 175 2 Release the carrier handle by pressing the latch in the...

Страница 190: ...ide Figure 141 2 5 SFF Drive Carrier Module in Open Position 4 Slide the carrier fully into the chassis until it is stopped by the camming lever on the bottom of the carrier Figure 142 Installing a 2...

Страница 191: ...into place CAUTION To ensure optimal cooling throughout the chassis drive blanks must be fitted to all unused drive slots in the 2U12 and 2U24 chassis Replacing a 2U RAID Controller or an Expansion IO...

Страница 192: ...183 and Installing a 2U RAID Controller or an Expansion IOM on page 184 for instructions on installing an additional controller module CAUTION When replacing a controller module ensure that less than...

Страница 193: ...or event notification With event notification configured and enabled use the disk management utility GUI to view the event log or run the CLI show events detail command to see details for events Check...

Страница 194: ...ve a controller module from a chassis or before you power off its chassis for maintenance repair or a move Use the disk management utility GUI or the CLI to perform a shutdown Using the Disk Managemen...

Страница 195: ...ment is on hand All modules must be in place when the system is in operation Comply with all ESD precautions Refer to ESD Precautions on page 165 for additional information Illustrations in the contro...

Страница 196: ...echanism but feature different faceplate geometry 7 Place both hands on the canister body and pull it straight out of the chassis such that the RAID controller or expansion IOM remains level during re...

Страница 197: ...e system automatically updates the partner controller Verifying Component Operation RAID Controller After replacing the controller module verify that the CRU OK LED rear panel illuminates green indica...

Страница 198: ...backup of the system 3 Record system settings for future use and label all cables 4 Prepare a suitable static protected work environment to accommodate chassis replacement Verifying Component Failure...

Страница 199: ...3 4 Remove the damaged storage chassis from the rack Preparing to Remove a Damaged 2U Chassis on page 186 5 Remove the two PSUs from the damaged chassis rear panel and install them in the replacement...

Страница 200: ...ssis 1 Disconnect the power cables and data cables between devices as needed a Between the cascaded chassis b Between the controller and peripheral SAN devices c Between the controller and the host NO...

Страница 201: ...Powering On Powering Off on page 105 Verifying Component Operation 1 Restart system devices by moving the power switch on the power supply to the On position in the following sequence a Expansion cha...

Страница 202: ...status for the system and its components If you discover a problem component follow the actions in its Recommendation field to resolve the problem As an alternative to using the disk management utili...

Страница 203: ...r 1 Verify that the anti tamper locks are not engaged The red arrows on the locks point inwards if the locks are disengaged as shown in Figure 146 If necessary unlock them by rotating counter clockwis...

Страница 204: ...drawer rails 2 Push the drawer in slightly 3 Release the drawer latches 4 Push the drawer all the way into the chassis making sure that it clicks home Replacing a 5U84 DDIC This procedure describes r...

Страница 205: ...assis Drive Slots 2 Open the relevant drawer per the instructions provided in Opening a 5U84 Drawer on page 191 3 Locate the DDIC to be replaced using any of the methods listed in step 1 above 4 Locat...

Страница 206: ...he slide latch towards the front of the chassis to release the DDIC from the drawer When the DDIC is released it pops up approximately a quarter inch from the drawer Figure 151 Removing a DDIC 1 6 Pul...

Страница 207: ...wn in Figure 153 b The Drive Fault LED is not lit 5 Close the drawer according to the instructions provided in Closing a 5U84 Drawer on page 192 Populating 5U84 Drawers General guidelines for populati...

Страница 208: ...on page 193 shows dual indexing of disk slots across front middle and back rows of a drawer The 5U84 ships with drawers installed in the chassis However to avoid shock and vibration issues during tra...

Страница 209: ...his is a representative example of removing the Drawer 1 bezel lower drawer from the chassis 1 Locate the anti tamper lock 2 Each and the bezel screws 4 each on the front of the 5U84 chassis drawer be...

Страница 210: ...driver unlock the anti tamper locks 2 each on the left and right side of the chassis bezel Figure 156 5U84 Chassis Unlock Anti tamper Locks 3 Using a T8 torque tip driver remove the four bezel screws...

Страница 211: ...s Bezel Drawer Latches 2 Each Figure 159 provides an illustration of the bezel removed from Drawer 1 lower drawer Figure 159 5U84 Chassis Bezel Removed Drawer 1 Lower Drawer Installing a 5U84 Bezel NO...

Страница 212: ...w studs on the front of the chassis drawer Figure 160 5U84 Chassis Bezel Removed Drawer 1 Screw Studs 2 Using a T20 torque tip driver unlock the anti tamper locks 2 each on the left and right side of...

Страница 213: ...the 5U84 chassis drawer slot Figure 162 5U84 Chassis Bezel Drawer Latches 2 Each 4 Using a T8 torque tip driver install the four bezel screws on left and right side of the bezel CAUTION Do not over ti...

Страница 214: ...a power supply unit significantly disrupts the chassis airflow Do not remove the PSU until you have received the replacement module It is important that all slots are filled when the chassis is in ope...

Страница 215: ...If replacing a single PSU via hot swap proceed to step 6 5 If replacing both PSUs verify that the chassis was shut down using management interfaces and that the chassis is powered off 6 Verify that th...

Страница 216: ...to the target slot on the chassis rear panel as shown in Figure 167 3 Slide the PSU into the slot until the latch clicks home 4 Connect the AC power cord 5 Move the PSU power switch to the On position...

Страница 217: ...fan significantly disrupts the chassis airflow Do not remove the fan until you have received the replacement module It is important that all slots are filled when the chassis is in operation 1 Identif...

Страница 218: ...it from the chassis See also Figure 170 Figure 170 Removing Fan 2 IMPORTANT The fan slot must not be empty for more than 2 minutes while the chassis is powered Installing a 5U84 System Fan 1 You can h...

Страница 219: ...management interfaces the disk management utility GUI or CLI determine if the health of the new fan is OK Verify that the Module OK LED is green and that the Ops panel states show no amber module faul...

Страница 220: ...agement utility GUI or the CLI In a dual controller environment if replacing both controller modules you must adhere to the instructions provided in Before You Begin on page 180 Do not remove a faulty...

Страница 221: ...controller However the procedure applies to all controller and expansion IOMs used in 5U84 chassis They all use the same latch mechanism but feature different face plate geometry as shown in Figure 1...

Страница 222: ...llowed by No 1 in Figure 173 on page 208 You should hear a click as the latch handle engages and secures the controller or expansion IOM to its connector on the back of the midplane 5 Reconnect the ca...

Страница 223: ...ironment to accommodate chassis replacement Verifying Component Failure The RAID chassis or expansion chassis CRU includes the chassis metal housing the module runner system the integrated Ops panel c...

Страница 224: ...1 Facing the front of the 5U chassis access the drawers and remove DDICs label each DDIC from the damaged chassis and place them in a static protected work area nearby 1 4 a Opening a 5U84 Drawer on...

Страница 225: ...efore removing the chassis from the rack A suitable mechanical lift is required to move a high density 5U84 chassis 1 Disconnect the power cables and data cables between devices as needed a Between th...

Страница 226: ...the TLA serial number from the OEM serial number This section provides a procedure for installing the replacement chassis in its rack location IMPORTANT Refer to Installing 5U Chassis page 64 when in...

Страница 227: ...8 provides the 2U chassis dimensions Table 48 QXS 2U Chassis Dimensions Specification Metric Unit Imperial Unit Overall chassis height 2U 87 9 mm 3 46 in Width across mounting flange located on front...

Страница 228: ...Chassis Weights CRU Component 2U12 kg lb 2U24 kg lb 5U84 kg lb Storage chassis empty 4 8 10 56 4 8 10 56 64 141 Disk drive carrier 0 9 1 98 0 3 0 66 0 8 1 8 Dummy disk drive carrier air management sle...

Страница 229: ...00 non precipitating 29 C Table 52 QXS 2U 5U Additional Environmental Requirements Specification Measurement Description Airflow System must be operated with low pressure rear exhaust installation Bac...

Страница 230: ...t exceeding 5Pa 0 5 mm H2O Table 52 continued QXS 2U 5U Additional Environmental Requirements Specification Measurement Description Table 53 QXS 2U PSU 580W Specifications Specification Measurement De...

Страница 231: ...urement Description Table 54 QXS 5U PSU 2114W Specifications Specification Measurement Description Maximum output power 2214W maximum continuous output power at high line voltage Voltage 12 V at 183 A...

Страница 232: ...220 QXS G2 Hardware Installation and Maintenance Guide...

Страница 233: ...for Radio Frequency Interference USA Federal Communications Commission FCC Notice This equipment has been tested and found to comply with the limits for a class A digital device pursuant to Part 15 o...

Страница 234: ...ng interference that may cause undesired operation European Regulations This equipment complies with European Regulations EN 55022 Class A Limits and Methods of Measurement of Radio Disturbance Charac...

Страница 235: ...safety precautions detailed in the preceding chapters weight restrictions handling batteries and lasers etc when dismantling and disposing of this equipment Table 57 AC Power Cord Specifications 1 Un...

Страница 236: ...224 QXS G2 Hardware Installation and Maintenance Guide...

Страница 237: ...G2 412 QXS G2 424 and QXS G2 484 system controllers contain a USB Universal Serial Bus Device interface pertaining to the Management Controller MC The Device interface is accessed via a port on the co...

Страница 238: ...le the management computer should detect a new USB device Using the Emulated Serial Port interface the QXS G2 312 QXS G2 324 QXS G2 412 QXS G2 424 and QXS G2 484 system controller presents a single se...

Страница 239: ...provides a native USB serial driver that supports the controller module s USB CLI port However if using an older version of Windows you should download and install the USB device driver using the pro...

Страница 240: ...228 QXS G2 Hardware Installation and Maintenance Guide For worldwide support http www quantum com ServiceandSupport Index aspx...

Страница 241: ...e host and controller module for setting network port IP addresses Create New Connection To create a new connection or open an existing connection HyperTerminal 1 From the Windows Control Panel select...

Страница 242: ...emulator and connect to the COM port 4 Set network port IP addresses using the CLI see Setting Network Port IP Addresses Using CLI Port and Cable on page 94 NOTE When using Windows 10 Server 2016 wit...

Страница 243: ...ter covers the QXS G2 312 QXS G2 324 QXS G2 412 QXS G2 424 and QXS G2 484 systems Locate SFP Transceivers Locate the qualified SFP options for your FC iSCSI controller canister within your product shi...

Страница 244: ...ost port mode set iscsi parameters show ports Install SFP Transceiver For each target CNC port perform the following procedure to install an SFP Refer to the Figure 179 when performing the steps NOTE...

Страница 245: ...qualified fiber optic interface cable into the duplex jack at the end of the SFP connector Verify Component Operation View the CNC port Link Status Link Activity LED on the controller face plate A gre...

Страница 246: ...234 QXS G2 Hardware Installation and Maintenance Guide...

Результаты поиска

Содержание QXS G2 Series

Отзывы:

Похожие инструкции для QXS G2 Series

Бренды по названию

Популярные бренды

Quantum QXS G2 Series, Руководство по установке и обслуживанию оборудования

Результаты поиска

Содержание QXS G2 Series

Отзывы:

Похожие инструкции для QXS G2 Series

Бренды по названию

Популярные бренды