HGST ULTRASTAR SERV24 User Manual Download Page 183

Page: 183 / 231

a) Open a browser window navigate to the enclosure IP address.
b) Login to Web UI using the username and password.

Figure 265: Log-in

Web UI will appear displaying the Dashboard Control Panel.

2. Navigate to Sensor using the Navigation menu.

3. Select a sensor to view the information for that particular component.

4. Change and save Thresholds as needed.

4.3.8

Viewing System Inventory using Web UI

1. Log-in to the Web UI.

a) Open a browser window navigate to the enclosure IP address.
b) Login to Web UI using the username and password.

Figure 266: Log-in

Web UI will appear displaying the Dashboard Control Panel.

183

System Management

User Guide

4.3

Web UI Overview

Summary of Contents for ULTRASTAR SERV24

Page 1: ...User Guide Ultrastar Serv24 S2122 N24 4 Version 1 3 August 2018 1ET1083 Long Live Data www hgst com...

Page 2: ...out 18 1 6 Ultrastar Serv24 Rack Requirements 18 1 7 List of Replaceable Components 19 1 8 LEDs 20 1 8 1 Chassis LEDs 20 1 8 2 PSU LED 22 1 8 3 Drive LEDs 22 1 9 Input Output 23 1 9 1 Front I O Ports...

Page 3: ...on 37 2 6 1 Rails Specification 37 2 6 2 Rails Layout 38 Chapter 3 Part Replacement 39 3 1 Drive Assembly Replacement 39 3 2 PSU Replacement 41 3 3 CMA Replacement 44 3 4 Rails Replacement 48 3 5 DIMM...

Page 4: ...193 4 4 1 Accessing IPMI on Linux 193 4 4 1 1 Creating a New User Using IPMItool 193 4 4 1 2 Configuring Network Settings Using IPMItool 195 4 4 1 3 Checking Enclosure Sensors using IPMItool 196 4 4...

Page 5: ...rnings and Cautions 209 Chapter 7 Regulatory Statements 210 7 1 Europe CE Declaration of Conformity 210 7 2 FCC Class A Notice 210 7 3 ICES 003 Class A Notice Avis NMB 003 Classe A 210 7 4 Japanese Co...

Page 6: ...pendix B System Operations Appendices 223 B 1 ESD 223 B 2 System Cooling 223 B 3 Power Requirements 223 B 4 Host Connectivity 224 Appendix C Compatibility 225 C 1 Supported Operating Systems List 225...

Page 7: ...the Supported Operating Systems List on page 225 Added the Supported Parts List on page 226 Added the VROC Key Replacement on page 157 Revision 1 2 February 2018 Updated the following List of Replacea...

Page 8: ...tact Platform Engineering technical support Please be prepared to provide the following information Serial Number S N product name model number and a brief description of the issue Email support hgst...

Page 9: ...HGST representative or on the Internet at support hgst com HGST may have patents or pending patent applications covering subject matter in this document The furnishing of this document does not give y...

Page 10: ...orted by this document SKUs 1ES1005 1ES1004 1ES0415 1ES0412 1ES1009 1ES1008 1ES1007 1ES1006 1ES1013 1ES1012 1ES1011 1ES1010 1ES1017 1ES1016 1ES1015 1ES1014 1ES1021 1ES1020 1ES1019 1ES1018 1ES1025 1ES1...

Page 11: ...ns The maximum Ultrastar Serv24 power cannot exceed 1000W 100 127VAC low line for low line input operation Not all of the Ultrastar Serv24 configurations can operate using Lowline input power The maxi...

Page 12: ...One dedicated 1 GbE port for IPMI 12 1 Ultrastar Serv24 Overview User Guide 1 1 Ultrastar Serv24 Description...

Page 13: ...1 2 Block Diagrams Figure 2 System Block Diagram Figure 3 CPU PCIe Switch NVMe Drive Interface Block Diagram 13 1 Ultrastar Serv24 Overview User Guide 1 2 Block Diagrams...

Page 14: ...Figure 4 I2C Block Diagram 14 1 Ultrastar Serv24 Overview User Guide 1 2 Block Diagrams...

Page 15: ...C Temperature 20 C per hour max 30 C per hour max Temperature Gradient 1 C per 300m above 950m 1 C per 300m above 12000m Temperature De rating 8 85 Non Condensing 8 93 Non Condensing Relative Humidity...

Page 16: ...Operational Specification 10G 5 ms duration 5 shocks positive and 5 negative in each of 3 axes 30 total shocks 35G 152 in sec trapezoid wave Shock 0 278G Random 5 500 Hz 30 minutes per axis 3 axes X...

Page 17: ...Intel Hyper Threading Technology per core The Ultrastar Serv24 uses a Lewisburg PCH combined with the Skylake Server and Workstation to provide more bandwidth and flexibility in the I O It allows for...

Page 18: ...uirements The Ultrastar Serv24 is designed to be installed into a rack that meets the EIA 310 standard at a minimum See EIA 310 Rack Standard of usable space measured from front rack frame to the rear...

Page 19: ...W 1EX1038 CMA 1EX1037 Rails 1EX1035 Fan module 1EX1177 Intel VROC Premium Hardware Key 1EX1026 SanDisk X400 SSD M 2 2280 Boot Drive 256GB 1EX1674 Heat Sink 1EX1030 Intel Xeon CPU 5120 w heat sink 1EX1...

Page 20: ...Ds Figure 6 Chassis LEDs Table 7 Chassis LEDs Action LED Number Off Power off Green Power on Power 1 Off Chassis is not being identified Blue Chassis is being identified ID 2 20 1 Ultrastar Serv24 Ove...

Page 21: ...I O LEDs Table 8 Rear I O LEDs Action LED Number Off No network connection Green right LED Network connection NIC LEDs 1 is established Green left LED Blinking Indicates activity on the network Off P...

Page 22: ...ilure or disconnection on PSU 1 8 3 Drive LEDs Figure 9 Drive LEDs Top View 1 2 Table 10 Drive LEDs Action LED Number Off No drive present Green Drive present Green Blinking Drive active Drive Link Ac...

Page 23: ...ing Drive locate 1 9 Input Output 1 9 1 Front I O Ports and Features Figure 10 Front I O Ports and Features Table 11 Front I O Ports and Features Port Number VGA Port 1 USB 2 0 x2 2 Power ID and Fault...

Page 24: ...USB 3 0 x2 7 PSU A 8 PSU B 9 1 10 List of Compatible Drives Table 13 Compatible Drives Part Number Volume Drive 1EX1018 1 6 TB SanDisk Skyhawk Ultra 1 7DW D ISE 1EX1022 1 6 TB SanDisk Skyhawk Ultra 1...

Page 25: ...nDisk Skyhawk 0 5DW D ISE 1EX1021 3 84 TB SanDisk Skyhawk 0 5DW D SE 1EX1025 6 4 TB HGST Ultrastar SN200 3DW D SE 1EX1024 7 68 TB HGST Ultrastar SN200 1DW D SE 1EX1026 256 GB SanDisk X400 SSD M 2 2280...

Page 26: ...r the NIC Other major system components such as the fan modules CPUs memory and M 2 Boot Drives are all contained under the chassis cover The remaining system parts are attached to exterior of the cha...

Page 27: ...2 1 2 Chassis Layout Figure 13 Chassis Layout 1 2 3 4 Table 15 Chassis Layout Feature Number Drive Slots 1 Rear I O 2 PSUs 3 Chassis Cover 4 27 2 Component Overviews User Guide 2 1 Chassis Description...

Page 28: ...yout Figure 14 Chassis Internal Layout 1 2 3 4 5 Table 16 Chassis Internal Layout Feature Number CPUs 1 NIC under the PCIe bracket 2 M 2 Boot Drives 3 DIMMs 4 Fans 5 28 2 Component Overviews User Guid...

Page 29: ...t The PSUs are 80 Plus Platinum certified and utilize C14 power cable receptacles 2 2 1 PSU Specifications Table 17 PSU Specification Summary Value Specifications 1000W low line 1200W high line Power...

Page 30: ...2 PSU Layout Figure 16 PSU Layout 1 2 3 4 5 Table 18 PSU Layout Feature Number Power Receptacle 1 Fan 2 Handle 3 Latch Release Lever 4 Connector 5 30 2 Component Overviews User Guide 2 2 PSU Descript...

Page 31: ...of the data storage drive and caddies the drive into the system It stabilizes the motion of the drive into the drive cage so that the drive properly mounts onto the drive board 2 3 1 Drive Specificati...

Page 32: ...t Figure 18 Drive Layout 1 2 3 4 5 6 Table 20 Drive Layout Feature Number Latch Release 1 Drive Carrier Arm 2 Drive Connector 3 Latch 4 Drive Carrier 5 Drive 6 32 2 Component Overviews User Guide 2 3...

Page 33: ...4 The fans are installed inside the chassis in close proximity to the drive bay to provide efficient cooling to all of the drives 2 4 1 Fan Specification Table 21 Fan Specification Summary Value Speci...

Page 34: ...2 4 2 Fan Layout Figure 20 Fan Layout 1 2 3 Table 22 Fan Layout Feature Number Release Latches 1 Fan 2 Connector 3 34 2 Component Overviews User Guide 2 4 Fan Description...

Page 35: ...losure to remain on during the service of hot swappable parts by keeping the cables from pulling out of the receptacles The Ultrastar Serv24 contains one CMA arm that can be unlatched from one side to...

Page 36: ...t Figure 22 CMA Layout 1 2 3 4 Table 24 CMA Layout Feature Number Rail and Rack Connectors left 1 Elbow 2 Baskets five total 3 Rail and Rack Connectors right 4 36 2 Component Overviews User Guide 2 5...

Page 37: ...ontain a locking mechanism on either side to secure the enclosure in the rack during basic servicing procedures The locks are easily unlatched to provide the option to pull the enclosure out of the ra...

Page 38: ...t 1 2 3 4 5 Table 26 Rails Layout Features Number Rack Mount Rail 1 Rear Rack Mounting Bracket 2 Front Rack Mounting Bracket 3 Inner Rail mounts to enclosure 4 CMA Mounting Bracket 5 38 2 Component Ov...

Page 39: ...e by identifying the illuminated amber LED on the drive carrier 2 Uninstall the drive assembly a Release the drive assembly by pressing the button on the drive carrier and lower the carrier release un...

Page 40: ...he button on the drive carrier Figure 26 Drive Assembly Installation b With the drive carrier button on top slide the drive into the open drive slot until the driver release engages 40 3 Part Replacem...

Page 41: ...hat the drive appears in the OS 3 2 PSU Replacement Table 28 Replacement Procedure Info Time Required of People Required Required Tools 5 minutes 1 None 1 Locate the failed PSU by identifying the illu...

Page 42: ...vice position 3 From the rear of the enclosure disconnect the power cord connected to the failed PSU a From the rear of the enclosure locate the power cable b Remove the power cord by pulling firmly b...

Page 43: ...Latch b Remove the PSU by pulling on the PSU handle until it is clear of the PSU bay Figure 30 Removing the PSU 5 Install the PSU a Press the PSU release latch and push the PSU fully into the PSU slo...

Page 44: ...uired of People Required Required Tools 7 minutes 1 None 1 Power off the enclosure 2 From the rear of the enclosure disconnect the all of the cables attached to the enclosure a Remove the 10G SFP cabl...

Page 45: ...as shown in the following example Figure 32 CMA in service position 4 Uncable the CMA a Open all of the baskets on the CMA Figure 33 Open Baskets b Remove one cable at a time until the CMA is empty 4...

Page 46: ...stall the CMA a The CMA has one arm that is connected to the rails and the enclosure chassis by four connectors The first step in installing the arm for the CMA is to first determine what orientation...

Page 47: ...lose all of the baskets f Reconnect the CMA at the elbow to connector A 8 Test for binding in the extension of the CMA by gently pulling the enclosure out of the rack to ensure the cables extend prope...

Page 48: ...trol b Select the option next to ACPI Shutdown and click Perform Action A message Are you sure to perform this operation will appear Figure 39 Are You Sure c Click OK The enclosure will Power Off Figu...

Page 49: ...Figure 41 Rear IO Ports 3 Place the CMA into the service position a Unlatch the CMA at the elbow connector by pressing the blue release button to unlatch the connector from the rail b Swing the CMA a...

Page 50: ...as shown in the following example Figure 42 CMA in service position 4 Uncable the CMA a Open all of the baskets on the CMA Figure 43 Open Baskets b Remove one cable at a time until the CMA is empty 50...

Page 51: ...re are three total connections that need to be removed one at the elbow and two at the opposite end Figure 44 Unlatching a CMA Connector 6 Remove the enclosure from the rack a Unlatch the enclosure fr...

Page 52: ...nner rail toward the front of the enclosure to unlock it from the pegs that secure it to the enclosure and pull it free b Repeat the previous step to remove the remaining inner rail 8 Uninstall the ra...

Page 53: ...on with the other to free the rail from the rack and remove it Figure 47 Rail Removal 9 Remove the inner rail a Extend the inner portion of the rails until they lock in place Figure 48 Inner Rail Remo...

Page 54: ...ng inner rail 10 Install the inner rail onto the chassis a Orient the inner rail so that the flat side is facing the enclosure b Align the keyholes on the inner rail to the mounting pegs on the side o...

Page 55: ...ure that the rails are installed in their assigned configuration Install the rails into whole rack unit U sections as seen in the following image The rails will not mount properly if staggered across...

Page 56: ...lowest hole in the U d Pull the rail toward the rack step with the support posts aimed at the holes on the rack stems until the tool less latching mechanism engages the rack Figure 53 Front Rail Mecha...

Page 57: ...Operation g Latch the front rail mount to the rack by sliding the latch over the rack holes 12 Slide the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until...

Page 58: ...Enclosure Installed 13 Install the CMA a The CMA has one arm that is connected to the rails and the enclosure chassis by four connectors The first step in installing the arm for the CMA is to first de...

Page 59: ...rly and return it back to its proper location 14 Cable the CMA a Unlatch the elbow side of the CMA arm and swing it forward by pressing the blue button that says push to unlatch it and swing it out b...

Page 60: ...of cables in the baskets check the connections to the rails and examine the joints of the CMA arm to ensure that they are all functioning properly 16 Make sure the CMA is in operational position by fo...

Page 61: ...anel page will appear Figure 62 Dashboard Control Panel c Review the Dashboard Control Panel to identify the failed enclosure 2 Locate verify the enclosure a In the Web UI click Server Identify The Da...

Page 62: ...cate that it is the enclosure being accessed 3 From the rear of the enclosure disconnect the all of the cables attached to the enclosure a Remove the 10G SFP cable from the NIC port 1 b Remove the Eth...

Page 63: ...elease button to unlatch the connector from the rail b Swing the CMA away from the enclosure c The arm should be extended away from the enclosure as shown in the following example Figure 65 CMA in ser...

Page 64: ...a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 66 Enclosure Latches 64 3 Part Replacement User Guide 3 5 DIMM Replacement...

Page 65: ...fety latches to unlock the enclosure and pull the enclosure out of the rack 6 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the...

Page 66: ...Figure 69 Removing the Enclosure Cover 7 Uninstall the air duct modules a Remove the small air duct module by sliding it towards the rear of the enclosure Figure 70 Small Air Duct Removal Slide 66 3 P...

Page 67: ...n place pull up the fully remove it Figure 71 Small Air Duct Removal Lift c Remove the large air duct module by sliding it up and out of the enclosure Figure 72 Large Air Duct Removal 8 Uninstall the...

Page 68: ...IMM from the slot Figure 74 Removing the DIMM 9 Install the DIMM a Slide the DIMM into the correct DIMM slot and carefully press down until seated Figure 75 Removing the DIMM b Secure the DIMM by pres...

Page 69: ...into the enclosure over the PSUs Figure 76 Large Air Duct Installation b Install the small air duct module by sliding it towards the front of the enclosure Figure 77 Small Air Duct Installation 69 3...

Page 70: ...he cover latch down until it locks Figure 78 Locking the Enclosure Cover 12 Slide the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in p...

Page 71: ...s Figure 81 Enclosure Installed 13 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 14 Connec...

Page 72: ...wer itself on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Pane...

Page 73: ...ser and browse to the IP Address of the enclosure The Web UI will appear Figure 84 Web UI b In the Web UI click Dashboard The Dashboard Control Panel page will appear Figure 85 Dashboard Control Panel...

Page 74: ...c Click Perform Action The Server Identification LED will illuminate to indicate that it is the enclosure being accessed 3 From the rear of the enclosure disconnect the all of the cables attached to t...

Page 75: ...3 Figure 87 Rear IO Ports 4 Place the CMA into the service position a Unlatch the CMA at the elbow connector by pressing the blue release button to unlatch the connector from the rail b Swing the CMA...

Page 76: ...llowing example Figure 88 CMA in service position 5 Remove the enclosure from the rack a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 89...

Page 77: ...fety latches to unlock the enclosure and pull the enclosure out of the rack 6 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the...

Page 78: ...p on the sheet metal carrier until it is clear of the enclosure 8 Uninstall the NIC a Remove the screws securing the NIC to the enclosure board Figure 93 Remove Screws from the NIC b From the rear of...

Page 79: ...e NIC a From the inside of the enclosure slide the NIC into place with the connectors sticking out of the NIC slot in the enclosure Figure 95 Install NIC into Place b Slide the NIC plate over the NIC...

Page 80: ...on the guides and pressing it into the bracket 11 Install the enclosure cover a Seat the cover into place over the enclosure by placing the cover on the chassis and sliding it towards the front of th...

Page 81: ...e front of the rack until they are locked in place Figure 98 Rack Mounted Rails Locked b Line up the enclosure with the rack mounted rails and slide it in until the first set of locks engage 81 3 Part...

Page 82: ...s Figure 100 Enclosure Installed 13 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 14 Conne...

Page 83: ...ower itself on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Pan...

Page 84: ...ess of the enclosure The Web UI will appear Figure 103 Web UI b In the Web UI click Dashboard The Dashboard Control Panel page will appear Figure 104 Dashboard Control Panel c Review the Dashboard Con...

Page 85: ...it is the enclosure being accessed 3 From the rear of the enclosure disconnect the all of the cables attached to the enclosure a Remove the 10G SFP cable from the NIC port 1 b Remove the Ethernet cabl...

Page 86: ...ton to unlatch the connector from the rail b Swing the CMA away from the enclosure c The arm should be extended away from the enclosure as shown in the following example Figure 107 CMA in service posi...

Page 87: ...nlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 108 Enclosure Latches 87 3 Part Replacement User Guide 3 7 CPU and Heat Sink Module Replacem...

Page 88: ...es to unlock the enclosure and pull the enclosure out of the rack 6 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the enclosure...

Page 89: ...Removing the Enclosure Cover 7 Uninstall the air duct modules a Remove the small air duct module by sliding it towards the rear of the enclosure Figure 112 Small Air Duct Removal Slide 89 3 Part Repla...

Page 90: ...ully remove it Figure 113 Small Air Duct Removal Lift c Remove the large air duct module by sliding it up and out of the enclosure Figure 114 Large Air Duct Removal 8 Uninstall the Heat Sink and CPU m...

Page 91: ...Heat Sink must be removed by loosening the Torx T30 screws in a specific order Use the following diagram to determine the specific order Figure 115 Heat Sink Removal Order 4 3 2 1 Pin Pin 91 3 Part Re...

Page 92: ...et alignment pin on the baseboard and the alignment hole on the Heat Sink b Carefully place the CPU and Heat Sink module in position on the socket alignment pins c The Heat Sink must be installed by t...

Page 93: ...ened to a torque setting of 12in lbs Do not over tighten Over tightening may cause damage to the parts Figure 117 Install CPU Figure 118 Heat Sink Installation Order 93 3 Part Replacement User Guide 3...

Page 94: ...1 2 3 4 Pin Pin 11 Install the air duct modules 94 3 Part Replacement User Guide 3 7 CPU and Heat Sink Module Replacement...

Page 95: ...nclosure over the PSUs Figure 119 Large Air Duct Installation b Install the small air duct module by sliding it towards the front of the enclosure Figure 120 Small Air Duct Installation 95 3 Part Repl...

Page 96: ...tch down until it locks Figure 121 Locking the Enclosure Cover 13 Slide the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in place Figur...

Page 97: ...24 Enclosure Installed 14 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 15 Connect the pow...

Page 98: ...on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Panel page wil...

Page 99: ...d browse to the BMC IP Address of the enclosure The Web UI will appear Figure 127 Web UI b In the Web UI click Dashboard The Dashboard Control Panel page will appear Figure 128 Dashboard Control Panel...

Page 100: ...lick Perform Action The Server Identification LED will illuminate to indicate that it is the enclosure being accessed 3 From the rear of the enclosure disconnect the all of the cables attached to the...

Page 101: ...gure 130 Rear IO Ports 4 Place the CMA into the service position a Unlatch the CMA at the elbow connector by pressing the blue release button to unlatch the connector from the rail b Swing the CMA awa...

Page 102: ...ng example Figure 131 CMA in service position 5 Remove the enclosure from the rack a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 132 En...

Page 103: ...y latches to unlock the enclosure and pull the enclosure out of the rack 6 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the enc...

Page 104: ...re 135 Removing the Enclosure Cover 7 Uninstall the air duct modules a Remove the small air duct module by sliding it towards the rear of the enclosure Figure 136 Small Air Duct Removal Slide 104 3 Pa...

Page 105: ...p the fully remove it Figure 137 Small Air Duct Removal Lift c Remove the large air duct module by sliding it up and out of the enclosure Figure 138 Large Air Duct Removal 8 Uninstall the Heat Sink an...

Page 106: ...b The Heat Sink must be removed by loosening the Torx T30 screws in a specific order Use the following diagram to determine the specific order Figure 139 Heat Sink Removal Order 4 3 2 1 Pin Pin 106 3...

Page 107: ...Removing the CPU from the Heat Sink with a Spudger b Insert the edge of a plastic spudger in between the CPU and Heat Sink and carefully pry the two components apart The plastic spudger may need to b...

Page 108: ...iner Figure 142 CPU and CPR Retainer Triangles c Unpack and remove the protective plastic tray that is attached to the bottom of the Heat Sink The protective plastic tray protects the thermal compound...

Page 109: ...Carefully place the CPU and Heat Sink module in position on the socket alignment pins c The Heat Sink must be installed by tightening the Torx T30 screws in a specific order Use the following diagram...

Page 110: ...e tightened to a torque setting of 12in lbs Do not over tighten Over tightening may cause damage to the parts Figure 144 Install CPU Figure 145 Heat Sink Installation Order 110 3 Part Replacement User...

Page 111: ...1 2 3 4 Pin Pin 12 Install the air duct modules 111 3 Part Replacement User Guide 3 8 Heat Sink Replacement...

Page 112: ...o the enclosure over the PSUs Figure 146 Large Air Duct Installation b Install the small air duct module by sliding it towards the front of the enclosure Figure 147 Small Air Duct Installation 112 3 P...

Page 113: ...over latch down until it locks Figure 148 Locking the Enclosure Cover 14 Slide the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in plac...

Page 114: ...igure 151 Enclosure Installed 15 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 16 Connect...

Page 115: ...itself on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Panel p...

Page 116: ...otal of two M 2 boot drives Only one drive may be changed at a time If both drives have failed replace the first one completely before removing and replacing the remaining drive 1 Identify the failed...

Page 117: ...entify The Dashboard Server Identify page will appear b In the Identify Actions section select UID LED ON and set the Server Identify Timeout Figure 156 Server Identify c Click Perform Action The Serv...

Page 118: ...his operation will appear Figure 158 Are You Sure c Click OK The enclosure will Power Off Figure 159 Power Off 4 Place the CMA into the service position a Unlatch the CMA at the elbow connector by pre...

Page 119: ...mple Figure 160 CMA in service position 5 From the rear of the enclosure disconnect the all of the cables attached to the enclosure a Remove the 10G SFP cable from the NIC port 1 b Remove the Ethernet...

Page 120: ...c Remove the power cord by pulling firmly but do not jerk it out of the unit 3 Figure 161 Rear IO Ports 6 Remove the enclosure from the rack 120 3 Part Replacement User Guide 3 9 M 2 Replacement...

Page 121: ...a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 162 Enclosure Latches 121 3 Part Replacement User Guide 3 9 M 2 Replacement...

Page 122: ...fety latches to unlock the enclosure and pull the enclosure out of the rack 7 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the...

Page 123: ...the Enclosure Cover 8 Uninstall the M 2 boot drive a Remove the middle PCIe bracket by carefully pulling up on the sheet metal carrier until it is clear of the enclosure Figure 166 PCIe Bracket Locati...

Page 124: ...he M 2 boot drive to the board Figure 167 M 2 Boot Drive Screw Removal c Remove the drive from the drive slot by pulling it out of the connector Figure 168 M 2 Boot Drive Removal 124 3 Part Replacemen...

Page 125: ...2 boot drive on the replacement enclosure 10 Install the M 2 boot drive a Slide the card edge of the drive into the connector with even pressure Figure 169 M 2 Boot Drive Installation 125 3 Part Repl...

Page 126: ...drive d Carefully press down on the sheet metal taking care to line up the board in the bracket with the connector on the board in the enclosure 11 Install the enclosure cover a Seat the cover into pl...

Page 127: ...the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in place Figure 172 Rack Mounted Rails Locked b Line up the enclosure with the rack mo...

Page 128: ...s Figure 174 Enclosure Installed 13 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 14 Conne...

Page 129: ...wer itself on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Pane...

Page 130: ...ailed enclosure a Open a browser and browse to the BMC IP Address of the enclosure The Web UI will appear Figure 177 Web UI b In the Web UI click Dashboard The Dashboard Control Panel page will appear...

Page 131: ...Server Identification LED will illuminate to indicate that it is the enclosure being accessed 3 Power Off the enclosure a In the Web UI click Power Control The Power Control page will appear Figure 18...

Page 132: ...rail b Swing the CMA away from the enclosure c The arm should be extended away from the enclosure as shown in the following example Figure 183 CMA in service position 5 From the rear of the enclosure...

Page 133: ...c Remove the power cord by pulling firmly but do not jerk it out of the unit 3 Figure 184 Rear IO Ports 6 Remove the enclosure from the rack 133 3 Part Replacement User Guide 3 10 Chassis Replacement...

Page 134: ...a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 185 Enclosure Latches 134 3 Part Replacement User Guide 3 10 Chassis Replacement...

Page 135: ...hes and pull out of the rack until the rail latches engage Figure 186 Enclosure c Slide the rail safety latches to unlock the enclosure and pull the enclosure out of the rack 7 Uninstall the PSU 135 3...

Page 136: ...Remove the PSU by pulling on the PSU handle until it is clear of the PSU bay Figure 188 Removing the PSU 8 Repeat the previous step to remove the remaining PSU 9 Identify the location of the PSU slot...

Page 137: ...latches 11 Repeat the previous step to install the remaining PSU into the new enclosure 12 Uninstall the drives a Release the drive by pressing the button on the drive carrier and lowering the carrie...

Page 138: ...Figure 190 Removing the Drive 13 Repeat the previous step to uninstall all of the drives from the enclosure 14 Install the drives a Open the drive carrier release by pressing the button on the drive c...

Page 139: ...ll make a connection with the connector on the drive board 15 Repeat the previous step to install all of the drives into the enclosure 16 Remove the enclosure cover a If the enclosure lock is secured...

Page 140: ...re 193 Removing the Enclosure Cover 17 Uninstall the air duct modules a Remove the small air duct module by sliding it towards the rear of the enclosure Figure 194 Small Air Duct Removal Slide 140 3 P...

Page 141: ...ace pull up the fully remove it Figure 195 Small Air Duct Removal Lift c Remove the large air duct module by sliding it up and out of the enclosure Figure 196 Large Air Duct Removal 18 Uninstall the D...

Page 142: ...e previous step to uninstall all of the DIMMs from the enclosure 20 Identify the location of the DIMM slots on the new enclosure Ensure that the DIMM is installed in the same locations as the failed e...

Page 143: ...side of the connector 22 Repeat the previous step to Install all of the DIMMs into the new enclosure 23 Uninstall the Heat Sink and CPU module a Locate the failed CPU and Heat Sink module CPU1 CPU0 14...

Page 144: ...moved by loosening the Torx T30 screws in a specific order Use the following diagram to determine the specific order Figure 200 Heat Sink Removal Order 4 3 2 1 Pin Pin 144 3 Part Replacement User Guid...

Page 145: ...Sink module a Locate the socket alignment pin on the baseboard and the alignment hole on the Heat Sink b Carefully place the CPU and Heat Sink module in position on the socket alignment pins c The He...

Page 146: ...e tightened to a torque setting of 12in lbs Do not over tighten Over tightening may cause damage to the parts Figure 202 Install CPU Figure 203 Heat Sink Installation Order 146 3 Part Replacement User...

Page 147: ...1 2 3 4 Pin Pin 26 Repeat the previous step to install the remaining heat sink containing the CPU 27 Uninstall the M 2 boot drive 147 3 Part Replacement User Guide 3 10 Chassis Replacement...

Page 148: ...sheet metal carrier until it is clear of the enclosure Figure 204 PCIe Bracket Location and Removal b Remove the screw that secures the M 2 boot drive to the board Figure 205 M 2 Boot Drive Screw Rem...

Page 149: ...ious step to uninstall the remaining M 2 boot drive 29 Identify the location of the first M 2 boot drive on the new enclosure 30 Install the M 2 boot drive a Slide the card edge of the drive into the...

Page 150: ...acket into place over the M 2 boot drive d Carefully press down on the sheet metal taking care to line up the board in the bracket with the connector on the board in the enclosure 31 Repeat the previo...

Page 151: ...e enclosure remove the screws that secure the NIC plate in place and remove the NIC plate c From inside the enclosure lift the NIC up and out of the enclosure Figure 210 Remove the NIC 33 Identify the...

Page 152: ...11 Install NIC into Place b Slide the NIC plate over the NIC ports and secure it in place with the provided screws c From inside of the enclosure secure the NIC to the enclosure board with the provide...

Page 153: ...to the enclosure over the PSUs Figure 213 Large Air Duct Installation b Install the small air duct module by sliding it towards the front of the enclosure Figure 214 Small Air Duct Installation 153 3...

Page 154: ...over latch down until it locks Figure 215 Locking the Enclosure Cover 37 Slide the enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in plac...

Page 155: ...igure 218 Enclosure Installed 38 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 39 Connect...

Page 156: ...t 3 Figure 219 Rear IO Ports 40 From the rear of the enclosure disconnect the all of the cables attached to the enclosure a Remove the 10G SFP cable from the NIC port 1 b Remove the Ethernet cable fro...

Page 157: ...s after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Panel page will appear Figure 221...

Page 158: ...ou sure to perform this operation will appear Figure 223 Are You Sure c Click OK The enclosure will Power Off Figure 224 Power Off 2 From the rear of the enclosure disconnect the all of the cables att...

Page 159: ...gure 225 Rear IO Ports 3 Place the CMA into the service position a Unlatch the CMA at the elbow connector by pressing the blue release button to unlatch the connector from the rail b Swing the CMA awa...

Page 160: ...ng example Figure 226 CMA in service position 4 Remove the enclosure from the rack a Unlatch the enclosure from the rack mounted by rotating the enclosure latches away from the enclosure Figure 227 En...

Page 161: ...y latches to unlock the enclosure and pull the enclosure out of the rack 5 Remove the enclosure cover a If the enclosure lock is secured use a Philips Screwdriver to unlock the latch b Unlatch the enc...

Page 162: ...Figure 231 VROC Connector Location b Pulling carefully on the VROC loop unplug the VROC key from the VROC connector on the motherboard 7 Install the VROC key onto the motherboard a Pressing firmly pl...

Page 163: ...enclosure into the rack a Pull the rack mounted rails out towards the front of the rack until they are locked in place Figure 233 Rack Mounted Rails Locked b Line up the enclosure with the rack mounte...

Page 164: ...igure 235 Enclosure Installed 10 Connect the CMA to the enclosure a Move the CMA arm inward towards the enclosure b Connect the CMA arm by pressing the CMA bracket into the rail connection 11 Connect...

Page 165: ...itself on about 20 seconds after it is connected to power b Once the enclosure is physically powered on from a laptop log into the Web UI c In the Web UI click Dashboard The Dashboard Control Panel p...

Page 166: ...e The IPMI management option uses IPMItool for enclosures that have a Linux operating system OS installed and IPMIutil for enclosures that have Windows OS installed firmware updates are completed usin...

Page 167: ...and Downloads page using a username and password Note If product has not been registered complete the registration first before continuing Figure 240 Customer Technical Support and Downloads 6 Access...

Page 168: ...9 Download the firmware by clicking Download next the latest firmware zip file Note In some cases more than one type of firmware package might need to be downloaded Ultrastar Serv24 contains firmware...

Page 169: ...handle out of band OOB management Web UI is intended to provide ease of control and customization while accessing the Ultrastar Serv24 This user interface is designed to easily create new users contr...

Page 170: ...as to offer It has the option to remain in the browser at all times for ease of navigation between different functions and can also be hidden to display only the Main Dashboard and Quick Buttons Menu...

Page 171: ...itions Each diagnostic that is displayed in the Dashboard Control Panel can be accessed quickly by clicking the cards 4 3 1 Web UI Supported Browsers Table 37 Supported Web UI Browsers by Operating Sy...

Page 172: ...247 User Management 3 Select an unassigned user card The User Management Configuration Page will appear 4 Configure the required User Settings Username create a Username Password create a Password Ena...

Page 173: ...to all functions within the Web UI The other user privilege levels contain limitations to what functions and features can be accessed or configured Figure 248 User Management Configuration Required F...

Page 174: ...Management Configuration Other User Privileges 6 From the Quick Buttons Bar select admin Sign out A Would you like to Sign out of this Session message will display 7 Click OK The current session will...

Page 175: ...hanging Network Settings in Web UI 1 Log in to the Web UI a Open a browser window navigate to the enclosure IP address b Login to Web UI using the username and password Figure 251 Log in Web UI will a...

Page 176: ...P Settings 3 Complete the necessary fields and save 4 3 4 Changing the DNS Configurations in Web UI 1 Log in to the Web UI a Open a browser window navigate to the enclosure IP address b Login to Web U...

Page 177: ...ion menu Figure 254 DNS Configuration 3 Complete the necessary fields and save 4 3 5 Updating BIOS Firmware in Web UI 1 Download the new BIOS image bin to a laptop from the HGST Support website 2 Log...

Page 178: ...Figure 255 Log in Web UI will appear displaying the Dashboard Control Panel 3 Navigate to Maintenance BIOS Firmware Update using the Navigation menu Figure 256 Navigate to BIOS Firmware Update 178 4...

Page 179: ...You Want to Continue 7 Click OK A Processing message will appear and the Uploading status bar in the background will display the progress of the update The Web UI session will be ended and the BIOS fi...

Page 180: ...rol Panel 3 Navigate to Maintenance Firmware Update using the Navigation menu Figure 260 Firmware Update 4 Select the checkbox next to Preserve all Configuration The Preserve Configuration table will...

Page 181: ...guration 5 Browse to the location of the new Firmware image 6 Select the image and click OK The image will appear in the Select Firmware Image section 7 Click Start firmware update A Do you want to co...

Page 182: ...s 10 Click OK The Web UI session will be ended 11 Log in to Web UI and navigate to Maintenance BMC Firmware Information Figure 264 BMC Firmware Information 12 Verify that the firmware version displays...

Page 183: ...n menu 3 Select a sensor to view the information for that particular component 4 Change and save Thresholds as needed 4 3 8 Viewing System Inventory using Web UI 1 Log in to the Web UI a Open a browse...

Page 184: ...2 Navigate to System Inventory using the Navigation menu Figure 267 Navigation Menu System Inventory 184 4 System Management User Guide 4 3 Web UI Overview...

Page 185: ...3 View the System Inventory Block Diagram by selecting the Block Diagram option Figure 268 System Inventory Information Block Diagram 185 4 System Management User Guide 4 3 Web UI Overview...

Page 186: ...ut View 4 3 9 Viewing System Logs and Reports in Web UI 1 Log in to the Web UI a Open a browser window navigate to the enclosure IP address b Login to Web UI using the username and password Figure 270...

Page 187: ...ogs a Select a Start Date and End Date from the Filter by Date droplists b Select an Event Type from the All Events droplist c Select a Sensor from the All Sensors droplist d Select the Timezone for t...

Page 188: ...n Menu Logs and Reports 6 Filter the System Logs a Select a Start Date and End Date from the Filter by Date droplists b Select an Alert from the Event Category droplist Figure 274 System Log Filter 7...

Page 189: ...enu Figure 275 Navigation Menu Logs and Reports 9 Filter the Audit Logs by selecting a Start Date and End Date from the Filter by Date droplists Figure 276 Audit Log Filter 10 Review the resulting Aud...

Page 190: ...eo Logs by selecting a Start Date and End Date from the Filter by Date droplists Figure 278 Video Log Filter 13 Review the resulting Video Logs 4 3 10 Identifying the Enclosure using Web UI 1 Log in t...

Page 191: ...displaying the Dashboard Control Panel 2 Navigate to Server Identify using the Navigation menu Figure 280 Server Identify 3 Select Option and perform action 4 3 11 Power Off Reset the Enclosure using...

Page 192: ...password Figure 281 Log in Web UI will appear displaying the Dashboard Control Panel 2 Navigate to Power Control using the Navigation menu Figure 282 Power Control 3 Select the option to power off or...

Page 193: ...c set of commands and options to interact with the enclosure This list of commands and options can be accessed from the Terminal by executing the command ipmiutil h or they can be viewed at IPMIutil C...

Page 194: ...l lan H IP ADDRESS U admin P admin user set name ID NUMBER USERNAME The new user will be created Figure 285 New User 5 Verify the new user has been created by issuing the command ipmitool l lan H IP...

Page 195: ...ss will be static or DHCP by issuing the command ipmitool l lan H IP ADDRESS U admin P admin lan set ipsrc STATIC or DHCP The Terminal will display set commands 4 Set a static IP address by issuing th...

Page 196: ...sensors by issuing the command ipmitool l lan H IP ADDRESS U USERNAME P PASSWORD sdr list The status of the enclosure sensors will be displayed Figure 289 Enclosure Sensors 3 Check the status of the...

Page 197: ...ORD sdr type fan The temperature of the enclosure fans will be displayed Figure 291 Fan Temperature Sensors 4 4 1 4 Viewing System Logs and Reports using IPMItool 1 Launch a Terminal by clicking Appli...

Page 198: ...l lan H IP ADDRESS U USERNAME P PASSWORD power off The system will power off 6 Reset the enclosure by issuing the command ipmitool l lan H IP ADDRESS U USERNAME P PASSWORD power reset The system power...

Page 199: ...IP ADDRESS U USERNAME P PASSWORD The status of the enclosure sensors will be displayed Figure 293 Enclosure Sensors 4 4 2 4 Viewing System Logs and Reports using IPMIutil 1 Launch a Command Prompt by...

Page 200: ...ower Off Reset the Enclosure using IPMIutil 1 Launch a Command Prompt by clicking Start Command Prompt The IPMIutil Terminal will appear 2 Determine if the enclosure should be powered off or restarted...

Page 201: ...h to the laptop being used 3 Launch a command line interface by doing one of the following For Linux OS Launch a Terminal by clicking Applications Accessories Terminal For Windows OS Launch a Command...

Page 202: ...atus will appear and the BIOS firmware will be updated after a few minutes Figure 299 Upgrade Status 5 The BIOS Firmware must be completed prior to the BMC Firmware update 4 5 3 Updating BMC Firmware...

Page 203: ...ng the specific image Figure 300 Upgrade BMC Firmware 2 Type Y and press Enter An Uploading Firmware Image message with status will appear and the BMC firmware will be updated after a few minutes Figu...

Page 204: ...s through the use of a tool or lock and key or other means of security and is controlled by the authority responsible for the location 5 3 Regulatory Statement Product Name Ultrastar Serv24 Regulatory...

Page 205: ...ns and Flicker EN 55024 KN35 EN 61000 4 2 ESD EN 61000 4 3 Radiated Immunity EN 61000 4 4 EFT EN 61000 4 5 Surge EN 61000 4 6 RF Common Mode EN 61000 4 8 Power Frequency Magnetic Field EN 61000 4 11 V...

Page 206: ...Authority or Mark Country Region SII Israel NRCS South Africa 206 5 Disclaimers User Guide 5 5 Country Certifications...

Page 207: ...product can result in lowered system performance or system failure Do not position the unit in an environment that has extreme high temperatures or extreme low temperatures Be aware of the proximity o...

Page 208: ...tions provided within the rackmount instructions Reliable grounding of rack mounted equipment should be maintained If installed in a closed or multi unit rack assembly the operating ambient temperatur...

Page 209: ...safety instructions and information The following safety symbols may be used throughout the documentation and may be marked on the product and or the product packaging CAUTION Indicates the presence...

Page 210: ...ce complies with Part 15 of the FCC Rules Operation is subject to the following two conditions 1 This device may not cause harmful interference 2 This device must accept any interference received incl...

Page 211: ...t this product may cause radio interference in which case the user may be required to take adequate measures Declaration of the Presence Condition of the Restricted Substances Marking Table 39 Declara...

Page 212: ...tricted substance exceeds the reference percentage value of presence condition 2 O Note O indicates that the percentage content of the restricted substance does not exceed the percentage of reference...

Page 213: ...ent Log SEL sel Configure Platform Event Filtering PEF pef Configure and connect IPMIv2 0 Serial over LAN sol Configure and connect with Tyan IPMIv1 5 Serial over LAN tsol Configure IPMIv1 5 Serial ov...

Page 214: ...ONE PASSWORD MD2 MD5 or OEM A authtype Remote session password P password Read password from IPMI_PASSWORD environment variable E Set local IPMB address m address Set destination channel for bridged r...

Page 215: ...m get IPMI events and display them event daemon getevt get IPMI events and display them event daemon getevent check and show the basic health of the IPMI BMC health HPM firmware update manager extensi...

Page 216: ...Z A 6 Standard IPMI Command Support List Table 40 CMD NetFn Commands IPM Device Global Commands 01h App Get Device ID 01h App Broadcast Get Device ID 02h App Cold Reset 03h App Warm Reset 04h App Get...

Page 217: ...hannel Access 42h App Get Channel Info 43h App Set User Access 44h App Get User Access 45h App Set User Name 46h App Get User Name 47h App Set User Password 48h App Activate Payload 49h App Deactivate...

Page 218: ...l 0Fh Chassis Get POH Counter Event Commands 00h S E Set Event Receiver 01h S E Get Event Receiver 02h S E Platform Event a k a Event Message PEF and Alerting Commands 10h S E Get PEF Capabilities 11h...

Page 219: ...Allocation Info 22h Storage Reserve SDR Repository 23h Storage Get SDR 25h Storage Partial Add SDR 26h Storage Delete SDR 27h Storage Clear SDR Repository 2Ah Storage Enter SDR Repository Update Mode...

Page 220: ...Connection Active 19h Transport Callback 1Ah Transport Set User Callback Options 1Bh Transport Get User Callback Options 21h Transport Set SOL Configuration Parameters 22h Transport Get SOL Configura...

Page 221: ...s information about current FW Sections img section info Displays information about current FW Versions img info Option to replace the Public Key in Existing Firmware replace publickey Option to prese...

Page 222: ...enter host name when using Network Medium u Option to enter UserName p Option to enter Password when using Network Medium kcs Option to use KCS medium 222 A System Management Appendices User Guide A 9...

Page 223: ...enclosure and all of its components In the case of a warning sensor threshold violation the OTP mechanism will report the warning information to the user through system management protocols If the cr...

Page 224: ...e specified maximum input current B 4 Host Connectivity The following table displays the list of power cables approved by HGST Table 42 Approved Power Cables Dimension overmold Part Number Type C13 to...

Page 225: ...ed Hat Enterprise Linux x86_64 7 4 VMD not supported Novell SUSE Linux Enterprise Server 12 SP1 VMD not supported Novell SUSE Linux Enterprise Server 12 SP2 Fully supported Novell SUSE Linux Enterpris...

Page 226: ...ld 6150 QMQ6 H0 Stepping 2 70 GHz 3 UPI 10 4GT s 18 Cores 24 75MB 165W Vendor Model Intel 6148 Intel Xeon Gold 6148 QMS1 H0 Stepping 2 40 GHz 3 UPI 10 4GT s 20 Cores 27 5MB 150W 1EX1031 Intel Xeon Gol...

Page 227: ...32GB DDR4 2666 ECC Reg 2Rx4 Vendor Model M393A2K43BB1 CRC Samsung DIMM RDIMM 16GB DDR4 2400 ECC Reg 2Rx8 Vendor Model M386A8K40BM1 CRC Samsung DIMM LRDIMM 64GB DDR4 2400 ECC Reg 4Rx4 Vendor Model M38...

Page 228: ...d Add in Cards Agile or Vendor Model Number Add in Card Vendor Model SAS 9300 8i Avago SAS HBA Adapter 12Gb s Mini SAS HD PCIe 3 0 x8 8 Port Vendor Model SAS 9305 16i Avago SAS HBA Adapter 12Gb s Mini...

Page 229: ...Port Vendor Model QLE3442 RJ SP Qlogic NIC Adapter 10Gb s RJ45 PCIe 3 0 x8 2 Port Vendor Model QLE3442 CU SP Qlogic NIC Adapter 10Gb s SFP PCIe 3 0 x8 2 Port Vendor Model QLE8362 CU Qlogic NIC Adapter...

Page 230: ...R1048 G0008 01 Celestica Expander 12Gb s PCIe 3 0 x8 2 port to 7 ports Table 48 Supported Cables Agile or Vendor Model Number Cable Type Cable Vendor Model 10123482 4030LF Rev 3 0 Mini SAS HD FCI SAS...

Page 231: ...l 31P1549 SFP 10G module Finisar Module Vendor Model 77P8042 SFP 10G module IBM Module Vendor Model FTLC9551REPM 100G Base SR4 SFP 100G module Finisar Module 1EX1027 Power Cable HGST AC Power Cord 2 5...

Search results

Summary of Contents for ULTRASTAR SERV24

Reviews:

Related manuals for ULTRASTAR SERV24

Brands by name

Popular brands

HGST ULTRASTAR SERV24, User Manual

Search results

Summary of Contents for ULTRASTAR SERV24

Reviews:

Related manuals for ULTRASTAR SERV24

Brands by name

Popular brands