IBM ESS 3000 Page 100 Hardware Planning And Installation Manual Download

Page: 100 / 108

non-quorum node

A node in a cluster that is not counted for the purposes of quorum determination.

O
OFED

See OpenFabrics Enterprise Distribution (OFED).

OpenFabrics Enterprise Distribution (OFED)

An open-source software stack includes software drivers, core kernel code, middleware, and user-

level interfaces.

P
pdisk

A physical disk.

PortFast

A Cisco network function that can be configured to resolve any problems that could be caused by the

amount of time STP takes to transition ports to the Forwarding state.

R
RAID

See redundant array of independent disks (RAID).

RDMA

See remote direct memory access (RDMA).

redundant array of independent disks (RAID)

A collection of two or more disk physical drives that present to the host an image of one or more

logical disk drives. In the event of a single physical device failure, the data can be read or regenerated

from the other disk drives in the array due to data redundancy.

recovery

The process of restoring access to file system data when a failure has occurred. Recovery can involve

reconstructing data or providing alternative routing through a different server.

recovery group (RG)

A collection of disks that is set up by IBM Spectrum Scale RAID, in which each disk is connected

physically to two servers: a primary server and a backup server.

remote direct memory access (RDMA)

A direct memory access from the memory of one computer into that of another without involving

either one's operating system. This permits high-throughput, low-latency networking, which is

especially useful in massively-parallel computer clusters.

RGD

See recovery group data (RGD).

remote key management server (RKM server)

A server that is used to store master encryption keys.

See recovery group (RG).

recovery group data (RGD)

Data that is associated with a recovery group.

RKM server

See remote key management server (RKM server).

S
SAS

See Serial Attached SCSI (SAS).

88 IBM Elastic Storage System 3000: Hardware Planning and Installation Guide

Summary of Contents for ESS 3000

Page 1: ...IBM Elastic Storage System 3000 Version 6 0 0 Hardware Planning and Installation Guide IBM SC28 3107 01...

Page 2: ...ct number 5765 DME IBM Spectrum Scale Data Access Edition for IBM ESS product number 5765 DAE IBM welcomes your comments see the topic How to submit your comments on page xii When you send information...

Page 3: ...2 Japan Voluntary Control Council for Interference VCCI Notice 12 Korea Notice 13 People s Republic of China Notice 13 Russia Notice 13 Taiwan Notice 13 United States Federal Communications Commission...

Page 4: ...nt Server 5148 21L 47 Connect power cords to rack PDUs 47 SAS Cabling for ESS 48 ESS Network Cabling 48 Connect AC wall power to the rack PDUs 48 Power On Switches Storage Enclosures and Servers 48 Co...

Page 5: ...Glossary 83 Index 91 Index 91 v...

Page 6: ...vi...

Page 7: ...canister 25 11 Rear view of an Elastic Storage System 3000 26 12 Sample illustration to show space around the rack 32 13 Acoustical Declaration with Noise Hazard Notice 34 14 Label showing the orienta...

Page 8: ...MS node 53 29 Rear view of the ESS 3000 enclosure 54 30 essutils tool 55 31 Canister SSR Tools menu options 55 32 Connecting a laptop to the serial port in the EMS node 59 33 Front and rear view of IB...

Page 9: ...characteristics of the enclosures 31 10 Rack space requirements for the Elastic Storage System 3000 32 11 Service clearance requirements 32 12 Clearances 33 13 Drive specifications 33 14 Shock testin...

Page 10: ...x...

Page 11: ...nt system elements that you must use literally such as commands flags values and selected menu options Depending on the context bold typeface sometimes represents path names directories or file names...

Page 12: ...tion Ctrl x indicates a control character sequence For example Ctrl c means that you hold down the control key while pressing c item Ellipses indicate that you can repeat the preceding item one or mor...

Page 13: ...M Japan Ltd 19 21 Nihonbashi Hakozakicho Chuo ku Tokyo 103 8510 Japan INTERNATIONAL BUSINESS MACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED...

Page 14: ...urce language which illustrate programming techniques on various operating platforms You may copy modify and distribute these sample programs in any form without payment to IBM for the purposes of dev...

Page 15: ...injury D002 2 Locate the IBM Systems Safety Notices with the user publications that were provided with your system hardware 3 Find the matching identification number in the IBM Systems Safety Notices...

Page 16: ...afety notices to ensure that you are in compliance Danger notices for the system Ensure that you are familiar with the danger notices for your system Use the reference numbers in parentheses at the en...

Page 17: ...s it is properly packaged secured on top of the supplied pallet R004 DANGER Main Protective Earth Ground This symbol is marked on the frame of the rack The PROTECTIVE EARTHING CONDUCTORS should be ter...

Page 18: ...aper manual must remain with machine in provided storage sleeve area Latest revision manual available on vendor s website Test verify stabilizer brake function before each use Do not over force moving...

Page 19: ...being raised Be sure winch is locked in position before releasing handle Read instruction page before operating this winch Never allow winch to unwind freely Freewheeling will cause uneven cable wrap...

Page 20: ...general rules to ensure safety to yourself and others Observe good housekeeping in the area where the devices are kept during and after maintenance Follow the guidelines when lifting any heavy object...

Page 21: ...s injury To inspect each node for unsafe conditions use the following steps If necessary see any suitable safety publications 1 Turn off the system and disconnect the power cord 2 Check the frame for...

Page 22: ...ur movement Movement can cause static electricity to build up around you Handle the device carefully holding it by its edges or frame Do not touch solder joints pins or exposed printed circuitry Do no...

Page 23: ...inie 2014 30 EU zur Angleichung der Rechtsvorschriften ber die elektromagnetische Vertr glichkeit in den EU Mitgliedsstaatenund h lt die Grenzwerte der EN 55032 Klasse A ein Um dieses sicherzustellen...

Page 24: ...ngen nach EN 55024 und EN 55032 Klasse A Japan Electronics and Information Technology Industries Association JEITA Notice This statement applies to products less than or equal to 20 A per phase This s...

Page 25: ...mmission FCC Notice This equipment has been tested and found to comply with the limits for a Class A digital device pursuant to Part 15 of the FCC Rules These limits are designed to provide reasonable...

Page 26: ...ed by using other than recommended cables and connectors or by unauthorized changes or modifications to this equipment Unauthorized changes or modifications could void the user s authority to operate...

Page 27: ...m 3000 can contain up to 24 NVMe attached SSD drives 12 drives half populated or 24 drives fully populated The drives are accessible from the front of ESS 3000 as shown in Figure 1 on page 15 Note The...

Page 28: ...ure based all inclusive software feature licensing NVMe transport protocol for high performance of 2 5 inch SFF NVMe attached flash drives Support for industry standard 2 5 inch NVMe attached SSD driv...

Page 29: ...8 x x Clustering Data Network 100 Gb EDR 100 GbE External Campus Network EMS ems1 HMC 1 HMC 1 HMC 1 C10 T3 C10 T4 10 0 0 1 C10 T2 Container Bridge Figure 4 ESS 3000 Sharing Existing Networks NVMe tran...

Page 30: ...EDR 2 x 100GE 2 x IB EDR 2 x 100GE 2 x IB EDR 2 x 100GE 2 x IB EDR 2 x 100GE HCA IB Enet I O Server Canister 1 I O Server Canister 1 ESS 3000 Hardware High Level Architecture and Topology ess30020 Fig...

Page 31: ...e 6 High level view of system management Drives ESS 3000 supports industry standard 2 5 inch NVMe attached SSD drive options The following storage capacities are supported 1 92 TB 3 84 TB 7 68 TB and...

Page 32: ...the canisters in the Elastic Storage System 3000 has a 1 Gb Ethernet SSR access port as shown in Figure 7 on page 20 Because canister 2 is inverted the plug of the Ethernet cable also needs to be inve...

Page 33: ...tion Before the IBM SSR installs the system you must plan the physical configuration and the initial data configuration Certain physical site specifications must be met before you can set up your syst...

Page 34: ...canister which means 48 DIMM slots per enclosure You can install two distinct memory configurations in those 48 DIMM slots Initially every ESS 3000 ships with the ACG1 feature Feature Memory per enclo...

Page 35: ...GB F1 dummy 32 GB Slots for CPU2 Content of each DIMM slot A0 32 GB 32 GB A1 dummy 32 GB B0 32 GB 32 GB B1 dummy 32 GB C0 32 GB 32 GB C1 dummy 32 GB D0 32 GB 32 GB D1 dummy 32 GB E0 32 GB 32 GB E1 du...

Page 36: ...the server canister chassis is oriented with the three PCIe HBAs in card risers to the right rear Three fan modules to the left sit atop the NVMe attached Flash drives at the front of the server cani...

Page 37: ...e independently if there is a loss of input power to the other power supply in the enclosure Plan to connect the power cords of the power supplies on the left side of the enclosures when viewed from t...

Page 38: ...Cord 4 3m 14 ft Drawer to IBM PDU 250V 10A with C13 C14 connectors 6671 Power Cord 2 7m 9 ft Drawer to IBM PDU 250V 10A with C13 C14 connectors 6672 Power Cord 2m 6 5 ft Drawer to IBM PDU 250V 10A wi...

Page 39: ...center POWER8 p8had p8had_specsheetpdu htm view kc Calculating the power load for 7188 or 9188 power distribution units https www ibm com support knowledgecenter POWER8 p8had p8had_pduloadcalculator h...

Page 40: ...trum Scale Frequently Asked Questions User interfaces The system provides the following user interfaces The management GUI which is a web accessible graphical user interface GUI that supports flexible...

Page 41: ...overloading of the circuits does not compromise the supply wiring or overcurrent protection To provide the correct power connection to a rack refer to the rating labels located on the equipment in the...

Page 42: ...cy of 50 Hz or 60 Hz IEC C14 standardized 10A x2 2000 W The power and thermal measurements that are shown in Table 7 on page 30 were obtained in the specific operating environment Table 7 Power consum...

Page 43: ...b EDR that are greater than or equal to 3 meter in length The maximum ambient operating temperature when using only copper cable is 35C which includes all Ethernet and InfiniBand cables less than 3 me...

Page 44: ...em 3000 use the recommended measurements that are given in the Table 11 on page 32 Table 11 Service clearance requirements Front1 Back 1200 mm 47 in 1000 mm 39 in 1Storage racks require larger service...

Page 45: ...Table 13 Drive specifications Model and type 2 5 inch drives Elastic Storage System 3000 with 24 2 5 inch drive slots 24 dual port NVMe drives 1 92 TB 3 84 TB 7 68 TB 15 36 TB Acoustical declaration w...

Page 46: ...ion specifications for Elastic Storage System 3000 enclosures Table 14 on page 35 and Table 15 on page 35 provide the shock and vibration testing results for your Elastic Storage System 3000 system 34...

Page 47: ...IP address for the second Ethernet port on the node or file module The storage system also has a management IP address which is known as the system IP address The addresses must be fixed addresses If...

Page 48: ...ice port 1Gbps which is used by the SSR to check hardware and set the IP address During production the customer will use the high speed network links 100Gb Ethernet or 100Gb InfiniBand to serve the IB...

Page 49: ...f onboard Ethernet ports Onboard Ethernet Port Speed Function 1 1 Gbps 10 Gbps Function Management IP 2 10 Gbps Not used 3 10 Gbps Not used 4 10 Gbps Not used TP Technician Port 1 Gbps Fixed IP addres...

Page 50: ...ters Each server canister has three PCIe interface slots to support extra adapters InfiniBand and Ethernet adapters are supported in any of the interface slots In addition each server canister contain...

Page 51: ...new ESS customers an EMS node is required and connections must be made from C10 T1 and C10 T2 to the management network VLAN For an existing ESS customer the SSR must add the new connection from C10 T...

Page 52: ...Rack Placement Table for 5148 21L For information on the rack placement table for 5148 21L see B 1 Rack Placement Table for 5148 21L Empty Rack Diagram For information on the empty rack diagram see B...

Page 53: ...ion of the foam packing away from the rear of the enclosure 8 Carefully cut open the bag covering the rear of the enclosure 9 Remove the left power supply unit PSU from the enclosure 10 Record the las...

Page 54: ...f the rack 1 Upper rail mounting bracket pin 2 Lower rail mounting bracket pin 3 Rack mounting screw hole 3 Ensure that the appropriate bracket pins are installed in the front and rear bracket of each...

Page 55: ...the holes in the rail bracket with the holes on the front and rear rack cabinet flanges Ensure that the rails are aligned on the inside of the rack cabinet 6 On the rear of the rail press the two bra...

Page 56: ...closure complete the following steps 1 On either side of the drive assemblies remove the enclosure end caps by grasping the handle and pulling the bottom of the end cap free then clearing the tab on t...

Page 57: ...ing the front of the enclosure 5 Re install the left and right end caps See Figure 24 on page 46 The left end cap has indicator windows that align with the status LEDs light emitting diodes on the edg...

Page 58: ...ort 1 of each server canister in the ESS 3000 system Connect Ethernet port 1 of each server canister in the system to the IP network that will provide connection to the system management interfaces as...

Page 59: ...e cable must be long enough to easily connect to a Notebook computer For information on how to set an IP on a Windows 10 computer see the instructions on the Change TCP IP settings page at the Microso...

Page 60: ...edefined outputType pdf modelPower_Systems modelSystem_i modelSystem_p modelSystem_z modelStorage modelHMC modelSolutions 12 platform Solutions mtmodel 51 47 084 installtype N predeforders PGH1201 7EP...

Page 61: ...subnet is 24 255 255 255 0 Keep the canister address below 128 Recommendation Example Subnet 192 168 45 0 24 IP Address 192 168 45 30 netmask 255 255 255 0 Logical interface enp29s0f0 Physical interf...

Page 62: ...tem did not come racked the SSR would rack the necessary components before the installation starts ESS 3000 needs to be placed in the rack location provided by the user For example EIA 1 2 and 3 4 Eac...

Page 63: ...ll automatically boot the operating system High Speed Network If available at the time of installation the SSR can plug in the high speed cables on the node side Otherwise this is the responsibility o...

Page 64: ...next step 5 Perform a ping test Using a terminal window not putty ping the default FSP network interface as shown in the following example ping 10 0 0 1 This should return a successful result If not i...

Page 65: ...n Advanced System Management Interface ASMI 1 Assign 169 254 3 1 netmask 255 255 255 0 24 to your laptop 2 Perform a ping test to the HMC2 port 169 254 3 147 3 Access 169 254 3 147 by using a web brow...

Page 66: ...ernet cable and switch and verify the laptop IP address If the issue persists see the ESS 3000 issues on page 61 topic e Open putty and set the target address as 10 0 0 102 24 port 22 for SSH accept a...

Page 67: ...31 on page 55 Figure 31 Canister SSR Tools menu options If a step fails go through the command output j To set the IP address of the management port see Ethernet 1 in Figure 15 on page 36 to the IP ad...

Page 68: ...to respond at initial start up process If there are any issues check Ethernet cable and switch and verify the laptop IP address If the issue persists see the ESS 3000 issues on page 61 topic e Open pu...

Page 69: ...er worksheet Press the Ctrl G keys to save the changes m To ping the EMS node with the target IP address select the 8 Ping tests option or highlight the option and press the Enter key n Click Back and...

Page 70: ...58 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 71: ...set to 10 0 0 100 netmask 255 255 255 0 Verify that you are plugged into the correct port C10 T4 on the EMS If the issues still persist shut down the node hold the white button until blinking again an...

Page 72: ...In the petitboot menu press the UP ARROW and or DOWN ARROW keys to select the redhat boot option Do not select the Rescue option If several redhat boot options besides the Rescue option are shown sel...

Page 73: ...the issues still persist shut down the node pull power and perform the following steps Connect a terminal to the canister monitor to VGA port and keyboard to USB port Put power back to the ESS 3000 un...

Page 74: ...62 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 75: ...oes not describe about the high speed network Customers must have that in place and ready to fill in or update the etc hosts file on the EMS node during the software installation For reference see Cha...

Page 76: ...g Note C10 T2 listed in the figure below would be used as the Container interface Figure 36 Port mapping on the back of the EMS The SSR connects to C10 T4 bottom port on the 1Gb card to access the ser...

Page 77: ...nt to point to C10 T4 and use the instructions in the Elastic Storage System 3000 installation SSR task on page 51 to log in perform the hardware checks and set the management IP address Reference Can...

Page 78: ...interface Port 1 Enter the IP address and subnet mask Ensure that each IP in this worksheet is on the same subnet Recommended subnet is 24 255 255 255 0 Keep the canister address below 128 Recommendat...

Page 79: ...ll out the essmgr yml file For more information on the essmgr yml file see the Quick Deployment guide in the IBM Elastic Storage System ESS 3000 Knowledge Center None There is no default IP set from m...

Page 80: ...and specification of each device The hardware location chart represents the rack into which the enclosures are installed Each row of the chart represents one Electronic Industries Alliance EIA 19 inch...

Page 81: ...EIA 3 EIA 2 EIA 1 Table 21 on page 69 lists the rack requirements for the different Elastic Storage System 3000 enclosure types Table 21 Enclosure models and rack requirements Device Machine type and...

Page 82: ...70 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 83: ...Guide SSR tasks This chapter illustrates various cabling scenarios of IBM Elastic Storage System 3000 For information on cables see Connections for Elastic Storage System 3000 on page 36 Copyright IBM...

Page 84: ...rio 1 IBM Elastic Storage System 3000 as a part of the entire ESS setup including ESS GL GS GH units with Protocol nodes and the EMS node 72 IBM Elastic Storage System 3000 Hardware Planning and Insta...

Page 85: ...Cabling Scenario 2 IBM Elastic Storage System 3000 as a part of the entire ESS setup with Protocol nodes and the EMS node Appendix B Cabling Guide SSR tasks 73...

Page 86: ...Cabling Scenario 3 High speed networking with IBM Elastic Storage System 3000 Protocol nodes and the EMS node 74 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 87: ...Cabling Scenario 4 High speed networking with IBM Elastic Storage System 3000 and the EMS node Appendix B Cabling Guide SSR tasks 75...

Page 88: ...Cabling Scenario 5 Management networking with IBM Elastic Storage System 3000 Protocol nodes and the EMS node 76 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 89: ...Cabling Scenario 6 Management networking with IBM Elastic Storage System 3000 and the EMS node Appendix B Cabling Guide SSR tasks 77...

Page 90: ...78 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 91: ...ible by touch but do not activate just by touching them Industry standard devices for ports and connectors The attachment of alternative input and output devices IBM Knowledge Center and its related p...

Page 92: ...80 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 93: ...ACHINES CORPORATION PROVIDES THIS PUBLICATION AS IS WITHOUT WARRANTY OF ANY KIND EITHER EXPRESS OR IMPLIED INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES OF NON INFRINGEMENT MERCHANTABILITY OR FI...

Page 94: ...s in source language which illustrate programming techniques on various operating platforms You may copy modify and distribute these sample programs in any form without payment to IBM for the purposes...

Page 95: ...essor complex CPC central processor complex CPC A physical collection of hardware that consists of channels timers main storage and one or more central processors cluster A loosely coupled collection...

Page 96: ...e management node and I O server nodes One management server is required for each ESS 3000 system composed of one or more building blocks encryption key A mathematical value that allows components to...

Page 97: ...d controls token management and quota management fileset A hierarchical grouping of files managed as a unit for balancing workload across a cluster See also dependent fileset independent fileset files...

Page 98: ...SD server for the GPFS cluster IP See Internet Protocol IP IP over InfiniBand IPoIB Provides an IP network emulation layer on top of InfiniBand RDMA networks which allows existing applications to run...

Page 99: ...management server MS MTU See maximum transmission unit MTU N Network File System NFS A protocol developed by Sun Microsystems Incorporated that allows any host in a network to gain access to another...

Page 100: ...s in the array due to data redundancy recovery The process of restoring access to file system data when a failure has occurred Recovery can involve reconstructing data or providing alternative routing...

Page 101: ...tion of STP is to prevent bridge loops and the broadcast radiation that results from them SSH See secure shell SSH STP See Spanning Tree Protocol STP symmetric multiprocessing SMP A computer architect...

Page 102: ...90 IBM Elastic Storage System 3000 Hardware Planning and Installation Guide...

Page 103: ...al device safety check 9 H high availability 35 homologation statement 3 I information overview xi inspections safety external device check 9 installation worksheet 63 interfaces 28 L license inquirie...

Page 104: ...SSR install 51 static sensitive devices 10 submitting xii T trademarks 2 82 V vibration specifications 34 W web documentation xi resources xi 92 IBM Elastic Storage System 3000 Hardware Planning and...

Page 105: ...al device safety check 9 H high availability 35 homologation statement 3 I information overview xi inspections safety external device check 9 installation worksheet 63 interfaces 28 L license inquirie...

Page 106: ...SSR install 51 static sensitive devices 10 submitting xii T trademarks 2 82 V vibration specifications 34 W web documentation xi resources xi 94 IBM Elastic Storage System 3000 Hardware Planning and...

Page 107: ......

Page 108: ...IBM Product Number 5765 DME 5765 DAE SC28 3107 01...

Search results

Summary of Contents for ESS 3000

Reviews:

Related manuals for ESS 3000

Brands by name

Popular brands

IBM ESS 3000, Hardware Planning And Installation Manual

Search results

Summary of Contents for ESS 3000

Reviews:

Related manuals for ESS 3000

Brands by name

Popular brands