background image

Red

paper

Front cover

IBM Power Systems S822LC
for High Performance Computing

Technical Overview and Introduction

Alexandre Caldeira

Volker Haug

Scott Vetter

Содержание S822LC

Страница 1: ...Redpaper Front cover IBM Power Systems S822LC for High Performance Computing Technical Overview and Introduction Alexandre Caldeira Volker Haug Scott Vetter...

Страница 2: ......

Страница 3: ...International Technical Support Organization IBM Power Systems S822LC for High Performance Computing September 2016 REDP 5405 00...

Страница 4: ...rs Restricted Rights Use duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp First Edition September 2016 This edition applies to the IBM Power Systems S822LC model 8335 GT...

Страница 5: ...overview 9 1 7 2 POWER8 processor core 11 1 7 3 Simultaneous multithreading 12 1 7 4 Memory access 13 1 7 5 On chip L3 cache innovation and Intelligent Cache 13 1 7 6 L4 cache and memory buffer 14 1 7...

Страница 6: ...versus x86 terminology 35 2 3 3 Error handling 35 2 3 4 Serviceability 36 2 3 5 Manageability 42 Appendix A Server racks and energy management 47 IBM server racks 48 Water cooling option 48 IBM 7014...

Страница 7: ...not in any manner serve as an endorsement of those websites The materials at those websites are not part of the materials for this IBM product and use of those websites is at your own risk IBM may us...

Страница 8: ...ww ibm com legal copytrade shtml The following terms are trademarks of the International Business Machines Corporation in the United States other countries or both AIX DS8000 Easy Tier EnergyScale IBM...

Страница 9: ...pp iOS Android Place a Sponsorship Promotion in an IBM Redbooks publication featuring your business or solution with a link to your web site Quali ed IBM Business Partners may place a full page promot...

Страница 10: ...THIS PAGE INTENTIONALLY LEFT BLANK...

Страница 11: ...configuration NVIDIA NVLink technology for exceptional processor to accelerator intercommunication Four SXM2 form factor connectors for the NVIDIA Tesla P100 GPU This publication is for professionals...

Страница 12: ...out Power Systems and PowerVM Volker is an IBM POWER8 Champion and a member of the German Technical Expert Council which is an affiliate of the IBM Academy of Technology The project that produced this...

Страница 13: ...Support Organization Dept HYTD Mail Station P099 2455 South Road Poughkeepsie NY 12601 5400 Stay connected to IBM Redbooks Find us on Facebook http www facebook com IBMRedbooks Follow us on Twitter ht...

Страница 14: ...xii IBM Power Systems S822LC for High Performance Computing...

Страница 15: ...r design to scale from single racks to hundreds simplicity of ordering and a strong innovation roadmap for GPUs The IBM Power System S822LC for High Performance Computing 8335 GTB server the first Pow...

Страница 16: ...B EM57 and 128 GB EM58 allowing for a maximum of 1024 GB DDR4 system memory The physical locations of the main server components are shown in Figure 1 2 Figure 1 2 Location of server main components T...

Страница 17: ...dundant cooling One front USB 2 0 port for general usage One rear USB 3 0 port for general usage One system port with RJ45 connector Two power supplies 1 1 1 Minimum features The minimum Power S822LC...

Страница 18: ...ency Tesla P100 is not only the most powerful but also the most architecturally complex GPU accelerator architecture ever built Key features of Tesla P100 include Extreme performance Powering HPC Deep...

Страница 19: ...precision allow GP100 to deliver great speedups for many Deep Learning algorithms These algorithms do not require high levels of floating point precision but they gain large benefits from the additio...

Страница 20: ...performance degradation above 35 degrees C if internal temperatures trigger a CPU clock reduction 1 60 degrees C 34 140 degrees F Relative humidity 8 80 8 80 Maximum dew point 24 degrees C 75 degrees...

Страница 21: ...and production workloads The Power S822LC server is a two single chip module SCM system Each SCM is attached to four memory riser cards that have buffer chips for the L4 Cache and four memory RDIMM s...

Страница 22: ...IMM RDIMM RDIMM RDIMM Buffer Cache L4 RDIMM RDIMM RDIMM RDIMM Buffer Cache L4 RDIMM RDIMM RDIMM RDIMM PHB 0 PHB 1 SMP A Bus Memory Controller PHB 0 PHB 1 PCIe Gen3 x8 CAPI support PCIe Gen3 x16 CAPI s...

Страница 23: ...Insulator SOI technology Each chip is 649 mm2 and contains 4 2 billion transistors As shown in Figure 1 5 the chip contains up to 12 cores two memory controllers Peripheral Component Interconnect Expr...

Страница 24: ...ding on chip encryption compression and random number generation accelerators CAPI which allows accelerators that are plugged into a PCIe slot to access the processor bus by using a low latency high s...

Страница 25: ...awareness Enhanced branch prediction which uses both local and global prediction tables with a selector table to choose the preferred predictor Improved out of order execution Two symmetric fixed poin...

Страница 26: ...h POWER8 core can present eight hardware threads Because there are multiple hardware threads per physical processor core additional instructions can run at the same time SMT is primarily beneficial in...

Страница 27: ...ment The memory buffer also has 16 MB of L4 cache Also the memory riser card houses four industry standard RDIMMs Each memory channel can address up to 64 GB Therefore the Power S822LC server can addr...

Страница 28: ...rs by moving the L3 cache on chip compared to L3 accesses on an external on ceramic Application Specific Integrated Circuit ASIC Bandwidth improvement A 2x bandwidth improvement occurs with on chip in...

Страница 29: ...amming model that makes parallel programming easier A programmer delimits regions of code that access shared data and the hardware runs these regions atomically and in isolation buffering the results...

Страница 30: ...effectively making the accelerator part of the coherency domain Figure 1 10 CAPI accelerator that is attached to the POWER8 processor The accelerator adapter implements the Power Service Layer PSL wh...

Страница 31: ...ng workloads Support for the GPU ISA means that programs running on NVLink connected GPUs can execute directly on data in the memory of another GPU as well as on local memory GPUs can also perform ato...

Страница 32: ...to GPU interconnect using NVLink 1 8 Memory subsystem The Power S822LC server is a two socket system that supports two POWER8 SCM processor modules The server supports a maximum of 32 DDR4 RDIMMs slot...

Страница 33: ...echnology same as the L3 cache which has lower latency than regular SRAM Each memory riser card has a buffer chip with 16 MB of L4 cache and a fully populated Power S822LC server two processors and ei...

Страница 34: ...memory modules be populated Memory features cannot be mixed The base memory is 128 GB with eight 16 GB 1600 MHz DDR3 memory modules EM55 Memory upgrades are not supported Table 1 5 shows the supporte...

Страница 35: ...z Core 1 32 B 1 32 B 3 259 GHz 208 57 GBps For the entire Power S822LC server populated with the two processor modules the overall bandwidths are shown in Table 1 7 Table 1 7 Power S822LC server total...

Страница 36: ...duplex and provides 64 GBps of I O connectivity to the PCIe slots SAS internal adapters and USB ports Some PCIe devices are connected directly to the PCIe Gen3 buses on the processors and other device...

Страница 37: ...he PCIe Interconnect each POWER8 processor has 32 PCIe lanes running at 9 6 Gbps full duplex The bandwidth formula is calculated as follows Thirty two lanes 2 processors 9 6 Gbps 2 128 GBps I O I O ba...

Страница 38: ...lots and connectors Table 1 9 shows the PCIe Gen3 slot configuration for the Power S822LC server Table 1 9 Power S822LC server PCIe slot properties Slot Description Card size CAPI capable Power limit...

Страница 39: ...interface and allows for point to point interconnections between devices by using a directly wired interface between these connection points A single PCIe serial link is a dual simplex connection tha...

Страница 40: ...twork LAN adapters for the Power S822LC servers Table 1 10 Supported LAN adapters 1 13 3 Compute Intensive Accelerator Compute Intensive Accelerators are GPUs that are developed by NVIDIA With NVIDIA...

Страница 41: ...tures A storage backplane for two 2 5 inch SFF Gen4 SATA HDDs or SDDs One integrated SATA disk controller without RAID capability The storage split backplane feature is not supported Table 1 14 on pag...

Страница 42: ...ons Figure 1 16 shows the logical connections of the integrated SATA disk controller Figure 1 16 Logical diagram for integrated SATA disk controller 1 14 1 Disk and media features The servers support...

Страница 43: ...re see the following website http www ibm com systems storage network IBM Storwize family The IBM Storwize family is the ideal solution to optimize the data architecture for business flexibility and d...

Страница 44: ...he IBM System Storage DS8000 storage system includes a range of features that automate performance optimization and application quality of service and also provide the highest levels of reliability an...

Страница 45: ...vailability and Serviceability The IBM Power Systems S822LC 8335 GTB server uses the Open Power Abstraction Layer OPAL baremetal firmware for a non virtualized configuration This chapter attempts to i...

Страница 46: ...processor based Linux on Power Systems servers 2 2 Service processor The service processor or BMC is the primary control for autonomous sensor monitoring and event logging features on the Power S822LC...

Страница 47: ...lity This chapter provides information about IBM Power Systems reliability availability and serviceability RAS design and features The elements of RAS can be described as follows Reliability Indicates...

Страница 48: ...support of a maximum of 12 cores because doing more work with less hardware in a system supports greater reliability The Power S822LC servers offer two processor socket offerings with 8 core and 10 co...

Страница 49: ...AL firmware provides a hypervisor and operating system independent layer that uses the robust error detection and self healing functions that are built into the POWER8 processor and memory buffer modu...

Страница 50: ...mplements the try again abilities of the memory buffer module Other processor chip functions Within a processor chip there are other functions besides just processor cores POWER8 processors have built...

Страница 51: ...o detect erroneous hardware operations Error checking hardware ranges from parity error detection that is coupled with Processor Instruction Retry and bus try again to ECC correction on caches and sys...

Страница 52: ...f advanced and complementary error detection logic that is built directly into hardware firmware and operating systems Power Systems servers can perform considerable self diagnosis Host Boot IPL In PO...

Страница 53: ...nd eSEL events 60 09 04 2015 15 12 27 Power Supply 0xcd Presence detected Asserted 61 09 04 2015 15 12 27 Power Supply 0xce Presence detected Asserted 62 09 04 2015 15 12 35 OEM record df 040020 0c220...

Страница 54: ...015 15 12 12 Fan 0xd5 Upper Critical going high Asserted 3c 09 04 2015 15 12 12 Fan 0xd5 Upper Non recoverable going high Asserted 3d 09 04 2015 15 12 12 Fan 0xd6 Upper Non critical going high Asserte...

Страница 55: ...not require tools Service Indicator LEDs The Service Indicator LED function is for scale out systems including Power Systems such as the Power S812LC server that can be repaired by clients In the Serv...

Страница 56: ...to diagnose and fix any problems that might occur with the system Because the information is electronically maintained changes because of updates or the addition of new capabilities can be used by se...

Страница 57: ...ice Network with IPMI Version 2 0 Service Indicator LEDs System attention and system identification front and back Host operating system Command line interface CLI The service processor is a controlle...

Страница 58: ...rehensively outlined in the firmware section of Fix Central found at the following website http www ibm com support fixcentral Figure 2 2 shows the ASM dashboard where service professionals can mainta...

Страница 59: ...ware update is completed no other activities can be performed in the Advanced System Management Interface If you want to proceed click OK Figure 2 4 Confirm firmware update mode 5 Select the firmware...

Страница 60: ...e updated as shown in Figure 2 6 on page 46 By default all the components are selected To update the firmware click Proceed Figure 2 6 Start the firmware upgrade 7 After the firmware update is complet...

Страница 61: ...l rights reserved 47 Appendix A Server racks and energy management This appendix provides information about the racking options and energy management related concepts that are available for the IBM Po...

Страница 62: ...the space that is available for mounting servers and other components IBM 7014 Model T00 rack The 1 8 meter 71 in Model T00 rack is compatible with past and present Power Systems servers The T00 rack...

Страница 63: ...of the rack It is hinged on the left side so that it can swing out of the way for easy access to the rack drawers when necessary The Ruggedized Rack Feature also includes hardware for bolting the rac...

Страница 64: ...of approximately 191 mm 7 5 in The ERG7 provides an attractive black full height rack door The door is steel with a perforated flat front surface The perforation pattern extends from the bottom to the...

Страница 65: ...the rear either a lockable perforated rear steel door EC02 or a lockable Rear Door Heat Exchanger RDHX 1164 95X is used Lockable optional side panels EC03 increase the rack s aesthetics help control a...

Страница 66: ...roach is to use fillers in the EIA units that are occupied by these PDUs to facilitate the correct airflow and ventilation in the rack Figure A 2 PDU placement and PDU view The PDU receives power thro...

Страница 67: ...ements and power cord details about the 7965 94Y rack see the Planning for power section in the IBM Power Systems Hardware IBM Knowledge Center website http www ibm com support knowledgecenter api red...

Страница 68: ...devices are followed Before placing the system into the service position be sure to follow the rack manufacturer s safety instructions regarding rack stability Useful rack additions This section highl...

Страница 69: ...t on center horizontal width between the vertical columns of holes on the two front mounting flanges and on the two rear mounting flanges Figure A 3 is a top view that shows the specification dimensio...

Страница 70: ...nagement The Power S822LC servers have features to help clients become more energy efficient EnergyScale technology enables advanced energy management features to conserve power dramatically and dynam...

Страница 71: ...er saver mode varies processor frequency and voltage based on the usage of the POWER8 processors Processor frequency and usage are inversely proportional for most workloads implying that as the freque...

Страница 72: ...nvalidated and the hardware clock is turned off in the core and in the cache Voltage is reduced to minimize leakage current Processor cores that are inactive in the system such as capacity on demand C...

Страница 73: ...ses reliability in energy management by having one controller in each processor that can perform certain functions independently of the others POWER8 also includes an internal voltage regulation capab...

Страница 74: ...60 IBM Power Systems S822LC for High Performance Computing...

Страница 75: ...es for IBM Power Systems Processors Including IBM POWER8 SG24 8171 You can search for view download or order these documents and other Redbooks Redpapers web docs draft and additional materials at the...

Страница 76: ...bsites are also relevant as further information sources IBM Fix Central website http www ibm com support fixcentral IBM Knowledge Center http www ibm com support knowledgecenter IBM Power Systems webs...

Страница 77: ...Related publications 63 https launchpad net ubuntu source openjdk 7 Help from IBM IBM Support and downloads ibm com support IBM Global Services ibm com services...

Страница 78: ...64 IBM Power Systems S822LC for High Performance Computing...

Страница 79: ......

Страница 80: ...ibm com redbooks Printed in U S A Back cover ISBN REDP 5405 00...

Отзывы: