manualshive.com logo in svg

Lenovo ThinkSystem SR665, Setup Manual

The Lenovo ThinkSystem SR665 setup manual is a comprehensive user guide providing detailed instructions for quick and hassle-free installation. This indispensable manual can be easily downloaded free of charge from our website, ensuring you have access to all the essential information you need to maximize your product's potential.

Share
Download
background image

Drive bay configurations and requirements

This topic describes the drive bay configurations supported by the server and the requirements for hardware 

configurations.

Depending on models, the server supports:

•  Up to twenty-four 2.5-inch or twelve 3.5-inch SAS/SATA/NVMe front drives
•  Up to eight 2.5-inch SAS/SATA/NVMe or four 3.5-inch SAS/SATA/NVMe middle drives
•  Up to eight 2.5-inch SAS/SATA or two 7mm SAS/SATA rear drives
•  Up to two M.2 SATA/NVMe drives

For a list of supported drives, see:  

https://static.lenovo.com/us/en/serverproven/index.shtml

Drive bay combinations

The following table shows the supported drive bay combinations for the chassis with 3.5-inch/2.5-inch front 
drive bays.

•  “Supported drive bay combinations (Two processors installed)” on page 144
•  “Supported drive bay combinations (One processor installed)” on page 146

Chapter 4

.

Server hardware setup

143

Summary of Contents for ThinkSystem SR665

Page 1: ...ThinkSystem SR665 Setup Guide Machine Types 7D2V and 7D2W ...

Page 2: ...n addition be sure that you are familiar with the terms and conditions of the Lenovo warranty for your server which can be found at http datacentersupport lenovo com warrantylookup First Edition June 2020 Copyright Lenovo 2020 LIMITED AND RESTRICTED RIGHTS NOTICE If data or software is delivered pursuant to a General Services Administration GSA contract use reproduction or disclosure is subject to...

Page 3: ...ront drive bays AnyBay 128 Chapter 4 Server hardware setup 133 Server setup checklist 133 Installation Guidelines 134 System reliability guidelines 135 Working inside the server with the power on 135 Handling static sensitive devices 136 Technical rules 136 Technical rules for processor and heat sink 139 Technical rules for memory modules 139 Drive bay configurations and requirements 143 Technical...

Page 4: ...ry 241 Configure the RAID arrays 242 Install the operating system 243 Back up the server configuration 243 Chapter 6 Resolving installation issues 245 Appendix A Getting help and technical assistance 250 Before you call 250 Collecting service data 251 Contacting Support 251 Appendix B Notices 253 Trademarks 254 Important notes 254 Particulate contamination 255 ASHRAE class compliance information 2...

Page 5: ...e it possible for you to customize the system hardware to meet your needs today and provide flexible expansion capabilities for the future The server comes with a limited warranty For details about the warranty see https support lenovo com us en solutions ht503310 For details about your specific warranty see http datacentersupport lenovo com warrantylookup Identifying your server When you contact ...

Page 6: ...work access label Quick response code The system service label which is on the top cover provides a quick response QR code for mobile access to service information Scan the QR code with a mobile device and a QR code reader application to get quick access to the Lenovo Service Web site for this server The Lenovo Service Information Web site provides additional information for parts installation and...

Page 7: ...s and documentation Features Performance ease of use reliability and expansion capabilities were key considerations in the design of your server These design features make it possible for you to customize the system hardware to meet your needs today and provide flexible expansion capabilities for the future Your server implements the following features and technologies Active Energy Manager Lenovo...

Page 8: ...rge data storage capacity and hot swap capability The hot swap server models support a maximum of fourteen 3 5 inch hot swap Serial Attached SCSI SAS hard disk drives or hot swap Serial ATA SATA hard disk drives The simple swap server models support a maximum of eight 3 5 inch simple swap SATA hard disk drives With the hot swap feature you can add remove or replace hard disk drives without turning...

Page 9: ...the fans fails ThinkSystem RAID support The ThinkSystem RAID adapter provides hardware redundant array of independent disks RAID support to create configurations The standard RAID adapter provides RAID levels 0 and 1 An optional RAID adapter is available for purchase UEFI compliant server firmware Lenovo ThinkSystem firmware is Unified Extensible Firmware Interface UEFI 2 5 compliant UEFI replaces...

Page 10: ...DS RDIMMs Note The operating speed and total memory capacity depend on the processor model and UEFI settings For a list of supported memory see the Lenovo ServerProven Web site https static lenovo com us en serverproven index shtml For technical rules for memory modules see Technical rules for memory modules on page 139 Internal drives Front Up to twenty four 2 5 inch hot swap SAS SATA NVMe drives...

Page 11: ... PCIe slots on page 148 Graphics processing unit GPU Your server supports the following GPUs Low profile half length double wide NVIDIA Tesla V100 V100S Full height full length single wide NVIDIA Quadro P620 NVIDIA Tesla T4 For technical rules for GPU see Technical rules for GPU on page 151 Input Output I O features Front One VGA connector optional One USB 3 1 Gen 1 connector and one USB 2 0 conne...

Page 12: ...30 16e SAS SATA 12Gb SFF HBA adapter Gen3 440 16e SAS SATA 12Gb SFF HBA adapter Gen4 440 16i SAS SATA 12Gb Internal CFF HBA adapter Gen4 Support the following RAID expander adapter 36i 12Gb Internal CFF RAID Expander Notes SFF Standard Form Factor This type of RAID HBA adapter is installed in the PCIe slots on the rear of the server CFF Custom Form Factor This type of RAID HBA adapter is installed...

Page 13: ...0 PLUS Platinum CAUTION 240 V dc input input range 180 300 V dc is supported in Chinese Mainland ONLY Power supply with 240 V dc input cannot support hot plugging power cord function Before removing the power supply with dc input please turn off server or disconnect dc power sources at the breaker panel or by turning off the power source Then remove the power cord Minimal configuration for debuggi...

Page 14: ...ormation on page 256 Management options Several management interfaces are available for managing your server The management options described in this section are provided to support the direct management of Lenovo servers Function Lenovo XClarity Administrator Lenovo XClarity Energy Manager Lenovo XClarity Provisioning Manager V3 Lenovo XClarity Essentials1 Lenovo XClarity Integrator2 Lenovo Capac...

Page 15: ...sually check the layout of room row and rack via 2D thermal map Show events and send e mail or SNMP trap notifications when certain faults occur or certain thresholds are reached Optimize energy efficiency by identifying hotspot or over cooling servers to optimize cooling efficiency and identifying low usage servers to save energy For more information about downloading installation and usage see h...

Page 16: ...logs as well You can also choose to view the service data that has been collected or to send the service data to Lenovo Support To learn more about Lenovo XClarity Essentials OneCLI see https datacentersupport lenovo com documents LNVO CENTER Documentation for Lenovo XClarity Essentials OneCLI is available at http sysmgt lenovofiles com help topic xclarity_essentials overview html Lenovo XClarity ...

Page 17: ...You can perform Simple RAID Setup and Advanced RAID Setup in LXPM also you can perform RAID configuration for different RAID adapters from UEFI Setup OS installation Use this function to deploy an operating system for the server with an easy to use Guided Install mode Operating systems can be installed using unattended mode after you choose the Operating System version and basic settings the devic...

Page 18: ... center planning by enabling IT administrators and pre sales to understand important parameters of different type of racks servers and other devices Lenovo Capacity Planner can dynamically calculate the power consumption current British Thermal Unit BTU and volt ampere VA rating at the individual server or rack level and therefore improves the efficiency of large scale deployments Lenovo Capacity ...

Page 19: ...ethod to do the erasing and allows the user to select different erasing levels Smart USB Protection Prohibit unauthorized access to the USB port of devices USB Data Safe Encrypt files to ensure data security on a USB storage device Note This tool is available in Chinese Mainland only More information about Lenovo Business Vantage is available at http support lenovo com cn lenovo wsi es es html Cha...

Page 20: ...16 ThinkSystem SR665 Setup Guide ...

Page 21: ...ver models with 2 5 inch front drive bays Eight 2 5 inch front drive bays on page 18 Eight 2 5 inch front drive bays with LCD diagnostics panel on page 19 Sixteen 2 5 inch front drive bays on page 20 Sixteen 2 5 inch front drive bays with LCD diagnostics panel on page 21 Twenty four 2 5 inch front drive bays on page 22 Backplane less on page 23 Server models with 3 5 inch front drive bays Eight 3 ...

Page 22: ...omponents on the front of the server Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Drive bay fillers 2 6 Front I O assembly with diagnostics panel 7 Rack latch right 8 Pull out information tab 9 Drive bays 8 10 Rack latch left Note For more information about each component see Front components overview on page 27 18 ThinkSystem ...

Page 23: ... front of the server Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Drive bay filler 6 Front I O assembly with LCD diagnostics panel 7 Rack latch right 8 Pull out information tab 9 Drive bays 8 10 Rack latch left Note For more information about each component see Front components overview on page 27 Chapter 2 Server components 19...

Page 24: ...r models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Drive bay filler 6 Front I O assembly with diagnostics panel 7 Rack latch right 8 Pull out information tab 9 Drive bays 16 10 Rack latch left Note For more information about each component see Front components overview on page 27 20 ThinkSystem SR665 Setup Guide ...

Page 25: ...ts on the front of the server Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Front I O assembly with LCD diagnostics panel 6 Rack latch right 7 Pull out information tab 8 Drive bays 16 9 Rack latch left Note For more information about each component see Front components overview on page 27 Chapter 2 Server components 21 ...

Page 26: ...n the front of server models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Front I O assembly with diagnostics panel 6 Rack latch right 7 Pull out information tab 8 Drive bays 24 9 Rack latch left Note For more information about each component see Front components overview on page 27 22 ThinkSystem SR665 Setup Guide ...

Page 27: ... Table 8 Components on the front of server models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive bay fillers 3 4 Front I O assembly with diagnostics panel 5 Rack latch right 6 Pull out information tab 7 Rack latch left Note For more information about each component see Front components overview on page 27 Chapter 2 Server components 23 ...

Page 28: ...erver models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Drive bay filler 6 Front I O assembly with diagnostics panel 7 Rack latch right 8 Pull out information tab 9 Drive bays 8 10 Rack latch left Note For more information about each component see Front components overview on page 27 24 ThinkSystem SR665 Setup Guide ...

Page 29: ...ront of server models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Drive activity LED 4 Drive status LED 5 Front I O assembly with diagnostics panel 6 Rack latch right 7 Pull out information tab 8 Drive bays 12 9 Rack latch left Note For more information about each component see Front components overview on page 27 Chapter 2 Server components 25 ...

Page 30: ...ss Table 11 Components on the front of server models Callout Callout 1 External diagnostics connector 2 VGA connector optional 3 Front I O assembly with diagnostics panel 4 Rack latch right 5 Pull out information tab 6 Drive bay fillers 12 7 Rack latch left Note For more information about each component see Front components overview on page 27 26 ThinkSystem SR665 Setup Guide ...

Page 31: ... 2 Drive activity LED left Solid green The drive is powered but not active Blinking green The drive is active External diagnostics connector The connector is for connecting an external diagnostics handset For more about its functions see LCD diagnostics panel handset on page 32 Front I O assembly with LCD diagnostics panel The assembly comes with an integrated LCD diagnostics panel that can be use...

Page 32: ...server cannot slide out especially in vibration prone areas For more information refer to the Rack Installation Guide that comes with your rail kit USB 3 1 Gen 1 connectors The USB 3 1 Gen 1 connectors can be used to attach a USB compatible device such as a USB keyboard USB mouse or USB storage device VGA connector The VGA connectors on the front and rear of the server can be used to attach a high...

Page 33: ...rity Controller management Depending on the setting this connector supports USB 2 0 function XClarity Controller management function or both If the connector is set for USB 2 0 function you can attach a device that requires a USB 2 0 connection such as a keyboard a mouse or a USB storage device If the connector is set for XClarity Controller management function you can attach a mobile device insta...

Page 34: ...be powered on standby state Fast blinking about four flashes per second Green The server is off but the XClarity Controller is initializing and the server is not ready to be powered on Off None There is no ac power applied to the server 4 Network activity LED When an OCP 3 0 Ethernet adapter is installed the network activity LED on the front I O assembly helps you identify the network connectivity...

Page 35: ...our server If the system error LED is lit one or more LEDs elsewhere in the server might also be lit to direct you to the source of the error Status Color Description Action On Yellow An error has been detected on the server Causes might include but not limited to the following errors A fan failure A memory error A storage failure A PCIe device failure A power supply failure A CPU error A system b...

Page 36: ...y access system information such as active errors system status firmware information network information and health information Where to find the LCD diagnostics panel Location Callout The LCD diagnostics panel is attached to the front of the server 1 The handle with which the panel can be pulled out and inserted into the rack Notes The panel can be inserted or pulled out regardless of the system ...

Page 37: ...ttached to the top or side of the rack This is helpful and can free up both hands for certain service tasks 3 The external diagnostics connector on the front of the server that can be used to connect an external LCD diagnostics handset Note Pay attention to the following steps when unplugging the external handset Step 1 Press the plastic clip on the plug in the shown direction Step 2 Gently pull o...

Page 38: ...erview 1 LCD display 2 Scroll buttons up down left right Press the scroll buttons to locate and select system information 3 Select button Press the select button to select from the menu options 34 ThinkSystem SR665 Setup Guide ...

Page 39: ...Options flow diagram The LCD diagnostics panel handset shows various system information Navigate through the options with the scroll keys Chapter 2 Server components 35 ...

Page 40: ... Home screen Active error quantity Note The Active Alerts menu displays only the quantity of active errors If no errors occur the Active Alerts menu will not be available during navigation 1 Active Alerts Details screen Error message ID Type Error Warning Information Occurrence time Possible sources of the error Active Alerts 1 Press to view alert details FQXSPPU009N Error 04 07 2020 02 37 39 PM C...

Page 41: ...ersion number Release date XCC Backup Active Build D8BT05I Version 1 00 Date 2019 12 30 XCC Network Information Sub Menu Example XCC hostname MAC address IPv4 Network Mask IPv4 DNS IPv6 Link Local IP Stateless IPv6 IP Static IPv6 IP Current IPv6 Gateway IPv6 DNS Note Only the MAC address that is currently in use is displayed extension or shared XCC Network Information XCC Hostname XCC xxxx SN MAC ...

Page 42: ...tive Sessions Sub Menu Example Quantity of active sessions Active User Sessions 1 Actions Sub Menu Example Several quick actions supported for users Restore XCC to Defaults Force XCC Reset Request XCC Reset Clear CMOS Request Virtual Reseat Modify XCC Static IPv4 Address Net mask Gateway Modify System Name Generate Download FFDC Service Data Remove FPGA Test Image Request XCC Reset This will reque...

Page 43: ...ht PCIe slots on page 40 Server model with four 2 5 inch rear drive bays and six PCIe slots on page 41 Server model with eight 2 5 inch rear drive bays and four PCIe slots on page 42 Server model with two 3 5 inch rear drive bays and four PCIe slots on page 43 Server model with four 3 5 inch rear drive bays and two PCIe slots on page 44 Chapter 2 Server components 39 ...

Page 44: ...6 on riser 2 assembly 7 PCIe slot 7 on riser 3 assembly 8 PCIe slot 8 on riser 3 assembly 9 Power supply 1 10 Power supply 2 optional 11 NMI button 12 USB 3 1 Gen 1 connector 13 VGA connector 14 USB 3 1 Gen 1 connectors 2 15 XClarity Controller Network connector 16 Ethernet connectors on OCP 3 0 Ethernet adapter optional two or four connectors may be available Note For more information about each ...

Page 45: ...ot 5 on riser 2 assembly 6 PCIe slot 6 on riser 2 assembly 7 2 5 inch rear drive bays 4 8 Power supply 1 9 Power supply 2 optional 10 NMI button 11 USB 3 1 Gen 1 connector 12 VGA connector 13 USB 3 1 Gen 1 connectors 2 14 XClarity Controller Network connector 15 Ethernet connectors on OCP 3 0 Ethernet adapter optional two or four connectors may be available Note For more information about each com...

Page 46: ...lot 6 on riser 2 assembly 5 2 5 inch rear drive bays 8 6 Power supply 1 7 Power supply 2 optional 8 NMI button 9 USB 3 1 Gen 1 connector 10 VGA connector 11 USB 3 1 Gen 1 connectors 2 12 XClarity Controller Network connector 13 Ethernet connectors on OCP 3 0 Ethernet adapter optional two or four connectors may be available Note For more information about each component see Rear components overview...

Page 47: ...lot 6 on riser 2 assembly 5 3 5 inch rear drive bays 2 6 Power supply 1 7 Power supply 2 optional 8 NMI button 9 USB 3 1 Gen 1 connector 10 VGA connector 11 USB 3 1 Gen 1 connectors 2 12 XClarity Controller Network connector 13 Ethernet connectors on OCP 3 0 Ethernet adapter optional two or four connectors may be available Note For more information about each component see Rear components overview...

Page 48: ... on riser 2 assembly 4 Power supply 1 5 Power supply 2 optional 6 NMI button 7 USB 3 1 Gen 1 connector 8 VGA connector 9 USB 3 1 Gen 1 connectors 2 10 XClarity Controller Network connector 11 Ethernet connectors on OCP 3 0 Ethernet adapter optional two or four connectors may be available Note For more information about each component see Rear components overview on page 45 44 ThinkSystem SR665 Set...

Page 49: ...net connectors for network connections One of the Ethernet connectors on the OCP 3 0 Ethernet adapter can also function as a management connector using the shared management capacity If the shared management connector fails traffic can automatically switch over to another connector on the adapter Hot swap drives and drive bays The drive bays on the front and rear of your server are designed for ho...

Page 50: ...ancy without turning off the server On each power supply there are three status LEDs near the power cord connector For information about the LEDs see Rear view LEDs on page 47 USB 3 1 Gen 1 connectors The USB 3 1 Gen 1 connectors can be used to attach a USB compatible device such as a USB keyboard USB mouse or USB storage device VGA connector The VGA connectors on the front and rear of the server ...

Page 51: ...u to the source of the error For more information see Diagnostics panel on page 29 2 3 Ethernet status LEDs The BMC management connector has two status LEDs Ethernet status LED Color Status Description 2 Ethernet link LED Green On Network link is established None Off Network link is disconnected 3 Ethernet activity LED Green Blinking Network link is connected and active None Off The server is disc...

Page 52: ...rs into the standby state while the other one delivers entire load When the power load increases the standby power supply will switch to active state to provide sufficient power to the server To disable zero output mode start the Setup utility go to Advanced Power Zero Output and select Disable If you disable zero output mode both power supplies will be in the active state Off The server is powere...

Page 53: ...2032 3 TPM module connector 4 M 2 power connector 5 Serial port module connector 6 Riser 1 power connector 7 Riser 1 slot 8 Internal USB connector 9 OCP 3 0 Ethernet adapter connector 10 7 mm SSD power connector 11 Front USB connector 12 PCIe connector 1 13 PCIe connector 2 14 PCIe connector 3 15 Front I O connector 16 Front VGA connector Chapter 2 Server components 49 ...

Page 54: ... 4 connector 26 External LCD connector 27 Backplane power connector 1 28 Fan 5 connector 29 Fan 6 connector 30 Riser 3 GPU power connector 31 Riser 3 power connector 32 PCIe connector 6 33 PCIe connector 8 34 PCIe connector 7 35 Power supply 1 connector 36 PCIe connector 4 37 PCIe connector 5 38 Power supply 2 connector 39 Riser 3 sideband connector 40 Riser 2 slot 41 Riser 2 power connector 50 Th...

Page 55: ... error LED 2 System ID LED 3 DIMM error LEDs 32 4 Fan error LEDs 6 5 FPGA error LED 6 FPGA heartbeat LED 7 FPGA power LED 8 BMC heart beat LED 1 System error LED When this yellow LED is lit one or more LEDs elsewhere in the server might also be lit to direct you to the source of the error For more information see Diagnostics panel on page 29 Chapter 2 Server components 51 ...

Page 56: ...s Color Description On Green FPGA running image is test build image Blinking Green Blinking blinking slowly about one flash per second Power Fault happened Blinking blinking rapidly about four flashes per second FPGA does not assert RSMRST_N Off None No Power Fault 6 FPGA heart beat LED The FPGA heart beat LED helps you identify different FPGA errors Status Color Description On or Off Green FPGA d...

Page 57: ...beat LED The XCC heart beat LED helps you identify the XCC status Status Color Description On Green The XCC is not alive Blinking Green The XCC is alive Off None The XCC is not alive Chapter 2 Server components 53 ...

Page 58: ...ut ordering the parts shown in Figure 11 Server components on page 55 https datacentersupport lenovo com us en products servers thinksystem sr665 7d2w parts Note Depending on the model your server might look slightly different from the illustration Some parts are available only on some models 54 ThinkSystem SR665 Setup Guide ...

Page 59: ...re identified as one of the following Tier 1 customer replaceable unit CRU Replacement of Tier 1 CRUs is your responsibility If Lenovo installs a Tier 1 CRU at your request with no service agreement you will be charged for the installation Chapter 2 Server components 55 ...

Page 60: ...on about ordering the parts shown in Figure 11 Server components on page 55 https datacentersupport lenovo com us en products servers thinksystem sr665 7d2w parts 1 Top cover 2 GPU air baffle filler 3 Add on GPU air baffle 4 GPU air baffle 5 Standard air baffle 6 Standard heat sink 7 Performance heat sink T shape 8 Processor 9 Fan module 10 Fan cage 11 Riser 1 2 assembly 3 FH 12 Internal SFF RAID ...

Page 61: ...ve backplanes top bottom 37 M 2 adapter 38 M 2 retainer clip 39 M 2 drive 40 8 x 2 5 inch front drive backplane 41 8 x 3 5 inch front drive backplane 42 12 x 3 5 inch front drive backplane 43 2 x 3 5 inch rear drive backplane 44 8 x 2 5 inch rear drive backplane 45 4 x 2 5 inch rear middle drive backplane 46 4 x 3 5 inch rear middle drive backplane 47 2 x 3 5 inch rear drive cage BP4 48 4 x 3 5 in...

Page 62: ...ng continued Index Description Tier 1 CRU Tier 2 CRU FRU Consumable and Structural parts 55 RAID super capacitor module 56 Memory module 57 Power supply unit 58 Power supply unit filler 59 58 ThinkSystem SR665 Setup Guide ...

Page 63: ...andards Association CSA For units intended to be operated at 115 volts Use a UL listed and CSA certified cord set consisting of a minimum of 18 AWG Type SVT or SJT three conductor cord a maximum of 15 feet in length and a parallel blade grounding type attachment plug rated 15 amperes 125 volts For units intended to be operated at 230 volts U S use Use a UL listed and CSA certified cord set consist...

Page 64: ...60 ThinkSystem SR665 Setup Guide ...

Page 65: ...re printed on the cables that come with the server and optional devices Use these identifiers to connect the cables to the correct connectors Ensure that the cable is not pinched and does not cover any connectors or obstruct any components on the system board Ensure that the relevant cables pass through the cable clips Note Disengage all latches release tabs or locks on cable connectors when you d...

Page 66: ...odels see Front view on page 17 When routing the callout 3 cable ensure that it is fixed on the upper frame of the cable retainer For details refer to on page Figure 12 VGA connector and front I O assembly cable routing From To 1 VGA cable on the left rack latch VGA connector on the system board 2 External diagnostics cable on the left rack latch External LCD connector on the system board 3 Front ...

Page 67: ...n the riser card on riser 1 or 2 assembly Riser cards Use this section to understand the cable routing for riser cards The server supports up to three riser cards riser card 1 riser card 2 and riser card 3 Among which only riser card 3 needs cable connections Riser card 3 power and sideband connection on page 64 Riser card 3 x8 x8 or x16 x16 PCIe cable connection on page 65 The riser card types va...

Page 68: ...er card 3 and x16 x16 PCIe riser card 3 are the same Figure 14 Riser card 3 power and sideband connection From To 1 Power connector on the riser card Riser 3 power connector on the system board 2 Sideband connector on the riser card Riser 3 sideband on the system board 64 ThinkSystem SR665 Setup Guide ...

Page 69: ...r card PCIe connector 1 2 on the system board 2 PCIe connectors 7 on the riser card PCIe connectors 7 on the system board 3 PCIe connector 8on the riser card PCIe connector 8 on the system board 4 PCIe connector 3 on the riser card PCIe connector 3 on the system board Note 3 and 4 are for x16 x16 PCIe riser card 3 only Chapter 3 Internal cable routing 65 ...

Page 70: ... shows the cable connection for scenario 1 The cable connections for other scenarios are the similar Note An extension cable is provided for each RAID super capacitor module for connection Connect the super capacitor cable to the super capacitor connector on the corresponding RAID adapter as shown Figure 16 Connecting the RAID super capacitor module to the RAID adapter From To RAID super capacitor...

Page 71: ...RAID HBA Expander adapters Note The illustration only involves power cable routing For signal cable routing see 2 5 inch 3 5 inch drive backplane signal on page 75 From To 1 Power connector on the CFF RAID HBA adapter RAID Pwr connector on the system board 1 Power connector on the CFF RAID Expander adapter EXP Pwr connector on the system board Chapter 3 Internal cable routing 67 ...

Page 72: ...n for the 7mm drives Note 7mm drives can only be installed in slot 6 Figure 17 Cable routing for 7mm drives From To 1 7mm signal cable PCIe connector 5 on the system board 2 Power cable 7mm power connector on the system board 68 ThinkSystem SR665 Setup Guide ...

Page 73: ...e drive cage scenario 4 The following illustration shows the cable connection for scenario 1 The cable connections for other scenarios are the same Figure 18 Cable routing for M 2 drives From To 1 M 2 signal cable PCIe connector 4 on the system board two processors installed PCIe connector 1 on the system board one processor installed 2 Power cable M 2 power connector on the system board Chapter 3...

Page 74: ...le drive backplanes on page 72 Rear drive backplanes on page 74 Front drive backplanes Figure 19 Power cabling for 3 5 inch chassis From To 1 Power 1 connector on backplane Front backplane 1 power connector on the system board 2 Power 2 connector on backplane Front backplane 2 power connector on the system board 70 ThinkSystem SR665 Setup Guide ...

Page 75: ...backplane 1 Front backplane 1 power connector on the system board 2 Power connector on backplane 2 Front backplane 2 power connector on the system board 3 Power connector on backplane 3 Front backplane 3 power connector on the system board Chapter 3 Internal cable routing 71 ...

Page 76: ...Middle drive backplanes Figure 21 Power cabling for 3 5 inch chassis From To 1 Power connector on backplane 5 Power connector on riser 1 or 2 assembly 72 ThinkSystem SR665 Setup Guide ...

Page 77: ...Figure 22 Power cabling for 2 5 inch chassis From To 1 Power connector on backplane 5 and backplane 6 Power connector 1 and power connector 2 on riser 1 or 2 assembly Chapter 3 Internal cable routing 73 ...

Page 78: ... the 4 x 2 5 inch rear drive backplane routing for the other rear drive backplanes are similar Figure 23 Power cabling for 3 5 inch chassis or 2 5 inch chassis From To 1 Power connector on backplane 4 Power connector on riser 1or 2 assembly 74 ThinkSystem SR665 Setup Guide ...

Page 79: ...on Storage controller Qty Type Config 1 Config 2 1 SFF 8i RAID HBA Config 3 1 SFF 16i RAID HBA Config 4 1 CFF 16i RAID HBA The following table shows the cable connections for all configurations Configuration 1 2 and 3 4 Front one 8 x 2 5 inch SAS SATA backplane Con fig Front BP System board Storage controller SFF 8i RAID HBA SFF 16i RAID HBA CFF 16i RAID HBA 1 BP 1 SAS PCIe 1 PCIe 2 2 BP 1 SAS Gen...

Page 80: ...Figure 24 Cable routing for configuration 4 76 ThinkSystem SR665 Setup Guide ...

Page 81: ...RAID HBA Config 3 1 CFF 16i RAID HBA Configuration 1 2 and 3 Front one 8 x 2 5 inch AnyBay backplane Config Front BP System board Storage controller SFF 8i RAID HBA SFF 16i RAID HBA CFF 16i RAID HBA 1 BP1 NVMe 0 1 PCIe 1 PCIe 2 BP1 NVMe 2 3 PCIe 3 BP1 NVMe 4 5 PCIe 7 BP1 NVMe 6 7 PCIe 8 BP1 SAS Gen 4 C 0 Gen3 C 0 C 1 2 BP1 NVMe 0 1 PCIe 1 PCIe 2 BP1 NVMe 2 3 PCIe 3 BP1 NVMe 4 5 PCIe 7 BP1 NVMe 6 7...

Page 82: ...tions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 25 Cable routing for configuration 2 78 ThinkSystem SR665 Setup Guide ...

Page 83: ...configured with one 8 x 2 5 inch AnyBay front drive backplane BP 1 Only onboard configuration is supported Note The AnyBay backplane BP 1 is used as a pure NVMe backplane Configuration 1 front drive bays only Config Front BP System board 1 BP1 NVMe 0 1 PCIe 1 PCIe 2 BP1 NVMe 2 3 PCIe 3 BP1 NVMe 4 5 PCIe 7 BP1 NVMe 6 7 PCIe 8 Chapter 3 Internal cable routing 79 ...

Page 84: ...The following illustration shows the cable routing for the configuration 1 Connections between connectors 1 1 2 2 3 3 n n Figure 26 Cable routing for configuration 1 80 ThinkSystem SR665 Setup Guide ...

Page 85: ... HBA Config 3 1 SFF 16i RAID HBA Config 4 1 CFF 16i RAID HBA The following table shows the cable connections for all configurations Configuration 1 2 3 and 4 front drive bays only Front two 8 x 2 5 inch SAS SATA backplanes BP 1 and BP 2 Con fig Front BP System board Storage controller SFF 8i RAID HBA SFF 16i RAID HBA CFF 16i RAID HBA 1 BP 1 SAS PCIe 1 PCIe 2 BP 2 SAS PCIe 4 PCIe 5 2 BP 1 SAS Gen 4...

Page 86: ...tions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 27 Cable routing for configuration 3 82 ThinkSystem SR665 Setup Guide ...

Page 87: ... Note The two AnyBay backplanes are used as two pure NVMe backplanes Configuration Storage controller Qty Type Config 1 1 PCIe switch The following table shows the cable connections for all configurations Configuration 1 front drive bays only Config Front BP System board Storage controller PCIe switch 1 BP 1 NVMe 0 1 PCIe 1 PCIe 2 BP 1 NVMe 2 3 PCIe 3 BP 1 NVMe 4 5 NVMe 6 7 C 0 C1 BP 2 NVMe 0 1 PC...

Page 88: ...The following illustration shows the cable routing for the configuration 1 Connections between connectors 1 1 2 2 3 3 n n Figure 28 Cable routing for configuration 1 84 ThinkSystem SR665 Setup Guide ...

Page 89: ...le shows the cable connections for all configurations Configuration 1 2 3 and 4 front drive bays only Front one 8 x 2 5 inch SAS SATA backplane and one 8 x 2 5 inch AnyBay backplane SFF 8i RAID HBA SFF 16i RAID HBA Config Front BP System board Storage controller SFF 8i RAID HBA SFF 16i RAID HBA 1 BP 1 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 2 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 2 NVMe 0 1 PCIe 1 PCIe 2 BP 2 NVMe ...

Page 90: ... PCIe 1 PCIe 2 BP 2 NVMe 2 3 PCIe 3 BP 2 NVMe 4 5 PCIe 7 BP 2 NVMe 6 7 PCIe 8 4 BP 1 SAS C 0 BP 2 SAS C 1 BP 2 NVMe 0 1 PCIe 1 PCIe 2 BP 2 NVMe 2 3 PCIe 3 BP 2 NVMe 4 5 PCIe 7 BP 2 NVMe 6 7 PCIe 8 The following illustration shows the cable routing for the configuration 1 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Figure 29 Cable routing for c...

Page 91: ...D HBA Config 4 1 CFF 16i RAID HBA The following table shows the cable connections for all configurations Configuration 1 2 3 and 4 front drive bays only Front one 8 x 2 5 inch SAS SATA backplane and one 8 x 2 5 inch NVMe backplane Con fig Front BP System board Storage controller SFF 8i RAID HBA SFF 16i RAID HBA CFF 16i RAID HBA 1 BP 1 SAS PCIe 4 PCIe 5 BP 2 NVMe 0 1 PCIe 1 PCIe 2 BP 2 NVMe 2 3 PCI...

Page 92: ...oard Storage controller SFF 8i RAID HBA SFF 16i RAID HBA CFF 16i RAID HBA 4 BP 1 SAS C 0 C 1 PCIe 6 MB BP 2 NVMe 0 1 PCIe 1 PCIe 2 BP 2 NVMe 2 3 PCIe 3 BP 2 NVMe 4 5 PCIe 7 BP 2 NVMe 6 7 PCIe 8 88 ThinkSystem SR665 Setup Guide ...

Page 93: ...tions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 30 Cable routing for configuration 3 Chapter 3 Internal cable routing 89 ...

Page 94: ... The following table shows the cable connections for all configurations Configuration 1 and 2 front drive bays only Front one 8 x 2 5 inch AnyBay backplane and one 8 x 2 5 inch pure NVMe backplane Con fig Front BP System board Storage controller PCIe switch SFF 8i RAID HBA SFF 16i RAID HBA 1 BP 1 NVMe 0 1 PCIe 1 PCIe 2 BP 1 NVMe 2 3 PCIe 3 BP 1 NVMe 4 5 NVMe 6 7 C 0 C 1 BP 1 SAS Gen 4 C 0 Gen3 C 0...

Page 95: ...Con fig Front BP System board Storage controller PCIe switch SFF 8i RAID HBA SFF 16i RAID HBA BP 2 NVMe 6 7 PCIe 8 Chapter 3 Internal cable routing 91 ...

Page 96: ...ns between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 31 Cable routing for configuration 2 92 ThinkSystem SR665 Setup Guide ...

Page 97: ...2 2 PCIe switch SFF 16i RAID HBA The following table shows the cable connections for all configurations Configuration 1 and 2 front drive bays only Front one 8 x 2 5 inch SAS SATA backplane and two 8 x 2 5 inch pure NVMe backplanes Co nfig Front BP System board Storage controller PCIe switch SFF 8i RAID HBA SFF 16i RAID HBA 1 BP 1 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 2 NVMe 0 1 PCIe 1 PCIe 2 BP 2 NVMe 2 ...

Page 98: ...Co nfig Front BP System board Storage controller PCIe switch SFF 8i RAID HBA SFF 16i RAID HBA BP3 NVMe 6 7 PCIe 8 94 ThinkSystem SR665 Setup Guide ...

Page 99: ...ns between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 32 Cable routing for configuration 2 Chapter 3 Internal cable routing 95 ...

Page 100: ...ype Config 1 2 CFF 16i RAID HBA CFF 36i RAID Expander Config 2 2 SFF 8i RAID HBA CFF 36i RAID Expander Config 3 3 SFF 8i RAID HBA Config 4 1 SFF 32i RAID Config 5 1 Rear 4 x 2 5 SAS SATA 2 CFF 16i RAID HBA CFF 36i RAID Expander Config 6 1 Rear 4 x 2 5 SAS SATA 2 SFF 8i RAID HBA CFF 36i RAID Expander Config 7 1 Rear 4 x 2 5 SAS SATA 3 SFF 8i RAID HBA Config 8 1 Rear 4 x 2 5 SAS SATA 1 SFF 32i RAID ...

Page 101: ...orage controller CFF 36i RAID Expander CFF 16i RAID HBA SFF 8i RAID HBA 1 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 RAID HBA C 0 C 1 PCIe 6 MB BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 2 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 RAID HBA Gen 4 C 0 Gen3 C 0 C 1 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 Chapter...

Page 102: ...AID 3 BP 1 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 2 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 3 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 4 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 98 ThinkSystem SR665 Setup Guide ...

Page 103: ...ions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 33 Cable routing for configuration 2 Chapter 3 Internal cable routing 99 ...

Page 104: ...FF 36i RAID Expander CFF 16i RAID HBA SFF 8i RAID HBA 5 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 RAID HBA C 0 C 2 C 1 C 3 PCIe 6 MB BP 4 SAS C 3 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 6 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS C 3 RAID HBA Gen 4 C 0 Gen3 C 0 C 1 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 ...

Page 105: ... BP 1 SAS Slot 2 Gen 4 C 0 C 1 Gen3 C 0 BP 2 SAS Slot 3 Gen 4 C 0 C 1 Gen3 C 0 BP 3 SAS Slot 4 Gen 4 C 0 C 2 Gen3 C 0 C 1 BP 4 SAS Slot 5 C 0 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 8 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS C 3 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 Chapter 3 Internal cable...

Page 106: ...ctions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 34 Cable routing for configuration 5 102 ThinkSystem SR665 Setup Guide ...

Page 107: ...2 5 inch SAS SATA backplane BP 1 and BP 2 and one 8 x 2 5 inch AnyBay backplanes BP 3 Below lists all supported configurations with these three front drive backplanes Note The AnyBay backplane BP3 is used as a pure NVMe backplane Configuration Storage controller Qty Type Config 1 2 SFF 8i RAID HBA Config 2 1 SFF 16i RAID HBA Config 3 1 CFF 16i RAID HBA Config 4 1 SFF 32i RAID Chapter 3 Internal ca...

Page 108: ... BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 2 BP 1 SAS Gen 4 C 0 Gen3 C 0 C 1 BP 2 SAS Gen 4 C 1 Gen3 C 2 C 3 BP 3 NVMe 0 1 PCIe 1 PCIe 2 BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 CFF 16i RAID HBA SFF 32i RAID HBA Co nfi g Front BP System board Storage controller CFF 16i RAID HBA SFF 32i RAID HBA 3 BP 1 BP 2 SAS C 0 C 2 C 1 C 3 PCI...

Page 109: ...BP 3 NVMe 2 3 PCIe 3 BP 3 NVMe 4 5 PCIe 7 BP3 NVMe 6 7 PCIe 8 Chapter 3 Internal cable routing 105 ...

Page 110: ...n shows the cable routing for the configuration 4 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Figure 35 Cable routing for configuration 4 106 ThinkSystem SR665 Setup Guide ...

Page 111: ...Rear 4 x 2 5 SAS SATA 2 SFF 8i RAID HBA CFF 36i RAID Expander Config 7 1 Rear 4 x 2 5 SAS SATA 4 SFF 8i RAID HBA Config 8 1 Rear 4 x 2 5 SAS SATA 1 SFF 32i RAID Config 9 3 Mid 4 x 2 5 SAS SATA Mid 4 x 2 5 SAS SATA Rear 8 x 2 5 SAS SATA 2 CFF 16i RAID HBA CFF 36i RAID Expander Config 10 3 Mid 4 x 2 5 SAS SATA Mid 4 x 2 5 SAS SATA Rear 4 x 2 5 SAS SATA 2 CFF 16i RAID HBA CFF 36i RAID Expander Config...

Page 112: ...D HBA Co nfig Front BP System board Storage controller CFF 36i RAID Expander CFF 16i RAID HBA SFF 8i RAID HBA SFF 32i RAID 1 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 RAID HBA C 0 C 1 PCIe 6 MB 2 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 RAID HBA Gen 4 C 0 Gen 3 C 0 C 1 3 BP 1 SAS Gen 4 C 0 Gen 3 C 0 C 1 BP 2 SAS Gen 4 C 0 Gen 3 C 0 C 1 BP 3 SAS Gen 4 C 0 C 2 Gen 3 C 0 C 1 4 BP 1 SAS C 0 BP 2 SAS C 1 BP...

Page 113: ... shows the cable routing for the configuration 4 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Figure 36 Cable routing for configuration 4 Chapter 3 Internal cable routing 109 ...

Page 114: ...FF 8i RAID HBA 5 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS C 3 PCIe 6 MB RAID HBA C 0 C 1 6 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS C 3 RAID HBA Gen 4 C 0 Gen 3 C 0 C 1 Front three 8 x 2 5 inch SAS SATA backplanes BP 1 BP 2 and BP 3 Rear one 4 x 2 5 SAS SATA BP 4 Configura tion option Front BP Rear BP Storage controller SFF 8i RAID HBA SFF 32i RAID 7 BP 1 SAS Slot 2 Gen 4 C 0 Gen 3 C...

Page 115: ...a tion option Front BP Rear BP Storage controller SFF 8i RAID HBA SFF 32i RAID BP 4 SAS Slot 6 Gen 4 C 0 Gen 3 C 0 C 1 8 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS C 3 Chapter 3 Internal cable routing 111 ...

Page 116: ...n shows the cable routing for the configuration 4 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Figure 37 Cable routing for configuration 5 112 ThinkSystem SR665 Setup Guide ...

Page 117: ... 2 BP 4 SAS 1 C 3 BP5 SAS SAS C 3 PCIe 6 MB RAID HBA C 0 C 1 Front three 8 x 2 5 inch SAS SATA backplanes BP 1 BP 2 and BP 3 Mid two 4 x 2 5 inch SAS SATA backplanes BP 5 and BP 6 Rear one 4 x 2 5 inch SAS SATA backplane BP 4 Con fig Front BP Mid Rear BP Sys tem board Storage controller CFF 36i RAID Expander CFF 16i RAID HBA SFF 8i RAID HBA 10 BP 1 SAS C 0 BP 2 SAS C 1 BP 3 SAS C 2 BP 4 SAS BP 5 S...

Page 118: ... shows the cable routing for the configuration 10 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Figure 38 Cable routing for configuration 10 114 ThinkSystem SR665 Setup Guide ...

Page 119: ...ations with these three front drive backplanes Note The AnyBay backplanes are used as pure NVMe backplanes Configuration Mid or Rear BP Storage controller Qty Type Qty Type Config 1 3 PCIe switch Config 2 2 Mid 4 x 2 5 NVMe 4 PCIe switch Depending on your server configurations refer to one of the following sections for cable routing information Configuration 1 front drive bays only Configuration 2...

Page 120: ...ler PCIe switch 1 BP1 NVMe 0 1 PCIe 1 PCIe 2 BP1 NVMe 2 3 PCIe 3 BP1 NVMe 4 5 NVMe 6 7 Slot 1 C 0 C 1 BP2 NVMe 0 1 NVMe 2 3 Slot 2 C 0 C 1 BP2 NVMe 4 5 PCIe 4 PCIe 5 BP2 NVMe 6 7 PCIe 6 BP3 NVMe 0 1 PCIe 7 BP3 NVMe 2 3 PCIe 8 BP3 NVMe 4 5 NVMe 6 7 Slot 4 C 0 C 1 The following illustration shows the cable routing for the configuration 4 Connections between connectors 1 1 2 2 3 3 n n 116 ThinkSystem...

Page 121: ...Figure 39 Cable routing for configuration 1 Chapter 3 Internal cable routing 117 ...

Page 122: ...r PCIe switch 2 BP 1 NVMe 0 1 NVMe 2 3 NVMe 4 5 NVMe 6 7 Slot 1 C 0 C 1 C 2 C 3 BP 2 NVMe 0 1 NVMe 2 3 NVMe 4 5 NVMe 6 7 Slot 2 C 0 C 1 C 2 C 3 BP 3 NVMe 0 1 NVMe 2 3 NVMe 4 5 NVMe 6 7 Slot 4 C 0 C 1 C 2 C 3 BP 5 NVMe 0 1 NVMe 2 3 BP 6 NVMe 0 1 NVMe 2 3 Slot 5 C 0 C 1 C 2 C 3 The following illustration shows the cable routing for the configuration 2 Connections between connectors 1 1 2 2 3 3 n n F...

Page 123: ...figurations Configuration 1 and 2 front drive bays only Front one 8 x 3 5 inch SAS SATA backplanes Con fig Front BP System board Storage controller SFF 16i RAID HBA 1 SAS 0 PCIe 1 SAS 1 PCIe 2 2 SAS 0 SAS 1 Gen 4 C 0 Gen3 C 0 C 1 The following illustration shows the cable routing for the configuration 2 the routing for configurations 1 is similar Connections between connectors 1 1 2 2 3 3 n n Note...

Page 124: ...Figure 41 Cable routing for configuration 2 120 ThinkSystem SR665 Setup Guide ...

Page 125: ...SAS SATA 2 SFF 8i HBA SFF 16i RAID HBA Config 8 1 Rear 4 x 2 5 SAS SATA 1 SFF 16i RAID HBA Config 9 1 Rear 4 x 2 5 SAS SATA 2 SFF 8i HBA SFF 16i RAID HBA Config 10 2 Mid 4 x 3 5 SAS SATA Rear 4 x 3 5 SAS SATA 1 SFF 8i HBA Config 11 2 Mid 4 x 3 5 SAS SATA Rear 4 x 3 5 SAS SATA 1 SFF 32i RAID Config 12 2 Mid 4 x 3 5 SAS SATA Rear 4 x 2 5 SAS SATA 1 SFF 8i HBA Config 13 2 Mid 4 x 3 5 SAS SATA Rear 4 ...

Page 126: ... 0 Gen3 C 0 C 1 The following illustration shows the cable routing for the configuration 2 the routing for configurations 1 is similar Connections between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Fig...

Page 127: ...S 2 C 1 BP 5 NVMe 0 1 PCIe 1 PCIe 2 BP 5 NVMe 2 3 PCIe 3 BP 6 NVMe 0 1 PCIe 7 BP 6 NVMe 2 3 PCIe 8 The following illustration shows the cable routing for the configuration 3 Connections between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector inform...

Page 128: ... SAS Gen 4 C 1 Gen3 C 2 C 3 5 7 9 SAS 0 SAS 1 Gen 4 C 0 Gen3 C 0 C 1 SAS 2 Gen 4 C 1 Gen3 C 2 BP 4 SAS Gen 4 C 0 Gen3 C 0 The following illustration shows the cable routing for the configuration 6 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the ca...

Page 129: ...Figure 44 Cable routing for configuration 6 Chapter 3 Internal cable routing 125 ...

Page 130: ...CIe 5 BP 4 SAS BP 5 SAS Gen 4 C 0 Gen3 C 0 C 1 11 13 SAS 0 SAS 1 C 0 SAS 2 BP 4 SAS C 1 BP 5 SAS C 2 The following illustration shows the cable routing for the configuration 10 the routing for the other configurations is similar Connections between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing informat...

Page 131: ...Chapter 3 Internal cable routing 127 ...

Page 132: ... Qty Type Qt y Type Config 1 1 SFF 16i RAID HBA Config 2 1 Rear 4 x 3 5 SAS SATA 1 SFF 16i RAID HBA Config 3 1 Rear 4 x 3 5 SAS SATA 2 SFF 8i HBA SFF 16i RAID HBA Config 4 2 Mid 4 x 3 5 SAS SATA Rear 4 x 3 5 SAS SATA 1 SFF 32i RAID Depending on your server configurations refer to one of the following sections for cable routing information Configuration 1 front drive bays only on page 129 Configura...

Page 133: ...nly Front one 12 x 3 5 inch AnyBay backplane Con fig Front BP System board Storage controller SFF 16i RAID HBA 1 NVMe 0 1 PCIe 1 PCIe 2 NVMe 2 3 PCIe 3 NVMe 4 5 PCIe 4 PCIe 5 NVMe 6 7 PCIe 6 NVMe 8 9 PCIe 7 NVMe 10 11 PCIe 8 SAS 0 SAS 1 Gen 4 C 0 Gen3 C 0 C 1 SAS 2 C 1 Chapter 3 Internal cable routing 129 ...

Page 134: ...AID 2 NVMe 0 1 PCIe 1 PCIe 2 NVMe 2 3 PCIe 3 NVMe 4 5 PCIe 4 PCIe 5 NVMe 6 7 PCIe 6 NVMe 8 9 PCIe 7 NVMe 10 11 PCIe 8 SAS 0 SAS 1 Gen 4 C 0 Gen3 C 0 C 1 SAS 2 BP 4 SAS Gen 4 C 1 Gen3 C 2 C 3 3 NVMe 0 1 PCIe 1 PCIe 2 NVMe 2 3 PCIe 3 NVMe 4 5 PCIe 4 PCIe 5 NVMe 6 7 PCIe 6 NVMe 8 9 PCIe 7 NVMe 10 11 PCIe 8 SAS 0 SAS 1 Gen 4 C 0 Gen3 C 1 SAS 2 Gen 4 C 1 Gen3 C 0 BP 4 SAS Gen 4 C 0 Gen3 C 0 130 ThinkSy...

Page 135: ...nch AnyBay backplane Rear one 4 x 3 5 inch AnyBay backplane Con fig Front BP Rear BP System board Storage controller SFF 32i RAID 4 NVMe 0 1 PCIe 1 PCIe 2 NVMe 2 3 PCIe 3 NVMe 4 5 PCIe 4 PCIe 5 NVMe 6 7 PCIe 6 NVMe 8 9 PCIe 7 NVMe 10 11 PCIe 8 SAS 0 SAS 1 C 0 SAS 2 BP 4 SAS C 1 BP 5 SAS C 2 Chapter 3 Internal cable routing 131 ...

Page 136: ...ctions between connectors 1 1 2 2 3 3 n n Note For models that support Gen 3 and Gen4 adapters at the same slot the illustration shows only the cable routing information for Gen 4 adapters the routing and connector information is similar for Gen 3 adapters Figure 46 Cable routing for configuration 4 132 ThinkSystem SR665 Setup Guide ...

Page 137: ...t cables and power cords to the server See Rear view on page 39 to locate the connectors See Cable the server on page 234 for cabling best practices d Power on the server See Turn on the server on page 234 Note You can access the management processor interface to configure the system without powering on the server Whenever the server is connected to power the management processor interface is avai...

Page 138: ...ect that might be too heavy for you If you have to lift a heavy object read the following precautions carefully Make sure that you can stand steadily without slipping Distribute the weight of the object equally between your feet Use a slow lifting force Never move suddenly or twist when you lift a heavy object To avoid straining the muscles in your back lift by standing or by pushing up with your ...

Page 139: ...e server must be installed when the server starts some servers might come with more than one air baffle Operating the server with a missing air baffle might damage the processor All processor sockets must contain either a socket cover or a processor with heat sink When more than one processor is installed fan population rules for each server must be strictly followed Working inside the server with...

Page 140: ... package and from your body Remove the device from the package and install it directly into the server without putting it down If it is necessary to put the device down put it back into the static protective package Never place the device on the server or on any metal surface When handling a device carefully hold it by the edges or the frame Do not touch solder joints pins or exposed circuitry Kee...

Page 141: ...pters Without GPUs CX6 adapter 8 x 2 5 8 x 3 5 16 x 2 5 Performance Standard 12 x 3 5 24 x 2 5 Performance Performance backplane less Performance Performance Technical rules for DIMMs The following figure and table help you locate the memory slots and channels on the system board Figure 47 Memory module locations Chapter 4 Server hardware setup 137 ...

Page 142: ...h two processors 14 30 16 32 3 19 1 17 10 26 12 28 7 23 5 21 13 29 15 31 4 20 2 18 9 25 11 27 8 24 6 22 Capacity DIMMs with different capacities only two different capacities can be mixed install the one with the highest capacity first Rank DIMMs with different ranks can be mixed install the one with the highest rank first Frequency DIMMs with different frequencies can be mixed in the same channel...

Page 143: ...ackplane less Performance Performance Notes for one processor configurations Internal CFF RAID expander is supported while internal RAID HBA adapters are not supported Up to four onboard NVMe signals are supported 7mm drives and M 2 drives are not supported when eight onboard SATA signals are configured M 2 or SATA drives can be connected to slimline x 4 connectors Notes for two processor configur...

Page 144: ...2 GB 64 GB RDIMM For a list of supported memory options see https static lenovo com us en serverproven index shtml Note The operating speed and total memory capacity depend on the processor model and UEFI settings The following figure and table help you locate the memory slots and channels on the system board Figure 48 Memory module locations 140 ThinkSystem SR665 Setup Guide ...

Page 145: ... two processors 14 30 16 32 3 19 1 17 10 26 12 28 7 23 5 21 13 29 15 31 4 20 2 18 9 25 11 27 8 24 6 22 Capacity DIMMs with different capacities only two different capacities can be mixed install the one with the highest capacity first Rank DIMMs with different ranks can be mixed install the one with the highest rank first Frequency DIMMs with different frequencies can be mixed in the same channel ...

Page 146: ...142 ThinkSystem SR665 Setup Guide ...

Page 147: ...h SAS SATA NVMe middle drives Up to eight 2 5 inch SAS SATA or two 7mm SAS SATA rear drives Up to two M 2 SATA NVMe drives For a list of supported drives see https static lenovo com us en serverproven index shtml Drive bay combinations The following table shows the supported drive bay combinations for the chassis with 3 5 inch 2 5 inch front drive bays Supported drive bay combinations Two processo...

Page 148: ...rs are installed Table 25 3 5 inch drive bay combinations matrix No Total drives Front bays Middle bays Rear bays 3 5 SAS SATA 3 5 Any Bay 3 5 SAS SATA 2 5 NVMe 2 5 SAS SATA 3 5 SAS SATA 1 8 8 2a 12 12 2b 14 12 2 2c 16 12 4 2d 16 12 4 4 2e 16 12 4 2f 20 12 4 4 2g 20 12 4 4 3a 12 12 3b 16 12 4 3c 20 12 4 4 144 ThinkSystem SR665 Setup Guide ...

Page 149: ...SATA 2 5 N VMe 2 5 Any Bay 2 5 SAS SATA 2 5 NVMe 2 5 SAS SATA 1 8 8 2 8 8 3 8 8 4 16 8 8 5 16 8 8 6 16 8 8 7 16 8 8 8 16 8 8 9 24 8 8 8 10 24 8 8 8 11a 24 8 8 8 11b 28 8 8 8 4 12a 24 8 8 8 12b 28 8 8 8 4 12c 36 8 8 8 4 4 4 12d 40 8 8 8 4 4 8 13a 24 8 8 8 13b 32 8 8 8 4 4 Chapter 4 Server hardware setup 145 ...

Page 150: ... requirements The following notes describe information that you must consider when you install a drive NVMe drives are only supported in the front drive bays bays 8 11 using the AnyBay backplane which offers 4 x NVMe capable drive bays With the 3 5 inch chassis the middle drive bays are 3 5 inch bays However the rear drive bays can be either 2 5 inch SAS SATA or 3 5 inch SAS SATA bays With the 2 5...

Page 151: ...ive cage Performance fans speed of 18000 RPM must be installed The processor TDP must be lower than or equal to 225 watts The ambient temperature must be lower than or equal to 35 C 95 F Chapter 4 Server hardware setup 147 ...

Page 152: ...t 6 riser 2 bracket 1FHHL 7 mm drives Compatibility of Riser cards and PCIe adapters Gen 3 riser cards can only support Gen 3 RAID adapters Gen 4 riser adapters can support both Gen 3 and Gen 4 RAID adapters Gen 3 and Gen 4 RAID adapters of the same group can be installed to the same riser card Gen 3 and Gen 4 RAID adapters of the different groups cannot be installed to the same riser card RAID ad...

Page 153: ...er models with 8 x 2 5 inch front drive bays SAS SATA Server models with 10 x 2 5 inch front drive bays 6 SAS SATA 4 AnyBay 10 AnyBay or 10 NVMe Server models with 4 x 3 5 inch front drive bays AnyBay Server models with 2 x 2 5 inch rear drive bays SAS SATA Supported slots for different PCIe adapters Slot priority for most PCIe adapters One processor installed Slot 2 slot 3 slot 1 Two processors i...

Page 154: ... configuration when 2 processors are installed 1 2 4 5 3 6 7 8 Slot priorities for most Mellanox NIC adapters If Slot priorities One processor is installed 1 2 3 Two processors are installed 1 4 2 5 3 6 7 8 Notes The following two Mellanox ConnectX 6 adapters are always installed on slot 1 Mellanox ConnectX 6 HDR IB 200GbE Single Port x16 PCIe Adapter Mellanox HDR Auxiliary x16 PCIe 3 0 Connection...

Page 155: ...d slots based on what is configured for your server GPU Type Maximum supported Tesla V100S V100 full height full length double wide 2 Tesla T4 Quadro P620 low profile half length single wide 8 Notes Use E 16 16 riser assembly riser 1 2 or riser 3C assembly for installing a double wide GPU Chapter 4 Server hardware setup 151 ...

Page 156: ...rocedure references any tasks that need to be performed to gain access to the component being replaced Installation procedures are presented in the optimum sequence to minimize work Attention To ensure the components you install work correctly without problems read the following precautions carefully Make sure the components you are installing are supported by your server For a list of supported o...

Page 157: ...ock the security bezel into place A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Use the key to unlock the security bezel Figure 49 Unlocking the security bezel Chapter 4 Server hardware setup 153 ...

Page 158: ...ttered metal burns or both S014 CAUTION Hazardous voltage current and energy levels might be present Only a qualified service technician is authorized to remove the covers where the label is attached Attention Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task A video for this task is available at Youtube https ...

Page 159: ...ewdriver to turn the cover lock to the unlocked position as shown b Press the release button on the cover latch and then fully open the cover latch c Slide the top cover to the rear until it is disengaged from the chassis Then lift the top cover off the chassis and place the top cover on a flat clean surface Remove the air baffle Use this information to remove the air baffle If you intend to insta...

Page 160: ...lation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 If there is a RAID super capacitor module installed on the air baffle disconnect the RAID super capacito...

Page 161: ...Figure 53 Removal of the air baffle extender Step 5 Grasp the air baffle and carefully lift it out of the server Figure 54 Removal of the standard air baffle Chapter 4 Server hardware setup 157 ...

Page 162: ...n Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or othe...

Page 163: ...1 Rotate the levers of the system fan cage to the rear of the server Step 2 Lift the system fan cage straight up and out of the chassis After you finish Begin installing any options that you have purchased Chapter 4 Server hardware setup 159 ...

Page 164: ...y fragile and easily damaged Contaminants on the processor contacts such as oil from your skin can cause connection failures Do not allow the thermal grease on the processor or heat sink to come in contact with anything Contact with any surface can compromise the thermal grease rendering it ineffective Thermal grease can damage components such as electrical connectors in the processor socket Do no...

Page 165: ...erformance heat sink Note For a new heat sink the thermal grease is pre applied to the heat sink Remove the protective cover and install the heat sink a Orient the heat sink with the screw holes on the processor plate The captive screws on the heat sink should align with the screw holes on the processor plate b Tighten all the captive screws in the installation sequence shown on the heat sink labe...

Page 166: ...Figure 58 Installing a standard heat sink Figure 59 Installing a performance heat sink After you finish Install any options that you have purchased 162 ThinkSystem SR665 Setup Guide ...

Page 167: ...tor contacts or allow these contacts to touch the outside of the memory module connector housing Handle memory modules with care never bend twist or drop a memory module Do not use any metal tools such as jigs or clamps to handle the memory modules because the rigid metals may damage the memory modules Do not insert memory modules while holding packages or passive components which can cause packag...

Page 168: ...ign the memory module with the slot and gently place the memory module on the slot with both hands c Firmly press both ends of the memory module straight down into the slot until the retaining clips snap into the locked position Note If there is a gap between the memory module and the retaining clips the memory module has not been correctly inserted In this case open the retaining clips remove the...

Page 169: ...6 GB using RDIMMs 128 GB using 3DS RDIMMs Maximum capacity 2 TB using RDIMMs 4 TB using 3DS RDIMMs Type depending on the model TruDDR4 3200 dual rank 16 GB 32 GB 64 GB RDIMM For a list of supported memory options see https static lenovo com us en serverproven index shtml Note The operating speed and total memory capacity depend on the processor model and UEFI settings The following figure and tabl...

Page 170: ...h two processors 14 30 16 32 3 19 1 17 10 26 12 28 7 23 5 21 13 29 15 31 4 20 2 18 9 25 11 27 8 24 6 22 Capacity DIMMs with different capacities only two different capacities can be mixed install the one with the highest capacity first Rank DIMMs with different ranks can be mixed install the one with the highest rank first Frequency DIMMs with different frequencies can be mixed in the same channel...

Page 171: ...planes or Anybay backplanes that support pure NVMe drives install the three backplanes to drive bays 0 7 drive bays 8 15 and drive bays 16 23 One SATA SAS 8 bay backplane and two Anybay 8 bay backplanes that support pure NVMe drives install the SATA SAS 8 bay backplane to drive bays 0 7 install the Anybay 8 bay backplanes to drive bays 8 15 and drive bays 16 23 One Anybay 8 bay backplane and two S...

Page 172: ...Step 2 Connect the cables to the backplane See 2 5 inch 3 5 inch drive backplane power on page 70 and 2 5 inch 3 5 inch drive backplane signal on page 75 168 ThinkSystem SR665 Setup Guide ...

Page 173: ...he chassis and press the backplane into position The release tabs will secure the backplane in place Step 4 Connect the cables to the system board See 2 5 inch 3 5 inch drive backplane power on page 70 and 2 5 inch 3 5 inch drive backplane signal on page 75 Install the front 3 5 inch drive backplane Use this information to install the front 3 5 inch drive backplane About this task Your server supp...

Page 174: ...c protective package that contains the new backplane to any unpainted surface on the outside of the server Then take the new backplane out of the package and place it on a static protective surface Step 2 Connect the cables to the backplane See 2 5 inch 3 5 inch drive backplane power on page 70 and 2 5 inch 3 5 inch drive backplane signal on page 75 Step 3 Install the 3 5 inch drive backplane Figu...

Page 175: ...c electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com alb...

Page 176: ...ve backplane After you finish Reinstall all the drives and fillers if any into the drive bays See Install a hot swap drive on page 197 Install the middle or rear 3 5 inch drive backplane Use this information to install the middle or rear 3 5 inch drive backplane 172 ThinkSystem SR665 Setup Guide ...

Page 177: ...ese devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the new backplane to any unpainted surface on the outside of the server Then...

Page 178: ...5 inch drive backplane Figure 67 Installing the 4 x 3 5 inch drive backplane After you finish Reinstall all the drives and fillers if any into the drive bays See Install a hot swap drive on page 197 174 ThinkSystem SR665 Setup Guide ...

Page 179: ...l installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the new backplane to any unpainted surface on ...

Page 180: ... the top Figure 69 7mm drive backplane installation top a Put the backplane down on the cage as shown b Install the two screws to secure the backplane in place Step 5 Connect the cables to the system board See 7mm drives on page 68 After you finish 1 Reinstall all the drives and fillers if any into the drive bays Install cable wall brackets Use this information to install the full height or half h...

Page 181: ...ces with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the new cable holder to any unpainted surface on the outside of the server Then take...

Page 182: ...Connect any cables See Chapter 3 Internal cable routing on page 61 178 ThinkSystem SR665 Setup Guide ...

Page 183: ...he middle C1 rearwall bracket on the right Server model with 4 x 2 5 inch rear drives The server requires 3 rearwall brackets A1 rearwall bracket on the left B1 rearwall bracket on the middle C2 rearwall bracket on the right Server model with 2 x 3 5 inch rear drives The server requires 2 rearwall brackets A1 rearwall bracket on the left B1 rearwall bracket on the middle Server model with 8 x 2 5 ...

Page 184: ...oss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch...

Page 185: ...ssembly on page 204 Install a B1 B2 rearwall bracket middle Use this information to install a B1 B2 rearwall on the middle of the rear chassis About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packa...

Page 186: ...acket Figure 72 B1 B2 rearwall bracket installation a Align the holes on the rearwall bracket and the chasiss Install the bracket to the chassis as shown b Install the two screws to secure the rearwall bracket After you finish Install a rear drive cage or riser assemblies See Install the 4 x 2 5 inch rear drive cage on page 191 Install the 2 x 3 5 inch rear drive cage on page 188 Install a riser 1...

Page 187: ...otective package that contains the new riser card to any unpainted surface on the outside of the server Then take the new riser card out of the package and place it on a static protective surface Step 2 Install a C1 rearwall bracket Note The illustration shows removing a C1 rearwall bracket The procedure is the same for removing the C2 rearwall bracket Figure 73 C1 rearwall bracket installation a ...

Page 188: ...ge 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list P...

Page 189: ...the bottom of the drive cage Rotate the backplane to vertical position so that the holes in the backplane pass through the pins on the drive cage and press the backplane into position The release latches will secure the backplane in place Figure 74 Installing the backplane Chapter 4 Server hardware setup 185 ...

Page 190: ...e cage Use this information to install the middle 3 5 inch drive cage About this task The middle drive cage is supported on some server models at certain conditions For detailed information see Drive bay configurations and requirements on page 143 Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt...

Page 191: ...e backplane with the studs at the bottom of the drive cage Rotate the backplane to vertical position so that the holes in the backplane pass through the pins on the drive cage and press the backplane into position The release latches will secure the backplane in place Figure 76 Installing the backplane Chapter 4 Server hardware setup 187 ...

Page 192: ...ive cage Use this information to install the 2 x 3 5 inch rear drive cage About this task The rear drive cage is supported on some server models at certain conditions For detailed information see Drive bay configurations and requirements on page 143 Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system ha...

Page 193: ...Install the 2 x 3 5 inch rear drive cage Figure 78 Installing the 2 x 3 5 inch rear drive cage 1 Align the rear drive cage with the chassis and lower the drive cage into the chassis Move the rear drive cage forward until it clicks into position 2 Twist and release the blue plunger to secure the drive cage Step 3 Check and ensure that the blue plungers are latched into place to secure the rear driv...

Page 194: ...www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Prepare your server 1 Install the required performance system fans See Install a system fan on page 223 2 Install the required riser brackets See Install a riser 1 2 assembly on page 204 3 Install the backplane to the drive cage See Install the middle or rear ...

Page 195: ...x 2 5 inch rear drive cage About this task The rear drive cage is supported on some server models at certain conditions For detailed information see Drive bay configurations and requirements on page 143 Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive...

Page 196: ...ackets if any and install the rearwall bracket required for installing 4 x 2 5 inch rear drive cage The rearwall bracket combinations matrix in the Rearwall bracket replacement section in Maintenance Manual Install a C1 C2 rearwall bracket right on page 182 Step 2 Install the 4 x 2 5 inch rear drive cage Figure 80 Installing the 4 x 2 5 inch rear drive cage 1 Align the rear drive cage with the cha...

Page 197: ...ese devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Prepare your server 1 Install the required performance system fans See Install a system fan on page 223 2 Install the required...

Page 198: ...s task The illustrations shows the installation for 7mm drive cage in a riser cage The installation of the independent single slot 7mm drive cage is the similar to the riser 1 2 assembly installation For details refer to Install a riser 1 2 assembly on page 204 Attention Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for ...

Page 199: ...ane to any unpainted surface on the outside of the server Then take the new backplane out of the package and place it on a static protective surface Step 2 Install the two 7mm backplanes See Install the 7mm drive backplanes on page 175 Step 3 Connect the cables to the backplanes See 7mm drives on page 68 Chapter 4 Server hardware setup 195 ...

Page 200: ...oles on the side brackets of the 7mm drive cage with the two holes on the front of the riser cage b Install the two screws to secure the 7mm drive cage in place Step 6 Connect any cables from the 7mm drive backplanes See 7mm drives on page 68 Step 7 Install the riser 2 assembly to the riser 2 slot on the system board See Install a riser 1 2 assembly on page 204 After you finish 1 Install all the d...

Page 201: ... task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the drive to any unpainted surface on the outside of the server Then take the drive out of the package and place it on a static protective surface Note Ensure that drive inst...

Page 202: ... handle to lock the drive in place Step 4 Check the drive status LED to verify that the drive is operating correctly a If the yellow drive status LED is lit continuously that drive is faulty and must be replaced b If the green drive activity LED is flashing the drive is being accessed Step 5 Continue to install additional hot swap drives if necessary 198 ThinkSystem SR665 Setup Guide ...

Page 203: ...static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku co...

Page 204: ...e surface Notes The M 2 adapter is shipped with a loose screw but the screw is not necessarily used for the installation The M 2 adapter you want to install might be different from the following illustrations but the installation method is the same Step 2 Adjust the retainer on the M 2 adapter to accommodate the particular size of the M 2 drive you wish to install See Adjust the retainer on the M ...

Page 205: ... 86 M 2 drive slot 1 Slot 0 2 Slot 1 Step 4 Install the M 2 adapter into the M 2 tray Figure 87 M 2 adapter installation a Open the retention clip on the air baffle b Insert the M 2 adapter at an angle of approximately 30 degrees into the tray as shown c Rotate the M 2 adapter downward until it snaps into place Chapter 4 Server hardware setup 201 ...

Page 206: ... catches on the lip of the retainer 2 c Slide the retainer forward toward the connector to secure the M 2 drive into place Step 6 Connect the cables to the system board See M 2 drives on page 69 After you finish Use the Lenovo XClarity Provisioning Manager to configure the RAID For more information see https sysmgt lenovofiles com help topic LXPMv3 LXPMv3_introduction html 202 ThinkSystem SR665 Se...

Page 207: ... static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku c...

Page 208: ...le until it is seated in place Install a riser 1 2 assembly Use this information to install a riser 1 assembly or riser 2 assembly About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until in...

Page 209: ...rface Step 2 Install a riser card in a riser cage Figure 90 Riser card installation three slot riser bracket Figure 91 Riser card installation one slot or two slot riser bracket a Align the holes in the riser card with the mounting studs on the bracket Install the riser card to the bracket in the direction as shown b Install the two screws to secure the riser card to the bracket Chapter 4 Server h...

Page 210: ... is securely seated and its bracket also is secured b Rotate the PCIe adapter retention latch to the closed position Step 4 Check the rearwall matrix and decide which rearwall brackets need to be installed or removed for installing your riser assemblies See Rearwall bracket combinations matrix on page 179 Rearwall bracket combinations matrix on page 179 and Install a B1 B2 rearwall bracket middle ...

Page 211: ...installation one slot riser bracket a Align the riser adapter in the riser cage with the PCIe riser slot on the system board Carefully press the PCIe adapter straight into the slot until it is securely seated and its bracket also is secured b Optional If there is one screw on the riser cage tighten the screw After you finish Refer to Chapter 3 Internal cable routing on page 61 to connect any cable...

Page 212: ...eLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the new riser card to any unpainted surface on the outside of the server Then take the new riser card out of the package and place it on a static protective surface Step 2 Install the riser card to the riser bracket See Step 2 Install a riser card on page 2...

Page 213: ...stall the internal CFF RAID HBA RAID expander adapter About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation ...

Page 214: ...s to the adapter After you finish Reinstall the components that you have been removed for installing the adapter Install a GPU adapter Use this information to install a GPU adapter About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive compo...

Page 215: ...bumlist show id_52339612 html Procedure Step 1 Depending on the GPU type install the GPU air baffle filler or extender for the GPU adapter For a double wide GPU adapter for example NVIDIA V100 GPU Figure 97 Installing the GPU air baffle filler for a double wide GPU adapter Chapter 4 Server hardware setup 211 ...

Page 216: ...on to install the OCP 3 0 Ethernet adapter About this task CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Attention Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task Prevent exposure to static electricity which might lead to system halt a...

Page 217: ...y seated and the thumbscrew is securely tightened Otherwise the OCP 3 0 Ethernet adapter will not get full connection and may not function If there is an OCP 3 0 Ethernet adapter installed when the system is powered off but still plugged in to AC power system fans will continue to spin at a much lower speed This is the system design to provide proper cooling for the OCP 3 0 Ethernet adapter Figure...

Page 218: ...Figure 100 Attaching the OCP label Step 5 Connect the cables to the OCP 3 0 Ethernet adapter 214 ThinkSystem SR665 Setup Guide ...

Page 219: ... specific topic for the installation procedure Install a super capacitor on the chassis on page 217 Install a super capacitor module on the air baffle on page 219 Install a super capacitor module on the 2 5 inch middle drive cage on page 220 Super capacitor locations in the system On the chassis On standard air baffle Chapter 4 Server hardware setup 215 ...

Page 220: ...On GPU air baffle 216 ThinkSystem SR665 Setup Guide ...

Page 221: ...packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the new RAID super capacitor modu...

Page 222: ...igure 101 Super capacitor box installation 1 Align the notch on the super capacitor holder with the pin on the chassis as shown 2 Pivot the super capacitor holder downward until the other side clicks into place 218 ThinkSystem SR665 Setup Guide ...

Page 223: ...tall a super capacitor module on the air baffle About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and ha...

Page 224: ... install a super capacitor module the 2 5 inch middle drive cage About this task Attention Read Installation Guidelines on page 134 to ensure that you work safely Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discha...

Page 225: ...3 Remove the metal cover Figure 105 Removing the metal cover 1 Pull out the blue plunger 2 Slide the metal cover out of the drive cage Step 4 Install the super capacitor module Figure 106 Installing the super capacitor module Chapter 4 Server hardware setup 221 ...

Page 226: ...allation Guidelines on page 134 to ensure that you work safely Power off the server and disconnect all power cords for this task Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding...

Page 227: ...nd handling these devices with an electrostatic discharge wrist strap or other grounding system Before installing a system fan ensure that you have selected the required system fan See Technical rules for system fans on page 152 S033 CAUTION Hazardous energy present Voltages with hazardous energy might cause heating when shorted with metal which might result in spattered metal burns or both S017 C...

Page 228: ...m fan installation Install the air baffle Use this information to install the air baffle About this task S033 CAUTION Hazardous energy present Voltages with hazardous energy might cause heating when shorted with metal which might result in spattered metal burns or both S017 CAUTION Hazardous moving fan blades nearby Attention Read Installation Guidelines on page 134 to ensure that you work safely ...

Page 229: ...our server b Ensure that all components have been reassembled correctly c Ensure that all cables inside the server have been properly routed and they will not interfere with installing the air baffle Step 2 Note the orientation of the air baffle Step 3 Align the tabs on both sides of the air baffle with the corresponding slots on both sides of the chassis Then lower the air baffle into the chassis...

Page 230: ...electrostatic discharge wrist strap or other grounding system A video for this task is available at Youtube https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Check your server and ensure that All cables adapters and other components are installed and seated correctly and that you have not left loose too...

Page 231: ...over latch is completely closed c Use a screwdriver to turn the cover lock to the locked position Install a hot swap power supply Use this information to install a hot swap power supply About this task The following describes the information that you must consider when installing a power supply The server is shipped with only one power supply by default In this case the power supply is non hot swa...

Page 232: ...or a list of supported optional devices for the server go to https static lenovo com us en serverproven index shtml Attach the power information label that comes with this option onto the existing label near the power supply Figure 111 Example power supply unit label on the top cover 228 ThinkSystem SR665 Setup Guide ...

Page 233: ...l current from power telephone and communication cables is hazardous To avoid a shock hazard Do not connect or disconnect any cables or perform installation maintenance or reconfiguration of this product during an electrical storm Connect all power cords to a properly wired and grounded electrical outlet Connect any equipment that will be attached to this product to properly wired outlets When pos...

Page 234: ...ables to devices 3 Attach signal cables to connectors 4 Attach power cords to outlet 5 Turn device ON 1 Turn everything OFF 2 Remove power cords from outlet 3 Remove signal cables from connectors 4 Remove all cables from devices 230 ThinkSystem SR665 Setup Guide ...

Page 235: ...uipment and result in data loss the damages and losses result from incorrect operation of the equipment will not be covered by the manufacturers warranty S035 CAUTION Never remove the cover on a power supply or any part that has this label attached Hazardous voltage current and energy levels are present inside any component that has this label attached There are no serviceable parts inside these c...

Page 236: ...t cables as described in the following table when installing moving or opening covers on this product or attached devices To Connect To Disconnect 1 Turn OFF all power sources and equipment that is to be attached to this product 2 Attach signal cables to the product 3 Attach power cords to the product For ac systems use appliance inlets For dc systems ensure correct polarity of 48 V dc connections...

Page 237: ...e https www youtube com playlist list PLYV5R7hVcs Bz2XIeLEfGcrLGm9TSY_yt Youku http list youku com albumlist show id_52339612 html Procedure Step 1 Touch the static protective package that contains the hot swap power supply to any unpainted surface on the outside of the server Then take the hot swap power supply out of the package and place it on a static protective surface Step 2 If there is a po...

Page 238: ...twork Connect the server to any storage devices Turn on the server After the server performs a short self test power status LED flashes quickly when connected to input power it enters a standby state power status LED flashes once per second The server can be turned on power LED on in any of the following ways You can press the power button The server can restart automatically after a power interru...

Page 239: ...ler can place the server in a standby state as an automatic response to a critical system failure Start an orderly shutdown using the operating system if supported by your operating system Press the power button to start an orderly shutdown if supported by your operating system Press and hold the power button for more than 4 seconds to force a shutdown When in a standby state the server can respon...

Page 240: ...236 ThinkSystem SR665 Setup Guide ...

Page 241: ...need to specify how Lenovo XClarity Controller will connect to the network Depending on how the network connection is implemented you might need to specify a static IP address as well The following methods are available to set the network connection for the Lenovo XClarity Controller if you are not using DHCP If a monitor is attached to the server you can use Lenovo XClarity Controller to set the ...

Page 242: ...tallation or update is performed by the Lenovo XClarity Controller collecting the update and then directing the update to the target subsystem or device Out of band updates have no dependency on an operating system executing on the core processor However most out of band operations do require the server to be in the S0 Working power state On Target update The installation or update is initiated fr...

Page 243: ...system firmware I O firmware and installed operating system driver updates Lenovo XClarity Essentials Bootable Media Creator Supports core system firmware and I O firmware updates You can update the Microsoft Windows operating system but device drivers are not included on the bootable image The latest firmware can be found at the following site https datacentersupport lenovo com products servers t...

Page 244: ...on can be used to update firmware and device drivers for your servers The update can be performed within the host operating system of the server in band or remotely through the BMC of the server out of band Specific details about updating firmware using Lenovo XClarity Essentials OneCLI is available at http sysmgt lenovofiles com help topic toolsctr_cli_lenovo onecli_c_update html Lenovo XClarity ...

Page 245: ...out optimizing memory performance and configuring memory is available at the Lenovo Press Web site https lenovopress com servers options memory In addition you can take advantage of a memory configurator which is available at the following site http 1config lenovo com memory_configuration For specific information about the required installation order of memory modules in your server based on the s...

Page 246: ...ultiple physical drives that uses a certain common method to distribute data across the drives A virtual drive also known as virtual disk or logical drive is a partition in the drive group that is made up of contiguous data segments on the drives Virtual drive is presented up to the host operating system as a physical disk that can be partitioned to create OS logical drives or volumes An introduct...

Page 247: ...ng system perform the following steps 1 Go to http datacentersupport lenovo com and navigate to the support page for your server 2 Click Documentation from the navigation pane 3 Select an operating system under Operating Systems to view the installation instructions Back up the server configuration After setting up the server or making changes to the configuration it is a good practice to make a c...

Page 248: ...244 ThinkSystem SR665 Setup Guide ...

Page 249: ...he CMOS battery 7 Try to power on the system by IPMI command through XCC or by the power button 8 Implement the minimum configuration one processor one DIMM and one PSU without any adapter and any drive installed 9 Reseat all power supplies and make sure that AC LEDs on the PSU rear side are lit 10 Replace the each power supply and check the power button function after installing the each one 11 I...

Page 250: ...tatus LED is lit remove the drive from the bay wait 45 seconds and reinsert the drive making sure that the drive assembly connects to the hard disk drive backplane 3 Observe the associated green hard disk drive activity LED and the yellow status LED and perform corresponding operations in different situations If the green activity LED is flashing and the yellow status LED is not lit the drive is r...

Page 251: ...ory modules are seated correctly You have installed the correct type of memory If you changed the memory you updated the memory configuration in the Setup utility All banks of memory are enabled The server might have automatically disabled a memory bank when it detected a problem or a memory bank might have been manually disabled There is no memory mismatch when the server is at the minimum memory...

Page 252: ...cable 5 If there is any cable damages then replace the cable Voltage planar fault is displayed in the event log Complete the following steps until the problem is solved 1 Revert the system to the minimum configuration See Specifications on page 5 for the minimally required number of processors and DIMMs 2 Restart the system If the system restarts add each of the removed items one at a time and res...

Page 253: ... Copyright Lenovo 2020 249 ...

Page 254: ... at the following location http thinksystem lenovofiles com help index jsp You can take these steps to try to solve the problem yourself Check all cables to make sure that they are connected Check the power switches to make sure that the system and any optional devices are turned on Check for updated software firmware and operating system device drivers for your Lenovo product The Lenovo Warranty ...

Page 255: ...he service technicians The Lenovo service technicians can start working on your solution as soon as you have completed and submitted an Electronic Service Request Collecting service data To clearly identify the root cause of a server issue or at the request of Lenovo Support you might need collect service data that can be used for further analysis Service data includes information such as event lo...

Page 256: ...252 ThinkSystem SR665 Setup Guide ...

Page 257: ...n new editions of the publication Lenovo may make improvements and or changes in the product s and or the program s described in this publication at any time without notice The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons The information contained in this document does not ...

Page 258: ... hard disk drive capacities assume the replacement of any standard hard disk drives and population of all hard disk drive bays with the largest currently supported drives that are available from Lenovo Maximum memory might require replacement of the standard memory with an optional memory module Each solid state memory cell has an intrinsic finite number of write cycles that the cell can incur The...

Page 259: ...nt of devices or parts on implementation of appropriate remedial measures to mitigate such environmental contamination Implementation of such remedial measures is a customer responsibility Table 31 Limits for particulates and gases Contaminant Limits Particulate The room air must be continuously filtered with 40 atmospheric dust spot efficiency MERV 9 according to ASHRAE Standard 52 21 Air that en...

Page 260: ... 85 maximum dew point 24 C 75 F ASHRAE class A4 8 90 maximum dew point 24 C 75 F Server off Ambient temperature 5 45 C 41 113 F Shipping or storage Ambient temperature 40 60 C 40 140 F Maximum altitude 3050 m 10 000 ft Relative humidity non condensing 8 90 Attention Airborne particulates and reactive gases acting alone or in combination with other environmental factors such as humidity or temperat...

Page 261: ...ons networks Further certification may be required by law prior to making any such connection Contact a Lenovo representative or reseller for any questions Electronic emission notices When you attach a monitor to the equipment you must use the designated monitor cable and any interference suppression devices that are supplied with the monitor Additional electronic emissions notices are available a...

Page 262: ...Taiwan BSMI RoHS declaration Taiwan import and export contact information Contacts are available for Taiwan import and export information 258 ThinkSystem SR665 Setup Guide ...

Page 263: ...t drive bays SAS SATA 115 Cable routing for server model with 24 x 2 5 inch front drive bays 16SAS SATA 8AnyBay 96 Cable routing for server model with 24 x 2 5 inch front drive bays 16SAS SATA 8NVMe 103 Cable routing for server model with 24 x 2 5 inch front drive bays 8SAS SATA 16NVMe 93 Cable routing for server model with 24 x 2 5 inch front drive bays SAS SATA 107 Cable routing for server model...

Page 264: ...208 super capacitor module on the air baffle 219 super capacitor module on the chassis 217 super capacitor module the 2 5 inch middle drive cage 220 system fan 223 system fan cage 222 top cover 225 internal cable routing 61 internal CFF RAID HBA Expander adapter power cable routing 67 internal RAID adapter installing 209 introduction 1 L LCD diagnostics panel 32 LCD diagnostics handset External 32...

Page 265: ...143 system board components 49 system board LED 51 System configuration ThinkSystem SR665 237 system error LED 51 system fan installing 223 system fan cage installing 222 removing 158 system ID LED 51 system power LED 51 system reliability guidelines 135 T Taiwan BSMI RoHS declaration 258 Taiwan import and export contact information 258 technical rules 136 Technical rules for GPU 151 Technical rul...

Page 266: ...262 ThinkSystem SR665 Setup Guide ...

Page 267: ......

Page 268: ......

Reviews:

No comments

Related manuals for ThinkSystem SR665

NEC Express5800/T110g-S Configuration Manual preview
Express5800/T110g-S
Brand: NEC Pages: 34
HYVE AQUILADSF Manual preview
AQUILADSF
Brand: HYVE Pages: 22
HPE 3PAR StoreServ Service And Upgrade Manual preview
3PAR StoreServ
Brand: HPE Pages: 174
Intermec Norand 6920 Reference Manual preview
Norand 6920
Brand: Intermec Pages: 232
Leightronix UltraNEXUS HD Blade Product Quick Manual preview
UltraNEXUS HD Blade
Brand: Leightronix Pages: 5
IBM totalstorage fastt100 Installation, User And Maintenance Manual preview
totalstorage fastt100
Brand: IBM Pages: 160
NEC Express5800/E120d-1 Update Manual preview
Express5800/E120d-1
Brand: NEC Pages: 4
NEC Express5800/B140a-T Important Notice preview
Express5800/B140a-T
Brand: NEC Pages: 19
NEC Express5800/B120f-h System Configuration Manual preview
Express5800/B120f-h
Brand: NEC Pages: 21
NEC Express5800/GT110b User Manual preview
Express5800/GT110b
Brand: NEC Pages: 27
NEC Express5800/GT110e-S Configuration Manual preview
Express5800/GT110e-S
Brand: NEC Pages: 34
NEC Express5800/B110d Configuration Manual preview
Express5800/B110d
Brand: NEC Pages: 19
NEC Express5800/A2040d User Manual preview
Express5800/A2040d
Brand: NEC Pages: 20
NEC Express5800/D120h Configuration Manual preview
Express5800/D120h
Brand: NEC Pages: 35
NEC Express5800/A1040e User Configuration Manual preview
Express5800/A1040e
Brand: NEC Pages: 59
NEC Express5800/A1040c Configuration Manual preview
Express5800/A1040c
Brand: NEC Pages: 79
NEC Express5800/A1040d Configuration Manual preview
Express5800/A1040d
Brand: NEC Pages: 83
NEC Express5800/B120e-h User Manual preview
Express5800/B120e-h
Brand: NEC Pages: 107

Brands by name

Popular brands

Load more brands