Lenovo ThinkAgile VX 4-Socket 4U Certified Node User Manual Download Page 51 | Manualshive

Page: 51 / 296

background image

Note:

Disengage all latches, release tabs, or locks on cable connectors when you disconnect cables. Failing

to release them before removing the cables will damage the cable connectors or the cable sockets on circuit
boards, which are fragile. Any damage to the cable connectors or cable sockets might require replacing the
cables or circuit boards.

Some options, such as RAID controllers, might require additional internal cabling. See the documentation
that is provided for the option to determine any additional cabling requirements and instructions.

Cable guides

Make sure that all the cables go through the cable guides, as shown in each cabling section.

Figure 29. Cable guide locations

Connecting cables

To connect cables, observe the following guidelines:

• Disconnect all power cords from the server before you connect or disconnect any internal cables.
• See the documentation that comes with any external devices for additional cabling instructions. It might

be easier for you to route cables before you connect the devices to the server.

.

Server components

41

«
...
49
50
51
52
53
...
»

Summary of Contents for ThinkAgile VX 4-Socket 4U Certified Node

Page 1: ...User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node Machine Types 7Z12 and 7Z13 ...

Page 2: ...ions where children are likely to be present In addition be sure that you are familiar with the terms and conditions of the Lenovo warranty for your server which can be found at http datacentersupport lenovo com warrantylookup First Edition May 2020 Copyright Lenovo 2020 LIMITED AND RESTRICTED RIGHTS NOTICE If data or software is delivered pursuant to a General Services Administration GSA contract...

Page 3: ...oller 67 Management options 68 Manage the lifecycle 68 Update the firmware 68 Configure the firmware 68 Memory configuration 70 Independent memory mode 70 Memory mirroring 70 Memory sparing 70 Configure DC Persistent Memory Module DCPMM 70 RAID configuration 75 Install the operating system 75 Back up the server configuration 76 Chapter 6 Installing and configuring vCSA 77 Using the ThinkAgile Inst...

Page 4: ...ot swap drive filler 151 I O tray replacement 152 Remove the I O tray 152 Install the I O tray 154 M 2 backplane replacement 163 Remove the M 2 backplane 163 Install the M 2 backplane 165 M 2 backplane retainer adjustment 166 M 2 drive replacement 166 Remove an M 2 drive 166 Install an M 2 drive 167 Memory air baffle replacement 170 Remove a memory baffle 170 Install a memory air baffle 171 Memory...

Page 5: ...procedures 249 Resolving suspected power problems 250 Resolving suspected Ethernet controller problems 250 Troubleshooting by symptom 251 Power on and power off problems 251 Processor problems 253 Memory problems 253 Hard disk drive problems 255 Monitor and video problems 257 Keyboard mouse or USB device problems 258 Optional device problems 259 Serial device problems 260 Intermittent problems 261...

Page 6: ...iv User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 7: ...veiligheidsvoorschriften Ennen kuin asennat tämän tuotteen lue turvaohjeet kohdasta Safety Information Avant d installer ce produit lisez les consignes de sécurité Vor der Installation dieses Produkts die Sicherheitshinweise lesen Prima di installare questo prodotto leggere le Informazioni sulla Sicurezza Les sikkerhetsinformasjonen Safety Information før du installerer dette produktet Antes de in...

Page 8: ...ck and key or other means of security and is controlled by the authority responsible for the location Important Electrical grounding of the server is required for operator safety and correct system function Proper grounding of the electrical outlet can be verified by a certified electrician Use the following checklist to verify that there are no potentially unsafe conditions 1 Make sure that the p...

Page 9: ...o the safety of any non Lenovo alterations 4 Check inside the server for any obvious unsafe conditions such as metal filings contamination water or other liquid or signs of fire or smoke damage 5 Check for worn frayed or pinched cables 6 Make sure that the power supply cover fasteners screws or rivets have not been removed or tampered with Copyright Lenovo 2020 vii ...

Page 10: ...viii User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 11: ... ensure that the ThinkAgile VX MT 7Z12 and 7Z13 Series servers are running the appropriate level of firmware for Lenovo XClarity Controller UEFI and PCIe devices Lenovo XClarity Integrator which provides VMware vCenter plug in functionality so that ThinkAgile VX MT 7Z12 and 7Z13 Series servers can be managed directly from vCenter The ThinkAgile VX MT 7Z12 and 7Z13 Series hardware is based on the S...

Page 12: ...ed by the Lenovo XClarity Administrator to communicate with the Lenovo XClarity Controller XCC on each of the ThinkAgile VX7820 Appliance VX 4U Certified Node A redundant 10Gbps Ethernet network is used for VMware vSphere management and vSAN data traffic The redundant Ethernet network is implemented through two top of rack switches such as the Lenovo ThinkSystem NE1032 RackSwitch For more informat...

Page 13: ...d memory controller and Ultra Path Interconnect UPI Two processors minimum expandable up to four Designed for the LGA 3647 0 socket Scalable up to 112 cores with four processors installed For a list of supported processors see http www lenovo com us en serverproven and the Processor Options section of the ThinkAgile VX7820 Appliance VX 4U Certified Node product guide on http lenovopress com Note I...

Page 14: ...DIMM or three dimensional stacking registered DIMM 3DS RDIMM Slots 24 dual inline in each compute tray 48 DIMMs maximum Supports depending on the model 16 GB 32 GB 64 GB RDIMMs 64 GB LRDIMMs 64 GB 128 GB 256 GB 3DS RDIMMs 128 GB 256 GB and 512 GB Intel OptaneTM DC persistent memory modules DCPMMs Drive expansion Up to six SAS SATA NVMe drive backplanes Up to 24 2 5 inch hot swap drive bays Up to 2...

Page 15: ...3 0 x16 low profile Slot 6 PCI Express 3 0 x16 low profile Slot 7 PCI Express 3 0 x8 low profile Note You should not install a RAID or Host Bus Adapter card in this slot Slot 8 ML2 x16 network adapter with NC SI support Slot 9 LOM adapter Slot 10 15 PCI Express 3 0 for PCIe riser card with the following slots available depending on the riser card installed 1 x8 x8 x8 x8 PCIe full height riser kit ...

Page 16: ...nctions and runs at 1 gigabit Gb speed USB 3 0 ports two Internal SAS HBAs The following RAID options are available for this server ThinkSystem 430 8i Internal SAS HBA ThinkSystem 430 16i Internal SAS HBA Fans Up to twelve 60 mm x 38 mm internal hot swap system fans dependent on server configuration The six upper fans are not installed for the four processor limited server configuration Only three...

Page 17: ...ystem which complies to 60364 1 IEC 2005 standard has to be present or installed Minimal configuration for debugging Two processors in processor sockets 1 and 2 Two DRAM DIMMs in slot 8 and 20 One power supply in slot 1 One drive with HBA adapter and backplane Six system fans fans 1 to 6 Acoustical noise emissions base configuration Sound power idling 6 1 bels Sound power operating 6 1 bels Note T...

Page 18: ...emperature might pose a risk to the server For information about the limits for particulates and gases see Particulate contamination ASHRAE compliance information see the Operating Environment section of ThinkAgile VX7820 Appliance VX 4U Certified Node product guide on http lenovopress com for specific ASHRAE compliance levels for each server configuration The server is designed for thermal compli...

Page 19: ...ogy for best performance 12 drive bays 6 of which support NVMe drives 4 NVMe with 3 processors and 2 NVMe with 2 processors 15 rear PCIe slots with 4 processors 10 rear slots with 3 processors 6 rear slots with 2 processors Filler installed in the upper tray area Processor configurations This configuration supports 2 or 4 processors 2 processor configuration Processors are installed in location 1 ...

Page 20: ...not connected if the 2x16 riser is used in riser slot 1 2 41 Not connected for 2x16 Slots 1 and 2 are not connected if the 2x16 riser is used in riser slot 1 3 41 4 41 5 I O tray 2 6 2 7 1 8 ML2 1 9 LOM 1 PCH 10 Riser 2 32 11 32 12 32 13 32 14 Not connected 15 ML2 Not connected 16 2x8 Riser 32 17 1 M 2 I O tray 1 PCH Storage adapter Upper tray Not connected 10 User Guide for ThinkAgile VX7820 Appl...

Page 21: ...y The server can support up to 12 NVMe drives depending on the server configuration The NVMe capable drive bays are the even numbered bays as shown in Figure 4 Drive bay locations on page 11 Bottom two drive bays support SAS or SATA drives only Regular 2 5 inch SAS SATA drive bays support only SAS or SATA drives however the Lenovo AnyBay drive bay design allows a choice of SATA SAS or U 2 NVMe PCI...

Page 22: ...Filler installed in the upper tray area Once upgraded the system will have 24 drive bays Processor configurations This configuration supports 2 3 or 4 processors 2 processor configuration Processors are installed in location 1 and 2 4 processor configuration Processors are installed in locations 1 2 3 and 4 Figure 5 Processor numbering viewed from the front of the server PCIe slot connectivity The...

Page 23: ...otes 1 In 2 processor configurations processors 3 and 4 are not installed this means slots 1 4 slot 10 13 and slot 16 are not connected 2 In 3 processor configurations processor 3 is not installed this means slot 10 13 and slot 16 are not connected Drive bays Drives are all located at the front of the server 12 drives at the front of the upper tray and 12 at the front of the lower tray Drives are ...

Page 24: ...o configure some of the bays with high performance PCIe SSDs while still using the other bays for high capacity HDDs which is an ideal solution for storage tiering Figure 6 Drive bay locations The following table lists the drive bays and NVMe support for the Four Socket Performance configuration Installed process ors Maximum storage Location of NVMe drives see Figure 6 Drive bay locations on page ...

Page 25: ...d the server will support 12 NVMe drives Processor configurations This configuration requires 4 processors installed in locations 1 2 5 and 6 Figure 7 Processor numbering viewed from the front of the server PCIe slot connectivity The following table shows the processor to PCIe slot connectivity PCIe slot Slot location Description 1 Riser 1 Not connected 2 Not connected Chapter 1 Introduction 15 ...

Page 26: ...S SATA backplane Supports 4 SAS or SATA drives AnyBay backplane Top two drives support SAS SATA or NVMe interface drives Lenovo AnyBay The server can support up to 12 NVMe drives depending on the server configuration The NVMe capable drive bays are the even numbered bays as shown in Figure 8 Drive bay locations on page 17 Bottom two drive bays support SAS or SATA drives only Regular 2 5 inch SAS S...

Page 27: ... a glance view of the managed hardware inventory and status Configuration management You can quickly provision and pre provision all of your servers using a consistent configuration Configuration settings such as local storage I O adapters boot settings firmware ports and Lenovo XClarity Controller and UEFI settings are saved as a server pattern that can be applied to one or more managed servers W...

Page 28: ...management xclarity integrators Lifecycle management Recommended procedure for updating a ThinkAgile VX cluster Step 1 Update the third party applications to ensure that vCenter integration is supported Guidance for VMware products is available at https kb vmware com s article 53710 For other third party products contact the vendor for guidance Step 2 Update vCenter The latest best recipe is avail...

Page 29: ...structions are available at https kb vmware com s article 2145116 Firmware updates Several options are available to update the firmware for the server Lenovo XClarity Administrator and Lenovo XClarity Integrator are the recommended tools for updates You can use the tools listed here to update to the most current firmware in a given Best Recipe for your appliance node and the devices that are insta...

Page 30: ...ing table to determine the best Lenovo tool to use for installing and setting up the firmware Note The server UEFI settings for option ROM must be set to Auto or UEFI to update firmware using Lenovo XClarity Administrator or Lenovo XClarity Essentials For more information see the following Tech Tip https datacentersupport lenovo com us en solutions ht506118 Tool In band update Out of band update O...

Page 31: ... changed the default to be the text based system setup you can bring up the Graphical User Interface from the text based system setup interface Additional information about using Lenovo XClarity Provisioning Manager Lite to update firmware is available at http sysmgt lenovofiles com help topic LXPM platform_update html Lenovo XClarity Controller If you need to install a specific update you can use...

Page 32: ...liance policy to managed endpoints Lenovo XClarity Administrator monitors changes to the inventory for those endpoints and flags any endpoints that are out of compliance Specific details about updating firmware using Lenovo XClarity Administrator are available at http sysmgt lenovofiles com help topic com lenovo lxca doc update_fw html Tech Tips Lenovo continually updates the support website with ...

Page 33: ...er in the customer label spaces 1 in the following illustration Note If any customer labels are placed on the removable front cover make sure that the cover is installed on the same server where the labels were originally added Figure 9 Location of the ID label for the ThinkAgile VX7820 Appliance VX 4U Certified Node QR code In addition the service label that is located at the top rear of the syst...

Page 34: ... before operating the system The label provides the default IPv4 host name and default IPv6 link local address of the XCC Figure 11 Network access tag Front view The front view of the server varies slightly by the model In some models components are replaced by fillers 24 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 35: ...f the server Callout Callout 1 2 5 inch drive bays 12 23 some models 7 Power button and power LED see Front operator panel on page 26 2 Fans 1 12 8 Front operator panel with pull out LCD display see LCD system information display panel on page 27 3 2 5 inch drive bays 0 11 9 USB 2 0 port 4 System error LED see Front operator panel on page 26 10 USB 2 0 Lenovo XClarity Controller management port 5 ...

Page 36: ...shows the controls and LEDs on the front operator panel Table 4 Operator panel front controls and indicators Callout Callout 1 Power button and power LED 3 System ID button LED 2 Network activity LED 4 System error LED 1 Power button and power LED Press this button to turn the server on and off manually The states of the power LED are as follows Off Power is not present or the power supply has fai...

Page 37: ...mation display panel on page 27 for more information LCD system information display panel The following section includes an overview of the LCD system information display panel which displays various types of information about the server The LCD system information display panel attached to the front of the server allows quick access to system status firmware network and health information Table 5 ...

Page 38: ...ple of the information on the display panel Table 6 LCD system information display information Callout Callout 1 System name 4 UEFI POST code 2 Ambient temperature 5 System status 3 Estimated power consumption Following is an illustration of the LCD system information display panel menu options flow 28 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 39: ... server Lenovo XClarity Controller LXCC system error log Note The Scroll Down button will only function for this menu option when errors occur A list of current errors reported by the system will be displayed System VPD information Machine type and serial number Universal Unique Identifier UUID string System firmware levels UEFI code level LXCC code level LXCC network information LXCC hostname LXC...

Page 40: ...rk connector RJ45 11 PCIe slot 11 on riser 2 24 Serial connector 12 PCIe slot 12 on riser 2 25 USB 3 0 connectors 2 13 PCIe slot 13 on riser 2 26 VGA video port Notes The following PCIe slots are assigned to components elsewhere in the server PCIe slot 18 is assigned to the RAID card in the lower tray PCIe slot 19 is assigned to the RAID card in the upper tray PCIe slot 20 is assigned to the M 2 b...

Page 41: ...ot meet the power supply input requirements for example a 1600 watt power supply will not run off of 120 vac input On The power supply is receiving the correct input power and the power supply is operating normally 2 Output status DC green The output status LED can be in one of the following states Off The server is powered off not connected to input power the power supply is not working correctly...

Page 42: ... LED is controlled by the Lenovo XClarity Controller Table 10 Adapter LEDs Callout Callout 4 Riser 1 to 4 fault LED 15 Adapter 15 fault LED 5 Adapter 1 fault LED 16 Adapter 16 fault LED 6 Adapter 2 fault LED 17 Adapter 17 fault LED 7 Adapter 3 fault LED 18 Riser 16 through 17 fault LED 8 Adapter 4 fault LED 22 3v fault system battery LED 9 Riser 10 to 15 fault LED 23 I O board fault LED 10 Adapter...

Page 43: ...NVMe connector 5 Hard disk drive Signal connector Storage board assembly connectors The following illustration shows the internal connectors on the storage board assembly Figure 18 Storage board assembly connectors Table 12 Storage board assembly connectors Callout Callout 1 NVMe connector 5 Hard disk signal connector J13 2 Fan signal connector J5 6 Hard disk drive power connector J2 3 Fan power c...

Page 44: ...or see Enable TPM TCM on page 159 3 PCIe3 x16 75 watt connector slot 6 10 Riser for slot 10 to 15 connectors Riser 2 4 PCIe3 x8 25 watt connector slot 7 11 M 2 SATA PCIe Hypervisor backplane connector see note following this table 5 Signal cable to power backplane 12 Riser for PCIe slot 16 to 17 Riser 3 6 Battery CR2032 13 SW2 Switch block 2 7 ML2 PCIe3 x16 connector slot 8 14 SW1 Switch block 1 N...

Page 45: ...h bypasses the power on password check the next time the server is turned on and starts the Lenovo XClarity Provisioning Manager Lite so that you can change or delete the power on password You do not have to move the switch back to the default position after the power on password is overridden Changing the position of this switch does not affect the administrator password check if an administrator...

Page 46: ...rs There are four PCIe3 x8 full height half length connectors each in the slot 1 to 4 riser connector 1 and slot 10 to 13 riser connector 2 risers Note Callouts for slots 1 to 4 apply when the riser is installed in the riser 1 position Callouts for slots 10 to 13 apply when the riser is installed in the riser 2 position Figure 20 Slot 1 to 4 and slot 10 to 13 riser connectors Table 15 Slot 1 to 4 ...

Page 47: ...x16 Slot 4 or 13 Slot 10 to 15 riser connectors There are five PCIe3 x16 full height half length connectors and one ML2 PCIe3 x16 connector in the slot 10 to 15 riser Figure 22 Slot 10 to 15 riser connectors Table 17 Slot 10 to 15 riser connectors Callout Callout 10 PCIe3 x16 Slot 10 13 PCIe3 x16 Slot 13 11 PCIe3 x16 Slot 11 14 PCIe3 x16 Slot 14 12 PCIe3 x16 Slot 12 15 ML2 PCIe3 x16 Slot 15 Slot 1...

Page 48: ...connectors Callout Callout 1 PCIe SAS interface from system board PCIE SAS only 5 Power for drive to backplane 2 or 5 BP 2 5 2 Drive power from system board POWER 6 PCI connector to RAID card 3 Drive signal from system board SIDEBAND 7 Power for drive to backplane 3 or 4 BP 3 4 4 Power for drive to backplane 1 or 6 BP 1 6 RAID card connectors The following illustration shows the internal connector...

Page 49: ... connectors type 2 Table 21 RAID card connectors type 2 Callout Callout 1 RAID flash power module J14 connector 3 RAID connector for drive backplane 1 or 4 SAS only 2 RAID connector for drive backplane 2 or 5 SAS only 4 PCI connector to storage interposer Drive backplane connectors The following illustration shows the internal connectors on the drive backplanes Note Some cable connectors have lock...

Page 50: ...ackplane power from interposer SAS NVMe drive backplane connectors Figure 28 SAS NVMe drive backplane connectors Table 23 SAS NVMe drive backplane connectors Callout Callout 1 Drive connector for SAS or NVMe drives 0 4 8 12 16 and 20 5 NVMe signal from compute board or storage tray 2 Drive connector for SAS or NVMe drives 2 6 10 14 18 and 22 6 SAS signal from RAID card 3 Drive connector for SAS dr...

Page 51: ...he documentation that is provided for the option to determine any additional cabling requirements and instructions Cable guides Make sure that all the cables go through the cable guides as shown in each cabling section Figure 29 Cable guide locations Connecting cables To connect cables observe the following guidelines Disconnect all power cords from the server before you connect or disconnect any ...

Page 52: ...to the cable sockets might require replacing the system board Cable routing for common components Use this section to understand how to route cables for common server components Cable routing for common components is different for the lower and upper trays Common cable routing lower tray on page 42 Common cable routing upper tray on page 44 Notes Make sure that all the cables go through the cable ...

Page 53: ...anel video port From Front panel video port To Lower system board in lower compute tray Front video connector see System board connectors on page 32 2 Front panel USB ports From Front USB ports To Lower system board in lower compute tray USB connector see System board connectors on page 32 Chapter 2 Server components 43 ...

Page 54: ...mpute tray is fully assembled 4 Fan cage cable From Fan cage connector underside of fan cage To Lower system board in lower compute tray Fan board power J56 and signal J40 connectors see System board connectors on page 32 Common cable routing upper tray The following illustration shows cable routing for common components in the upper tray Note In this illustration the image on the left shows the t...

Page 55: ...oard assembly in upper tray fan power J3 and signal J5 connectors see Storage board assembly connectors on page 33 Note Do not route the fan cage cable in the upper tray through any cable guides Cable routing for drives Use this section to understand how to route cables for drives and related components Cable routing for drive components is different for the lower and upper trays Drive cable routi...

Page 56: ...sconnect the cable Drive cable routing lower tray Cable routing for drive components is different for SAS and NVMe drives Common drive cables lower tray SAS drive cables lower tray NVMe drive cables lower tray Common drive cables lower tray Common drive cables are used by both SAS and NVMe drives Figure 33 Cable routing common drive cables lower tray 46 User Guide for ThinkAgile VX7820 Appliance T...

Page 57: ...module From RAID flash power module connector To RAID card flash power module J14 connector see RAID card connectors on page 38 3 Drive power From Lower system board in lower compute tray POWER connector see System board connectors on page 32 To Storage interposer POWER connector see Storage interposer connectors on page 38 4 Drive signal From Lower system board in lower compute tray Signal connec...

Page 58: ...ector see RAID card connectors on page 38 To Drive backplane connector SAS see Drive backplane connectors on page 39 3 RAID interface for drive backplane 2 From RAID card C2 connector see RAID card connectors on page 38 To Drive backplane connector SAS see Drive backplane connectors on page 39 4 RAID interface for drive backplane 3 From RAID card C1 connector see RAID card connectors on page 38 To...

Page 59: ...nectors on page 39 2 Drive backplane 2 NVMe From Upper system board in lower compute tray NVMe connector see System board connectors on page 32 To Drive backplane 2 connector NVMe see Drive backplane connectors on page 39 3 Drive backplane 3 NVMe From Lower system board in lower compute tray NVMe connector see System board connectors on page 32 To Drive backplane 3 connector NVMe see Drive backpla...

Page 60: ...es are used by both SAS and NVMe drives Note In this illustration the image on the left shows the tray right side up and the image on the right shows the tray upside down Figure 36 Cable routing common drive cables upper tray with compute system board 50 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 61: ...kplanes 4 5 and 6 From Storage interposer see Storage interposer connectors on page 38 For drive backplane 4 use interposer BP 3 4 connector For drive backplane 5 use interposer BP 2 5 connector For drive backplane 6 use interposer BP 1 6 connector To Drive backplane connector Power see Drive backplane connectors on page 39 2 RAID flash power module From RAID flash power module connector To RAID c...

Page 62: ...er compute tray Signal connector see System board connectors on page 32 Storage board assembly in upper tray hard disk drive signal J13 connector see Storage board assembly connectors on page 33 To Storage interposer SIDEBAND connector see Storage interposer connectors on page 38 SAS drive cables upper tray Common drive cables are used by only SAS drives Note In this illustration the image on the ...

Page 63: ... interposer PCIe connector see Storage interposer connectors on page 38 2 RAID interface for drive backplane 4 From RAID card C3 connector see RAID card connectors on page 38 To Drive backplane connector SAS see Drive backplane connectors on page 39 3 RAID interface for drive backplane 5 From RAID card C2 connector see RAID card connectors on page 38 To Drive backplane connector SAS see Drive back...

Page 64: ...uting NVMe drive cables upper tray with system board Note In this illustration the image on the left shows the tray right side up and the image on the right shows the tray upside down Figure 41 Cable routing NVMe drive cables upper tray with storage board assembly 54 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 65: ... 39 3 Drive backplane 4 NVMe From Upper system board in upper compute tray NVMe connector see System board connectors on page 32 Storage board assembly in upper tray NVMe connector see Storage board assembly connectors on page 33 To Drive backplane 4 connector NVMe see Drive backplane connectors on page 39 Parts list Use the parts list to identify each of the components that are available for this...

Page 66: ...Figure 42 Server components 56 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 67: ...alls a structural component at your request you will be charged for the service Table 32 Parts listing Index Description Tier 1 CRU Tier 2 CRU FRU Consuma ble and Structural part For more information about ordering the parts shown in Figure 42 Server components on page 56 https datacentersupport lenovo com us en products solutions and software thinkagile vx vx7820 7z13 parts display compatible htt...

Page 68: ...ller four slot 28 Riser bracket slot 14 to 15 28 PCIe filler two slot 28 Riser bracket filler slot 14 to 15 29 PCIe filler single slot 30 Riser 1 slot 1 to 4 Riser 2 slot 10 to 13 31 Bezel front 32 Cover front 33 Storage drive 34 Drive filler single bay or four bay 35 Chassis 36 Cable covers 37 M 2 retainer 38 M 2 backplane 39 M 2 drive 40 I O tray 41 Adapter LOM or ML2 x16 network 42 Power supply...

Page 69: ...Chapter 2 Server components 59 ...

Page 70: ...ded to be operated at 115 volts Use a UL listed and CSA certified cord set consisting of a minimum 18 AWG Type SVT or SJT three conductor cord a maximum of 15 feet in length and a parallel blade grounding type attachment plug rated 15 amperes 125 volts For units intended to be operated at 230 volts U S use Use a UL listed and CSA certified cord set consisting of a minimum 18 AWG 16 AWG for the 200...

Page 71: ... Default Password Virtual Machines VMs IPV4 Host name User ID Password vCenter Server XClarity Adminis trator XClarity Integrator Infrastructure Domain DNS NTP Default Gateway Virtual Network For information about the VMKernal see https docs vmware com en VMware vSphere 6 5 com vmware vsphere networking doc GUID D4191320 209E 4CB5 A709 C8741E713348 html Physical Adapters VMkernal Adapter vSwitch 0...

Page 72: ...ter Services vMotion Provision ing FT Logging Manage ment vSphere Replica tion vSphere Replica tion NFC vSAN Adapter 1 Adapter 2 62 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 73: ...er See the Rack Installation Guide that comes with the optional rail kit b Connect the Ethernet cables and power cords to the server See Rear view on page 29 to locate the connectors See Cable the server on page 64 for cabling best practices c Power on the server See Power on the server connect input power on page 64 3 Connect the Lenovo XClarity Controller to the management network See Set the ne...

Page 74: ...o a power source to the data network and to storage In addition you will need to connect the server to the management network Connect to power Connect the server to power Connect to the network Connect the server to the network Connect to storage Connect the server to any storage devices Power on the server connect input power After the server performs a short self test power status LED flashes qu...

Page 75: ...om the server power status LED off you must disconnect all power cables S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source To pl...

Page 76: ...66 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 77: ...default IPv4 address and the IPv6 Link Local Address LLA is provided on the Lenovo XClarity Controller Network Access label that is affixed to the Pull Out Information Tab If you are using the Lenovo XClarity Administrator Mobile app from a mobile device you can connect to the Lenovo XClarity Controller through the Lenovo XClarity Controller USB connector on the front of the server For the locatio...

Page 78: ...the firmware for the server Refer to Firmware updates on page 19 Configure the firmware Several options are available to install and set up the firmware for the server Important Do not configure option ROMs to be set to Legacy unless directed to do so by Lenovo Support This setting prevents UEFI drivers for the slot devices from loading which can cause negative side effects for Lenovo software suc...

Page 79: ...ide or remove it The only alternative is to replace the I O tray Lenovo XClarity Essentials OneCLI You can use the config application and commands to view the current system configuration settings and make changes to Lenovo XClarity Controller and UEFI The saved configuration information can be used to replicate or restore other systems For information about configuring the server using Lenovo XCl...

Page 80: ...Reference for memory mirroring requirements and recommended memory module population sequences Memory sparing In memory sparing mode one memory rank serves as a spare for other ranks on the same channel in case they fail The spare rank is held in reserve and not used as active memory until a failure is indicated with reserved capacity subtracted from the total available memory in the system After ...

Page 81: ...managed with the following tools Lenovo XClarity Provisioning Manager Lite LXPM To open LXPM power on the system and press F1 as soon as the logo screen appears If a password has been set enter the password to unlock LXPM Go to UEFI Setup System Settings Intel Optane DCPMMs to configure and manage DCPMMs For more details see https sysmgt lenovofiles com help topic LXPM using_LXPM html Note If the ...

Page 82: ...100 Memory Mode Go to Goals Memory Mode input the memory percentage and reboot the system Notes Before changing from one mode to another 1 Make sure the capacity of installed DCPMMs and DRAM DIMMs meets system requirements for the new mode see the Memory Population Reference 2 Back up all the data and delete all the created namespaces Go to Namespaces View Modify Delete Namespaces to delete the cr...

Page 83: ...y to not interleaved will turn the displayed App Direct regions from one region per processor to one region per DCPMM Regions After the memory percentage is set and the system is rebooted regions for the App Direct capacity will be generated automatically Select this option to view the App Direct regions Namespaces App Direct capacity of DCPMMs requires the following steps before it is truly avail...

Page 84: ...e locked units needs to be disabled before the units are available for access or secure erase Always make sure to keep records of the slot number of locked DCPMMs and corresponding passphrases In the case the passphrases are lost or forgotten the stored data cannot be backed up or restored but you can contact Lenovo service for administrative secure erase After three failed unlocking attempts the ...

Page 85: ...nown as RAID drive group is a group of multiple physical drives that uses a certain common method to distribute data across the drives A virtual drive also known as virtual disk or logical drive is a partition in the drive group that is made up of contiguous data segments on the drives Virtual drive is presented up to the host operating system as a physical disk that can be partitioned to create O...

Page 86: ...ion Make sure that you create backups for the following server components Management processor You can back up the management processor configuration through the Lenovo XClarity Controller interface For details about backing up the management processor configuration see http sysmgt lenovofiles com help topic com lenovo systems management xcc doc NN1ia_c_ backupthexcc html Alternatively you can use...

Page 87: ...ile VX You can use the ThinkAgile Installer to deploy the vSAN cluster to an existing vCenter instance or to install a vCenter instance and deploy a vSAN cluster on a new instance Use the following procedures to download and run the installer Downloading and setting up the ThinkAgile Installer The ThinkAgile Installer is available on the Lenovo website It is downloaded as an OVA template that can ...

Page 88: ... to be added to the VSAN cluster Hostname static IPv4 address gateway netmask DNS server and credentials for the ESXi hosts that are to be used to deploy the VSAN VM data network to which to attach the vCenter Data store to deploy the VM for vCenter User name and password for the vCenter that you want to deploy Ensure that you meet the following prerequisites before running the ThinkAgile Installe...

Page 89: ...browser navigate to the IPv4 address shown in the VM console Note When you run ThinkAgile Installer for the first time you will need to accept the certificate The certificate is a value security certificate and all communication is encrypted However it is signed by a Lenovo certificate authority therefore it is not listed in the trusted certificate chain for your browser The ThinkAgile Installer p...

Page 90: ...hoose to deploy a new vCenter appliance specify the following information Host network settings Specify the network settings for the host on which the vCenter appliance will be deployed such as IPv4 address or hostname access credentials port group and datastore name vCenter appliance network settings Specify the network settings to be applied to the vCenter appliance such as the IPv4 address or h...

Page 91: ...Installer https datacentersupport lenovo com us en solutions ht505302 2 Download the vCSA Installer to a Microsoft Windows based system that has network connectivity to the ThinkAgile VX7820 Appliance VX 4U Certified Node 3 Mount the vCSA ISO image file 4 On the mounted ISO image locate and run the installer executable at vcsa ui installer win32 installer exe 5 Click Install to begin the Stage 1 i...

Page 92: ...ance VX 4U Certified Node to an existing instance of vCSA 1 Optionally Create a new Datacenter For instructions on creating a Datacenter see the following website Create a Datacenter and Add Hosts in the vSphere Web Client 2 Create a Cluster For instructions on creating a cluster see the following website Create a Cluster 3 Add the ThinkAgile VX7820 Appliance VX 4U Certified Node to the Cluster 82...

Page 93: ...ion to deploy Lenovo XClarity Administrator Before you begin make sure that you download the Lenovo XClarity Administrator from Lenovo Refer to the ThinkAgile VX Best Recipe to determine the correct version to be downloaded Complete the following steps to deploy Lenovo XClarity Administrator Step 1 Download the Lenovo XClarity Administrator image The currently supported version of Lenovo XClarity ...

Page 94: ...ified Node servers are install to Lenovo XClarity Administrator Information about adding rack views to Lenovo XClarity Administrator is available at http sysmgt lenovofiles com help topic com lenovo lxca doc racks_manage html Step 5 Add the ThinkAgile VX7820 Appliance VX 4U Certified Node firmware packages to the Lenovo XClarity Administrator firmware repository From the Lenovo XClarity Administra...

Page 95: ...servers in the cluster Note Consider creating the server pattern from an existing server to save time Information about creating a server pattern is available at http sysmgt lenovofiles com help topic com lenovo lxca doc server_addserverpattern html Chapter 7 Installing and setting up Lenovo XClarity Administrator 85 ...

Page 96: ...86 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 97: ... xclarity integrators Deploying Lenovo XClarity Integrator for VMware vCenter Use this procedure to install Lenovo XClarity Integrator for VMware vCenter Complete the following steps to deploy Lenovo XClarity Integrator Step 1 Download the Lenovo XClarity Integrator image The currently supported version of Lenovo XClarity Integrator is available at the following website https datacentersupport len...

Page 98: ...m help topic com lenovo lxci_vcenter doc vcenter_install_config_lxci html Step 4 Manage the cluster by requesting access to the BMC of each of the servers in the cluster Information about requesting BMC access is available at http sysmgt lenovofiles com help topic com lenovo lxci_vcenter doc vcenter_using_vsphere_web_ client_request_imm_access html 88 User Guide for ThinkAgile VX7820 Appliance Thi...

Page 99: ... notices carefully Attention Prevent exposure to static electricity which might lead to system halt and loss of data by keeping static sensitive components in their static protective packages until installation and handling these devices with an electrostatic discharge wrist strap or other grounding system Read the safety information and guidelines to ensure that you work safely A complete list of...

Page 100: ... latch and so on Orange on a component or an orange label on or near a component indicates that the component can be hot swapped if the server and operating system support hot swap capability which means that you can remove or install the component while the server is still running Orange can also indicate touch points on hot swap components See the instructions for removing or installing a specif...

Page 101: ...c discharge wrist strap or other grounding system While the device is still in its static protective package touch it to an unpainted metal surface on the outside of the server for at least two seconds This drains static electricity from the package and from your body Remove the device from the package and install it directly into the server without putting it down If it is necessary to put the de...

Page 102: ...talling the Mellanox Innova 2 Flex Open Programmable SmartNIC in an 8 socket configuration A maximum of four of these adapters can be installed in the system The Mellanox Innova 2 Flex Open Programmable SmartNIC can be installed only in adapter slots 2 3 4 6 7 11 12 13 14 15 16 or 17 The installation of two of these adapters in contiguous sequential slots is not supported For example you cannot in...

Page 103: ...pter slot and close the retention latch 2 Install the riser See Install the riser for slot 1 to 4 riser 1 on page 218 3 Install the I O tray and connect all cables See Install the I O tray on page 154 If you are instructed to return the adapter follow all packaging instructions and use any packaging materials for shipping that are supplied to you Remove a PCIe adapter from slot 5 to 8 PCIe adapter...

Page 104: ...ideo of the installation process is available YouTube Youku Figure 44 Adapter removal slot 5 to 8 Step 1 Disengage the adapter from its connector on the I O tray and lift the adapter from the I O tray After you remove the adapter If you are removing the adapter and are not installing a replacement 1 Install a filler in the empty adapter slot 2 Install the riser for slot 1 to 4 riser 1 See Install ...

Page 105: ...isconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove a LOM adapter from slot 9 1 Disconnect and label all cables connected to adapters in the I O tray then remove the I O tray See Remove the I O tray on page 152 2 Remove the riser for slot 10 to 15 riser 2 t...

Page 106: ...o return the adapter follow all packaging instructions and use any packaging materials for shipping that are supplied to you Remove a PCIe adapter from slot 10 to 15 PCIe adapters in slot 10 to 15 are in an I O tray riser that is accessed from the rear of the server After removing the I O tray and the riser for slot 10 to 15 riser 2 open the retention latch and remove the adapter from the riser S0...

Page 107: ...move the adapter If you are removing the adapter and are not installing a replacement 1 Install a filler in the empty adapter slot and close the retention latch 2 Install the riser See Install the riser for slot 10 to 15 riser 2 on page 220 3 Install the I O tray and connect all cables See Install the I O tray on page 154 Remove an I O adapter from slot 16 to 17 I O adapters in slot 16 to 17 are i...

Page 108: ... 17 Watch the procedure A video of the installation process is available YouTube Youku Figure 47 Adapter removal slot 16 to 17 Step 1 Open the retention latch Step 2 Disengage the adapter from its connector on the riser and lift the adapter from the riser After you remove the adapter If you are removing the adapter and are not installing a replacement 1 Install a filler in the empty adapter slot a...

Page 109: ...sure that all power cords are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you install an adapter in slot 1 to 4 make sure that there is no filler installed in the slot where you are installing the adapter Important You must remove the riser from the I O tray ...

Page 110: ... O tray Note You should not install a RAID or Host Bus Adapter card in slot 7 S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source...

Page 111: ...the chassis and connect all cables See Install the I O tray on page 154 Install a LOM adapter in slot 9 The LOM adapter in slot 9 is in the I O tray that is accessed from the rear of the server With the I O tray removed from the chassis insert the adapter in the I O tray S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical curr...

Page 112: ... tray If you have no other operations to perform on the I O tray after installing a LOM adapter in slot 9 1 Install the riser for slot 10 to 15 riser 2 See Install the riser for slot 10 to 15 riser 2 on page 220 2 If any of the adapters in the I O tray have internal cables make sure that they are connected before installing the I O tray 3 Install the I O tray in the chassis and connect all cables ...

Page 113: ...are installing an adapter in slot 14 or 15 the slot 14 to 15 riser bracket or a riser bracket filler must be installed on the riser See Install the riser bracket for slot 14 to 15 riser 2 on page 221 Complete the following steps to install an adapter in slot 10 to 15 Watch the procedure A video of the installation process is available YouTube Youku Figure 51 Adapter installation slot 10 to 15 Step...

Page 114: ...ical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you install an adapter in slots 16 or 17 make sure that there is no filler installed in the slot where you are installing the adapter Important You must r...

Page 115: ... riser in the I O tray 2 Install the riser in the I O tray See Install the riser for slot 16 to 17 riser 3 on page 222 3 Install the I O tray in the chassis and connect all cables See Install the I O tray on page 154 System battery CR2032 replacement The system battery is in the I O tray that is accessed from the rear of the server Remove the system battery The system battery is in the I O tray th...

Page 116: ... ion battery To avoid possible explosion do not burn the battery Exchange it only with the approved part Recycle or discard the battery as instructed by local regulations on page 89 on page 65 Consider the following when replacing the system battery Lenovo has designed this product with your safety in mind The lithium battery must be handled correctly to avoid possible danger If you replace the ba...

Page 117: ...he top of the battery toward the center of the I O tray to disengage if from the battery retainer Step 2 Lift the battery from the retainer using your fingers After removing the system battery dispose of it according to local regulations Install the system battery The system battery is in the I O tray that is accessed from the rear of the server With the I O tray removed from the chassis insert th...

Page 118: ...change it only with the approved part Recycle or discard the battery as instructed by local regulations on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure When replacing the system battery in the server You must replace the system battery with another battery of the same type from the same manufacturer Afte...

Page 119: ...nd reset the system date and time See Chapter 5 System configuration on page 67 System board replacement System boards are in the upper or lower compute tray that is accessed from the front of the server Important Before you return the system board make sure that you install the CPU socket covers from the new system board To replace a CPU socket cover 1 Take a socket cover from the CPU socket asse...

Page 120: ... installed See Remove a compute tray on page 115 3 If you are removing the lower system board in a compute tray first remove the upper system board or filler from the compute tray to access connectors on the lower system board See the steps below or Remove a system board filler on page 113 Complete the following steps to remove the compute system board Watch the procedure A video of the replacemen...

Page 121: ... covers from the new system board To replace a CPU socket dust cover 1 Take a dust cover from the CPU socket assembly on the new compute system board and orient it correctly above the CPU socket assembly on the removed system board 2 Gently press down the dust cover legs to the CPU socket assembly pressing on the edges to avoid damage to the socket pins You might hear a click on the dust cover is ...

Page 122: ...e Youku Figure 56 System board installation Step 1 Align the system board with the rails in the compute tray then insert the system board making sure that the rails on the board fit into the slots in the compute tray Step 2 Slide the system board into the compute tray until the release clips snap into the locked position Step 3 Connect all cables on the system board that go to connectors in the co...

Page 123: ...are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove the system board filler 1 Remove the front cover See Remove the front cover on page 131 2 Remove compute tray where the system board filler is installed See Remove a compute tray on page 115 Complete ...

Page 124: ...ute tray and pushing it in until it locks in place S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 CAUT...

Page 125: ...ay replacement Use the following procedures to remove and install the upper or lower compute tray Remove a compute tray The upper and lower compute trays are accessed from the front of the server Open the release levers to extract the compute tray pressing the release tabs when it stops to remove the tray fully from the chassis S002 CAUTION The power control button on the device and the power swit...

Page 126: ...Press the button on each release lever then simultaneously rotate the release levers until they are perpendicular to the chassis Step 2 Pull compute tray forward evenly until it stops then close the release levers Step 3 Press the release tabs on either side of the tray then slide the tray fully forward evenly and remove it from the chassis Attention Be prepared to support the full weight of the c...

Page 127: ...lane see Hard disk drive backplane replacement on page 139 Hard disk drives and fillers see Hot swap drive replacement on page 147 Cables see Internal cable routing on page 40 If you are instructed to return the compute tray follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install a compute tray Install a compute tray by inserting it in the fr...

Page 128: ...tray when it is fully removed from the chassis Watch the procedure A video of the replacement process is available YouTube Youku Figure 61 Compute tray installation fully removed Important Compute trays must be inserted in the chassis with the processors and memory visible from the top Step 1 Align the compute tray with its opening in the front of the chassis and insert Step 2 Fully open the compu...

Page 129: ... Compute System Board 01CV978 installed in each compute tray Remove a fan Press the fan handle inward toward the center of the server to release the latch then pull the fan forward halfway and wait 10 seconds before pulling the fan out of the server Fans are hot swap devices that can be removed while the server is powered on S017 CAUTION Hazardous moving fan blades nearby Keep fingers and other bo...

Page 130: ...packaging materials for shipping that are supplied to you Install the fan Insert the fan in the front of the chassis then push it in until it stops and the latch engages Fans are hot swap devices that can be installed while the server is powered on S017 CAUTION Hazardous moving fan blades nearby Keep fingers and other body parts away Note The ThinkAgile VX7820 Appliance VX 4U Certified Node suppor...

Page 131: ...s or optional storage tray that are accessed from the front of the server Procedures for removing and installing fan cages in the upper and lower trays are different Remove a fan cage upper tray The upper fan cage is in the upper compute tray or optional storage tray that is accessed from the front of the server After removing the upper tray turn it upside down press the fan cage releases and pull...

Page 132: ...orage tray See Remove a compute tray on page 115 or Remove a storage tray on page 232 3 If you removed a compute tray and it has one or more compute system boards or a system board filler installed remove the system boards or system board filler See Remove a system board on page 109 or Remove a system board filler on page 113 4 Turn the tray upside down Complete the following steps to remove an up...

Page 133: ...an cage Attention To prevent damage to components lift the fan cage straight up when removing it from the tray Step 4 Lift and remove the fan cage from the tray If you are instructed to return the fan cage follow all packaging instructions and use any packaging materials for shipping that are supplied to you Remove a fan cage lower tray The lower fan cage is in the lower compute tray that is acces...

Page 134: ...age 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove a lower fan cage 1 Remove the front cover See Remove the front cover on page 131 2 Remove the lower compute tray See Remove a compute tray on page 115 Complete the following steps to remove a lower fan cage Watch the procedure A video of the installation...

Page 135: ...e tray Step 4 Lift and remove the fan cage from the tray If you are instructed to return the fan cage follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install a fan cage upper tray The upper fan cage is in the upper compute tray or optional storage tray that is accessed from the front of the server Fan cages are keyed and each can be used only...

Page 136: ...m the device ensure that all power cords are disconnected from the power source on page 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Complete the following steps to install an upper fan cage Watch the procedure A video of the installation process is available YouTube Youku 126 User Guide for ThinkAgile VX7820 Appliance ...

Page 137: ...in the underside of the fan cage Step 5 Insert the fan cage fully down into the tray making sure that you do not pinch any cables Important Make sure the that fan cage goes beneath all tabs on the tray when pushing it backward The fan cage must be pushed fully back until it is against the tray bulkhead along its entire length Step 6 Slide the fan cage back into the tray with the fan cage under the...

Page 138: ...e compute tray is installed in the chassis Install the lower fan cage by connecting the fan cage electrical connector inserting the fan cage in the lower compute tray and pushing it backward until it locks in place S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have mor...

Page 139: ...ully down into the tray making sure that you do not pinch any cables Important Make sure the that fan cage goes beneath all tabs on the tray when pushing it backward The fan cage must be pushed fully back until it is against the tray bulkhead along its entire length Step 6 Slide the fan cage back into the tray with the fan cage under the tabs on the tray 2 in above illustration Push the fan cage b...

Page 140: ...ront bezel Watch the procedure A video of the replacement process is available YouTube Youku Figure 68 Front bezel removal Step 1 Press the release tabs on each front bezel Step 2 Pull the bezel forward and remove it from the server Install the front bezel Install each front bezel by sliding the bezel into position and pressing until it is fully seated Complete the following steps to install the f...

Page 141: ...r replacement Use the following procedures to remove and install the front cover Remove the front cover Remove the front cover by pressing the release buttons and pulling the cover from the front of the server Complete the following steps to remove the front cover Watch the procedure A video of the replacement process is available YouTube Youku Chapter 9 Hardware replacement procedures 131 ...

Page 142: ...upplied to you If you are planning to recycle the front cover follow the instructions in Disassemble the front cover for recycle on page 268 for compliance with local regulations Install the front cover Install the front cover by sliding the cover into position and pressing until it is fully seated Complete the following steps to install the front cover Watch the procedure A video of the replaceme...

Page 143: ...e that the pull tab on the operator panel used to access the LCD system information display panel passes through the hole for the display panel on the front cover Step 2 Press and slide the cover toward the rear of the server until the release clips engage Chapter 9 Hardware replacement procedures 133 ...

Page 144: ...m the device ensure that all power cords are disconnected from the power source on page 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove the front operator panel 1 Remove the front cover See Remove the front cover on page 131 2 Remove the compute tray See Remove a compute tray on page 115 3 If an upper sys...

Page 145: ...om the compute tray If you are instructed to return the front operator panel or cable follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install the front operator panel The front operator panel is in the lower compute tray that is accessed from the front of the server Insert the front operator panel in the front of the lower compute tray and pr...

Page 146: ...s fully assembled Step 3 Press the front operator panel in until the locking tabs engage Step 4 If you are replacing the operator panel cable route the front operator panel cable to its connector on the lower system board and connect it See Cable routing for common components on page 42 After you have installed the front operator panel 1 Make sure that all cables are correctly routed See Cable rou...

Page 147: ...e power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove the front USB VGA assembly 1 Remove the front cover See Remove the front cover on page 131 2 Remove the lower compute tra...

Page 148: ...ute tray that is accessed from the front of the server Position the front USB VGA assembly in the lower compute tray and secure with two screws then route the front USB VGA assembly cables to the lower compute system board and connect S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device...

Page 149: ... 45 2 Install the lower fan cage See Install a fan cage lower tray on page 128 3 If you removed an upper system board or system board filler install it See Install a system board on page 111 4 Install the compute tray then install the front cover See Install a compute tray on page 117 and Install the front cover on page 132 Hard disk drive backplane replacement Hard disk drive backplanes are in th...

Page 150: ...te tray on page 115 or Remove a storage tray on page 232 4 Turn the tray upside down 5 Remove the fan cage See Remove a fan cage upper tray on page 121 Complete the following steps to remove the hard disk drive backplane Watch the procedure A video of the replacement process is available YouTube Youku Step 1 Disconnect all power and data cables on the hard disk drive backplane that go to the stora...

Page 151: ...the backplane follow all packaging instructions and use any packaging materials that are provided Remove a hard disk drive backplane lower tray Hard disk drive backplanes in the lower compute tray are accessed from the front of the server There are several types of hard disk drive backplane Steps that are backplane specific are noted in each procedure S002 CAUTION The power control button on the d...

Page 152: ...d harnesses to provide a clear path for accessing the hard disk drive backplanes and their connectors Complete the following steps to remove the hard disk drive backplane Watch the procedure A video of the replacement process is available YouTube Youku Step 1 Disconnect all power and data cables on the hard disk drive backplane that go to the storage interposer or to connectors on the system board...

Page 153: ...ou are instructed to return the backplane follow all packaging instructions and use any packaging materials that are provided Install a hard disk drive backplane upper tray Hard disk drive backplanes in the upper compute tray or optional storage tray are accessed from the front of the server There are several types of hard disk drive backplane Steps that are backplane specific are noted in each pr...

Page 154: ...Cable routing for drives on page 45 Step 2 Install the hard disk drive backplane Note Existing cables might need to be removed from their retaining clips or moved to the side to install the backplane Figure 78 Hard disk drive backplane upper installation Align the backplane with its location in the tray then insert the backplane and push it down until it is fully seated Step 3 Connect the data cab...

Page 155: ...ll the front cover See Install the front cover on page 132 Install a hard disk drive backplane lower tray Hard disk drive backplanes in the lower compute tray are accessed from the front of the server There are several types of hard disk drive backplane Steps that are backplane specific are noted in each procedure S002 CAUTION The power control button on the device and the power switch on the powe...

Page 156: ... hard disk drive backplane being installed in the lower compute tray Step 4 Route and connect the data cables See Cable routing for drives on page 45 Route each SAS data cable to its connector on the RAID adapter then connect the SAS cable Route each NVMe data cable to its connector on the compute system board then connect the NVMe cable Step 5 Make sure that all power cables are connected to the ...

Page 157: ...re you remove a hot swap drive 1 Identify the failed drive to be replaced by referring to Identify a failed data drive 2 Prepare for replacing the failed drive by referring to either of the following For a failed cache drive Replace a failed cache drive For a failed capacity drive Replace a failed capacity drive 3 Make sure you save the data on your drive especially if it is part of a RAID array b...

Page 158: ...e the drive out of the drive bay Figure 81 3 5 inch hot swap drive removal After you remove the hot swap drive 1 Install a replacement hot swap drive or filler in the empty drive bay 2 If you are instructed to return the drive assembly follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install a hot swap drive Insert the hot swap drive in its ba...

Page 159: ...ap drive Watch the procedure A video of the installation process is available YouTube Youku Note NVMe drive bays can accept either SATA SAS or NVMe drives Step 1 Determine the drive type that you can install in the bay SATA SAS or NVMe based on labels stenciled on the front of the server and any drive bay labels that might have been applied to the front of the server These labels correspond to the...

Page 160: ...rive activity LED is flashing the drive is being accessed Step 5 If you are installing additional hot swap drives do so now After you have installed all hot swap drives 1 Install fillers in all empty drive bays 2 If the server is configured for RAID operation you might have to reconfigure your disk arrays after you install hot swap drives See RAID configuration on page 75 for information Remove a ...

Page 161: ... bay filler hot swap drives or single bay fillers Install a hot swap drive filler Insert the hot swap drive filler in the bay and press it in place Hot swap drive fillers can be of two types single bay fillers and four bay fillers Complete the following steps to install a hot swap drive filler Watch the procedure A video of the installation process is available YouTube Youku Chapter 9 Hardware rep...

Page 162: ...the I O tray The I O tray is accessed from the rear of the server Open the release levers to extract the I O tray If you are removing the I O tray as part of a maintenance replacement you must transfer system identification information to the new I O tray S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to ...

Page 163: ...Clarity Administrator make sure that you unmanage the server before removing the I O tray Then you can manage the server again after the new I O tray has been replaced Complete the following steps to remove the I O tray Watch the procedure A video of the installation process is available YouTube Youku Figure 86 I O tray removal Step 1 Press the button on each release lever then simultaneously rota...

Page 164: ...he device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 Before you install the I O tray 1 Make sure that all cables adapters and other components in the ...

Page 165: ...e Universal Unique Identifier UUID on page 155 and Update the DMI SMBIOS data on page 157 Enable TPM TCM See Enable TPM TCM on page 159 Optionally enable Secure Boot Update the Universal Unique Identifier UUID The Universal Unique Identifier UUID must be updated when the system board is replaced Use the Lenovo XClarity Provisioning Manager Lite to update the UUID in the UEFI based server Lenovo XC...

Page 166: ... and password default values onecli config set SYSTEM_PROD_DATA SYsInfoUUID uuid_value user user_id password password Example that uses the user ID and password default values onecli config set SYSTEM_PROD_DATA SysInfoUUID uuid_value Online KCS access unauthenticated and user restricted You do not need to specify a value for access_method when you use this access method Example onecli config set S...

Page 167: ...s OneCLI to update the asset tag for the server To download Lenovo XClarity Essentials OneCLI go to the following site https datacentersupport lenovo com solutions HT116433 Lenovo XClarity Essentials OneCLI sets the DMI in the Lenovo XClarity Controller Select one of the following methods to access the Lenovo XClarity Controller and set the DMI Operate from the target system such as LAN or keyboar...

Page 168: ...ig set SYSTEM_PROD_DATA SysInfoSerialNum s n onecli config set SYSTEM_PROD_DATA SysEncloseAssetTag asset_tag Online KCS access unauthenticated and user restricted You do not need to specify a value for access_method when you use this access method Note The KCS access method uses the IPMI KCS interface which requires that the IPMI driver be installed The following commands are examples of not using...

Page 169: ...ke sure that the TPM TCM policy is set correctly CAUTION Take special care when setting the TPM TCM policy If it is not set correctly the system board can become unusable Set the TPM TCM policy By default a replacement system board is shipped with the TPM TCM policy set to undefined You must modify this setting to match the setting that was in place for the system board that is being replaced You ...

Page 170: ...i exe misc ospower reboot imm userid password ip_address 4 Read back the value to check whether the change has been accepted OneCli exe config show imm TpmTcmPolicy override imm userid password ip_address Notes If the read back value is matched it means the TPM_TCM_POLICY has been set correctly imm TpmTcmPolicy is defined as below Value 0 use string Undefined which means UNDEFINED policy Value 1 u...

Page 171: ...r Lite or through the Lenovo XClarity Controller 2 Switch the hardware jumpers on the system board Notes If the Physical Presence Policy has been disabled 1 Set the hardware Physical Presence jumper on the system board to assert Physical Presence 2 Enable the Physical Presence Policy using either F1 UEFI Settings or Lenovo XClarity Essentials OneCLI Assert Physical Presence through the Lenovo XCla...

Page 172: ...rid password password override To set the TPM version to version 1 2 asu64 set TPMVersion TPMVersion Update to TPM1 2 compliant host ip_address user userid password password override where userid and password are the credentials used to the BMC Lenovo XClarity Controller interface of your server The default user ID is USERID and the default password is PASSW0RD zero not an uppercase o ip_address i...

Page 173: ...rity Essentials OneCLI s se et t command see http sysmgt lenovofiles com help topic toolsctr_cli_lenovo onecli_r_set_command html M 2 backplane replacement The M 2 backplane is in the I O tray that is accessed from the rear of the server Remove the M 2 backplane The M 2 backplane is in the I O tray that is accessed from the rear of the server After removing the I O tray and one of the risers to ga...

Page 174: ...e backplane at the same time Note Pull the M 2 backplane straight up when removing it from the system board After you remove the M 2 backplane If you are replacing the M 2 backplane remove any M 2 drives installed in the backplane See Remove an M 2 drive on page 166 If you are removing the M 2 backplane and are not installing a replacement 1 Install the riser that you removed to access the M 2 bac...

Page 175: ...urrent from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 Before you install the M 2 backplane install the M 2 drives See Install an M 2 drive on page 167 Complete the following steps to install the M 2 backplane Watch the procedure A video of the installation process is available YouTube Youku Figure 89 M 2 backplane installation Step 1 Align ...

Page 176: ...ove it toward the connector until it is in the large opening of the keyhole then remove the retainer from the backplane Step 3 Insert the retainer into the keyhole that matches the size of your M 2 drive then and slide it back until the retainer locking nubs are in the holes M 2 drive replacement M 2 drives are in the I O tray that is accessed from the rear of the server Remove an M 2 drive The M ...

Page 177: ...ackplane has two M 2 drives both will release when you slide the retainer Step 2 Rotate the M 2 drive up from the backplane and pull it away from the connector After removing the M 2 drive if you are removing the M 2 backplane and drives and not installing a replacement 1 Install the riser that you removed to access the M 2 backplane See Install the riser for slot 10 to 15 riser 2 on page 220 or I...

Page 178: ... that the retainer on the M 2 backplane is in the correct keyhole to accommodate the size of the M 2 drive you are installing See M 2 backplane retainer adjustment on page 166 Complete the following steps to install an M 2 drive in the M 2 backplane Watch the procedure A video of the installation process is available YouTube Youku on page 91 Step 1 Locate the connector on each side of the M 2 back...

Page 179: ...e in the small holes on the M 2 backplane when the retainer is slid forward Slide the retainer forward toward the connector until you hear a soft click sound After installing an M 2 drive 1 Install the M 2 backplane See Install the M 2 backplane on page 165 2 Install the riser that you removed to access the M 2 backplane See Install the riser for slot 10 to 15 riser 2 on page 220 or Install the ri...

Page 180: ...fles are installed See Remove a compute tray on page 115 3 If you are removing a memory air baffle from the lower compute system board remove the upper system board or system board filler See Remove a system board on page 109 Complete the following steps to remove a memory air baffle Watch the procedure A video of the replacement process is available YouTube Youku Note Each memory air baffle spans...

Page 181: ...ol button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 on page 91 CAUTION Make sure that all server power cords are disconnected from thei...

Page 182: ...ach memory module aligns with its connector then seat each memory module See Install a memory module on page 175 for more information Note Make sure that all memory module connector retaining clips spanning the memory air baffle are closed even if a memory module is not installed in the connector Step 5 After installing each memory baffle make sure that it is secure by attempting to lift and remov...

Page 183: ...ng storage Never touch the gold memory module connector contacts or allow these contacts to touch the outside of the memory module connector housing Handle memory modules with care never bend twist or drop a memory module Note The same procedure is used to remove memory modules and memory module fillers Before you remove a memory module 1 If you are removing a DCPMM in App Direct or Mixed memory m...

Page 184: ... removed 1 See the Memory Population Reference for the required installation order of the remaining memory modules 2 Install a memory module filler that was originally provided with the PHM option in any vacant memory module connectors 3 Install the upper system board or system board filler if it was removed See Install a system board on page 111 4 Install the compute tray where the system board i...

Page 185: ... and understand the installation order depending on whether you are implementing memory mirroring memory rank sparing or independent memory mode See the Memory Population Reference for specific information about installing memory modules and their required installation order based on the number of processors and memory modules in your system and the memory mode that you are implementing See http w...

Page 186: ...d to install memory modules and memory module fillers Figure 97 Memory module locations Table 34 Memory module locations on each system board DIMM number System board 1 lower board lower tray DIMM number System board 2 upper board lower tray DIMM number System board 3 lower board upper tray DIMM number System board 4 upper board upper tray DIMM number 1 1 25 49 73 2 2 26 50 74 3 3 27 51 75 4 4 28 ...

Page 187: ... 91 Attention Memory modules are sensitive to static discharge and require special handling In addition to the standard guidelines for Handling static sensitive devices on page 91 Always wear an electrostatic discharge strap when removing or installing memory modules Electrostatic discharge gloves can also be used Never hold two or more memory modules together so that they touch Do not stack memor...

Page 188: ...tray on page 117 3 Install the front cover See Install the front cover on page 132 4 Power on the system 5 If you have installed a DCPMM a Update the system firmware to the latest version see Firmware updates on page 19 b Make sure that the firmware on all of the DCPMM units is the latest version If not update it to the latest version see https sysmgt lenovofiles com help index jsp topic 2Fcom len...

Page 189: ...ulation Reference to install all the DCPMMs and DRAM DIMMs see Install a memory module on page 175 8 Disable security on all the installed DCPMMs see Configure DC Persistent Memory Module DCPMM on page 70 9 Make sure the DCPMM firmware is the latest version If not update it to the latest version see https sysmgt lenovofiles com help index jsp topic 2Fcom lenovo lxca doc 2Fupdate_fw html 10 Configu...

Page 190: ... to safely lift this part or unit S036 18 32 kg 39 70 lb 32 55 kg 70 121 lb on page 89 on page 65 Before you remove a midplane remove the server from the rack See the Rack Installation Instructions at https thinkagile lenovofiles com help topic com lenovo thinkagile 7Z12 pdf_files html The following illustration identifies the various midplanes that might be installed in your server Figure 99 Midp...

Page 191: ...ore removing a midplane make sure that all of the compute or storage trays the I O tray and all of the power supplies are removed from the server or slid partially out Step 1 Remove the eight screws and lift the top cover from the server Step 2 Grasp each midplane by the handle and pull it up and out of the server If you are instructed to return a midplane follow all packaging instructions and use...

Page 192: ... is authorized to remove the covers where the label is attached S037 CAUTION The weight of this part or unit is more than 55 kg 121 2 lb It takes specially trained persons a lifting device or both to safely lift this part or unit S036 18 32 kg 39 70 lb 32 55 kg 70 121 lb on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing ...

Page 193: ...in the correct orientation so that the cover pins that help support the backplanes engage Step 3 Position the top cover on the server and secure it with eight screws Install the server in the rack and replace all components See the Rack Installation Instructions at https thinkagile lenovofiles com help topic com lenovo thinkagile 7Z12 pdf_files html Power supply replacement Use the following proce...

Page 194: ...n this product or attached devices To Connect To Disconnect 1 Turn everything OFF 2 Attach all cables to devices 3 Attach signal cables to connectors 4 Attach power cords to outlet 5 Turn device ON 1 Turn everything OFF 2 Remove power cords from outlet 3 Remove signal cables from connectors 4 Remove all cables from devices S002 CAUTION The power control button on the device and the power switch on...

Page 195: ...teps to remove a power supply Watch the procedure A video of the installation process is available YouTube Youku Step 1 Remove the power supply Figure 102 Power supply removal Press the release latch toward the left and pull the drive to remove it from the drive bay After you remove the power supply 1 Install a replacement power supply or filler in the empty bay 2 If you are instructed to return t...

Page 196: ...table when installing moving or opening covers on this product or attached devices To Connect To Disconnect 1 Turn everything OFF 2 Attach all cables to devices 3 Attach signal cables to connectors 4 Attach power cords to outlet 5 Turn device ON 1 Turn everything OFF 2 Remove power cords from outlet 3 Remove signal cables from connectors 4 Remove all cables from devices S035 CAUTION Never remove t...

Page 197: ...alled the number of power supplies that are required to keep the system operating which power supply bays are populated and the number of input power sources as shown in the following table Table 36 Power supply redundancy Total number of processors installed in server Minimum number of power supplies to maintain system operation Bays where power supplies are installed Power supply redundancy sing...

Page 198: ...the procedure A video of the installation process is available YouTube Youku Step 1 Install the power supply in the bay Figure 103 Power supply installation Insert the power supply in its bay and press it in until the release tab locks After you have installed a power supply 1 Connect the power cable to the power supply 2 Route the power cable through the cable hook and loop on the rear of the ser...

Page 199: ... bays Install a power supply filler Insert the power supply filler in the bay and press it in place Complete the following steps to install a power supply filler Figure 105 Power supply filler installation Step 1 Insert the filler in the power supply in the bay and press it in place Install fillers in all empty hard disk drive bays Chapter 9 Hardware replacement procedures 189 ...

Page 200: ...or and heat sink known as a processor heat sink module PHM a processor and a heat sink All of these tasks require a Torx T30 driver S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that ...

Page 201: ...page 131 2 Remove the compute tray where the system board is installed See Remove a compute tray on page 115 3 If you are removing a PHM from the lower system board remove the upper system board or system board filler See Remove a system board on page 109 Figure 106 Processor locations on system board Figure 107 Processor layout for multi processor systems viewed from front of server Complete the ...

Page 202: ...sor heat sink module in the removal sequence shown on the heat sink label b Lift the processor heat sink module from the processor socket After you remove a PHM If you are removing the PHM as part of a system board replacement set the PHM aside If you are replacing the processor or heat sink separate the processor and its retainer from the heat sink 192 User Guide for ThinkAgile VX7820 Appliance T...

Page 203: ...r retainer will be removed and discarded in a later step and replaced with a new one If you are replacing the processor you will be reusing the heat sink Wipe the thermal grease from the bottom of the heat sink using an alcohol cleaning pad If you are replacing the heat sink you will be reusing the processor Wipe the thermal grease from the top of the processor using an alcohol cleaning pad If you...

Page 204: ...on in the socket See http www lenovo com us en serverproven for a list of processors supported for your server All processors on the system board must have the same speed number of cores and frequency Before you install a new PHM or replacement processor update your system firmware to the latest level See Update the firmware on page 68 Optional devices available for your system might have specific...

Page 205: ...sor retainer Figure 111 Installing a processor retainer 1 Position the processor on the new retainer so that the triangular marks align then insert the unmarked end of the processor into the retainer 2 Holding the inserted end of the processor in place flex the opposite end of the retainer down and away from the processor until you can press the processor under the clip on the retainer To prevent ...

Page 206: ...ion label from the old heat sink and place it on the new heat sink in the same location The label is on the side of the heat sink closest to the triangular alignment mark If you are unable to remove the label and place it on the new heat sink or if the label is damaged during transfer write the processor serial number from the processor identification label on the new heat sink in the same locatio...

Page 207: ... and the heat sink or align the triangular mark on the processor retainer with the notched corner of the heat sink b Insert the processor retainer clips into the holes on the heat sink c Press the retainer into place until the clips at all four corners engage Chapter 9 Hardware replacement procedures 197 ...

Page 208: ...deo of the installation process is available YouTube Youku Step 1 Remove the processor socket cover if one is installed on the processor socket by placing your fingers in the half circles at each end of the cover and lifting it from the system board Step 2 Install the processor heat sink module on the system board 198 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified N...

Page 209: ... screws until they stop then visually inspect to make sure that there is no gap between the screw shoulder beneath the heat sink and the processor socket For reference the torque required for the nuts to fully tighten is 1 4 1 6 newton meters 12 14 inch pounds After you install a PHM 1 Install the upper system board or system board filler if it was removed See Install a system board on page 111 2 ...

Page 210: ...mpact your RAID configurations Back up your data before you begin to avoid any data loss due to a RAID configuration change Remove a RAID HBA card upper tray The upper RAID HBA card is in the upper compute tray or optional storage tray that is accessed from the front of the server After removing the upper tray and fan cage disconnect the RAID HBA card from the storage interposer then disconnect th...

Page 211: ... to remove an upper tray RAID HBA card Watch the procedure A video of the installation process is available YouTube Youku Figure 117 Upper tray RAID HBA card removal Step 1 Disconnect the RAID HBA card from the storage interposer Step 2 Rotate the RAID HBA card upward and disconnect the three cables Step 3 Remove the RAID HBA card from the tray If you are instructed to return the RAID HBA card fol...

Page 212: ...a lower RAID HBA card 1 Place the node in maintenance mode by referring to Enter Maintenance Mode 2 Power off the node Disconnect all the cables if the cable management arm is not in place 3 Remove the front cover See Remove the front cover on page 131 4 Remove the lower compute tray or extract it to the maintenance position See Remove a compute tray on page 115 5 Remove the lower fan cage See Rem...

Page 213: ...e tray that is accessed from the front of the server Connect the three cables to the RAID HBA card then connect the RAID HBA card to the storage interposer in the upper tray S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all elect...

Page 214: ...rds or system board filler See Install a system board on page 111 4 Install the upper compute tray or storage tray See Install a compute tray on page 117 or Install the storage tray fully removed on page 234 5 Install the front cover See Install the front cover on page 132 6 Verify the RAID HBA adapter is replaced successfully The amber warning light on the node front panel is off All of the drive...

Page 215: ... the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 Complete the following steps to install a lower RAID HBA card Watch the procedure A video of the installat...

Page 216: ...ion then install the front cover See Install a compute tray on page 117 and Install the front cover on page 132 3 Verify the RAID HBA adapter is replaced successfully The amber warning light on the node front panel is off All of the drives are shown as online Verify that the LSI HBA card is detected by running the lls sp pc cii command root host lspci grep i SAS3408 00 05 0 Serial Attached SCSI co...

Page 217: ...ensure that all power cords are disconnected from the power source on page 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove an upper RAID flash power module 1 Remove the front cover See Remove the front cover on page 131 2 Remove the upper compute tray or storage tray See Remove a compute tray on page 115 ...

Page 218: ...ray that is accessed from the front of the server After removing the lower compute tray and fan cage disconnect the RAID flash power module cable then remove the module from the compute tray S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To...

Page 219: ...power module upward Step 2 Disconnect the cable from the RAID flash power module Step 3 Remove the RAID flash power module from the compute tray If you are instructed to return the RAID flash power module follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install a RAID flash power module upper tray The upper RAID flash power module is in the up...

Page 220: ...to secure it After you have installed the upper RAID flash power module 1 Make sure that all cables are correctly routed See Cable routing for drives on page 45 and Cable routing for common components on page 42 2 Install the upper fan cage See Install a fan cage upper tray on page 125 3 Turn the tray right side up 4 If you removed one or more system boards or a compute system board filler from a ...

Page 221: ... following steps to install a lower tray RAID flash power module Watch the procedure A video of the installation process is available YouTube Youku Figure 124 Lower RAID flash power module installation Step 1 Insert the end of the RAID flash power module into its carrier on the compute tray Step 2 Connect the cable to the RAID flash power module Step 3 Rotate the RAID flash power module down into ...

Page 222: ...e riser for slot 16 to 17 riser 3 on page 217 and Install the riser for slot 16 to 17 riser 3 on page 222 Remove the riser for slot 1 to 4 riser 1 The riser for adapter slot 1 to 4 riser 1 is in the I O tray that is accessed from the rear of the server After removing the I O tray loosen the two captive screws that secure the riser to the I O tray then remove the riser from the I O tray S002 CAUTIO...

Page 223: ... the rear edge of the riser below the midplane connectors Screw location is labelled Remove screw before installing riser on the I O board 2 Install the I O tray fillers in adapter slot 1 to 4 item 4 in the figure 3 Install the I O tray and connect all cables See Install the I O tray on page 154 If you are instructed to return the riser follow all packaging instructions and use any packaging mater...

Page 224: ...more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove the riser for slot 10 to 15 disconnect and label all cables connected to adapters in the I O tray ...

Page 225: ...figure This screw replaces the captive screw on the rear edge of the riser below the midplane connectors Screw location is labelled Remove screw before installing riser on the I O board 2 Install the I O tray fillers in adapter slot 10 to 15 item 4 in the figure 3 Install the I O tray and connect all cables See Install the I O tray on page 154 If you are instructed to return the riser follow all p...

Page 226: ...t and label all cables connected to adapters in the I O tray then remove the I O tray See Remove the I O tray on page 152 2 Remove the riser for slot 10 to 15 riser 2 See Remove the riser for slot 10 to 15 riser 2 on page 213 3 Remove any adapters that are installed in slots 14 or 15 See Remove a PCIe adapter from slot 10 to 15 on page 96 Complete the following steps to remove the riser bracket Fi...

Page 227: ...N The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected fro...

Page 228: ...riser below the midplane connectors Screw location is labelled Remove screw before installing riser on the I O board 2 Install the I O tray fillers in adapter slot 16 to 17 item 4 in the figure 3 Install the I O tray and connect all cables See Install the I O tray on page 154 If you are instructed to return the riser follow all packaging instructions and use any packaging materials for shipping th...

Page 229: ...aptive riser screw will be installed 2 Make sure that there are no I O tray fillers installed in riser slot 1 to 4 3 Install adapters in the riser See Install a PCIe adapter in slot 1 to 4 on page 99 4 If any of the adapters in the riser have internal cables make sure that they are connected before installing the riser in the I O tray Complete the following steps to install the riser for slot 1 to...

Page 230: ...ent from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 Before you install the riser for slot 10 to 15 1 Make sure that there is no screw item 2 in following figure securing the I O tray board to the I O tray in a location where a captive riser screw will be installed 2 Make sure that there are no I O tray fillers installed in riser slot 10 to 1...

Page 231: ...ge 154 Install the riser bracket for slot 14 to 15 riser 2 The riser bracket for slot 14 to 15 is attached to the slot 10 to 15 I O tray riser that is accessed from the rear of the server After removing the I O tray and the riser for slot 10 to 15 riser 2 slide the bracket onto the riser until the retaining clips engage S002 CAUTION The power control button on the device and the power switch on th...

Page 232: ...ore installing the riser in the I O tray 3 Install the riser in the I O tray See Install the riser for slot 10 to 15 riser 2 on page 220 4 Install the I O tray in the chassis and connect all cables See Install the I O tray on page 154 Install the riser for slot 16 to 17 riser 3 The riser for adapter slot 16 to 17 is in the I O tray that is accessed from the rear of the server After installing adap...

Page 233: ... installing the riser in the I O tray Complete the following steps to install the riser for slot 16 to 17 Watch the procedure A video of the installation process is available YouTube Youku Figure 132 Riser for slot 16 to 17 installation Step 1 Align the riser with the I O tray making sure that the sides of the riser fit into the guides on the I O tray and insert the riser then press the riser into...

Page 234: ...ice also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 on page 65 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this procedure Before you remove the storage board assembly 1 Remove the front cover See Remove the front cover...

Page 235: ...ge board assembly follow all packaging instructions and use any packaging materials for shipping that are supplied to you Install a storage board assembly Install a storage board assembly by inserting it in the storage tray pushing it in until it locks in place and connecting the cables S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the...

Page 236: ...r See Install the storage tray fully removed on page 234 and Install the front cover on page 132 Storage interposer replacement Storage interposers are in the upper and lower trays that are accessed from the front of the server Procedures for removing and installing storage interposers in the upper and lower trays are different Remove a storage interposer upper tray The upper storage interposer is...

Page 237: ...age 115 or Remove a storage tray on page 232 3 If you removed a compute tray and it has one or more compute system boards installed or a system board filler remove the system boards or system board filler See Remove a system board on page 109 4 Turn the tray upside down and remove the upper fan cage See Remove a fan cage upper tray on page 121 Complete the following steps to remove an upper storag...

Page 238: ...e and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords are disconnected from the power source on page 89 CAUTION Make sure that all server power cords are disconnected from their power sources before performing this proce...

Page 239: ...or shipping that are supplied to you Install a storage interposer upper tray The upper storage interposer is in the upper compute tray or optional storage tray that is accessed from the front of the server Insert the storage interposer in the upper tray and secure it with the screw then connect the RAID card to the storage interposer and install the fan cage and upper tray S002 CAUTION The power c...

Page 240: ...er storage interposer 1 Install the upper fan cage See Install a fan cage upper tray on page 125 2 Turn the tray right side up 3 If you removed one or more system boards or a compute system board filler from a compute tray install the system boards or system board filler See Install a system board on page 111 4 Install the upper compute tray or storage tray See Install a compute tray on page 117 o...

Page 241: ...ing steps to install a lower storage interposer Watch the procedure A video of the installation process is available YouTube Youku Figure 138 Lower storage interposer installation Step 1 Position the storage interposer over the pins on the compute tray then slide the storage interposer to the right Step 2 Secure the storage interposer with the screw Step 3 Rotate the RAID card down and align it wi...

Page 242: ...t plan to replace it relocate or backup data that is on the hard disk drives in the storage tray before removing it S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current from the device ensure that all power cords ...

Page 243: ...it stops then close the release levers Step 3 Press the release tabs on either side of the tray then slide the tray fully forward evenly and remove it from the chassis Attention Be prepared to support the full weight of the storage tray when you remove it from the chassis Do not use the release levers as handles to support the storage tray Figure 140 Storage tray removal from chassis Chapter 9 Har...

Page 244: ...moved Install the storage tray when it is fully removed by inserting it in the front of the chassis pushing it in until it stops and closing the release levers S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the electrical current supplied to the device The device also might have more than one power cord To remove all electrical current ...

Page 245: ...e chassis install the front cover See Install the front cover on page 132 TCM TPM adapter replacement for Chinese Mainland only Use this information to remove and install the TCM TPM adapter sometimes called a daughter card Remove the TCM TPM adapter for Chinese Mainland only Use this information to remove the TCM TPM adapter on page 89 on page 65 on page 91 Before removing the TCM TPM adapter rem...

Page 246: ...l the TCM TPM adapter for Chinese Mainland only Use this information to install the TCM TPM adapter on page 89 on page 65 on page 91 Before installing the TCM TPM adapter touch the static protective package that contains the new TCM TPM adapter to any unpainted surface on the outside of the server Then take the new TCM TPM adapter out of the package and place it on a static protective surface To i...

Page 247: ...rom the front of the server This optional component is used only in some server configurations Remove the upper tray filler The upper tray filler is at the front of the server Remove the four screws securing the upper tray filler then slide the filler out of the front of the server S002 CAUTION The power control button on the device and the power switch on the power supply do not turn off the elec...

Page 248: ...12 pdf_files html Complete the following steps to remove the upper tray filler Figure 144 Upper tray filler removal Note Before removing the upper tray filler make sure that the lower compute tray is removed from the server Step 1 Remove the four screws two each side securing the upper tray filler Step 2 Slide the upper tray filler out of the front of the server If you are instructed to return an ...

Page 249: ...d To remove all electrical current from the device ensure that all power cords are disconnected from the power source S037 CAUTION The weight of this part or unit is more than 55 kg 121 2 lb It takes specially trained persons a lifting device or both to safely lift this part or unit S036 18 32 kg 39 70 lb 32 55 kg 70 121 lb on page 89 on page 65 CAUTION Make sure that all server power cords are di...

Page 250: ... components have been reassembled correctly and that no tools or loose screws are left inside your server 2 Properly route and secure the cables in the server Refer to the cable connecting and routing information for each component 3 If you have removed the server front cover reinstall it See Install the front cover on page 132 4 Reconnect external cables and power cords to the server Attention To...

Page 251: ...latest version of ThinkSystem M 2 with Mirroring Enablement Kit Firmware is applied to avoid virtual disk array missing after system board replacement 6 Exit maintenance mode Chapter 9 Hardware replacement procedures 241 ...

Page 252: ...242 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 253: ... If you are using some other management application begin with the Lenovo XClarity Controller event log Event logs An alert is a message or other indication that signals an event or an impending event Alerts are generated by the Lenovo XClarity Controller or by UEFI in the servers These alerts are stored in the Lenovo XClarity Controller Event Log If the server is managed by the Lenovo XClarity Ad...

Page 254: ...peeds and component status The Lenovo XClarity Controller provides various interfaces to systems management software and to system administrators and users to enable remote management and control of a server The Lenovo XClarity Controller monitors all components of the server and posts events in the Lenovo XClarity Controller event log Figure 147 Lenovo XClarity Controller event log For more infor...

Page 255: ...oting server beyond providing power status See Front operator panel on page 26 for additional information 2 Network activity LED Indicates activity between the server and the Ethernet LAN Not used for troubleshooting server beyond providing network activity status See Front operator panel on page 26 for additional information 3 System ID LED blue This LED is used as a presence detection LED You ca...

Page 256: ...LEDs 5 6 7 8 10 11 12 13 14 15 16 17 24 25 26 27 28 Adapter error LEDs Indicates a malfunction for the associated adapter 1 Check the Lenovo XClarity Controller event log and the system error log for information about the error 2 Save the log if necessary and clear the log afterward 4 9 18 Riser error LEDs Indicates a malfunction for the associated riser or an adapter installed in the riser 1 Chec...

Page 257: ...he error 2 Save the log if necessary and clear the log afterward Power supply LEDs This topic provides the location of the power supply LEDs The following minimum configuration is required for the ac power LED on the power supply to be lit Power supply Power cord Appropriate input power from the power source The following minimum configuration is required for the server to start Two processors ins...

Page 258: ...er supply Off On On The power supply has failed Replace the power supply On Off Off The system is shut down server is connected to power The system has overloaded the power supply Remove options or add an additional power supply If the system is shut down this is a normal condition The LED may go off for less than a minute then back on during an input power cycle operation initiated by the Lenovo ...

Page 259: ...y Controller event log and the system error log for information about the error 2 Save the log if necessary and clear the log afterward 3 Light path diagnostic switch Press the light path diagnostic switch to turn on the system board LEDs 4 Processor 1 error LED LED on An error has occurred with processor 1 See Processor problems on page 253 5 Memory module 1 through 24 error LEDs LED on An error ...

Page 260: ... diagnose and resolve a suspected power problem Step 1 Check the event log and resolve any errors related to the power Note Start with the event log of the application that is managing the server For more information about event logs see Event logs on page 243 Step 2 Check for short circuits for example if a loose screw is lying on a circuit board Step 3 Remove the adapters and disconnect the cabl...

Page 261: ...ff make sure that the hub and network are operating and that the correct device drivers are installed Step 6 Check for operating system specific causes of the problem and also make sure that the operating system drivers are installed correctly Step 7 Make sure that the device drivers on the client and server are using the same protocol If the Ethernet controller still cannot connect to the network...

Page 262: ...er to ensure that it is working properly Server does not power on no indication that input power is connected to the server Complete the following steps until the problem is resolved 1 Check the power LED on the front operator panel 2 Check the power supply LEDs 3 Check the front operator panel LCD display for error indications 4 Check the event log for any events related to the server not powerin...

Page 263: ... 1 Check the light path diagnostics LEDs and the XCC event log and resolve any errors that have occurred 2 Make sure that the server supports all the processors and that the processors match in speed and cache size You can view processor details from system setup To determine if the processor is supported for the server see http www lenovo com us en serverproven 3 Trained technician only Make sure...

Page 264: ...y diagnostics When you start a server and press F1 the Lenovo XClarity Provisioning Manager Lite interface is displayed by default You can perform memory diagnostics from this interface From the Diagnostic page click Run Diagnostic Memory test 4 Check the POST error log If a DIMM was disabled by a systems management interrupt SMI replace the DIMM If a DIMM was disabled by the user or by POST resea...

Page 265: ...45 seconds and reinsert the drive making sure that the drive assembly connects to the hard disk drive backplane 3 Observe the associated green hard disk drive activity LED and the yellow status LED If the green activity LED is flashing and the yellow status LED is not lit the drive is recognized by the controller and is working correctly Run the diagnostics tests for the hard disk drives When you ...

Page 266: ...View the Lenovo XClarity Controller event log for events related to power supplies or vibration issues and resolve those events View the storage subsystem log for events related to the storage subsystem and resolve those events A replacement hard disk drive does not rebuild Complete the following steps until the problem is solved 1 Make sure that the hard disk drive is recognized by the adapter th...

Page 267: ...rectly to the correct connector on the rear of the server 3 Make sure that the correct server is controlling the monitor if applicable 4 Make sure that The server is turned on If there is no power to the server the screen will be blank The monitor cables are connected correctly If monitors are connected to both the front and rear video ports only the monitor connected to the front video port will ...

Page 268: ...splayed update the server firmware to the latest level See Update the firmware on page 68 Keyboard mouse or USB device problems Use this information to solve problems related to a keyboard mouse or USB device Note The ThinkAgile VX7820 Appliance VX 4U Certified Node server only supports connection of a USB keyboard and mouse USB device does not work on page 258 All or some keys on the keyboard do ...

Page 269: ... Complete the following steps until the problem is resolved 1 Make sure that the proper drivers are installed on the server See the product documentation for the USB device for information about device drivers 2 Use the Lenovo XClarity Provisioning Manager Lite to make sure that the device is configured correctly 3 If the USB device is plugged into a hub or the console breakout cable unplug the de...

Page 270: ...server and press F1 to display the system setup interface Whenever memory or any other device is changed you must update the configuration Make sure that the latest levels of device firmware and device drivers are installed See https datacentersupport lenovo com us en solutions ht505302 2 Reseat the device that you just installed 3 Replace the device that you just installed A Lenovo optional devic...

Page 271: ...rect device drivers are installed See the manufacturer s website for documentation 2 For a USB device a Make sure that the device is configured correctly Restart the server and press F1 to display the Lenovo XClarity Provisioning Manager Lite system setup interface Then click System Settings Devices and I O Ports USB Configuration b Connect the device to another port If using a USB hub remove the ...

Page 272: ... to resolve issues related to power System error LED is on and event log Power supply has lost input is displayed To resolve the problem ensure that 1 The power supply is properly connected to a power cord 2 The power cord is connected to a properly grounded electrical outlet for the server 3 The power source is providing a correct level of power to the power supply check the power supply label fo...

Page 273: ... XClarity Essentials OneCLI onecli config show EnableDisableAdapterOptionROMSupport bmc xcc_userid xcc_password xcc_ipaddress To recover a system that hangs during the boot process with Legacy Option ROM settings see the following Tech Tip https datacentersupport lenovo com us en solutions ht506118 If legacy Option ROMs must be used do not set slot Option ROMs to Legacy on the Devices and I O Port...

Page 274: ...t which the connection breaks down 4 Restart the compute node remotely through the management interface 5 If the problem remains verify that any new software has been installed and configured correctly 6 Contact your place of purchase of the software or your software provider Server is unresponsive cannot press F1 to start System Setup Configuration changes such as added devices or adapter firmwar...

Page 275: ...ssor event log for rising temperature events If there are no events the compute node is running within normal operating temperatures Note that you can expect some variation in temperature Cracked parts or cracked chassis Contact Lenovo Support Software problems Use this information to solve software problems 1 To determine whether the problem is caused by the software make sure that The server has...

Page 276: ...266 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 277: ...embling the system board 1 Remove the system board from the server see Remove a system board on page 109 2 Refer to local environmental waste or disposal regulations to ensure compliance Complete the following steps to disassemble the system board Step 1 Remove the following components as illustrated Use a Torx T10 screwdriver and remove the Torx T10 screw 1 in the following illustration Use a scr...

Page 278: ...e on the system board filler Figure 152 Disassembling the system board filler Step 2 Separate the latches from the system board filler After disassembling the system board filler comply with local regulations when recycling Disassemble the front cover for recycle Follow the instructions in this section to disassemble the front cover before recycling Before disassembling the front cover 1 Remove th...

Page 279: ...ont cover Step 2 Separate the sheet metal shield from the front cover Figure 154 Separating sheet metal shield on the front cover Step 3 Use a screwdriver and remove the four screws then remove the four latch springs Appendix A Hardware disassembling for recycle 269 ...

Page 280: ... Removing latch springs on the front cover After disassembling the front cover comply with local regulations when recycling 270 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 281: ...at the following location http thinksystem lenovofiles com help index jsp You can take these steps to try to solve the problem yourself Check all cables to make sure that they are connected Check the power switches to make sure that the system and any optional devices are turned on Check for updated software firmware and operating system device drivers for your Lenovo product The Lenovo Warranty t...

Page 282: ...o collect new data Lenovo XClarity Controller You can use the Lenovo XClarity Controller web interface or the CLI to collect service data for the server The file can be saved and sent to Lenovo Support For more information about using the web interface to collect service data see http sysmgt lenovofiles com help topic com lenovo systems management xcc doc NN1ia_c_ servicesandsupport html For more ...

Page 283: ...e com s article 2072796 Contacting Support You can contact Support to obtain help for your issue You can receive hardware service through a Lenovo Authorized Service Provider To locate a service provider authorized by Lenovo to provide warranty service go to https datacentersupport lenovo com serviceprovider and use filter searching for different countries For Lenovo support telephone numbers see ...

Page 284: ...274 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 285: ...n new editions of the publication Lenovo may make improvements and or changes in the product s and or the program s described in this publication at any time without notice The products described in this document are not intended for use in implantation or other life support applications where malfunction may result in injury or death to persons The information contained in this document does not ...

Page 286: ...rive bays with the largest currently supported drives that are available from Lenovo Maximum memory might require replacement of the standard memory with an optional memory module Each solid state memory cell has an intrinsic finite number of write cycles that the cell can incur Therefore a solid state device has a maximum number of write cycles that it can be subjected to expressed as total bytes...

Page 287: ...fficiency or greater using high efficiency particulate air HEPA filters that meet MIL STD 282 The deliquescent relative humidity of the particulate contamination must be more than 60 2 The room must be free of conductive contamination such as zinc whiskers Gaseous Copper Class G1 as per ANSI ISA 71 04 19853 Silver Corrosion rate of less than 300 Å in 30 days 1 ASHRAE 52 2 2008 Method of Testing Ge...

Page 288: ...RoHS declaration Taiwan import and export contact information Contacts are available for Taiwan import and export information 278 User Guide for ThinkAgile VX7820 Appliance ThinkAgile VX 4 Socket 4U Certified Node ...

Page 289: ...ent 240 components server 23 compute tray installing 117 removing 115 replacing 115 Configuration ThinkAgile VX7820 Appliance VX 4U Certified Node 67 configure the firmware 68 connector battery 34 drive backplane 39 internal 32 interposer 38 I O tray 34 I O tray riser 35 RAID card 38 riser 35 storage board assembly connectors 33 system board 32 contamination particulate and gaseous 276 control pan...

Page 290: ... hard disk drive backplane upper tray installing 143 removing 139 hard disk drive problems 255 hard disk drive status LED 26 hardware service and support telephone numbers 273 heat sink installing 193 removing 190 replacing 190 help 271 hot swap drive installing 148 removing 147 replacing 147 Hot swap drive 147 hot swap drive filler installing 151 removing 150 replacing 147 I identifying the serve...

Page 291: ... 152 I O tray connectors 34 I O tray riser connectors 35 I O tray switches 35 IPv4 host name default 23 IPv6 link local address default 23 K keyboard problems 258 L label machine type model 23 MTM 23 network access 23 service 23 LCD system information display panel 27 LCD system information display panel 27 LED 26 for hard disk drive activity 26 for hard disk drive status 26 for solid state drive ...

Page 292: ...ces 259 PCIe 259 power 250 262 power on and power off 251 processor 253 serial device 260 software 265 USB device 258 video 257 procedure update a ThinkAgile VX cluster 18 processor installing 193 removing 190 replacing 190 processor heat sink module replacing 190 processor problems 253 processor heat sink module installing 193 removing 190 Q QR code 23 R RAID card connectors 38 RAID controller ca...

Page 293: ...9 fan cage 121 filler System board 112 front bezel 130 front cover 131 front operator panel 134 front USB VGA assembly 137 hard disk drive backplane 139 heat sink 190 hot swap drive 147 hot swap drive filler 147 I O tray 152 M 2 backplane 163 M 2 drive 166 memory air baffle 170 memory module 173 microprocessor 190 microprocessor heat sink module 190 midplane 179 PHM 190 power supply 183 power supp...

Page 294: ...mmunication regulatory statement 277 telephone numbers 273 top cover installing 181 removing 179 TPM 159 TPM 1 2 161 TPM 2 0 161 TPM policy 159 TPM version 161 trademarks 276 troubleshooting 253 257 259 265 by symptom 251 hard disk drive problems 255 intermittent problems 261 keyboard problems 258 memory problems 253 mouse problems 258 network problems 262 observable problems 263 power on and powe...

Page 295: ......

Page 296: ......

Reviews:

No comments

Related manuals for ThinkAgile VX 4-Socket 4U Certified Node

Univerge SV9100

Brand: NEC Pages: 78

UNIVERGE SL2100

Brand: NEC Pages: 34

Brand: NEC Pages: 2

Brand: NEC Pages: 975

SUPERSERVER 5018D-MHR7N4P

Brand: Supermicro Pages: 126

ProLiant DL180 Gen10

Brand: HPE Pages: 10

Brand: Supero Pages: 112

FusionServer 5288 V6

Brand: xFusion Digital Technologies Pages: 131

UniServer R4900 G3

Brand: H3C Pages: 24

Brand: Conceptronic Pages: 127

Brand: HIKVISION Pages: 13

Brand: SmartSight Pages: 113

Brand: Hamlet Pages: 76

Brand: Docomo Pages: 61

SuperServer 2029BT-HNC0R

Brand: Supermicro Pages: 148

Brand: TRENDnet Pages: 9

Picturall MST02-R1

Brand: Analog way Pages: 4

NetNumen U31 R06

Brand: Zte Pages: 128

Brands by name

0 1 2 3 4 5 6 7 8 9 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z

Popular brands

Load more brands