H3 Falcon PCIe User Manual Download

Page: 11 / 58

⚫

Jumpers

Jumper

Function

Setting

Reserved Jumper

J7, J8

PCIe bifurcation

Config.

Pin 1, 2

1 x16 lanes

Pin 2, 3

2 x8 lanes

Pin 1, 2

Pin 2, 3

4 x4 lanes

Mode

Config.

Pin 1, 2

Host mode

Pin 2, 3

Target mode

2.3 Compatible Devices

Devices

Accelerator

▪

Nvidia A10

▪

Nvidia RTX A4000

Storage

▪

Intel® P4500, P4600, P4800X

▪

Samsung PM1725a, PM1725b, PM1733, PM1735

NIC

▪

Mellanox ConnectX®-4, ConnectX®-5, ConnectX®-6

Devices listed above are devices that have been tested. Standard PCIe devices may also be compatible.

If you can’t find your device on the list or is planning to use our products with special PCIe device, please contact

[email protected]

for help.

Summary of Contents for Falcon PCIe

Page 1: ...Version 1 0 February 20th 2022 Falcon PCIe Expansion Solution User Manual Falcon 4109 Falcon 4118 ...

Page 2: ...roducts superior in the like product For you to have the good understanding to Falcon Composable PCIe Expansion Solutions please read the user manual and operate according to the suggested steps for each feature If you have any questions when using our machine please feel free to contact us We are more than happy to serve you constantly Technical Support support h3platform com FAQ https www h3plat...

Page 3: ... may be trademarks of their respective owners Notes Cautions and Warning Note A NOTE indicates important information that helps you make better use of your product Caution A CAUTION indicates either potential damage to hardware or loss of data and tells you how to avoid the problem Warning A WARNING indicates a potential for property damage personal injury or death ...

Page 4: ...le Devices 6 3 Requirements 7 3 1 CPU 7 3 2 Host OS 7 3 3 BIOS 7 3 4 Web Browsers 7 4 Graphical User Interface 8 4 1 Log In 8 4 2 Functions 8 4 2 1 Overview 9 4 2 2 Resource Management 13 4 2 3 Port Configuration 17 4 2 4 Monitor 20 4 2 5 System Health 22 4 2 6 Chassis 25 4 2 7 Maintenance 25 4 2 8 Event Logs 27 4 2 9 Setting 28 5 LCD 36 5 1 Operation 36 5 2 Menu 37 5 2 1 Power control 38 5 2 2 Po...

Page 5: ...ault 42 6 Part Replacement 43 6 1 Fans 43 6 2 Power Supply Unit 43 7 Operational Safety 44 8 Trouble Shooting 45 PCIe out of resource 45 GPU P2P underperforming 46 PCIe link health error 46 Failure to assign remove a device 47 Information does not display properly on GUI 48 Failure to access GUI 48 Device Link Down 48 Host Link Down 49 ...

Page 6: ...1 ...

Page 7: ...dynamically You could activate the Advanced mode with Premium License Please contact sales h3platform com for license purchase Standard mode Advanced mode System monitor Power control from GUI Download system performance data from GUI Firmware update User management Limited to single host System monitor Power control from GUI Download system performance data from GUI Firmware update User managemen...

Page 8: ...0x38mm 6700 RPM hot swap Operating Temp 0 C 35 C 32 F 95 F Dimension 174 H x 320 W x 466 D mm Weight 12 75 Kg Falcon 4118 Model Falcon 4118 BMC mCPU Aspeed AST2500 PCIe Switch PEX 88096 PCIe 4 0 PCIe Slots 16x PCIe4 0 x8 FHFL and 2x PCIe 4 0x16 low profile Slot Power 75 W slot 1 2 7 and 8 both drawers supports 225W PCIe 8 pin power Host Interface SFF 8644 connectors Fan 6x 120x120x38mm 6700 RPM ho...

Page 9: ... 1200 W AC Input 100 127V 200 240V 100 240V DC Output 12V 12Vsb 82A 98A 2 1A Efficiency 94 at full load Lifespan 250 000 hrs Operating Env 0 C 56 C 85 relative humidity non condensing 80 PLUS Certified Platinum Host Adapter Form factor PCIe Low profile MD2 PCIe 4 0 x16 Connector Quad SFF 8644 PCIe switch PEX 88032 PCIe 4 0 Dimension 160 L x 68 H mm ...

Page 10: ... 4 x4 lanes 6 SFF 8644 connector 1 See LED 1 signal indication 7 PCIe Switch Status Good Error 8 SFF 8644 connector 2 See LED 1 signal indication 9 SFF 8644 connector 3 See LED 1 signal indication 1 SFF 8644 connectors 2 Connection LED 3 PCIe link LED 4 Heartbeat LED 5 Jumpers Each jumper has 3 pins Please pay attention to the labels on PCB sign indicates pin 1 Image 1 illustrates pin number Image...

Page 11: ... Compatible Devices Devices Accelerator Nvidia A10 Nvidia RTX A4000 Storage Intel P4500 P4600 P4800X Samsung PM1725a PM1725b PM1733 PM1735 NIC Mellanox ConnectX 4 ConnectX 5 ConnectX 6 Devices listed above are devices that have been tested Standard PCIe devices may also be compatible If you can t find your device on the list or is planning to use our products with special PCIe device please contac...

Page 12: ...12G PCI Access Control Services ACS disability Mozilla Firefox Please update to the latest version Google Chrome Please update to the latest version Advanced mode is not limited to the OS listed above The listed OS are recommended as they have been tested to support PCIe device hot plug Also requires a vacant PCIe x16 slot on the host server for host adapter card installation PCIe Gen3 or later En...

Page 13: ...me you access the GUI you will be asked to log in Please enter your Username and Password 4 2 Functions The menu at the top or top left corner of the page shows all the available functions Please find details of each function in the relative section ...

Page 14: ...vides PCIe device usage and host port usage information Usage of specific device types GPU NVMe SSD FPGA and NIC features can be accessed with Premium License activated Used indicates the number of devices that are currently assigned to hosts e g Used 2 of 10 There are 10 devices installed in Falcon system 2 of them are assigned to the host s ...

Page 15: ...ere 1 Graph title GPU Utilization Rate 2 Utilization rate The average GPU utilization scaled from 0 100 3 Bar graph Utilization rate of the GPUs displayed in bar graphs 4 Device number Displayed as Drawer slot E g 1 1 represents device on slot 1 of drawer 1 5 Display period The graph will display the utilization rate of the GPUs in the past hours 1 12 24 or 72 hours options available 6 Download Do...

Page 16: ... point Move the mouse over the curve to see this menu 7 Traffics Select traffic types to display on the throughput graph There are three types Ingress Egress and Sum 8 Display period The graph will display the throughput rate of the devices in the past hours 1 12 24 or 72 hours options available 9 Download Download the PCIe throughput data Up to the past 72 hours PCIe Link Health 1 Graph title PCI...

Page 17: ...e Mac address Mac address of this machine Firmware BMC firmware version System up time Time since the system is powered on Last login The last user Last login time The time of last login Online user The number of users currently online IP address The IP address of this machine Falcon System will shut down automatically when the system detects any device temperature 85 C for over 10 seconds This te...

Page 18: ...ocate This button is used when allocating resources See Device Allocation section for detail 3 Drawer 1 PCIe ports PCIe ports topology of drawer 1 4 Drawer 2 PCIe ports PCIe ports topology of drawer 2 Falcon 4118 5 System mode Displays the current system mode of the drawer 6 Refresh Refresh the topology display 7 Legends Help users to identify components in the topology view 8 Port label Port labe...

Page 19: ...of the PCIe slot Displayed in PCIe generation x lanes 8 Device details Click on the drop down icon to show detailed information of the device 9 Refresh Click to refresh the list 10 Port label Port label diagram that helps users to identify PCIe ports 1 Color tag Indicates which host that the device is assigned too The color is corresponding to the color frame of host port E g the device is assigne...

Page 20: ...nfirmation window will pop up Click Yes to proceed Then Click OK to finish the process 1 Select the host 2 Select the available device 3 Click Allocate to assign Users could select multiple devices at a time for batch assignment The link icon and color tag should appear when the device is successfully assigned ...

Page 21: ...to unassign devices Confirmation window will pop up Click Yes to proceed Then Click OK to finish the process 1 Click the link icon next to the target device Users can only unassign one device at a time The link icon and color tag should disappear when the device is successfully unassigned ...

Page 22: ... settings See Configure Ports section for details 2 Drawer 1 PCIe ports PCIe ports topology of drawer 1 3 Drawer 2 PCIe ports PCIe ports topology of drawer 2 Falcon 4118 4 Legends Help users to identify components 5 System mode Displays the current system mode of the drawer 6 Port label Port label diagram that helps users to identify PCIe ports 7 Import Export Import a past configuration setting o...

Page 23: ...ly to apply the configuration or Undo to discard the configuration Confirmation window will pop up Click Yes to proceed Then Click OK to finish the process The text in Red indicates that the configuration is not yet applied The text should turn Black when the configuration is successfully applied ...

Page 24: ...hassis Side Connector 0 1 for host 1 H1 0 Connector 2 3 for host 1 H1 1 Host port 4x4 configuration Host side Use connector 0 on the HBAs Chassis Side Connector 0 for host 1 H1 0 Connector 1 for host 1 H1 1 Connector 2 for host 1 H1 2 Connector 3 for host 1 H1 3 Please play attention to the direction which the HBA is installed in the following diagrams If your HBA is installed in a different direc...

Page 25: ...1 3 Drawer 2 PCIe port PCIe ports topology of drawer 2 Falcon 4118 4 Legends Help users to identify components 5 System mode Displays the current system mode of the drawer 6 Port label Port label diagram that helps users to identify PCIe ports Traffic The real time traffics will show up in the right side of every PCIe port 1 Ingress traffic PCIe switch to device traffics 2 Egress traffic Device to...

Page 26: ... link speed The maximum link speed of the PCIe port Link speed display format PCIe generation x Lanes E g Nvidia A100 is a PCIe Gen4 x16 device the current link speed should display G4x16 Max link speed should be G4x16 the current link speed is depending on the device If the link speed does not meet the specification try power cycle the PCIe slot ...

Page 27: ...tion for details 2 Drawer 2 device temp see Device Temperature Graph section for details Falcon 4118 3 Chassis temp see Chassis Temperature Graph section for details 4 Power consumption see Power Consumption Graph section for details 5 Fan speed See Fan Speed Graph section for details 6 Port label Port label diagram that helps users to identify PCIe ports 7 Time period Select time interval for all...

Page 28: ...s at the specific time will be shown in the black menu Chassis Temperature Graph 1 Temperature Temperature scale in degree Celsius 2 Time Time scale in hours 3 Components Components in the chassis each given a color tag E g PCIe switch 2 Blue 4 Temperature curve Temperature curves of the components colors are corresponding to the component color tag E g Blue curve temperature of PCIe switch 2 in t...

Page 29: ...consumption of all components at the specific time will be shown in the black menu Fan Speed Graph 1 Fan speed Fan speed scale in RPM 2 Time Time scale in hours 3 Fans Fans each given a color tag E g Fan 1 2 Blue 4 Fan speed curve Fan speed curves of the fans colors are corresponding to the color tags E g Blue curve fan speed of fan 1 2 in the given time period 5 Instantaneous fan speed Hover the ...

Page 30: ...ntrol the power of drawer 2 Falcon 4118 4 Apply Apply power settings 4 2 7 Maintenance Users can view the current firmware information of BMC and PCIe switches and or update the firmware of Falcon PCIe Expansion System from the Maintenance page 1 BMC firmware Displays BMC firmware version 2 PCIe switch firmware Displays PCIe switch firmware version 3 Update Install Update firmware see P 26 Firmwar...

Page 31: ...model then select Firmware for download item Download the firmware file to your management device i e your PC When the firmware file is downloaded users can update the firmware from Falcon GPU System GUI Go to Maintenance page click Update Install button to upload the file Upload the firmware img file The confirmation message will pop up confirm that you have disconnected all host machines then cl...

Page 32: ...evels or using the search bar 1 Log Categories Filter logs by categories 2 Logs Event logs ordered from newest to oldest 3 Search bar Search for specific logs 4 Download Download all event logs in csv format 5 Refresh Refresh the logs 6 Page Select pages of logs Logs in bold text are unread logs Error highest severity events that may damage the system Warning moderate severity events that requires...

Page 33: ... Setting 1 Time zone Set modify system time zone 2 Sync with NTP server Sync the system with a NTP server Requires NT server IP address 3 Manual Setting Set modify date and time with calendar tool After modifying NTP server IP please click Sync Now the NTP server IP will be updated immediately Please click Apply after modifying time settings in order to keep the settings ...

Page 34: ...tically Use custom DNS server Requires DNS server address 3 Apply Apply the new settings 1 Search bar Search for specific user account 2 User accounts Shows the username user role and UUID of each account 3 Action Edit user account Change password Delete user account 4 Create account Create new user account admin account cannot be deleted See User Roles and Authorities section P 27 for user role d...

Page 35: ... O O O X Read System Settings O O X X Read Maintenance Info O O X X Read Security Logs O O X X User Account Management O O X X Modify System Setting O O X X Maintenance Operation O O X X Premium License Setting O X X X 1 Set up ELK server Check the box to enable ELK server setting Requires ELK server IP and TCP port 2 Send test log Send a test log to the ELK server 3 Apply Apply ELK server setting...

Page 36: ...wer Above information would be shown on the PCIe tree depend on the OS to help users identifying PCIe switches when there are multiple of them 1 License information Current software license details 2 Activate License Activate premium license key This feature is for advanced users with higher level of knowledge and familiarity to PCIe Incorrect setting may cause system error Drawer 2 option does no...

Page 37: ...ion Users can set the MMIO size that each device is able to reserve from host machines The set MMIO size refers to the MMIO size that every device can reserve The MMIO size should be equal or greater than the memory size of your PCIe device Drawer 2 for Falcon 4118 E g If the MMIO size for drawer 1 is set to be 64GB the total MMIO size that drawer 1 reserves from the host will be 32GB x 8devices 2...

Page 38: ...s every time for PCIe scan after re allocating devices Thermal Control Users can set the fan speed and temperature threshold for Falcon GPU system Please set a number that suits your devices spec This number effects the overheat protection mechanism explained as following Continue to next page Please monitor device and chassis temperature closely when setting the parameters Any damage caused by ov...

Page 39: ...c slot will be turned off instead of the entire drawer However when two or more devices components reach the threshold for 10 seconds simultaneously the entire drawer will be turned off There will always be a Fatal threshold equal to the critical threshold 3 C When any device component reaches the fatal threshold the entire drawer will shut down immediately The output limit applies to all fans tog...

Page 40: ...ill be recognized as unsafe site This certificate will expire when IP or domain of this machine changes Upload Upload an SSL certificate for this machine Users will have to install the certificate on every machine that needs to access the Falcon GUI via public internet To allow Falcon GUI to be accessed via open network it is recommended to register an SSL certificate from a certification authorit...

Page 41: ...ake LCD Enter sub menu Select Right Enter sub menu Left Back Up Down 5 1 Operation 1 Functions List of functions accessible from LCD module 2 Cursor Indicating which function is being selected Press button to enter the sub menu 3 Scrollbar Use and button to scroll up and down ...

Page 42: ...awer 1 device ports Drawer 1 host ports Drawer 2 device ports Falcon 4118 Drawer 2 host ports Falcon 4118 Devices Drawer 1 device ports Traffics Status Device name Temperature Drawer 2 device ports Falcon 4118 Hosts Drawer 1 host ports Attached devices Drawer 2 host ports Falcon 4118 Health PSU PSU status Fan RPM Temperature PCIe switch temperature Device temperature Network IP address Subnet mask...

Page 43: ...rm No to decline 5 2 2 Power reset Power reset runs a full power cycle restart on the selected drawer Select a drawer to power reset press to proceed Select Yes to confirm No to decline 5 2 3 System View system information including Serial number Firmware version and System mode S N Chassis serial number FW VER Firmware version D1 Drawer 1 D2 Drawer 2 ...

Page 44: ... H2 Falcon 4118 5 2 5 Devices View device performance including Traffics Status Device name and Temperature Device slot includes drawer 1 1 1 1 8 and drawer 2 2 1 2 8 Falcon 4118 Display Status Drawer slot PCIe Gen x Lanes Status AVL The device is available ATT The device is attached to a host MTY The device slot is empty ERR Device error OFF The device slot is turned off Display Host port number ...

Page 45: ...s LINK The device is available UNLK The device is attached to a host Attached devices display Drawer Slot s ATT Attached devices Display Status PSU PCIe Gen x Lanes Status GOOD PSU working well EMPTY PSU socket empty or not detected There is no space between two slot numbers when multiple devices are attached E g D1 12 indicates that device 1 and device 2 of drawer 1are both attached PSU numbers F...

Page 46: ...ature in C of PCIe switches and devices Switch including PCIe switch 1 and PCIe switch 2 Falcon 4118 Device slot includes drawer 1 1 1 1 8 and drawer 2 2 1 2 8 Falcon 4118 Display Fan RPM SW PCIe switch Fan numbers Falcon 4109 Falcon 4118 ...

Page 47: ...each digit with and When selecting DHCP the system will generate an IP address automatically 5 2 10 Reset to default Reset Falcon system IP address Gateway and GUI Log in account to default Select Yes to start reset No to decline Default IP address 169 254 100 100 Default gateway 0 0 0 0 Log in username admin lower case Log in password admin lower case Default IP address Gateway and Log in account...

Page 48: ...re not warranted See Hardware Specification for details Remove the top cover to replace fans Front cover for Falcon 4118 The fans can be hot plugged User Simply remove the fan that is out of order 6 2 Power Supply Unit Please select the suitable power supply units for replacement damages caused by incompatible power supply units are not warranted Lift the handle and press the release button to unl...

Page 49: ...ng the top cover Especially when installing replacing devices for the riser slot Please power off the drawer before you draw them out of the chassis Power off the drawer from GUI Chassis see P 25 or from LCD Power control P 35 Falcon 4109 Falcon 4118 Falcon 4109 Falcon 4118 ...

Page 50: ...o to the BIOS Advanced a Advanced PCIe PCI PnP configuration Above 4G Decoding to Enabled b Advanced PCIe PCI PnP Configuration MMIOHBase to 56T c Advanced PCIe PCI PnP Configuration MMIO High Size to 512G or higher 3 Connect the GPU expansion chassis to the server and see if the server boots properly Example 2 Intel Xeon Phi Server 1 Temporarily remove the connection of GPU expansion chassis unpl...

Page 51: ...for Intel platforms Disable IOMMU for AMD platforms PCIe link health error If you find the status of PCIe link health showing Error there may be Physical signal issue or PCIe TLP Transaction Layer Packet error between the PCIe slot and your PCIe device It may have an impact on performance e g latency and bandwidth but no data information is lost and PCIe fabric remains reliable Such errors are cor...

Page 52: ... removing the device If it still fails please check your hardware Make sure the device is in good condition Make sure that the power cable is properly connected Make sure that the device is properly plugged into the PCIe slot Clean the PCIe slot and the gold finger of the device Power cycle the device Make sure that the host is properly linked to Falcon chassis Make sure that the mini SAS HD cable...

Page 53: ...e IP address of Falcon PCIe Expansion system or GUI log in identity Check the LCD on the chassis for the IP address If that does not help Try reset Falcon PCIe Expansion system to default See Reset to default section P 39 Device Link Down When device links are not detected by the Falcon PCIe Expansion system the link speed data will not display on the GUI Monitor page Please check if the device is...

Page 54: ...hines to the Falcon PCIe chassis please boot up the Falcon PCIe Expansion system first Only boot up the host machines after Falcon PCIe Expansion system is ready Please visit FAQ https www h3platform com knowledge base faq Or contact sales h3platform com if you have any question Make sure the teeth are hooked into the openings so that the SFF 8644 connectors are properly connected The Falcon syste...

Page 55: ...rranty period H3 Platform shall at its option and expense except for shipping cost repair the defective product or part deliver to the customer an equivalent product or part to replace the defective item All products that are replaced will become the property of H3 Platform Replacement products may be new or reconditioned Warranty does not apply if 1 The warranty period is expired 2 The warranty l...

Page 56: ...en all files are properly installed H3 Platform does not warrant that the software will be uninterrupted or error free In the event H3 Platform software fails to execute its programming instructions during the warranty period the customer s remedy will be either 1 replacement of the H3 Platform software or 2 a refund upon return of the product and all copies of software as well as installation ins...

Page 57: ...s or such items not designed for use with the product The H3 Platform Software Product Limited Warranty does not apply to software used with products not covered by any of the H3 Platform limited warranties offered as part of your purchase of an H3 Platform product TO THE EXTENT ALLOWED BY LOCAL LAW THE ABOVE WARRANTIES ARE EXCLUSIVE AND NO OTHER WARRANTY OR CONDITION WHETHER WRITTEN OR ORAL IS EX...

Page 58: ...NCLUDING LOST PROFIT OR DATA OR OTHER DAMAGE WHETHER BASED IN CONTRACT TORT OR OTHERWISE Some countries states or provinces do not allow the exclusion or limitation of incidental or consequential damages so the above limitation or exclusion may not apply to you THE WARRANTY TERMS CONTAINED HERE EXCEPT TO THE EXTENT LAWFULLY PERMITTED DO NOT EXCLUDE RESTRICT OR MODIFY AND ARE IN ADDITION TO THE MAN...

Search results

Summary of Contents for Falcon PCIe

Reviews:

Related manuals for Falcon PCIe

Brands by name

Popular brands

H3 Falcon PCIe, User Manual

Search results

Summary of Contents for Falcon PCIe

Reviews:

Related manuals for Falcon PCIe

Brands by name

Popular brands