Alpha Data ADM-PCIE-9V3 User Manual Download Page 1

Page: 1 / 40

ADM-PCIE-9V3

User Manual

Document Revision: 2.7

14th December 2018

Summary of Contents for ADM-PCIE-9V3

Page 1: ...ADM PCIE 9V3 User Manual Document Revision 2 7 14th December 2018 ...

Page 2: ... without prior written consent from Alpha Data Parallel Systems Ltd Head Office Address 4 West Silvermills Lane Edinburgh EH3 5BD UK Telephone 44 131 558 2600 Fax 44 131 558 2700 email sales alpha data com website http www alpha data com US Office 611 Corporate Circle Suite H Golden CO 80401 303 954 8768 866 820 9956 toll free sales alpha data com http www alpha data com All trademarks are the pro...

Page 3: ...2 5 Ultraport SlimSAS 10 3 2 6 DDR4 SDRAM Reference Clocks 10 3 3 PCI Express 11 3 4 DDR4 SDRAM 11 3 5 QSFP28 12 3 6 OpenCAPI Ultraport SlimSAS 13 3 7 System Monitor 14 3 7 1 System Monitor Status LEDs 15 3 8 USB Interface 16 3 9 Configuration 16 3 9 1 Configuration From Flash Memory 16 3 9 1 1 Building and Programming Configuration Images 17 3 9 2 Configuration via JTAG 17 3 10 GPIO Connector 18 ...

Page 4: ...Table 15 Complete Pinout Table 21 List of Figures Figure 1 ADM PCIE 9V3 Product Photo 1 Figure 2 Thermal Performance 3 Figure 3 Optional Blower 4 Figure 4 Full Height Heat Sink 4 Figure 5 ADM PCIE 9V3 Block Diagram 5 Figure 6 Switches 6 Figure 7 Backside LEDs 7 Figure 8 Front Panel LEDs 7 Figure 9 Clock Topology 8 Figure 10 Si5328 Block Diagram 10 Figure 11 QSFP Locations 12 Figure 12 OpenCAPI Loc...

Page 5: ...ECC 16GB 8GB per bank default rated at 2400MT s 32GB option rated at 1866MT s Two QSFP28 zQSFP sites capable of data rates up to 28 Gbps per channel 112 Gbps per cage One 8 lane Ultraport SlimSAS connector that is compliant with OpenCAPI Optional timing input Front panel and rear edge JTAG access via USB port FPGA configurable over USB JTAG and SPI configuration flash XCVU3P 2FFVC1517I FPGA Voltag...

Page 6: ...er If the application requires a low profile bracket and the order quantity is high contact sales alpha data com to get the correct bracket fitted before shipping 2 2 3 Power Requirements The PCIe Specification permits a standard low profile half length PCIe card to dissipate up to 25 W of power drawn from the PCIe slot The ADM PCIE 9V3 may consume more than 25 W of power for larger user FPGA desi...

Page 7: ... in still air The power dissipation can be estimated by using the Alpha Data power estimator in conjunction with the Xilinx Power Estimator XPE downloadable at http www xilinx com products technology power xpe html Download the UltraScale tool and set the Device to Virtex UltraScale VU3P FFVC1516 2 Industrial Set the ambient temperature to your system ambient and select User Override for the Effec...

Page 8: ...ink plug in the small power connector into the connector in the corner of the board It is possible to alternatively ship a fan that fits in the adjacent PCIe card slot if required for mechanical fit contact sales alpha data com for details Figure 3 Optional Blower 2 5 Full Height Heat Sink For customers with full height chassis that would like better thermal performance and to baffle the empty spa...

Page 9: ...S connector also capable of 28G channel an input for a timing synchronization input a 12 pin header for general purpose use clocking control pins debug etc and a robust system monitor XCVU3P 2 FFVC1517I 0 4 5 7 8 11 12 15 x16 PCIe Gen3 4 Edge QSFP28 Cage 4x28 Gbps max QSFP28 Cage 4x28 Gbps max DDR4 Bank 1 DDR4 2400 1866 8 16GB DDR4 Bank 0 DDR4 2400 1866 8 16GB System Monitor MGT MGT HPIO HPIO HRIO...

Page 10: ...Pin AW27 0 SW1 3 OFF Service Mode Regular Operation Firmware update service mode SW1 4 OFF JTAG Source JTAG to FPGA from USB JTAG to FPGA from debug header SW2 1 ON HOST_I2 C_EN Sysmon over PCIe I2C Sysmon isolated SW2 2 ON CAPI_VP D_EN OpenCAPI VPD available OpenCAPI VPD isolated SW2 3 ON CAPI_VP D_WP CAPI VPD is write protected CAPI VPD is writable SW2 4 ON Reserved Reserved Reserved Table 3 Swi...

Page 11: ...FF State D4 DONE FPGA is configured FPGA is not configured D8 USER_LED_G0 User defined 0 pin AT27 User defined 1 pin AT27 D2 USER_LED_G1 User defined 0 pin AU27 User defined 1 pin AU27 D7 USER_LED_R User defined 0 pin AU23 User defined 1 pin AU23 D5 Status 0 See Status LED Definitions D6 Status 1 See Status LED Definitions D9 Front_LED_0 User defined 1 pin AH24 User defined 0 pin AH24 D10 Front_LE...

Page 12: ... Bank 64 Card Edge PCIe Ref Clock 100MHz 25MHz 30ppm Source Si5338 Clock Synth NB6L11S Fanout CAPI 161 1328125MHz Factory Default MGTREFCLK0_125 CAPI 161 1328125MHz Factory Default MGTREFCLK0_124 NB6L11S Fanout Memory Interface Clock 300Mhz IO Bank 44 Memory Interface Clock 300Mhz IO Bank 64 NB6L11S Fanout PCIe Ref Clock MGTREFCLK0_224 PCIe Ref Clock MGTREFCLK0_226 CAPI Cable Clock 156 25MHz when ...

Page 13: ..._0C MGTREFCLK0_127 LVDS U33 U34 Table 8 QSFP28 Reference Clocks The QSFP28 cages are also located such that they can be clocked from a Si5328 jitter attenuator clock multiplier If jitter attenuation is required please see the reference documentation for the Si5328 https www silabs com Support 20Documents TechnicalDocs Si5328 pdf The Si5328 is configured with a 114 285MHz oscilator on XA and XB The...

Page 14: ...em monitor This can be done using the Alpha Data API or over USB with the appropriate Alpha Data Software tools Signal Target FPGA Input I O Standard P pin N pin GTY_CLK_1B MGTREFCLK0_125 HCSL AE33 AE34 GTY_CLK_1C MGTREFCLK0_124 HCSL AJ33 AJ34 CAPI_CLK_C MGTREFCLK1_125 HCSL AC33 AC34 CAPI_CLK_D MGTREFCLK1_124 HCSL AG33 AG34 Table 10 SlimSAS Reference Clocks OpenCAPI 3 2 6 DDR4 SDRAM Reference Cloc...

Page 15: ...ng setting if a user experiences link errors or training issues with their system within the IP core generator change the mode to Advanced and open the GT Settings tab change the form factor driven insertion loss adjustment from Add in Card to Chip to Chip See Xilinx PG239 for more details 3 4 DDR4 SDRAM Two banks of DDR4 SDRAM memory are soldered down to the board While the factory default is 8GB...

Page 16: ...able at the end of this document The notation used in the pin assignments is QSFP0 and QSFP1 with locations clarified in the diagram below Use the QSFP _SEL_1V8_L in conjunction with the OPTICAL_SCL_1V8 and OPTICAL_SDA_1V8 pins as detailed in Complete Pinout Table to communicate with QSFP28 register space Note The LP_MODE Low Power Mode to each QSFP28 cage is tied to ground Figure 11 QSFP Location...

Page 17: ...API compliant interfaces running at 200G 8 chanels at 25G Please contact support alpha data com or your IBM representative for more details on OpenCAPI and its benefits The SlimSAS connector can also be used to connect multiple 9V3 cards within a chassis Figure 12 OpenCAPI Location Page 13 Functional Description ad ug 1322_v2_7 pdf ...

Page 18: ... Index Purpose Description ETC ETC Elapsed time counter seconds EC EC Event counter power cycles 12 0V ADC00 Board Input Supply 3 3V ADC01 Board Input Supply 3 3V ADC02 Board Input Auxilary Power Supply 3 3V PSU0OK Internal logic voltage 2 5V ADC03 Clock and DRAM Voltage Supply 1 8V PSU0OK FPGA IO Voltage VCCO 1 8V ADC04 Transceiver Power AVCC_AUX 1 2V ADC05 DDR4 SDRAM and FPGA memory I O 1 2V ADC...

Page 19: ...by Powered off Flashing Green Flashing Red together Attention critical alarm active Flashing Green Flashing Red alternating Service Mode Flashing Green Red Attention alarm active Red Missing application firmware or invalid firmware Flashing Red FPGA configuration cleared to protect board Table 14 Status LED Definitions Page 15 Functional Description ad ug 1322_v2_7 pdf ...

Page 20: ... two main ways of configuring the FPGA on the ADM PCIE 9V3 From Flash memory at power on as described in Section 3 9 1 Using USB cable connected at either USB port Section 3 9 2 3 9 1 Configuration From Flash Memory The FPGA can be automatically configured at power on from two 256 Mbit QSPI flash memory device configured as an x8 SPI device Micron part numbers MT25QU256ABA8E12 1SIT These flash dev...

Page 21: ...BITSTREAM CONFIG SPI_FALL_EDGE YES current_design set_property BITSTREAM CONFIG UNUSEDPIN Pullnone current_design set_property CFGBVS GND current_design set_property CONFIG_VOLTAGE 1 8 current_design set_property BITSTREAM CONFIG OVERTEMPSHUTDOWN Enable current_design Generate an MCS file with these properties write_cfgmem format MCS size 64 interface SPIx8 loadbit up 0x0000000 directory to file f...

Page 22: ...ular options The direct connect GPIO signals are limited to 1 8V by a quickswitch 74CBTLVD3861BQ in order to protect the FPGA from overvoltage on IO pins This quickswitch allows the signals to travel in either direction with only 4 ohms of series impedance and less than 1ns of propagation delay The nets are directly connected to the FPGA after the quickswitch Direct connect signal names are labele...

Page 23: ... 3 11 User EEPROM A 2Kb I2C user EEPROM is provided for storing MAC addresses or other user information The EEPROM is part number M24C02 RMC6TG The address pins A2 A1 and A0 are all strapped to a logical 0 Write protect WP Serial Clock SCL and Serial Data SDA pin assignments can be found in Complete Pinout Table with the names SPARE_WP SPARE_SCL and SPARE_SDA respectively WP SDA and SCL signals al...

Page 24: ...ADM PCIE 9V3 User Manual Page Intentionally left blank Page 20 Functional Description ad ug 1322_v2_7 pdf ...

Page 25: ...API_I2C_SCL_1V8 1 8 C28 CAPI_I2C_SDA_1V8 1 8 AV37 CAPI_RX0_N MGT AV36 CAPI_RX0_P MGT AU39 CAPI_RX1_N MGT AU38 CAPI_RX1_P MGT AE39 CAPI_RX10_N MGT AE38 CAPI_RX10_P MGT AR39 CAPI_RX2_N MGT AR38 CAPI_RX2_P MGT AN39 CAPI_RX3_N MGT AN38 CAPI_RX3_P MGT AL39 CAPI_RX7_N MGT AL38 CAPI_RX7_P MGT AJ39 CAPI_RX8_N MGT AJ38 CAPI_RX8_P MGT AG39 CAPI_RX9_N MGT AG38 CAPI_RX9_P MGT AW34 CAPI_TX0_N MGT AW33 CAPI_TX0...

Page 26: ... 2 G9 DDR4_0_A1 1 2 D9 DDR4_0_A10 1 2 H11 DDR4_0_A11 1 2 E8 DDR4_0_A12 1 2 J11 DDR4_0_A13 1 2 C9 DDR4_0_A14 1 2 B11 DDR4_0_A15 1 2 K12 DDR4_0_A16 1 2 H9 DDR4_0_A17 1 2 G11 DDR4_0_A2 1 2 D11 DDR4_0_A3 1 2 E12 DDR4_0_A4 1 2 G10 DDR4_0_A5 1 2 F10 DDR4_0_A6 1 2 J9 DDR4_0_A7 1 2 J8 DDR4_0_A8 1 2 F12 DDR4_0_A9 1 2 C12 DDR4_0_ACT_N 1 2 H7 DDR4_0_ALERT_N 1 2 F8 DDR4_0_BA0 1 2 H8 DDR4_0_BA1 1 2 D10 DDR4_0_...

Page 27: ...5 1 2 H22 DDR4_0_DM6 1 2 N22 DDR4_0_DM7 1 2 J13 DDR4_0_DM8 1 2 L10 DDR4_0_DQ0 1 2 L9 DDR4_0_DQ1 1 2 M15 DDR4_0_DQ10 1 2 M17 DDR4_0_DQ11 1 2 M14 DDR4_0_DQ12 1 2 N18 DDR4_0_DQ13 1 2 N16 DDR4_0_DQ14 1 2 N17 DDR4_0_DQ15 1 2 F15 DDR4_0_DQ16 1 2 E16 DDR4_0_DQ17 1 2 F14 DDR4_0_DQ18 1 2 E17 DDR4_0_DQ19 1 2 N9 DDR4_0_DQ2 1 2 G16 DDR4_0_DQ20 1 2 F17 DDR4_0_DQ21 1 2 E15 DDR4_0_DQ22 1 2 G17 DDR4_0_DQ23 1 2 A1...

Page 28: ..._DQ37 1 2 E21 DDR4_0_DQ38 1 2 G20 DDR4_0_DQ39 1 2 M10 DDR4_0_DQ4 1 2 D18 DDR4_0_DQ40 1 2 B22 DDR4_0_DQ41 1 2 A19 DDR4_0_DQ42 1 2 A18 DDR4_0_DQ43 1 2 C19 DDR4_0_DQ44 1 2 B19 DDR4_0_DQ45 1 2 A22 DDR4_0_DQ46 1 2 C18 DDR4_0_DQ47 1 2 G22 DDR4_0_DQ48 1 2 J20 DDR4_0_DQ49 1 2 K11 DDR4_0_DQ5 1 2 H19 DDR4_0_DQ50 1 2 J19 DDR4_0_DQ51 1 2 H18 DDR4_0_DQ52 1 2 J18 DDR4_0_DQ53 1 2 G21 DDR4_0_DQ54 1 2 K18 DDR4_0_D...

Page 29: ...DR4_0_DQ7 1 2 H14 DDR4_0_DQ70 1 2 J15 DDR4_0_DQ71 1 2 L17 DDR4_0_DQ8 1 2 M16 DDR4_0_DQ9 1 2 L12 DDR4_0_DQS0_C 1 2 M12 DDR4_0_DQS0_T 1 2 L14 DDR4_0_DQS1_C 1 2 L15 DDR4_0_DQS1_T 1 2 E13 DDR4_0_DQS2_C 1 2 F13 DDR4_0_DQS2_T 1 2 A15 DDR4_0_DQS3_C 1 2 B15 DDR4_0_DQS3_T 1 2 E22 DDR4_0_DQS4_C 1 2 F22 DDR4_0_DQS4_T 1 2 B21 DDR4_0_DQS5_C 1 2 C21 DDR4_0_DQS5_T 1 2 K20 DDR4_0_DQS6_C 1 2 K21 DDR4_0_DQS6_T 1 2 ...

Page 30: ...4_1_A15 1 2 AT11 DDR4_1_A16 1 2 AL8 DDR4_1_A17 1 2 AP11 DDR4_1_A2 1 2 AU9 DDR4_1_A3 1 2 AT10 DDR4_1_A4 1 2 AL12 DDR4_1_A5 1 2 AM12 DDR4_1_A6 1 2 AM10 DDR4_1_A7 1 2 AL11 DDR4_1_A8 1 2 AP7 DDR4_1_A9 1 2 AV9 DDR4_1_ACT_N 1 2 AR10 DDR4_1_ALERT_N 1 2 AN11 DDR4_1_BA0 1 2 AR9 DDR4_1_BA1 1 2 AP12 DDR4_1_BG0 1 2 AN10 DDR4_1_BG1 1 2 AW13 DDR4_1_C0 1 2 AU10 DDR4_1_C1 1 2 AW11 DDR4_1_C2 1 2 AN7 DDR4_1_CK_C 1 ...

Page 31: ... AM14 DDR4_1_DQ11 1 2 AL15 DDR4_1_DQ12 1 2 AM17 DDR4_1_DQ13 1 2 AL17 DDR4_1_DQ14 1 2 AM13 DDR4_1_DQ15 1 2 AR15 DDR4_1_DQ16 1 2 AP14 DDR4_1_DQ17 1 2 AT15 DDR4_1_DQ18 1 2 AR14 DDR4_1_DQ19 1 2 AH10 DDR4_1_DQ2 1 2 AP17 DDR4_1_DQ20 1 2 AN16 DDR4_1_DQ21 1 2 AN17 DDR4_1_DQ22 1 2 AN15 DDR4_1_DQ23 1 2 AU15 DDR4_1_DQ24 1 2 AT17 DDR4_1_DQ25 1 2 AV15 DDR4_1_DQ26 1 2 AT16 DDR4_1_DQ27 1 2 AV14 DDR4_1_DQ28 1 2 A...

Page 32: ...2 1 2 AW22 DDR4_1_DQ43 1 2 AU18 DDR4_1_DQ44 1 2 AT22 DDR4_1_DQ45 1 2 AW21 DDR4_1_DQ46 1 2 AU19 DDR4_1_DQ47 1 2 AH19 DDR4_1_DQ48 1 2 AJ22 DDR4_1_DQ49 1 2 AH12 DDR4_1_DQ5 1 2 AF21 DDR4_1_DQ50 1 2 AH22 DDR4_1_DQ51 1 2 AF20 DDR4_1_DQ52 1 2 AJ19 DDR4_1_DQ53 1 2 AH21 DDR4_1_DQ54 1 2 AJ21 DDR4_1_DQ55 1 2 AM19 DDR4_1_DQ56 1 2 AK20 DDR4_1_DQ57 1 2 AM22 DDR4_1_DQ58 1 2 AL22 DDR4_1_DQ59 1 2 AG10 DDR4_1_DQ6 1...

Page 33: ...DQS0_T 1 2 AL16 DDR4_1_DQS1_C 1 2 AK16 DDR4_1_DQS1_T 1 2 AT13 DDR4_1_DQS2_C 1 2 AR13 DDR4_1_DQS2_T 1 2 AV17 DDR4_1_DQS3_C 1 2 AU17 DDR4_1_DQS3_T 1 2 AP22 DDR4_1_DQS4_C 1 2 AN22 DDR4_1_DQS4_T 1 2 AV21 DDR4_1_DQS5_C 1 2 AV22 DDR4_1_DQS5_T 1 2 AH20 DDR4_1_DQS6_C 1 2 AG20 DDR4_1_DQS6_T 1 2 AL21 DDR4_1_DQS7_C 1 2 AK21 DDR4_1_DQS7_T 1 2 AH15 DDR4_1_DQS8_C 1 2 AH16 DDR4_1_DQS8_T 1 2 AR11 DDR4_1_ODT 1 2 A...

Page 34: ...FPGA_FLASH_DQ6 1 8 AG28 FPGA_FLASH_DQ7 1 8 AH24 FRONT_LED_0 1 8 AJ23 FRONT_LED_1 1 8 F30 GP0_1V8_N 1 8 G30 GP0_1V8_P 1 8 H31 GP1_1V8_N 1 8 J31 GP1_1V8_P 1 8 N34 GTY_CLK_0B_PIN_N MGT_REFCLK N33 GTY_CLK_0B_PIN_P MGT_REFCLK U34 GTY_CLK_0C_PIN_N MGT_REFCLK U33 GTY_CLK_0C_PIN_P MGT_REFCLK AE34 GTY_CLK_1B_PIN_N MGT_REFCLK AE33 GTY_CLK_1B_PIN_P MGT_REFCLK AJ34 GTY_CLK_1C_PIN_N MGT_REFCLK AJ33 GTY_CLK_1C_...

Page 35: ...EFCLK J1 PCIE_RX0_N MGT J2 PCIE_RX0_P MGT L1 PCIE_RX1_N MGT L2 PCIE_RX1_P MGT AJ1 PCIE_RX10_N MGT AJ2 PCIE_RX10_P MGT AL1 PCIE_RX11_N MGT AL2 PCIE_RX11_P MGT AN1 PCIE_RX12_N MGT AN2 PCIE_RX12_P MGT AR1 PCIE_RX13_N MGT AR2 PCIE_RX13_P MGT AU1 PCIE_RX14_N MGT AU2 PCIE_RX14_P MGT AV3 PCIE_RX15_N MGT AV4 PCIE_RX15_P MGT N1 PCIE_RX2_N MGT N2 PCIE_RX2_P MGT R1 PCIE_RX3_N MGT R2 PCIE_RX3_P MGT U1 PCIE_RX...

Page 36: ...N_P MGT AP4 PCIE_TX12_PIN_N MGT AP5 PCIE_TX12_PIN_P MGT AT4 PCIE_TX13_PIN_N MGT AT5 PCIE_TX13_PIN_P MGT AU6 PCIE_TX14_PIN_N MGT AU7 PCIE_TX14_PIN_P MGT AW6 PCIE_TX15_PIN_N MGT AW7 PCIE_TX15_PIN_P MGT M4 PCIE_TX2_PIN_N MGT M5 PCIE_TX2_PIN_P MGT P4 PCIE_TX3_PIN_N MGT P5 PCIE_TX3_PIN_P MGT T4 PCIE_TX4_PIN_N MGT T5 PCIE_TX4_PIN_P MGT V4 PCIE_TX5_PIN_N MGT V5 PCIE_TX5_PIN_P MGT AB4 PCIE_TX6_PIN_N MGT A...

Page 37: ...39 QSFP0_RX0_N MGT G38 QSFP0_RX0_P MGT E39 QSFP0_RX1_N MGT E38 QSFP0_RX1_P MGT C39 QSFP0_RX2_N MGT C38 QSFP0_RX2_P MGT B37 QSFP0_RX3_N MGT B36 QSFP0_RX3_P MGT D31 QSFP0_SEL_1V8_L 1 8 F36 QSFP0_TX0_N MGT F35 QSFP0_TX0_P MGT D36 QSFP0_TX1_N MGT D35 QSFP0_TX1_P MGT C34 QSFP0_TX2_N MGT C33 QSFP0_TX2_P MGT A34 QSFP0_TX3_N MGT A33 QSFP0_TX3_P MGT F33 QSFP1_MODPRS_L 1 8 R39 QSFP1_RX0_N MGT R38 QSFP1_RX0_...

Page 38: ... SI5328_1V8_SCL 1 8 L29 SI5328_1V8_SDA 1 8 M30 SI5328_REFCLK_IN_N 1 8 LVDS M29 SI5328_REFCLK_IN_P 1 8 LVDS L34 SI5328_REFCLK_OUT0_PIN_N MGT_REFCLK L33 SI5328_REFCLK_OUT0_PIN_P MGT_REFCLK R34 SI5328_REFCLK_OUT1_PIN_N MGT_REFCLK R33 SI5328_REFCLK_OUT1_PIN_P MGT_REFCLK AT25 SPARE_SCL 1 8 AT26 SPARE_SDA 1 8 AP23 SPARE_WP 1 8 AW24 SRVC_MD_L_1V8 1 8 Y10 TCK 1 8 AC9 TDI 1 8 Y9 TDO 1 8 AD10 TMS 1 8 AT27 U...

Page 39: ...ADM PCIE 9V3 User Manual Pin Number Signal Name Bank Voltage AW27 USR_SW1 1 8 Table 15 Complete Pinout Table Page 35 Complete Pinout Table ad ug 1322_v2_7 pdf ...

Page 40: ...ated pinout appendix to include refclk100_pin and front_LED nets 3 Apr 2018 2 3 K Roth Added Si5328 diagram Si5328 Block Diagram Added link to Xilinx documents for vivado hardware manager in Configuration via JTAG updated Avr2util download link to point to current version 15 May 2018 2 4 K Roth Updated Switch Functions to include SW2 Updated USB Interface description to mention rear USB connector ...

Search results

Summary of Contents for ADM-PCIE-9V3

Reviews:

Related manuals for ADM-PCIE-9V3

Brands by name

Popular brands

Alpha Data ADM-PCIE-9V3, User Manual

Search results

Summary of Contents for ADM-PCIE-9V3

Reviews:

Related manuals for ADM-PCIE-9V3

Brands by name

Popular brands