Xilinx Zynq UltraScale+ User Manual Download Page 24

Page: 24 / 86

Zynq Ult VCU TRD User Guide

UG1250 (v2019.1) May 29, 2019

www.xilinx.com

Chapter 3:

APU Software Platform

Media Framework

The main goal of the media framework is to discover the device topology of a video pipeline

and to configure it at run time. To achieve this, pipelines are modeled as an oriented graph

of building blocks called

entities

connected through pads.

X-Ref Target - Figure 3-2

Figure 3-2:

VL42 Driver Stack

vcu_qt/vcu_gst_app

vcu_gst_lib

libv4lsubdev

libv4I2

libmediactl

User Space

Kernel Space

/dev/v4I-subdev*

/dev/video*

/dev/media*

DMA Engine

Channel

DMA

V4L2 subdev

XVIPP Driver

TPG

Fmbuf Wr

VPSS

(Scaler Only)

Fmbuf Wr

TPG Capture Pipeline

HDMI Rx Capture Pipeline

CSK-2 Rx Capture Pipeline

SDI Rx Capture Pipeline

SDI Rx

Fmbuf Wr

IMX274

MIPI

CSI-2

Demosaic

Gamma

VPSS

CSC

VPSS

Scaler

Fmbuf Wr

VTC

HDMI Rx

X19930-120118

VPSS

(Scaler Only)

vcu_apm_lib

vcu_video_lib

SCD

SCD Pipeline

Send Feedback

Summary of Contents for Zynq UltraScale+

Page 1: ...Zynq UltraScale MPSoC ZCU106 Video Codec Unit Targeted Reference Design User Guide UG1250 v2019 1 May 29 2019...

Page 2: ...capture and HDMI display with SDSoC support and SDI video display designs Updated with complete VCU TRD design details and with design components for the audio and streaming feature Added 1080p30 mul...

Page 3: ...APU Software Platform Introduction 21 Software Architecture 22 GUI Application vcu_qt 33 GStreamer Application vcu_gst_app 41 GStreamer Interface Library vcu_gst_lib 42 AXI Performance Monitor APM Li...

Page 4: ...1250 v2019 1 May 29 2019 www xilinx com Appendix B Additional Resources and Legal Notices Xilinx Resources 84 Solution Centers 84 Documentation Navigator and Design Hubs 84 References 85 Please Read I...

Page 5: ...VCU to arrive at optimal configurations for encoder and decoder blocks The TRD demonstrates the following hard block features in the PS and PL VCU hard block capable of performing up to 4K 3840 x 216...

Page 6: ...ls gives an overview of the design modules and design components that make up this reference design Chapter 3 APU Software Platform describes the APU software platform covering the middleware and oper...

Page 7: ...32KB D Cache w ECC 32KB I Cache w Parity Memory Mgmt Unit Trace Macro Cell GIC 400 SCU CCI SMMU 1MB L2 w ECC 1 2 3 4 Real Time Processing Unit ARM Cortex R5 Vector Floating Point Unit 32KB D Cache w E...

Page 8: ...memory Secure Digital embedded Multimedia Card SD eMMC inter IC I2C and general purpose I O GPIO Platform management unit PMU Configuration security unit CSU 6 port DDR controller with error correctio...

Page 9: ...mory pipeline includes VCU encode decode Video frames are read from DDR memory processed by the VCU and written back to memory Display pipeline reading video frames from memory and sending them to a m...

Page 10: ...HDMI TX SDI TX DP and I2S TX interfaces Transcode pipeline to transfer the file from the HOST machine to the client board zcu106 through PCIe XDMA bridge interface in the PL The file is passed to the...

Page 11: ...DI Tx Design HDMI Tx Design HDMI Rx Design Capture Pipeline FB Write VPSS Scaler Only SDI RX Source FB Write TPG AXI DMA 10G Ethernet SS FB Write Encode SDX Accelerator File System SATA USB SD Audio F...

Page 12: ...rd Figure 1 4 shows the block diagram of the TRD along with the board components Key Features Target platforms and extensions ZCU106 evaluation board see ZCU106 Evaluation Board User Guide UG1244 Ref...

Page 13: ...019 1 Ref 4 PetaLinux tools 2019 1 Hardware interfaces and IP GPU Video inputs TPG HDMI RX MIPI CSI 2 RX File source SD card SATA and USB 3 0 drives SDI RX Stream In Video outputs DisplayPort TX contr...

Page 14: ...rks libraries Video Video4Linux V4L2 Media controller Audio libalsa Display Direct Rendering Manager Kernel Mode Setting DRM KMS X Server X Org Graphics Qt5 OpenGL ES2 User application APU GStreamer b...

Page 15: ...Zynq UltraScale VCU TRD User Guide 15 UG1250 v2019 1 May 29 2019 www xilinx com Chapter 1 Introduction Pixel formats NV12 NV16 XV15 XV20 Send Feedback...

Page 16: ...lemented in the PL The video stored in SD cards or USB SATA drives is decoded and displayed on HDMI TX The module can Stream in encoded data through an Ethernet interface and decode and display it on...

Page 17: ...ting the performance increase using high level synthesis HLS to create RTL from a C algorithm and automatically inserts data movers along with the required drivers PL SDI Video Capture and SDI Display...

Page 18: ...XV20 pixel format This is the new design approach proposed to use PL_DDR for decoding and PS_DDR for encoding so that DDR bandwidth would be enough to support high bandwidth VCU applications requiring...

Page 19: ...u_apm_lib and vcu_video_lib libraries and provides a GUI to control and visualize various parameters of this design The GUI is supported only on DP vcu_video_lib Library that configures various video...

Page 20: ...ser Guide 20 UG1250 v2019 1 May 29 2019 www xilinx com Chapter 2 Targeted Reference Design Details PL Vivado Vivado IP integrator design that integrates the capture processing encode decode and displa...

Page 21: ...ware platform which is further subdivided into a middleware layer an operating system OS layer and an application stack see Figure 3 1 The two layers are examined in conjunction because they interact...

Page 22: ...vides a stable well defined API to user space Includes device drivers and kernel frameworks subsystems Accesses the hardware X Ref Target Figure 3 1 Figure 3 1 APU Linux Software Platform Mali 400 vcu...

Page 23: ...in the device tree The framework exposes the following device node types to user space to control certain aspects of the pipeline Media device node dev media Video device node dev video V4L subdevice...

Page 24: ...through pads X Ref Target Figure 3 2 Figure 3 2 VL42 Driver Stack vcu_qt vcu_gst_app vcu_gst_lib libv4lsubdev libv4I2 libmediactl User Space Kernel Space dev v4I subdev dev video dev media DMA Engine...

Page 25: ...devices are modeled as subdevice nodes and DMA channels as video nodes A media device node is created that allows the user space application to configure the video pipeline and its subdevices through...

Page 26: ...ing their own APIs However both are commonly referred to as a single framework namely DRM KMS This split is advantageous especially for SoCs that often have dedicated hardware blocks for display and g...

Page 27: ...OMX integration layer IL is integrated on top of CtrlSW and the GStreamer framework is used to integrate the OMX IL component and other multimedia elements see the OpenMAX website Ref 5 OpenMAX Open M...

Page 28: ...ard are enumerated starting from zero In this TRD design sound cards are created for the HDMI RX capture pipeline the HDMI TX playback pipeline and the I2S RX and SDI RX Capture and I2S TX and SDI TX...

Page 29: ...d with the source and destination addresses of the memory ranges to copy In a read case the PCIe Endpoint block driver running on the client allocates the destination buffer in the client DDR and pass...

Page 30: ...s refer to Xilinx Answer 71435 XDMA Host Application The XDMA host application transfers files from host to client in chunks of buffers using DMA memory based transfers The application receives transc...

Page 31: ...D Table 3 1 Software Stack Components Component Description Kernel drivers This layer contains the kernel drivers for HDMI Test Pattern Generator TPG IMX274 sensor driver MIPI CSI 2 RX Subsystem Xilin...

Page 32: ...detail in subsequent sections GUI application vcu_qt GStreamer interface library vcu_gst_lib Video library vcu_video_lib AXI Performance Monitor APM library vcu_apm_lib GStreamer command line applicat...

Page 33: ...ured video stream The GUI shown in Figure 3 9 contains the following control elements displayed on top of the video output area Control bar top Video info panel top right System performance panels bot...

Page 34: ...SATA drives Test Pattern Generator TPG Implemented in the PL CSI Implemented in the PL option MIPI MIPI CSI model LI IMX274 MIPI FMC v1 1 SDI Implemented in the PL Stream In Stream from network or In...

Page 35: ...tions Every ten seconds playback preset changes and plays in a loop until you click the stop button If no source is connected an error popup displays If any error returns in any playback the demo skip...

Page 36: ...ipeline is in the stop state X Ref Target Figure 3 11 Figure 3 11 Encoder Parameter Panel X19923 112718 Table 3 3 Encoder Parameter Panel Settings Encoder Parameter Setting Encoder This can be either...

Page 37: ...tributed over the frame as regularly as possible If slice size is also defined more slices can be produced to fit the slice size requirement Range 4 22 4Kp resolution with HEVC codec 4 32 4Kp resoluti...

Page 38: ...r the recorded file The list is dynamically populated based on mounted storage devices Supported storage devices include SD cards and USB SATA drives Note Because of speed and storage constraints usin...

Page 39: ...he idr value gop length X Ref Target Figure 3 13 Figure 3 13 Stream Out Panel Table 3 5 Stream Out Panel Settings Parameter Setting SINK Provides the sink option for the stream out case It is set to P...

Page 40: ...g Panel Table 3 6 Audio Settings Parameter Setting Enable Audio Enable or disable audio in pipeline Format Audio format Currently S24_32LE format is supported Channel Number of audio channels Currentl...

Page 41: ...app is a command line multi threaded Linux application that uses the vcu_gst_lib interface similar to vcu_qt The difference is to manually feed the input configuration and run the pipeline each time w...

Page 42: ...line parameters such as resolution format and source type v4l2src filesrc set encoder parameters start and stop the pipeline calculate FPS perform error handling calculate bit rate for file stream in...

Page 43: ...ed video Note Use omxh264dec for H264 decoding and omxh265dec for H265 decoding omxh26xenc Encoder omxh26xenc is a hardware accelerated video encoder that encodes raw video frames Example pipeline gst...

Page 44: ...io x raw format S24_32LE rate 48000 channels 2 queue audioconvert audioresample faac aacparse mpegtsmux filesink location out ts This pipeline shows the audio captured from an ALSA device that deliver...

Page 45: ...ww xilinx com Chapter 3 APU Software Platform HDMI Here only half of each stream is displayed to showcase eight different streams on a single screen X Ref Target Figure 3 18 Figure 3 18 Multi Stream 3...

Page 46: ...nt In the case of a raw processed pipeline the video capture device v4l2src video processing accelerator VCU element and kmssink plugin use DMABUF framework for sharing buffers between peer elements s...

Page 47: ...uses the DMA_BUF framework and reads the kernel buffer for encoding In the playback side 1 The decoder driver allocates DMA buffer 2 The gst omx plug in exports the file descriptor FD to the kmssink p...

Page 48: ...e configuration for video capture The vcu_video_lib library exports and imports the following interfaces TPG video source controls to vcu_gst_lib library CSI video source controls to vcu_gst_lib libra...

Page 49: ...e entities pads and links Configure sub devices Set media bus format Set dimensions width height The video_lib library sets the media bus format and video resolution on each sub device source and sink...

Page 50: ...locks and resets of the system as well as system power management In the pre configuration stage the PMU executes the PMU ROM and releases the reset of the configuration security unit CSU It then ente...

Page 51: ...PSoC Technical Reference Manual UG1085 Ref 8 Global Address Map For more information on system addresses see chapter 8 in Zynq UltraScale MPSoC Technical Reference Manual UG1085 Ref 8 Memory The DMA i...

Page 52: ...arated into two sub images or planes In NV12 and XV15 formats chroma planes are sub sampled in both the horizontal and vertical dimensions by a factor of 2 That is to say for a 2x2 square of pixels th...

Page 53: ...umber of bytes from the first pixel of a line to the first pixel of the next line of video In the simplest case the stride equals the width multiplied by the bits per pixel converted to bytes For exam...

Page 54: ...PI FMC image sensor daughter card X Ref Target Figure 5 1 Figure 5 1 Hardware Block Diagram ZCU106 Ethernet Source Sink IMX274 Sensor LI IMX274MIPI FMC Accerlator VCU SDX Bypass Filter Accelerator DP...

Page 55: ...of 10G 25G Ethernet Subsystem IP that receives video data over Ethernet and AXI DMA IP that writes it to memory The SDI RX capture pipeline in PL consists of the SDI RX Subsystem and Video Processing...

Page 56: ...or data transfers between the host system memory and the Endpoint In the card to host direction the XDMA block moves data from the Endpoint PS DDR to the host memory through PCIe The block diagram hig...

Page 57: ...r reset signal Interconnect and peripheral reset signals are generated using proc_sys_rst IP in the PL The VCU Reset in PCIe design is gated with the link_up signal of the PCIe Endpoint block X Ref Ta...

Page 58: ...4 Lite based register interface The Video Timing Controller VTC generates video timing signals including horizontal and vertical sync and blanking signals The timing signals are converted to AXI4 Stre...

Page 59: ...e register interface The Video PHY Controller VPHY enables plug and play connectivity with Video Transmit or Receive Subsystems The interface between the media access control MAC and physical PHY laye...

Page 60: ...kes AXI4 Stream input data from the VPSS and converts it to memory mapped AXI4 format The output is connected to the HP1 high performance PS PL interface via an AXI interconnect For each video frame t...

Page 61: ...lter array overlaid on the silicon substrate enables CMOS image sensors to measure local light intensities that correspond to different wavelengths However the sensor measures the intensity of only on...

Page 62: ...apture pipelines all the IPs in this pipeline are configured to transport 2ppc enabling up to 2160p60 performance SDI RX Capture Pipeline The SDI RX capture pipeline is shown in Figure 5 6 The serial...

Page 63: ...engine that fetches data from memory and forwards it to the A V buffer manager The video layer can consist of up to three channels depending on the chosen pixel format whereas the graphics layer is a...

Page 64: ...col It does not support multi stream transport or other optional features The DisplayPort controller is responsible for managing the link and physical layer functionality The controller packs video da...

Page 65: ...rs The AXI4 Stream output interface is a 48 bit bus that transports 2 ppc for up to 2160p60 performance It is connected to the HDMI TX Subsystem input interface A GPIO is used to reset the subsystem b...

Page 66: ...transceivers as the physical layer The Video Mixer enables you to mix video layers and allows mixing up to four streaming or memory layers Each layer can be up to 4K resolution and can perform color s...

Page 67: ...reference clock to the transceiver is provided by the Si570 programmable oscillator available on the ZCU106 board For more information see 10G 25G High Speed Ethernet Subsystem Product Guide PG210 Re...

Page 68: ...high speed Ethernet Subsystem and AXI DMA each shared with the Ethernet 10G input capture pipeline Refer to Ethernet 10G Input Capture Pipeline for more information and for the configuration of each...

Page 69: ...Audio Formatter provides high bandwidth direct memory access between memory and AXI4 Stream target peripherals Initialization status and management registers are accessed through an AXI4 Lite slave in...

Page 70: ...5 13 The processing pipeline with a dummy SDx accelerator is entirely generated by the SDSoC tool based on the C code description The accelerator function which is simply copying the input data is tra...

Page 71: ...at help in reducing the compressed stream size thereby saves bandwidth The Video Scene Change Detection on IP core can read up to eight video streams in memory mode and one video stream in stream mode...

Page 72: ...e MPSoC VCU H 264 H 265 Video Codec unit Address Map Table 5 1 shows the address map for various IP blocks used in PL for the VCU TRD full fledged design Table 5 1 Address Map for IP Blocks of the VCU...

Page 73: ...udio Clock Recovery 0x00_A029_0000 64K Audio Formatter1 0x00_A005_2000 4K Audio Formatter2 0x00_A005_1000 4K AXI GPIO 0x00_A005_3000 4K AXI Interrupt Controller 0x00_A005_5000 4K HDMI ACR Control 0x00...

Page 74: ...HDMI Frame Buffer Write 1 0x00_A02C_0000 64K HDMI 1 4 2 0 Receiver Subsystem v2 0 0x00_A000_0000 64K HDMI 1 4 2 0 Transmitter Subsystem v2 0 0x00_A002_0000 128K Video Mixer 0x00_A007_0000 64K Video Pr...

Page 75: ...000 4K HDMI frame buffer read 0x00_A004_0000 64K Video frame buffer read 0x00_A00F_0000 64K HDMI frame buffer write 0 0x00_A001_0000 64K TPG frame buffer write 0x00_A00C_0000 64K Video frame buffer wr...

Page 76: ...rupt ID HDMI CTL IIC 94 HDMI Frame Buffer Read 89 HDMI Frame Buffer Write_0 90 HDMI Frame Buffer Write_1 108 HDMI Frame Buffer Write_2 109 HDMI RX 91 HDMI TX 93 Interrupt Controller 107 MIPI Frame Buf...

Page 77: ...I2C Controller 94 HDMI 1 4 2 0 Transmitter Subsystem v2 0 93 Video Mixer 95 HDMI Frame Buffer Read 89 HDMI Frame Buffer Write 0 90 HDMI 1 4 2 0 Receiver Subsystem v2 0 91 Video PHY Controller 92 VCU...

Page 78: ...traScale VCU TRD User Guide 78 UG1250 v2019 1 May 29 2019 www xilinx com Chapter 5 Hardware Platform VCU 94 Video Mixer 95 Table 5 9 Interrupt ID Map for SDI Design Cont d IP Core Interrupt ID Send Fe...

Page 79: ...ay interface Options HDMI SDI or DP Out Type Options display record and stream Display Rate Pipeline frame rate Options 30 or 60 Exit Tells the application that common configuration is finished Input...

Page 80: ...Raw Tells the pipeline to run the raw or processed pipeline Options True False Width Width of the live source Options 3840 1920 Height Height of the live source Options 2160 1080 Accelerator Flag Ena...

Page 81: ...der latency mode Options normal sub_frame Low Bandwidth If enabled decreases the vertical search range used for P frame motion estimation to reduce the bandwidth Options True False GoP Mode Group of P...

Page 82: ...configuration is finished Record Configuration Starting point of a record configuration Record Num Starting nth record configuration Options 1 8 Out File Name Record file path Options media usb abc ts...

Page 83: ...els Options 2 Volume Sets the volume level Options 0 0 10 0 Source Required audio source Options HDMI SDI and I2S Renderer Required audio sink Options HDMI SDI I2S and DP Exit Indicates to the applica...

Page 84: ...deos and support resources which you can filter and search to find information To open the Xilinx Documentation Navigator DocNav From the Vivado integrated design environment IDE select Help Documenta...

Page 85: ...Development Kit XSDK 5 OpenMAX website 6 Advanced Linux Sound Architecture ALSA project homepage 7 Zynq UltraScale MPSoC Software Developer Guide UG1137 8 Zynq UltraScale Device Technical Reference M...

Page 86: ...assumes no obligation to correct any errors contained in the Materials or to notify you of updates to the Materials or to product specifications You may not reproduce modify distribute or publicly di...

Search results

Summary of Contents for Zynq UltraScale+

Reviews:

Related manuals for Zynq UltraScale+

Brands by name

Popular brands

Xilinx Zynq UltraScale+, User Manual

Search results

Summary of Contents for Zynq UltraScale+

Reviews:

Related manuals for Zynq UltraScale+

Brands by name

Popular brands