ADLINK Technology EVA SDK User Manual Download Page 76

Page: 76 / 117

Image Processing Plugins

4.1.36 nvh265enc

NVENC H.265 Video Encoder.

Note

1. Only supported on NVIDIA platforms.

2. Only supported on Windows. Refer to the EVA SDK Installation Guide for Windows to install the NVIDIA

Solution and the third-party plugin before using nvh265enc.

3. It is recommended to use the nvv4l2h265enc element in the NVIDIA DeepStream on Linux.

4.1.36.1 Example Using gst-launch

gst-launch-1.0 videotestsrc num-buffers=2000 ! nvh265enc ! h265parse !
matroskamux ! filesink location=videotestsrc.mkv

4.1.36.2 Hierarchy

GObject

╰

──

GInitiallyUnowned

╰

──

GstObject

╰

──

GstElement

╰

──

GstVideoEncoder

╰

──

GstNvBaseEnc

╰

──

GstNvH265Enc

Summary of Contents for EVA SDK

Page 1: ...EVA SDK User s Manual Manual Rev 1 2 Revision Date July 20 2021 Part Number 50M 00010 1020...

Page 2: ...nd function and does not represent a commitment on the part of the manufacturer In no event will the manufacturer be liable for direct indirect special incidental or consequential damages arising out...

Page 3: ...Element Descriptions 76 6 Connector Plugins 103 6 1 Element Descriptions 103 7 Convert for OpenVINO Model 105 7 1 Model Optimizer Developer Guide 105 7 2 Converting a Caffe Model 105 7 3 Converting a...

Page 4: ...iv Preface This page intentionally left blank...

Page 5: ...be categorized as follows Image video capture Supports different image capture solutions with different camera standards Image processing Provides the required algorithms needed before or after the in...

Page 6: ...with a single image source on an ADLINK edge AI platform camera e g Neon 1 Application 1 captures the image from a GigE camera 2 Application 1 does AI inference on Nvidia TensorRT via adrt plugin 3 Ap...

Page 7: ...drt plugin 7 Application 1 does the inference result interpretation with the adtranslator plugin Notes The examples using gst launch in this User s Manual are for reference only The property values su...

Page 8: ...4 Introduction This page intentionally left blank...

Page 9: ...K EVA SDK If the software has no corresponding libraries the script will not set up the corresponding environment variables Note The environment variables are removed when closing the terminal On Wind...

Page 10: ...6 Setting Environment Variables This page intentionally left blank...

Page 11: ...tail 3 1 1 pylonsrc This is a GStreamer element that uses Basler s USB3 Vision cameras and Basler s GigE Vision cameras for image capture When multiple cameras are available use the camera parameter t...

Page 12: ...mera source or image resolution to make the rendered screen output smoother 3 1 1 1 Examples Using gst launch Displaying a pylon video with ycbcr422_8 format gst launch 1 0 pylonsrc pixel format ycbcr...

Page 13: ...amera will remember this setting and will use values from the previous runs if you relaunch without specifying this parameter Reconnect the camera or use the reset parameter to reset Range 0 to 10000...

Page 14: ...tting will have no effect if limitbandwidth is set to off Note that the camera will remember this setting and will use values from the previous runs if you relaunch without specifying this parameter R...

Page 15: ...Flags Read Write Default value 5000k autoexposure autoexposure gchararray off once continuous Controls whether the camera will try to adjust the exposure settings Setting this parameter to anything b...

Page 16: ...balanceblue balanceblue gdouble Specifies the blue color balance The autowhitebalance setting must be set to off for this property to have any effect Note that this value gets saved on the camera and...

Page 17: ...ue gdouble Specifies the green hue Note that this value gets saved on the camera and running this plugin again without specifying this value will cause the previous value to be used Use the reset para...

Page 18: ...the blue saturation Note that this value gets saved on the camera and running this plugin again without specifying this value will cause the previous value to be used Use the reset parameter or reconn...

Page 19: ...is sent to the computer The value of this parameter will be saved to the camera but it will be set to 0 every time this plugin is launched without specifying this parameter Reconnect the camera or us...

Page 20: ...ines the pixel format for sending frames The default is auto which will query the camera for supported pixel formats and allow GStreamer to negotiate the format with downstream elements Flags Read Wri...

Page 21: ...Flags Read Write Default value 99999 centerx centerx gboolean true false Setting this to true will center the horizontal offset and cause the plugin to ignore the offsetx value Flags Read Write Defau...

Page 22: ...gdouble 105 to 1000000 Sets the lower limit for the auto exposure function Flags Read Write Default value 999 gainupperlimit gainupperlimit gdouble 0 to 12 00921 Sets the upper limit for the auto gain...

Page 23: ...ctors Flags Read Write Default value default transformation00 transformation00 gdouble Gain00 transformation selector Range 8 0 to 7 96875 Flags Read Write Default value 999 transformation01 transform...

Page 24: ...n21 transformation selector Range 8 0 to 7 96875 Flags Read Write Default value 999 transformation22 transformation22 gdouble Gain22 transformation selector Range 8 0 to 7 96875 Flags Read Write Defau...

Page 25: ...upported on Linux It is recommended to use the glimagesink or d3dvideosink elements to display the window on Windows 3 1 3 rtspsrc This element receives data over a network via RTSP RFC 2326 For more...

Page 26: ...he window on Windows 3 1 5 multifilesrc This element reads buffers from sequentially named files If used together with an image decoder the caps property or a capsfilter must be used to force to caps...

Page 27: ...t of time handling frames If the camera s frame rate is too fast the pipeline will drop a lot of frames resulting in rendering delays However the rendering and AI inference performance of each platfor...

Page 28: ...2rgb element after the hikplugin element for decoding 3 1 7 2 Hierarchy GObject GInitiallyUnowned GstObject GstElement GstBaseSrc GstPushSrc AdCamerahik 3 1 7 3 Pad Templates src ANY Presence always D...

Page 29: ...era gint If multiple cameras are enumerated this will allow selection of a specific camera Range 0 to 10000 Flags Read Write Default value 1 continuous continuous gboolean true false Toggles between t...

Page 30: ...d Write Default value 0 gain gain gdouble Sets the gain added to a camera before sending a frame to the computer GigE 0 96 100000 USB3 0 4 1000 Range 0 100000 0 Flags Read Write Default value 0 Note S...

Page 31: ...l format pixel format gchararray Sets the camera output data pixel format value Valid values include YUV422_8_UYVY BayerRG8 BayerBG8 BayerGB8 BayerGR8 RGB8Packed and Mono8 If an invalid value is passe...

Page 32: ...1 0 flirplugin videoconvert glimagesink Use the pixel format property if the camera module supports other pixel fomats The example is YUV422_8_UYVY gst launch 1 0 flirplugin pixel format YUV422_8_UYVY...

Page 33: ...r the camera can adjust the exposure settings Flags Read Write Default value Once autogain autogain gchararray Off Once Continoust Controls whether the camera can adjust the gain settings Flags Read W...

Page 34: ...inuous gboolean true false Toggles between triggered and continuous mode True is for continuous mode Flags Read Write Default value false exposure exposure gdouble Sets the custom exposure value GigE...

Page 35: ...e Default value 0 gamma gamma gdouble Sets the gamma correction value GigE 0 5 4 USB3 0 25 4 Range 0 to 1000 0 Flags Read Write Default value 1 width gain gint64 Sets the image width Range 0 to 10000...

Page 36: ...ist of the supported formats The default pixel format is NULL and is retrieved from the camera during initialization Flags Read Write Default value NULL setfps setfps gboolean true false Setting this...

Page 37: ...bva GStreamer 1 14 5 1 16 2 libav bmp encoder avdec_bmp libva GStreamer 1 14 5 1 16 2 libav bmp decoder videorate gstvideorate GStreamer 1 14 5 1 16 2 Drops duplicates adjusts timestamps on video fram...

Page 38: ...A DeepStrea m 5 1 5 0 Encodes H 264 video streams via V4L2 API nvv4l2h265enc nvvideo4linux2 NVIDA DeepStrea m 5 1 5 0 Encodes H 265 video streams via V4L2 API rotate geometrictransform GStreamer 1 14...

Page 39: ...t launch 1 0 filesrc location frameJ1 jpeg jpegdec videoconvert avenc_bmp filesink location frameJB1 bmp gst launch 1 0 filesrc location frameJ1 jpeg jpegdec videoconvert filesink location frameJR1 ra...

Page 40: ...k location framePJ1 jpeg Similarly png images can be transcoded to tiff bmp or raw image formats gst launch 1 0 filesrc location frameP1 png pngdec videoconvert avenc_tiff filesink location framePT1 t...

Page 41: ...1 tiff gst launch 1 0 filesrc location frameT1 tiff gdkpixbufdec videoconvert avenc_bmp filesink location frameTB1 bmp gst launch 1 0 filesrc location frameT1 tiff gdkpixbufdec videoconvert filesink l...

Page 42: ...e It is recommended to use gdkpixbufdec to decode tiff files in Linux 4 1 9 1 Example Using gst launch This pipeline transcodes a bmp to jpeg and saves it as a jpeg file gst launch 1 0 multifilesrc lo...

Page 43: ...aussianblur videoconvert xvimagesink To change the frame rate to 5 before applying xvimagesink add the videorate element before it The videorate will change the frame rate to adopt the coming media ty...

Page 44: ...e to operate anywhere in a pipeline For more details refer to the official documentation at https gstreamer freedesktop org documentation videoscale index html gi language c videoscale page 4 1 12 1 E...

Page 45: ...e buffering in the encoder This may cause problems with pipeline stalling in non trivial pipelines because the encoder latency is often considerably higher than the default size of a simple queue elem...

Page 46: ...entation libav avdec_h264 html gi language c 4 1 16 1 Example Using gst launch This pipeline decodes an H 264 video file gst launch 1 0 filesrc location sample_1080p_h264 mp4 qtdemux h264parse queue a...

Page 47: ...inux It is recommended to use the glimagesink or d3dvideosink elements to display the window on Windows 4 1 18 msdkh264enc This element is an Intel Media SDK H 264 encoder Notes 1 Only supported on In...

Page 48: ...irection sink src video x h264 framerate 0 1 2147483647 1 width 1 2147483647 height 1 2147483647 stream format byte stream alignment au profile string high string main string baseline string constrain...

Page 49: ...ariable Bitrate cqp 3 Constant Quantizer avbr 4 Average Bitrate la_vbr 8 VBR with look ahead Non HRD compliant icq 9 Intelligent CQP vcm 10 Video Conferencing Mode Non HRD compliant la_icq 11 Intellig...

Page 50: ...convergence convergence guint The AVBR convergence per 100 frames Range 0 to 65535 Flags Read Write Default value 0 rc lookahead rc lookahead guint Number of frames to look ahead for rate control Ran...

Page 51: ...rames Range 0 to 2147483647 Flags Read Write Default value 1 i frames i frames guint Number of I frames between IDR frames Range 0 to 2147483647 Flags Read Write Default value 0 b frames b frames guin...

Page 52: ...ve I Frame Insertion control Flags Read Write Default value 32 off GstMsdkEncAdaptiveI Members 0 auto SDK decides what to do 32 off Disable Adaptive I frame insertion 16 on Enable Adaptive I frame ins...

Page 53: ...ng Members 1 none default 3 side by side 7 top bottom rc lookahead ds rc lookahead ds GstMsdkEncRCLookAheadDownsampling Downsampling mode in look ahead bitrate control Flags Read Write Default value 0...

Page 54: ...e structure Flags Read Write Default value false 4 1 19 msdkh265enc This element is an Intel Media SDK H 265 encoder Notes 1 Only supported on Intel platforms 2 Refer to the EVA SDK Installation Guide...

Page 55: ...nterlace mode progressive Presence always Direction sink src video x h265 framerate 0 1 2147483647 1 width 1 2147483647 height 1 2147483647 stream format byte stream alignment au profile main Presence...

Page 56: ...rate cqp 3 Constant Quantizer avbr 4 Average Bitrate la_vbr 8 VBR with look ahead Non HRD compliant icq 9 Intelligent CQP vcm 10 Video Conferencing Mode Non HRD compliant la_icq 11 Intelligent CQP wit...

Page 57: ...ult value 0 convergence convergence guint The AVBR convergence per 100 frames Range 0 to 65535 Flags Read Write Default value 0 rc lookahead rc lookahead guint Number of frames to look ahead for rate...

Page 58: ...s Read Write Default value 1 i frames i frames guint Number of I frames between IDR frames Range 0 to 2147483647 Flags Read Write Default value 0 b frames b frames guint Number of B frames between I a...

Page 59: ...eosink elements to display the window on Windows 4 1 20 2 Hierarchy GObject GInitiallyUnowned GstObject GstElement GstVideoDecoder GstMsdkDec GstMsdkH264Dec 4 1 20 3 Pad Templates sink video x h264 wi...

Page 60: ...K Installation Guide to install OpenVino toolkits and Media SDK for GStreamer before using the MSDK plugin 4 1 21 1 Example Using gst launch This pipeline decodes and displays an H 265 format image gs...

Page 61: ...ys Direction sink src video x raw format string NV12 framerate 0 1 2147483647 1 width 16 2147483647 height 16 2147483647 interlace mode progressive Presence always Direction src 4 1 21 4 Properties ha...

Page 62: ...decodes and displays an H 265 format image gst launch 1 0 videotestsrc admetawriter devinfo dev1 txt count frame TRUE videoconvert ximagesink Note The element xvimagesink is only supported on Linux I...

Page 63: ...B16 RGB15 GRAY8 NV12 NV21 GRAY16_LE GRAY16_BE width 0 2147483647 height 0 2147483647 framerate 0 1 2147483647 1 Presence always Direction sink 4 1 22 4 Properties devinfo devinfo gchararray Provides t...

Page 64: ...enum describing how the device communicates V4L2 10 serial Vision Device serial identifier AD0123456 11 status DeviceStatus enum OASYS defined OK 12 stream_id Stream publisher ID Stream1 13 uri Video...

Page 65: ...R AYUV YUY2 YVYU UYVY I420 YV12 RGB16 RGB15 GRAY8 NV12 NV21 GRAY16_LE GRAY16_BE width 0 2147483647 height 0 2147483647 framerate 0 1 2147483647 1 Presence always Direction sink src video x raw format...

Page 66: ...FrameData structure The idx is the order of the input video frame Class information is from the ClassificationResult structure Box information is from the DetectionBoxResult structure Seg information...

Page 67: ...tly only able to write box or class metadata Only one inference result can be written at a time To write multiple inference results to metadata the element can be duplicated as many times as necessary...

Page 68: ...ta Flags Read Write Default value 0 class DataTypePattern Members 0 class Class data id class prob 1 box Box data id x1 y1 x2 y2 prob id id guint Id of inference data Flags Read Write Default value 0...

Page 69: ...width so the value is between 0 and 1 Flags Read Write Default value 0 0 x2 x2 gfloat The X axis coordinate is at the right side of the bounding box It should be a ratio of the width so the value is b...

Page 70: ...coding engine available on Jetson and dGPU platforms by interfacing with libv4l2 plugins The plugin accepts RAW data in I420 format It uses the NVENC hardware engine to encode RAW input Encoded output...

Page 71: ...The element xvimagesink is only supported on Linux It is recommended to use the glimagesink or d3dvideosink elements to display the window on Windows 4 1 30 gamma This element performs gamma correcti...

Page 72: ...age c 4 1 32 1 Example Using gst launch This pipeline flips the test image 90 degrees clockwise gst launch 1 0 videotestsrc videoflip method clockwise videoconvert xvimagesink Note The element xvimage...

Page 73: ...in the NVIDIA DeepStream on Linux 4 1 34 1 Example Using gst launch gst launch 1 0 filesrc location 4k_h264 mkv matroskademux h264parse nvdec videoconvert glimagesink sync false 4 1 34 2 Hierarchy GO...

Page 74: ...e 1 Only supported on NVIDIA platforms 2 Only supported on Windows Refer to the EVA SDK Installation Guide for Windows to install the NVIDIA Solution and the third party plugin before using nvh264enc...

Page 75: ...444 width 16 4096 height 16 4096 framerate 0 1 2147483647 1 interlace mode string progressive string mixed string interleaved Presence always Direction sink src video x h264 width 16 4096 height 16 40...

Page 76: ...NVIDIA Solution and the third party plugin before using nvh265enc 3 It is recommended to use the nvv4l2h265enc element in the NVIDIA DeepStream on Linux 4 1 36 1 Example Using gst launch gst launch 1...

Page 77: ...te 0 1 2147483647 1 Presence always Direction sink src video x h265 width 16 4096 height 16 4096 framerate 0 1 2147483647 1 stream format byte stream profile string main Presence always Direction src...

Page 78: ...74 Image Processing Plugins This page intentionally left blank...

Page 79: ...5 3 5 ADLINK inference s fcn translator adtrans_openpose_py python ADLINK 3 5 3 5 ADLINK inference s openpose translator adrt adrt ADLINK 3 5 3 5 Initializes TensorRT inference engine plugins to infe...

Page 80: ...otestsrc advino model classification xml device GPU adtrans_classifier class num 1000 label label txt ximagesink The following pipeline translates the TensorRT inference result using the classificatio...

Page 81: ...list will be seen as an inference result 5 1 2 1 Examples Using gst launch The following pipeline translates the OpenVINO inference result using the ssd type model into a human readable format gst lau...

Page 82: ...lways Direction src 5 1 2 4 Properties batch num batch num gint The number of the deep learning model batch Range 1 to 256 Flags Read Write Default value 1 max count max count gint The maximum count o...

Page 83: ...e format gst launch 1 0 videotestsrc advino model segment xml device GPU adtrans_segment blob height 1024 blob width 2048 only max prob true ximagesink The following pipeline translates the TensorRT i...

Page 84: ...256 Flags Read Write Default value 1 class num class num gint The class number of the deep learning model Range 1 to 65535 Flags Read Write Default value 1 blob width blob width gint The blob width o...

Page 85: ...g pipeline translates the OpenVINO inference result using the yolov3 type model into a human readable format gst launch 1 0 videotestsrc advino model yolov3 xml device GPU adtran_yolo label yolo_VINO_...

Page 86: ...arning model Range 1 to 256 Flags Read Write Default value 1 class num class num gint The class number of the deep learning model Range 1 to 65535 Flags Read Write Default value 80 blob size blob size...

Page 87: ...116 90 156 198 373 326 input width input width gint Input width of image Range 1 to 65535 Flags Read Write Default value 416 input height input height gint Input height of image Range 1 to 65535 Flag...

Page 88: ...anslates the TensorRT inference result using the fcn type model into a human readable format gst launch 1 0 videotestsrc adrt model FCN Alexnet Cityscapes SD_tx2 engine scale 1 0 mean 0 0 0 device 0 b...

Page 89: ...efault value 21 input width input width gint The FCN deep learning model input width Range 1 to 65535 Flags Read Write Default value 1024 input height input height gint The FCN deep learning model inp...

Page 90: ...a human readable format Used mode reference at https github com Daniil Osokin lightweight human pose estimation pytorch gst launch 1 0 videotestsrc adrt model pose b1 engine scale 0 0039 mean 128 128...

Page 91: ...can also define blob sizes by WxH for example 26x42 52x68 13x19 Flags Read Write Default value NULL input width input width gint Input width of the pose estimation deep learning model Range 1 to 6553...

Page 92: ...on NVIDIA platforms This element must connect with adtranslator plugins to handle inference data or the pipeline will have unexpected results 5 1 7 1 Example Using gst launch gst launch 1 0 videotests...

Page 93: ...ly to input data Scale is to normalize the input to corresponding range Please refer to https en wikipedia org wiki Feature_scaling Flags Read Write Default value 1 0 mean mean gchararray Sets the mea...

Page 94: ...5 MYRIAD Reference OpenVINO support devices link Some MYRIAD boards contain multiple MYRIAD cores so users can inference a specific core or select one via the OpenVINO inference scheduler The followi...

Page 95: ...plates sink video x raw format BGR width 0 2147483647 height 0 2147483647 framerate 0 1 2147483647 1 video x raw ANY format BGR width 0 2147483647 height 0 2147483647 framerate 0 1 2147483647 1 Presen...

Page 96: ...ce device gchararray Deep learning device ex CPU GPU or MYRIAD User may also using following command to list available devices python3 c from openvino inference_engine import IECore print IECore avail...

Page 97: ...ch meta data 5 1 9 1 Example Using gst launch gst launch 1 0 advideobatch name mix videoconvert ximagesink videotestsrc pattern red admetawriter devinfo dev1 txt count frame TRUE mix sink_0 videotests...

Page 98: ...with admeta data in its stream Note The fps of the stream at sink pad will be equal to the number of batched streams multiplied with the fps of the src stream For example a stream of 30fps with a batc...

Page 99: ...th the following parameters 1 Onnx model path 2 Scale must be a float number 3 Mean must be a string of 3 numbers representing a vector of mean values used in training 4 Standard deviation must be a s...

Page 100: ...ob size 38 32 57 19 32 57 admetadrawer videoconvert ximagesink gst launch 1 0 videotestsrc pattern snow adonnx model yolov3 onnx scale 0 0039 mean 128 128 128 adtran_yolo label yolo_RT_labels txt blob...

Page 101: ...s Read Write Default value NULL scale scale gfloat Sets the scale to apply to input data to normalize the input of the corresponding range Refer to https en wikipedia org wiki Feature_scaling Flags Re...

Page 102: ...rite Default value gpu 0 inps inps gint Specifies which input size is used Only use with a model that does not list input size of the input layer If set to 0 uses the model defined size Flags Read Wri...

Page 103: ...videoconvert ximagesink Note The element ximagesink is only supported on Linux It is recommended to use the glimagesink or d3dvideosink elements to display the window on Windows 5 1 12 2 Hierarchy GOb...

Page 104: ...mask mask gchararray Yolo mask Flags Read Write Default value 0 1 2 3 4 5 6 7 8 anchor anchor gchararray Yolo anchor Flags Read Write Default value 0 573 0 677 1 87 2 06 3 34 5 47 7 88 3 53 9 77 9 17...

Page 105: ...erence result using classification model from Microsoft Custom Vision into a human readable format gst launch 1 0 videotestsrc adonnx model classification onnx mean 0 0 0 std 1 1 1 rgbconv False nchw...

Page 106: ...102 Image Analytic Plugins 5 1 13 4 Properties batch num batch num gint The number of the deep learning model batch Range 1 to 256 Flags Read Write Default value 1...

Page 107: ...s located in EVA_INSTALLED_PATH bin third party for testing For more details refer to the official documentation at https github com aler9 rtsp simple server Set publishUser and publishPass as adlinkp...

Page 108: ...rtspclientsink location rtsp adlinkpublisher adlinkpublisher SERVERIP 8554 streamName The relevant rtspsrc to receive the data gst launch 1 0 rtspsrc location rtsp adlinkreader adlinkreader SERVERIP 8...

Page 109: ...penvinotoolkit org latest _docs_MO_DG_prepare_model_convert_model_Convert_Model_From_Caff e html 7 3 Converting a TensorFlow Model For more details on converting a TensorFlow Model refer to the offici...

Page 110: ...106 Convert for OpenVINO Model This page intentionally left blank...

Page 111: ...ster samples opensource trtexec If TensorRT is installed trtexec is prebuilt and installed in the following path On Linux usr src tensorrt bin trtexec On Windows TensorRT install Path bin trtexec exe...

Page 112: ...he Tensorflow models must be converted to a UFF model or an ONNX model 8 2 1 Convert Tensorflow to UFF For more details on converting a TensorFlow model stream to a UFF mode refer to the official docu...

Page 113: ...more information fp16 Set model to fp16 precision buildOnly Reduce 10s for testing model and only build and save engine Optional 8 3 2 Convert googlenet caffe model to FP32 or FP16 engine file Downlo...

Page 114: ...w maximum input batch size 4 saveEngine googlenet engine Save the final engine to googlenet engine file fp16 Set model to fp16 precision verbose Display more information buildOnly Reduce 10s for testi...

Page 115: ...16 8 3 3 3 Parameters Parameter Description uff Load uff file path uffInput Input 3 300 300 Set input layer name and input layer dimension output NMS Set output layer name workspace 3000 Set GPU memor...

Page 116: ...s To avoid electrical shock and or damage to equipment Keep equipment away from water or liquid sources Keep equipment away from high heat or high humidity Keep equipment properly ventilated do not bl...

Page 117: ...el 1 408 360 0200 Toll Free 1 800 966 5200 USA only Fax 1 408 360 0222 Email info adlinktech com ADLINK Technology China Co Ltd Address 300 Fang Chun Rd Zhangjiang Hi Tech Park Pudong New Area Shangha...

Search results

Summary of Contents for EVA SDK

Reviews:

ADLINK Technology EVA SDK, User Manual, Page: 76 / 117

Search results

Summary of Contents for EVA SDK

Reviews: