Nielsen ID3 User Manual Download Page 2

Page: 2 / 67

ID3 Tag Validator Application User Manual

Release 1.8 Revision F

Nielsen Confidential

Nielsen and the Nielsen Logo are trademarks or registered trademarks of The Nielsen Company (US), LLC.

Other company names, products and services may be trademarks or registered trademarks of their respective
companies.

This documentation contains the intellectual property and proprietary information of The Nielsen Company (US), LLC.
Publication, disclosure, copying, or distribution of this document or any of its contents is prohibited.

Detailed Revision History

Revision

Date

Changes Made

Responsible Engineer

2015-01-26

Initial Draft

Lois Price

2015-05-15

Revised for Release 1.5

Lois Price

Pre-release

2016-02-03

Revised for beta release of 1.7

Lois Price
Lore Eargle (editor)

2016-02-08

Production release of 1.7

Lois Price
Lore Eargle (editor)

2017-04-05

Internal Release of 1.8: added MPEG-DASH

Lois Price

Lore Eargle (editor)

2017-06-29

Updated Figure 3 – Summary Reports Contents and
Figure 5 – SID List Contents

Lois Price

Lore Eargle (editor)

Harish GP (reviewer)

2017-09-27

Clarified examples of –d, -df, and –dr input segment
designators

Lois Price

Lore Eargle (editor)

Harish GP (reviewer)

Julia Wen (reviewer)

Summary of Contents for ID3

Page 1: ...Nielsen Confidential ID3 Tag Validator Application User Guide Release 1 8 Revision F...

Page 2: ...this document or any of its contents is prohibited Detailed Revision History Revision Date Changes Made Responsible Engineer A 2015 01 26 Initial Draft Lois Price B 2015 05 15 Revised for Release 1 5...

Page 3: ...put Folder Specifier o 14 Report Type Specifier p 14 Program Number Specifier n 15 Using Validator Output 15 Validator Output Summary Report 16 Overview 16 File Header 17 Description and Example 17 Di...

Page 4: ...Owner ID Check 39 Test 26 Result Metadata PES PTS Check 39 Test 27 Result PES Header Test 40 Test 28 Result PES Incomplete Tag Test 41 Tests 29 and 30 Result PMT Elementary Stream and Descriptor Chec...

Page 5: ...round 59 Symptoms 59 Suggestions for Follow up 59 Missing or Invalid INFO Tag Settings 59 Background 59 Missing or Invalid Breakout Type Setting 60 Background 60 Suggestions for Follow up 60 Unexpecte...

Page 6: ...ielsen Confidential Table 2 p Option Settings 14 Table 3 Default Output Files 16 Table 4 Interval in Seconds 19 Table 5 NAES 2 EDU Description 47 Table 6 NAES 6 EDU Description 48 Table 7 Comma Delimi...

Page 7: ...plication or the log file of a mobile device that processes Nielsen ID3 tags option r Process specific MPEG DASH segments in a folder guided by the user s command line instructions to guarantee that t...

Page 8: ...Glossary on page 62 Related Documents Apple iOS Developer Library Timed Metadata for HTTP Live Streaming https developer apple com library ios documentation AudioVideo Conceptual HTTP _Live_Streaming_...

Page 9: ...the primary tool used in evaluating the files that you deliver to Nielsen for analysis Before delivering the test files to Nielsen you may use Validator to process the files yourself Note the followi...

Page 10: ...CM to ID3 SDK and stamps each buffer with the current time Ensure the current time advances by 1 second for every second of audio processed Potential and most frequent error The application feeds syst...

Page 11: ...014 E for a detailed explanation of emsg box format and signaling The DASHEventMessageBox scheme_id_uri should be set to the null terminated string www nielsen com id3 v1 The value field must be set t...

Page 12: ...Confidential Figure 1 is a visual representation of the tagging process for HLS and MPEG DASH Streams The green boxes represent the functionality your application provides The blue box represents func...

Page 13: ...ng the ID3 tags p 6 Note Validator locates and analyzes all ID3 Tags in the log file and ignores all text entries between tags o Use t for a transport stream with Nielsen ID3 tags muxed into it Nielse...

Page 14: ...st option For a single file holding a set of MPEG DASH segments where some of the segments hold emsg boxes containing Nielsen ID3 tags you can use the d command as follows NielsenId3TagValidator d ful...

Page 15: ...g fields o Breakout type o List of PC Program Content and FD Final Distributor SIDs o Results of the 30 tests described in Test Results on page 21 Log File always generated regardless of p setting the...

Page 16: ...to report only on ID3 tags associated with program 2 in a multi program transport stream Note that Validator lists identifying information for all programs in the PAT and PMT even if the program numb...

Page 17: ...re 2 below shows an example of a Summary Report generated when the application was run with the m option The Summary Report has four major components which are described in detail in the sections that...

Page 18: ...0 1 INFO Tag SDK Version Test Pass 0 0 1 INFO Tag Distributor ID Test Pass 0 0 1 MPEG 2 Tag Placement Test No Status 0 0 0 MPEG 2 Tag Time Test Pass 0 0 6 MPEG 2 PES Packet Header Test Pass 109 81 0 M...

Page 19: ...r all of the watermarked content that it processed Note The reported duration in seconds is calculated by subtracting the tag time of the first decoded ID3 tag from the tag time of the last decoded ID...

Page 20: ...AES 6 PC FD PC FD1 FD2 FD3 PC Only 4 N A 4 8 N A N A N A PC FD 4 4 4 8 4 8 N A N A FD Only N A 2 N A 4 8 N A N A FD FD N A 2 N A 4 8 4 8 N A FD FD FD N A 2 N A 4 8 4 8 4 8 PC FD FD 4 4 4 8 4 8 4 8 N A...

Page 21: ...the SID List of the Summary Report may well be reflected in a reduced tally of DATA tags See Test 1 Result DATA Tag Count on page 21 INFO Tag Listing Description and Example The INFO Tag listing is th...

Page 22: ...s but it is not an indication that the test failed Test 1 Result DATA Tag Count Format Format count of DATA tags minimum DATA tags required 0 Count Number found in stream Minimum Required Minimum requ...

Page 23: ...g Validator inspects the placement of the first three forward slash characters after the www nielsen com string in the Nielsen ID3 tag It also verifies the correct placement of the last characters in...

Page 24: ...ted 0 Not used Maximum Tolerated Maximum number of errors tolerated before failure is declared Meaning The sequence number check indicates the number of times that Validator detected a sequence number...

Page 25: ...imes and watermark times of the two systems however should be close A tag overlap or a tag gap of up to 9 seconds may exist at the point of the fail over If there are sequence number errors but no tag...

Page 26: ...e gap error Test 5 Result Tag Duration Test Format Format number of tag duration errors 0 maximum tolerated Number of Errors Number of errors detected 0 Not used Maximum Tolerated Maximum number of er...

Page 27: ...Nielsen timestamp both derived from recent Nielsen watermarks Consequently the following must occur If the ID3 tag includes valid PC watermarks the PC CID SID must be set to the value of the PC water...

Page 28: ...f CID timestamp errors 0 maximum tolerated Number of Errors Number of errors detected 0 Not used Maximum Tolerated Maximum number of errors tolerated before failure is declared Meaning Important The P...

Page 29: ...tween one of the tag s EDUs and the CID timestamp Valid EDUs within the tag should fall within 10 seconds before or after the sum of the CID timestamp plus the tag offset For example if the CID timest...

Page 30: ...t on page 30 The PCM to ID3 SDK may have reset the offset to 0 because watermark SIDs are in transition Although reported as an error a setting of 0 in this case is acceptable if it lasts for only one...

Page 31: ...setting are tallied as watermark time warnings These observations regarding watermarks are counted as warnings not errors because ID3 tag inserter applications are not responsible for generating water...

Page 32: ...a SID changes Note that a level change from PC FD followed by a change from FD PC may be counted as two warnings although it may indicate only one error If test 11 fails take this double counting int...

Page 33: ...the number of INFO tags that occurred during the entire length of processed content as compared to the minimum number of INFO tags that must be seen for the test to pass Periodic Reports The INFO tag...

Page 34: ...correctly setting the currentTime argument to the InputAudioData function The currentTime value must advance 1 second for each second of audio content processed If the value of currentTime is not corr...

Page 35: ...plication logs the following error message 53 INFO tag is missing or late sequence number xx See Possible Causes of Failure for Test 12 on page 33 Tests 14 and 15 INFO TAG CID and Offset Tests Format...

Page 36: ...six of the seven INFO tag fields Vendor ID Device Type Device ID System Version SDK Version Distributor ID This is the only field for which a specific string is required In most cases it should contai...

Page 37: ...Tag Distributor ID Test Pass 0 0 1 Figure 6 INFO Tag Tests Test 22 Result Tag Placement Check Tests 22 through 30 are MPEG 2 transport stream checks Validator runs the tests only if the content being...

Page 38: ...d tag times are out of sync 0 maximum tolerated Number of Errors Number of errors detected 0 Not used Maximum Tolerated Maximum number of errors tolerated before failure is declared Meaning PTS Presen...

Page 39: ...or counts the number of PES packet headers that are associated with the Nielsen ID3 tag metadata stream There should be at least one of these PES packet headers every 10 seconds If non Nielsen ID3 tag...

Page 40: ...ror check Validator does not log a message when it detects a Nielsen owner ID Example The example below shows that Validator found 319 Nielsen owner IDs In most cases the total number is equal to the...

Page 41: ...Errors Number of errors detected 0 Not used Maximum Tolerated Maximum number of errors tolerated before failure is declared Meaning Each Nielsen metadata PES packet header must have a stream ID of 0x...

Page 42: ...r An incomplete payload cannot be decoded Each Nielsen ID3 tag spans two transport packets If the size of the payload as specified in the PES header is correct but the second transport packet contribu...

Page 43: ...PMT elementary stream loop Each time that the decoder finds a stream of type 0x15 in the loop Validator counts it as a metadata PES Each time that it finds an ID3 tag descriptor Validator counts it a...

Page 44: ...you called Validator with the d option the EMSG string is displayed but the segment names are not displayed Tag Number 7 Transport Pid 259 0x103 PTS seconds 65 PC CID Ma0O2J3MrcrfObLJMbVUYQ 9020 28008...

Page 45: ...roper crediting The PTS should advance at roughly the same rate as the First and Last tag times PC CID The PC CID is a 24 byte string that holds two encrypted values SID and timestamp Under ideal cond...

Page 46: ...e watermarks to which the PC CID SID corresponds They are not derived from the current time that your application supplies to the PCM to ID3 SDK Seconds since FD Day Start FD CID Offset The FD CID off...

Page 47: ...red to your application On average Nielsen ID3 tags should be 10 seconds long If the PCM to ID3 SDK receives 10 or more EDUs in less than 10 seconds however the tag ends promptly after the tenth EDU i...

Page 48: ...mark SID 9020 SID for source watermark is 9020 The PC CID is also 9020 Stacked U Watermark is unstacked which means that all reported fields are generally more reliable than those of a stacked S water...

Page 49: ...rce watermark is 9120 The FD CID is also 9120 Stacked U Watermark is unstacked which means that all reported fields are generally more reliable than those of a stacked S watermark TS TIC 180039658 Raw...

Page 50: ...663930087 STD FD 8107 6 01 15 2016 09 01 27 TIME N2 9999 U 663930089 STD FD 8107 6 01 15 2016 09 01 29 TIME N2 9999 S 663929984 STD UK 3993 2 01 15 2016 08 59 44 TIME Figure 7 Tags txt File In example...

Page 51: ...an indication that the ID3 tag was dropped or delivered out of order sometime after the PCM to ID3 SDK generated it It is acceptable for INFO Tags to be delivered out of order However DATA tags shoul...

Page 52: ...y watermarked the tag offsets should be spaced at intervals of 12 seconds or less Watermark PC SID 18 1444645043 155 9020 2800909011 9020 2800851248 57763 9120 2800909007 9120 2800851248 57759 168 The...

Page 53: ...The PC CID Timestamp is the decrypted timestamp component of the 24 byte PC Content ID Under ideal conditions the PC CID timestamp is a raw NAES 2 timestamp that translates to the start of the current...

Page 54: ...decoding errors may cause one or more bits of the timestamp to be incorrectly set In a spreadsheet you can easily detect the faulty timestamp FD CID SID The FD CID SID is the decrypted SID component...

Page 55: ...ve shows the tag with sequence number 21 which has a badly decoded FD timestamp As a result the Comments field in this row reports FD CID TS inconsistency Ignore this error because the CID is correct...

Page 56: ...DK before the ID3 tag was packaged and released If this value is not set correctly the Nielsen ID3 tags may fall out of sync with the audio stream and result in improper crediting The PTS should advan...

Page 57: ...such cases the tag time is not properly advancing Symptoms If there is a real problem with tag times not advancing properly the Summary Report shows that the Tag Duration test failed as shown in the e...

Page 58: ...llback function to deliver ID3 tags to the calling application The call to InputAudioData does not have a one to one relationship with the SDK response in delivering an ID3 tag In fact typically many...

Page 59: ...scenario to determine if more than an acceptable number of tags are dropped Improper Set up of MPEG Multiplexer Background If you do not properly set up the PMT of the transport stream to indicate th...

Page 60: ...audio and ID3 tags it reports a synchronization error if the tags are separated by more than 15 seconds from the corresponding audio as shown in this example MPEG 2 Missing PES PTS Test Fail 110 0 4 S...

Page 61: ...breakout type Unexpected SIDs in Tags Background Unexpected SIDs refers to SIDs that are valid but are not expected to appear More specifically there is nothing wrong with the SID itself The ID3 tags...

Page 62: ...etailed tag listing file Find the last ID3 tag that includes 2 or more 2 unstacked EDUs for the same SID Identify the name of the segment that holds that tag Identify the names of segments that fall 1...

Page 63: ...p for both CIDs is rolled back to the most recently crossed broadcast day start 3 a m relative to the watermark timestamp of the most recent EDU of the same type in the ID3 tag D decoder An applicatio...

Page 64: ...d the PC offset should be within 15 seconds of the most recent FD EDU in the current ID3 tag In the _tags txt file the FD offset is labeled Seconds Since FD Day Start H HLS HTTP Live Streaming The pro...

Page 65: ...ermark information to Nielsen crediting systems downstream for measuring audience viewing on browsers and mobile devices P PAT Program Association Table A component of an MPEG 2 transport stream that...

Page 66: ...starts at 0 when the PCM to ID3 SDK is first activated The sequence number increases by one each time a new tag is released Both INFO and DATA tags are assigned sequence numbers from the same continu...

Page 67: ...PID MPEG 2 uses the PID to link transport packets to the elementary PESs to which the packets belong The transport PID reported in each ID3 tag listing in Validator _tags txt output file identifies t...

Search results

Summary of Contents for ID3

Reviews:

Brands by name

Popular brands

Nielsen ID3, User Manual

Search results

Summary of Contents for ID3

Reviews:

Brands by name

Popular brands