OPTICOM OPERA Broadcast User Manual Download Page 129

C H A P T E R   6 :   T E L E P H O N Y   B A N D   V O I C E   Q U A L I T Y  

T E S T I N G  

123 

accompanied cultural differences. For example, a noisy floor may be more 
annoying if there are more silent intervals during a telephone conversation.  

The PSQM algorithm was one out of several proposals that had been brought 
before  study group 12 of ITU-T in 1995 for the purpose of international 
verification. Further proposals were the EPR Algorithm ("Expert Pattern 
Recognition"), which consisted of measures of the "LPC Cepstrum Function", 
"Information Index", and the "Coherence Function" (CHF). In a test series 
conducted by the Japanese phone corporation NTT, including listening tests in 
Japan and Italy, the highest correlation was achieved with PSQM results, when 
compared to the subjective tests. Consequently, PSQM was recommended by 
the ITU-T in 1996 for the objective quality measurement of telephone band 
speech codecs. Since then, PSQM has been used intensively for R&D as well as 
field applications in networks.  

6.5.2 Signal Acquisition 

PSQM can be used for online as well as for offline measurements. As 

Figure 

6.4

, shows, two kinds of signal sources be used. All OPERA™ versions can 

assess files. The version including a voice board can issue test calls to acquire 
the data, but the actual evaluation is also performed offline. For the data 
acquisition use 

OptiCall™

 which is described in chapter 4.2. Online 

measurements with phone lines are currently not supported. However, if there 
is an audio interface option included in your OPERA™ system – in addition to or 
instead of a POTS telephony board – evaluations of audio sources (e.g. VoIP 
terminals) with PSQM are available. 

 

PSQM

Files

(always)

Audio

(if audio board

included)

Sources:

Phone Line

(if voice board

included)

OptiCall.exe

Offline

Online

 

Figure 6.4: 

Kinds of signal sources for the PSQM measurement 

When using the file based version of the Telecom Version of OPERA™ (OPR-
100-xxx-x), no telephony interfaces are provided. Instead of this, use audio files 
as input. Supported file formats are 

WAVE

 files containing either plain PCM and 

a-law

 

or 

µ

-law

. The PSQM algorithm is defined for sampling rates of 

8 kHz

 and 

16 kHz

As described in Chapter 4, select the files you want to assess in the first two 
steps of the measurement set-up wizard. 

OPERA™ systems equipped with audio interfaces may be used for online 
measurements in real time too. Use our OptiCall program for the data 
acquisition if the system is equipped with telephone interfaces. OptiCall may 
also be used together with the audio interfaces. Please refer to chapter 4 for 
details on the data acquisition.  

File Based Version 

Version with 
Telephony and/or 
Audio Interfaces 

Summary of Contents for OPERA Broadcast

Page 1: ...User Manual Version 3 5...

Page 2: ...C O N T E N T S ii Document Version as of 20 Dez 02...

Page 3: ...11 1 2 1 OPERA Measurement System 11 1 2 2 OPERA Software Suite 12 2 Test Methodology 13 2 1 Assessing Quality 13 2 2 Advanced Audio Measurements Employing Perceptual Modeling 15 2 3 International St...

Page 4: ...m Batch Files 81 4 5 1 Syntax of the Command Line Parameters in a Batch File 81 4 5 2 How to Use a Configuration File 82 4 5 3 Example RunPsqm bat 83 5 Wide Band Audio Quality Testing 85 5 1 What To K...

Page 5: ...4 Diagram Types 144 6 6 5 Command Line Arguments 148 6 7 Echo Measurement 148 6 7 1 Fundamentals of the Echo Measurement Algorithm 148 6 7 2 Interpretation of Echo Parameters 149 6 7 3 Signal Acquisi...

Page 6: ...Algorithm 178 PSQM Algorithm 179 PESQ Algorithm 180 Echo Algorithm 181 8 2 Hardware 181 POTS Telephony Board 181 Audio Interface Option LynxONE 182 Audio Interface Option Digigram 183 OPERA Workstati...

Page 7: ...ponsibility for the lawful usage of our machines as stipulated in the laws rules and stipulations applicable Complaints Concerning Defects Any complaints concerning deficiencies in quality and perform...

Page 8: ...PERA Software In such a case as either party vests any patent rights in any enhancements and new features in the OPERA Software the relevant party upon request of the other party shall offer to the ot...

Page 9: ...r faulty or incomplete programming by licensee or in case of any modification to the OPERA Software along with the Accompanying Hardware carried out by Licensee or a third party b if the OPERA Softwar...

Page 10: ...as far as possible effect the economic intentions expressed therein In the case of a gap of these terms this shall apply accordingly Applicable law and Place of Jurisdiction If the customer is either...

Page 11: ...o the corresponding literature indicated by a number in squared brackets How this Manual is Organized This manual is organized in seven chapters Chapter 1 Introduction will briefly explain the OPERA m...

Page 12: ...ventions in order to ease the understanding of the operation For instance all menu options that can be selected will be in bold style all command line parameters that you might need to type will be pr...

Page 13: ...web page please make sure to contact the OPTICOM support with a detailed bug report Note In case of hardware problems with your PC workstation please refer to the hardware documentation of the OEM man...

Page 14: ...P R E F A C E 8...

Page 15: ...h and wide band audio signals by modelling the human ear OPERA is your digital ear OPERA is not only suitable to assess a single processing device with OPERA you can achieve a comprehensive analysis o...

Page 16: ...of the Telecom Version are at the time ITU T P 862 PESQ ITU T P 861 PSQM PSQM PSQM improved for GSM Echo measurement with real speech Delay measurement Interfaces to file wav loop start a b E1 T1 VoIP...

Page 17: ...software If assessment of speech quality is required the Telecom configuration is provided Audio quality can be measured using the broadcast configuration Combinations of telecom and broadcast configu...

Page 18: ...Software Suite which is delivered without any hardware A description of the corresponding hardware requirements can be found in the Installation and Setup chapter in Section 3 1 2 and in the Technica...

Page 19: ...e assessment of speech codecs since 1993 Within the ACR test method the ITU five grade impairment scale is applied see Table 2 1 In the telecommunication environment testing is done without a comparis...

Page 20: ...elevant recommendation is known as BS 1116 titled Methods for the Subjective Assessment of small Impairments in Audio Systems including Multichannel Sound Systems ITUR1116 which was issued by the ITU...

Page 21: ...bit rate audio coding applications still remains questionable as there are no clearly defined example distortions in such a case The scale was derived from telephone speech quality and is not well def...

Page 22: ...re 2 1 The underlying concept for perceptual measurement The evaluation of the internal representation is often related to an estimate of the masked threshold This estimate is based on data found in a...

Page 23: ...f this new model were presented A Basic version featuring a low complexity approach and an Advanced version for higher accuracy at the trade off of higher complexity After thorough verification the mo...

Page 24: ...tive experiment of the corresponding listening test To decide which one is the proper experiment remember to ask yourself the following questions Is this an assessment of wide band audio signals music...

Page 25: ...1116 Setup 5 0 4 0 3 0 2 0 1 0 wide band audio YES comparison with reference YES PEAQ Figure 2 3 Illustration of the principle of BS 1116 DUT device under test y ORIGINAL TEST 5 4 3 2 1 P 800 Setup w...

Page 26: ...tly already defined by the algorithm that shall be used for the evaluation of the recorded data PEAQ according to ITU R BS 1387 for example requires 48kHz sample rate although the implementation in OP...

Page 27: ...similar the enhanced signal sounds to the clean reference This also implies that based on the measurement no statement can be made weather the enhanced signal sounds better than the original signal o...

Page 28: ...C H A P T E R 2 W H I C H M E A S U R E M E N T D O I N E E D 22...

Page 29: ...have purchased 3 1 OPERA Software Suite 3 1 1 Unpacking the Software After unpacking the Software Suite please check the delivery for completeness first The delivery should include the following parts...

Page 30: ...the case memory will be swapped out to the harddisk and accessing this part of the memory is a million times slower than accessing real memory If the OPERA Software Suite requires too much memory adj...

Page 31: ...ned to work on 220V as well as on 110V AC Please check for any transportation damages Installation and Setup The OPERA system comes completely pre configured so no additional setup procedure is requir...

Page 32: ...packing the System After unpacking the system check the delivery for completeness first The delivery should include the following parts Portable industry type of computer system with integrated 15 TFT...

Page 33: ...to the parallel interface port of the computer if it is not yet there connect the power cable to the side panel of the main unit and switch on the system For details please refer to the hardware docu...

Page 34: ...C H A P T E R 3 I N S T A L L A T I O N A N D S E T U P 28...

Page 35: ...ns of the plug in specific topics are found in Chapter 5 or Chapter 6 respectively Users of the Software Suite may skip the explanation of the data acquisition process since this feature is not availa...

Page 36: ...codecs 4 2 1 POTS Telephony Interfaces The POTS telephony interfaces of the OPERA system OPR 101 xxx x are accessible from the rear of the system or in the case of the portable version from the side p...

Page 37: ...the optional high quality sound board that is used for data acquisition Since both ports of OPERA provide two POTS interfaces each the splitter cables shown in Figure 4 2 are used to split up the sig...

Page 38: ...of the POTS telephony board with the optional high quality sound board Next the volume should be adjusted This needs to occur in two locations First the hardware control on the right side of the port...

Page 39: ...before June 2000 a Digigram audio board is installed If the manufacturing date of the system is after June 2000 it may be equipped with LynxONE audio boards For information concerning the audio interf...

Page 40: ...re 4 5 Here select whether the digital AES EBU input connections or the analog inputs shall be used then click on the corresponding buttons in the Record Source field In the field below titled Clock S...

Page 41: ...equency If feeding back the digital outputs to the digital inputs is desired supply an external clock to one of the synch inputs of the board To use OptiCall for the data acquisition as described late...

Page 42: ...multi cables the Audio Port is a 25 pin female D connector and the MIDI Clock port is a high density 15 pin female D connector see Figure 4 7 Audio Port MIDI Clock Port Figure 4 7 Mounting bracket of...

Page 43: ...are used to synchronize the LynxONE with external equipment The connectors support TTL level signals and should be connected with 75 ohm coaxial cable Connect the CLOCK IN connector to the clock outpu...

Page 44: ...Figure 4 8 and Figure 4 9 represent the typical settings for a measurement with analog input signals As shown in the example in Figure 4 8 the Source for the Sample Clock is Internal the Reference is...

Page 45: ...signal in Figure 4 10 the Source for the Sample Clock is set to Digital the Reference is set to Auto and Auto Selection is turned on Figure 4 11 shows the mixer settings for the test signal The Sourc...

Page 46: ...nterface is in the connected state immediately Using the audio interfaces OptiCall can also be used for the assessment of wide band audio codecs like e g MPEG or AAC codecs multimedia terminals or any...

Page 47: ...xplorer If a valid file is selected some information on the encoding of the data below of the edit field will be observed In case there is no information displayed the file does either not exist or is...

Page 48: ...Audio Standard View all unnecessary parameters are left out e g phone numbers and the names of some of the field in order to be more meaningful for pure audio applications are modified Essentially the...

Page 49: ...e monitoring output It may seem unusual but from the drop down list box for the interface selection you can also choose telephony interfaces Do not select those when in the audio view Nothing serious...

Page 50: ...hone call on the specified interface and start transmission as well as recording of the audio data as soon as the call is answered by the terminating side which may be another OPERA system or just ano...

Page 51: ...e echo from the network as well as the side tone desired echo with short delay The near end signal is usually used for echo measurement This operating mode is called the Loop mode Figure 4 15 shows th...

Page 52: ...ch interface each file was recorded Following is a short text that describes the type of interface Names starting with POTS refer to the analog loop start interfaces Names starting with Audio identify...

Page 53: ...ay run on the same remote or local system as well This operation mode is called the Origin mode Figure 4 17 shows the OptiCall window after Origin had been selected The operation is the same as in the...

Page 54: ...rd the echo signal Of course it is advisable that the calling person is either switching off the microphone or at least covering it with the hands Figure 4 18 The OptiCall window in termination mode T...

Page 55: ...n Since OptiCall will only perform the data acquisition you will have to employ the OPERA program for the actual measurement In OPERA the following assignments must be made to achieve proper results w...

Page 56: ...t 9 interfaces and the calls will be terminated on the interfaces selected under Termination plus the next 9 consecutive interfaces To distinguish the recorded files Bulksetxx will be appended to the...

Page 57: ...interfaces if the device under test operates with very low levels like e g head or hand set interfaces on telephones mobile phones In this case OptiCall makes the recording with a bit resolution of 24...

Page 58: ...ile name Play file used on calling side RefFileTermination file name Play file used on terminating side DestinationPath drive path UNC path Destination directory for recorded files RootFilename root f...

Page 59: ...rop down list box as if to change the terminating or the originating interface Find that all entries in the list box start with a number This number is the index of the line In the future a command li...

Page 60: ...t be fully synchronous Take care to not overload the analog inputs Using the mixer controls on Windows based PCs will in general degrade the signal quality since the attenuation amplification is appli...

Page 61: ...international standards like ITU R BS 1387 PEAQ or ITU T P 861 PSQM usually describe a measurement algorithm and do not take into account all the constraints of a realistic measurement situation For...

Page 62: ...generate a test signal to be seen on the top of Figure 4 22 to remote control the system Remote Control and also to control it by other programs and scripts Scripting Interface Scripts written in an...

Page 63: ...user interface realized under Windows NT for a typical measurement situation As mentioned before there are currently three standardized measurement schemes implemented in OPERA PEAQ PESQ and PSQM whic...

Page 64: ...urement drag the slider with the mouse and scroll through the signal Other options to move the slider are to click into the grey area besides it or to click once onto the slider to give it the input f...

Page 65: ...used by the algorithm by clicking on Properties This will bring up another dialog allowing enterance of some algorithm specific parameters These parameters will be explained in detail in the chapters...

Page 66: ...is a step by step guide through the set up making sure that all parameters that are required to obtain correct results are set Measurement Setup Wizard Step 1 Physical Input 1 With the first wizard s...

Page 67: ...aw or law at 8bit per sample mono The supported sample rates depend on the selected algorithm Figure 4 28 Selection of Input 1 After having selected the input source click on Next to get to the next w...

Page 68: ...r channel Also in the case that the device under test is swapping the channels the multiplexer can be used to correct this The chapter on applications will show other useful example settings Figure 4...

Page 69: ...easurement Setup Wizard Step 4 Delay compensation and signal preprocessing In this wizard step various parameters of the signal preprocessing applied to reference and test signal can be influenced bef...

Page 70: ...can be compensated for automatically is 1000 ms for PSQM and PEAQ offline measurements For PEAQ online measurements it is 500 ms PESQ uses a different time alignment algorithm and can compensate for t...

Page 71: ...akes effect when the Snap Mode of the Automatic Delay Compensation has been chosen When performing Snap Again the delay is re determined and used for the delay compensation from now on Figure 4 33 Too...

Page 72: ...Example 2 Maximum Range of the delay compensation auto auto 10 s auto 10 or s Figure 4 34 Total range of the delay compensation using only Automatic Delay Compensation Example 1 and using Static Delay...

Page 73: ...ied to all input signals before they are processed by the measurement algorithm If the signals show a DC bias the filters will need approximately 2000 samples to settle DC filtering may be used safely...

Page 74: ...ring over a long period of time Figure 4 36 Toolbar button for the Reset Averaged Values command To stop the measurement before the entire files are processed do so by either selecting Measurement Sto...

Page 75: ...zard style dialog similar to Figure 4 39 will appear This dialog presents all the measurement diagrams that are provided by the currently selected algorithm This first step of the wizard is also refer...

Page 76: ...Signal dialog see Figure 4 40 allows a selection of channels and input signals required for the results in one diagram Modify the selection by clicking with the left mouse button on any of the option...

Page 77: ...a is shown on the screen Usually this is identical to selecting the units of the diagram axes For the time signals choose between a binary linear representation in which the input signals are always s...

Page 78: ...r signals for the second diagram pane Note At any time while the wizard dialog is active click Back to go to the previous wizard step or click on Cancel to leave the wizard without performing any chan...

Page 79: ...ime in the measurement Table 4 1 Interpretation of the displayed values Figure 4 42 General information related to the current measurement settings 4 4 5 Setting Markers in Diagrams In most diagrams s...

Page 80: ...click on the Format button and the dialog shown in Figure 4 45 will appear Here change the Sample Rate the number of Channels or the bit resolution Bits per Sample of your digital signal format When...

Page 81: ...s of the Input Files OPERA allows the use of a certain part of the input signals for the measurement only To use this feature select the menu option Measurement Trigger The dialog shown in Figure 4 46...

Page 82: ...en as shown in Figure 4 47 By clicking on the Print button on the toolbar the printing will start Print the diagram view from the menu option File Print or by selecting the toolbar button showing the...

Page 83: ...depicts the result of this command Figure 4 48 Exported measurement graph using the Edit Copy printable menu option 4 4 11 Summary of the Menu Options File Menu File Print Prints the diagrams Use the...

Page 84: ...d using printer friendly colours Edit Paste This menu option currentlyhas no function View Menu View Toolbar Switches the toolbar on and off View Status bar Switches the status bar on and off View Res...

Page 85: ...mal Mode of the Automatic Delay Compensation When activated the current delay compensation is used for the rest of the measurement Measurement Snap Again Snap Again takes effect when the Snap Mode of...

Page 86: ...C H A P T E R 4 G E T T I N G T O K N O W T H E O P E R A F R A M E W O R K 80 Help About OPERA Opens the About box that contains license information the version number and the copyrights...

Page 87: ...ist of parameters in a configuration file and start OPERA like opera Exec Cfg name of the configuration file For a description how to use configuration files see paragraph 4 5 2 In the following the s...

Page 88: ...t of the measurement in ms EndTime End time Specify end point of the measurement in ms Channel 0 Relate to reefernce 1 realte to test Relate start and end point to the beginning of the reference or th...

Page 89: ...al rem rem File2 File that contains the test signal rem rem Outputfile Results are stored in this file If rem it exists already results are rem appended to it otherwise it will be rem newly created re...

Page 90: ...C H A P T E R 4 G E T T I N G T O K N O W T H E O P E R A F R A M E W O R K 84...

Page 91: ...ening tests of this type This also defines the applicability of wideband audio tests like PEAQ They can be applied wherever a subjective test according to BS 1116 would be applied In addition to that...

Page 92: ...and test file for each algorithm which are used for demonstrating file based measurements These files are called AlgorithmRef wav and AlgorithmTest wav where Algorithm must be replaced by PEAQ PSQM PE...

Page 93: ...potential users of such a system The development therefore was influenced by the feedback of the users group with respect to their requirements This feedback resulted in a list of applications which...

Page 94: ...ers Figure 5 3 Perceptual model PEAQ Basic Following the signal flow from the input signal to the final calculation of the excitation pattern the processing starts by a transformation of the input sig...

Page 95: ...r distortions BandwidthTestB frequency response etc RelDistFramesB Frequency of audible distortions Total NMRB Noise to mask ratio MFPDB Detection probability ADBB EHSB Harmonic structure of the error...

Page 96: ...ccuracy of the algorithm was slightly improved compared to the Basic version For more detailed information about the Advanced Version see the paper of the ITU R recommendation BS 1387 located in the a...

Page 97: ...are specific to the PEAQ algorithm The Listening level is shown for informational purpose only It is set fixed to 92dB SPL This value is used to scale the binary representation of the audio data as t...

Page 98: ...is performed the results will be displayed There are ten diagram types available for the PEAQ algorithm that will be described in this section see Figure 5 6 Figure 5 6 Result Type Window Display of t...

Page 99: ...Current point of time in the measurement Table 5 3 Interpretation of the displayed values Figure 5 7 Display of the current measurement settings of the PEAQ algorithm Timesignals To choose this diagr...

Page 100: ...on the screen Usually this is identical to selecting the units of the diagram axes For the time signals choose between a binary linear representation in which the input signals are always scaled to 32...

Page 101: ...e signal the test signal or the difference between the reference spectrum and the test spectrum Noise Available result styles for the spectra see Figure 5 12 are a linear frequency scale a Bark scale...

Page 102: ...C H A P T E R 5 W I D E B A N D A U D I O Q U A L I T Y T E S T I N G 96 Figure 5 11 Select Signal Dialog Figure 5 12 Result Style Dialog Figure 5 13 The Spectra diagram...

Page 103: ...own in Figure 5 14 the reference signal excitation the test signal excitation and the difference between both excitations can be selected The difference is calculated by subtraction in the linear doma...

Page 104: ...ximum level of a not audible error and the actual noise error signal NMR is defined as the ratio between the error signal to the masked threshold The masked threshold is estimated for each band of the...

Page 105: ...shed simultaneous masking and temporal masking In case of simultaneous masking the masker and the maskee are present at the same time and are quasi stationary In case of temporal masking the masker an...

Page 106: ...s less than the sum of the loudness of all its components In the context of audio quality measurement the loudness of the unwanted distortion added to the reference signal the noise loudness is reduce...

Page 107: ...A P T E R 5 W I D E B A N D A U D I O Q U A L I T Y T E S T I N G 101 Figure 5 21 Select Signal window of the loudness diagram type Figure 5 22 The Loudness result styles Figure 5 23 The Loudness diag...

Page 108: ...am is for orientation merely The first 11 bars AvgBwRef NLoudB represent the MOVs according to BS 1387 Table 5 4 may help relating the OPERA names to the according names of the ITU recommendation Figu...

Page 109: ...r DI and ODG can only be compared quantitatively but not qualitatively Figure 5 25 shows two curves that represent the relation between the quality and the DI value left diagram and the relation betwe...

Page 110: ...e diagram type NMR vs Time Select between the NMR value vs time of the right and or the left channel see Figure 5 27 The resulting diagram will look similar to the one shown in Figure 5 28 As in the O...

Page 111: ...s that should be reported as the result of a measurement Please note that the Model Output Variables and ODG diagram may contain slightly different values for the ODG and DI than the Final Result diag...

Page 112: ...are defined by BS 1387 For a detailed explanation refer to the ITU R recommendation in the Appendix of this manual The results are shown framewise and are averaged since the beginning of the measureme...

Page 113: ...se parameters essentially follow the settings of the algorithm parameter dialog and are listed with a short comment on their usage in the following Keyword Add Parameter Comment Version 0 Basic 1 Adva...

Page 114: ...ined in detail Figure 5 32 The Input 1 dialog step of the Measurement Setup Wizard Then we choose from the OPERA framework the menu option Measurement Algorithm Parameters the PEAQ algorithm and use t...

Page 115: ...led when a file based measurement is performed In addition to this we switch on the functions Static Gain Compensation and Remove DC From Signals Now start the measurement by clicking on the Finish bu...

Page 116: ...de In some measurement cases notice that the displayed value for the detected delay in the field to the right of the diagram panes changes When stopping the permanent delay detection while keeping a c...

Page 117: ...the name Example 2 Now select from the menu option Measurement Algorithm Parameters the algorithm PEAQ Take notice of the following warning message and click on the Continue button Now commence the m...

Page 118: ...izard before the actual measurement starts is the Signal Preprocessing Window shown in Figure 5 41 If there is a delay between the reference and the test signal it should be compensated automatically...

Page 119: ...anel contains the parameter Status which is a measure of the reliability of the delay compensation For measurements with the PEAQ algorithm this value should not fall below approx 80 When the status v...

Page 120: ...rking correctly The following will explain the mentioned batch files and explain how to save result values in your own result file The following lines are the contents of the file RunPeaqBasic bat Ple...

Page 121: ...tputfile Results are stored in this file If it rem exists already results are appended to it rem otherwise it will be newly created rem rem echo echo ConfPeaqBasic V1 0 c OPTICOM 1999 echo echo echo o...

Page 122: ...asic 1 2 Conformance BS 1387 lrefpip wav 2 Conformance BS 1387 lcodpip wav 3 call RunPeaqBasic 1 2 Conformance BS 1387 mrefcla wav 2 Conformance BS 1387 mcodcla wav 3 call RunPeaqBasic 1 2 Conformance...

Page 123: ...gy to estimate the subjective quality of speech codecs with less subjective testing In the past the most widely used objective speech quality measure was the Signal to Noise Ratio SNR S N However it w...

Page 124: ...s between two implementations The following chapter explains PSQM in more detail PSQM is chosen here as an example for all modern perception based measurement methods Both algorithms will be explained...

Page 125: ...omparing a coded signal characterized as output y t in Figure 6 1 to a source signal input x t For this reason experimental parameters subject properties and listening conditions have to be taken into...

Page 126: ...1 6 4 PSQM or PESQ which one shall I use Since there are two speech quality algorithms available as ITU recommendations now the user has to decide which one to use This decision however can be based o...

Page 127: ...neous approach has been presented that would allow for high correlation with both speech and music signals without adapting algorithm parameters BEER95 Figure 6 3 depicts a detailed block diagram to c...

Page 128: ...k diagram of the PSQM Algorithm according to ITUT861 The following blocks are intended to represent the cognitive part of the modelling The asymmetry processing should take into account that distortio...

Page 129: ...st calls to acquire the data but the actual evaluation is also performed offline For the data acquisition use OptiCall which is described in chapter 4 2 Online measurements with phone lines are curren...

Page 130: ...background noise that produces masking effects Even in silent environments this noise is in most cases higher than 30 dBA This effect is modelled by adding background Hoth noise to the reference as w...

Page 131: ...am is described first see Figure 6 7 The meaning of these values is as shown in Table 6 1 Displayed Values Interpretation Time The time when the measurement has been finished Hoth Noise Setting of the...

Page 132: ...d input signals is defined that will be shown together in one diagram Modify the selection by clicking with the left mouse button on any of the option buttons This will add or remove the check mark in...

Page 133: ...nary linear representation in which the input signals are always scaled to 32768 32767 or a dB FS full scale dBov scale Note Independent of the input data format samples are always converted to 16bit...

Page 134: ...yles for the spectra see Figure 6 12 are a linear frequency scale or a Bark scale as outlined in P 861 The Y axis is always scaled in dB SPL Note that the Y axis is depending on the setting of the lis...

Page 135: ...representation see Section 2 2 of a signal The Bark scale is used for the horizontal axis the vertical axis is scaled in dB SPL The curves are shown on a frame by frame basis without any averaging As...

Page 136: ...presents the diagram of the Raw PSQM There are several values shown the description of the result values is given in Table 6 2 below The PSQM value indicates the degree of subjective quality degradati...

Page 137: ...according to P 861 silence weight 0 2 PSQM W4 PSQM according to P 861 silence weight 0 4 PSQM Silence PSQM value of the silence intervals Table 6 2 MOVs used by the OPERA PSQM version and their inter...

Page 138: ...ble 6 3 Results calculated by the OPERA PSQM version and their interpretation Table 6 4 shows the simple interpretation of the MOS P 801 MOS Value Interpretation 5 Excellent 4 Good 3 Fair 2 Poor 1 Bad...

Page 139: ...OS value This diagram type see Figure 6 18 shows the PSQM MOS over all frames in the history buffer This figure is not an averaged value and no weighting factor is used for this result value When scro...

Page 140: ...cribed in Chapter 4 specific commands will be described in this section PSQM currently interprets the following algorithm specific command line switches If no switches are specified the default settin...

Page 141: ...Are the correct files used Are the listening level and the upper frequency limit set up properly check under Algorithm Properties If Delay Tracking is enabled it will discard all frames for which it c...

Page 142: ...C H A P T E R 6 T E L E P H O N Y B A N D V O I C E Q U A L I T Y T E S T I N G 136...

Page 143: ...system that implemented an easy way to solve the varying delay issue in most cases without loosing the option of realtime operation Although this feature failed for some signals it was the only avail...

Page 144: ...anges from 1 0 worst up to 4 5 best This may surprise at first glance since the ITU scale ranges up to 5 0 but the explanation is simple PESQ simulates a listening test and is optimized to reproduce t...

Page 145: ...est is required This will compensate for differences in the MOS scales used by the listeners and by PESQ The actual scale varies slightly between listening tests which means in test 1 File a may be gr...

Page 146: ...load on the network To optimize the latency of VoIP equipment adaptive algorithms are used to automatically adjust the size of the jitter buffer to what ever is required by the network These adaptati...

Page 147: ...ore almost every mobile phone and VoIP terminal has a built in AGC Automatic Gain Control or ALC Automatic Level Control These mechanisms both target the same problem They amplify or attenuate the inp...

Page 148: ...oo late and the signal is already transmitted This false or slow detection of active speech and silence is characterised by two parameters Front End Clipping FEC and Hold Over Time HOT The Hold Over T...

Page 149: ...PERA It takes the time aligned signals from PESQ realigns them if required and calculates an almost ideal VAD on both signals It is much easier for the measurement algorithm to calculate a VAD than fo...

Page 150: ...own in Figure 6 22 differs significantly from the Timesignal displays of the other algorithms since it shows the entire waveforms in one diagram instead of the framewise displays of the other algorith...

Page 151: ...mum delay in milliseconds as well as the delay jitter in milliseconds Figure 6 23 PESQ Final Result diagram Using PESQ on OPERA you can even analyze the behaviour of adaptive jitter buffers Of course...

Page 152: ...Figure 6 26 represents the Delay vs Time diagram It shows the time on the x axis and the delay in ms on the y axis Especially with packet networks significant variations of the delay during a call can...

Page 153: ...e to the discrete delay values occurring in the measurement and the resolution of the algorithm This diagram together with the Delay vs Time diagram defines the full dynamic characteristic of the dela...

Page 154: ...gure 6 29 Two wires are used to transmit voice in one direction and the two other wires are used for the opposite direction In contrast the subscriber network is built with using only two wires to tra...

Page 155: ...all phones The algorithm to calculate the echo on a telephone line uses real speech as the stimulus for the measurement Consequently the values obtained by this algorithm may not be compared to an Ech...

Page 156: ...s up to 1000ms Choosing a lower value will lead to a faster calculation of the echo delay Figure 6 31 Echo algorithm properties 6 7 5 Specific Settings for the Echo Measurement When performing echo me...

Page 157: ...dialog 6 7 6 Diagram Types Chapter 4 explained how to select a measurement algorithm and how to start a measurement Once a measurement is performed results can be displayed This section describes the...

Page 158: ...able 6 6 Interpretation of the displayed values Figure 6 34 Display of the current measurement settings of the Echo algorithm To choose this diagram type highlight the radio button next to Timesignals...

Page 159: ...on the screen is selected Usually this is identical to selecting the units of the diagram axes For time signals choose between a binary linear representation in which the input signals are always sca...

Page 160: ...he ERL diagram type The horizontal axis is a time axis for the amount of the delay the vertical axis shows the attenuation of the echo signal given in dB Optionally you can overlay the diagram with th...

Page 161: ...Result Values Interpretation Min Delay Echos with values smaller than this value will not be regarded as echo Frame Size Size of one frame ERLmom Attenuation of the echo in the momentary window with l...

Page 162: ...6 7 7 Command Line Arguments In addition to the command line arguments described in Chapter 4 there is one specific command for the Echo algorithm This command is to be used together with the Algorith...

Page 163: ...e g D test 050700 Now press the start button The connection will be established the speech sample will be sent and the test samples will be saved in the location we have defined above Note The WAVE fi...

Page 164: ...rties dialog Take notice of the warning message and click on the Continue button Now start the analysis by clicking on the toolbar button shown in Figure 6 41 for example The Measurement Setup Wizard...

Page 165: ...n this example D test 050700 Line0 wav see Figure 6 43 When clicking on the Next button the Input Mapping dialog will appear Here the left signal of Input 1 as the left channel of the reference signal...

Page 166: ...mpensation function and the Remove DC from Signals option see Figure 6 44 are selected Finally click the Finish button to start the actual analysis After the computation of the delay has been finished...

Page 167: ...P T E R 6 T E L E P H O N Y B A N D V O I C E Q U A L I T Y T E S T I N G 161 Figure 6 44 Signal Preprocessing dialog of the Measurement Setup Wizard Figure 6 45 Resulting view on the measurement res...

Page 168: ...edure as the first input signal see Figure 6 47 in this example DefaultRefFile wav As the second input signal the stored file is selected whose name ends with the string Line1 In this example the file...

Page 169: ...C H A P T E R 6 T E L E P H O N Y B A N D V O I C E Q U A L I T Y T E S T I N G 163 Figure 6 47 First step of the Measurement Setup Wizard Figure 6 48 Second step of the Measurement Setup Wizard...

Page 170: ...n in the Signal Preprocessing Dialog the echo is measured After this process has been completed the diagram types to display can be chosen In this example the Result Summary type for the upper diagram...

Page 171: ...ushd echo TODO change working dir according to where echo OPERA exe is echo Processing file 1 Opera Exec Algorithm Name PSQM Input Inp 0 File 1 Inp 1 File 2 Mux InpRefLeft 0 ChannelRefLeft 0 InpTestLe...

Page 172: ...cessing dialog are made First the Automatic delay compensation is switched on The left channel is used for this function and it is run in normal mode In the Command Line Parameters functionality there...

Page 173: ...6 8 3 More Examples More example setups especially some more exotic applications can be found in our paper OPERA Application Notes which is attached to this manual Carefully look at the paper since so...

Page 174: ...C H A P T E R 6 T E L E P H O N Y B A N D V O I C E Q U A L I T Y T E S T I N G 168...

Page 175: ...ber of command line parameters These parameters may also be written into a batch file and contain comments For information about the syntax of batch files please refer to the corresponding help topic...

Page 176: ...t channel of the reference signal InpTestLeft 0 1 Input used to form the left channel of the test signal InpTestRight 0 1 Input used to form the right channel of the test signal ChannelRefLeft 0 for l...

Page 177: ...ne parameters 7 2 3 How to Use a Configuration File A configuration file can be created containing e g frequently used settings of parameters Inside the configuration file all parameters listed above...

Page 178: ...ODG float LogInterval duration Logging intervals in s LogFileName FileName Name of the logfile PSQM currently interprets the following algorithm specific command line switches If no switches are spec...

Page 179: ...ll Termination Terminate the call Origin Originate the call Cfg file name Read more parameters from configuration file Phonenumber phone number Phone number to dial RefFileOrigin file name Play file u...

Page 180: ...nterface and bit 1 is the terminating interface Enter 3 for both interfaces Host hostname Name of the OPERA system on which the program should execute This parameter is subject to a special network li...

Page 181: ...the reference signal rem rem File2 File that contains the test signal rem rem Outputfile Results are stored in this file If rem it exists already results are rem appended to it otherwise it will be r...

Page 182: ...the batch file you find some parameters like e g phone numbers etc which you should adjust according to your needs BulkDemo pl This is a PERL script which generates the file names required for the ev...

Page 183: ...CM 8 or 16 bit Maximum Duration of Measurement Signals File based As limited by WAVE format Online Limitation of current version 8 32 2 ple bitspersam fs e g 12 4 h at 48 kHz 16 bit resolution Maximum...

Page 184: ...ay Tracking 512 samples Available Measurement Results Timesignal Spectrum Excitation Noise to Mask Ratio NMR averaged Noise to Mask Ratio NMR vs Time Masked Threshold Loudness Objective Difference Gra...

Page 185: ...lation changes where the reference contains little or no modulations AmodDif2B RModDifA RMS value of the averaged noise loudness with emphasis on introduced components NloudB NLA Averaged Linear Disto...

Page 186: ...re MOS vs Time Delay between the Reference Signal and the output signal of the device under test Attenuation of the test signal compared to the reference signal Modulation of the reference and test si...

Page 187: ...oop start interfaces Power Environment Power requirements 5V 12V 12V 0 8A typ 0 3A 0 1A 0 1A Operating temp humidity noncondensing 0 C 50 C 5 80 non condensing Storage temp humidity noncondensing 20 C...

Page 188: ...2k Output impedance Balanced 100 unbalanced 50 Output drive capability 600 impedance 0 16 F capacitance A D and D A converters 24bit 128x oversampling sigma delta Bit depth 8 16 24 or 32 bit file type...

Page 189: ...S11 synchronization yes Connector 62 pin SUB D Audio Specifications Sampling frequencies available Programmable from 6 kHz to 50 kHz in steps of 0 02 Hz A D and D A converter resolutions 20 bits PCM r...

Page 190: ...C H A P T E R 8 T E C H N I C A L S P E C I F I C A T I O N S 184...

Page 191: ...n of Speech Quality PESQ The New ITU Standard for End to End Speech Quality Assessment Part II Psychoacoustic Model J Audio Eng Soc Vol 50 No 10 2002 BENJ02 BENJAMIN E Evaluating Digital Audio Artifac...

Page 192: ...ces Which Use Low Bit Rate Audio Coding 94th AES Convention Berlin 1993 Preprint 3616 KEYH96 KEYHL M HERRE J SCHMIDMER Ch NMR Measurements on Multiple Generations Audio Coding 96th AES Convention Amst...

Page 193: ...t 1 A method for subjective assessments of very low bit rate audio 104th AES Convention Amsterdam 1998 STOL99 STOLL G Beerends J Bitto R Brandenburg K Colomes C Feiten B Keyhl M Schmidmer C Sporer T T...

Page 194: ...a computational model for use in transmission planning May 2000 ITUT420 ITU T Recommendation E 420 Checking the Quality of the International Telephone Service General Considerations 1988 Extract from...

Page 195: ...726 Bit rate of 32 kbit s also possible with 16 24 and 40 kbit s AES The Absolute Error Score AES is derived from a formula developed especially for evaluating the quality of the results obtained from...

Page 196: ...parameter which measures the headroom of inaudible coding artefacts to the threshold when these artefacts become audible CTI Computer Telephony Integration Short for computer telephony integration whi...

Page 197: ...g MUSHRA Stands for Multiple Stimulus With Hidden Reference Anchors A new draft ITU recommendation on testing very low bit rate audio NMR The measurement scheme NMR Noise to Masked Ratio BRAN87 evalua...

Page 198: ...a subjective listening test Offline measurements Measurement procedure which does not interact with the ongoing programme transmission Online measurements Measurement procedure which relies on the ong...

Page 199: ...ys perfectly PESQ is not designed for streaming applications which is it s only drawback This is why it cannot fully replace PSQM With PSQM and PESQ there are now two standards that cover the entire p...

Page 200: ...d of a telephone conversation In general this is the party which receives a phone call In older OPERA versions this was the Called telephone line VAD Voice Activity Detection VAD is part of most VoIP...

Page 201: ...10 11 29 40 54 55 57 108 9 117 124 177 H HOT 142 43 180 I Inp 81 82 114 165 170 71 175 InpRefLeft 81 83 114 165 170 175 InpRefRight 81 114 170 InpTestLeft 81 83 114 165 170 175 InpTestRight 81 114 17...

Page 202: ...62 164 166 170 72 Signal 5 9 13 15 18 20 21 32 35 37 39 42 45 48 52 57 58 60 67 70 74 79 81 83 85 89 93 101 103 106 108 110 12 114 117 119 24 126 29 135 46 149 54 157 62 164 65 170 174 75 178 83 187 S...

Page 203: ...of the Art Voice Quality Testing White Paper OPERA Application Notes 1 ITU T Recommendation P 861 ITU T Recommendation P 862 White Paper PEAQ The ITU Standard for Objective Measurement of Perceived Au...

Page 204: ......

Reviews: