YOKOGAWA DL850E, User Manual

Page: 255 / 270

App-21
IM DL850E-01EN
Appendix
Computation Format (Resolution) of the Frequency Module
The computation flow on the frequency module is indicated below.
2400 LSB/div
Binary
↓
Float
Float
↓
16 bits Binary
Computation
(floating point)
2400 LSB/div
Floating point data
Offset
Range value
Value/Div
−
Input
32-bits counter
ACQ Memory
A (Float)
Floating-point computation
Frequency
Period
...
50 ns resolution
C (Float)
÷ ÷
D (Float)
E (Binary)
Computations
1 LSB weight
Internally, all calculations
are performed using
floating-point values.
The frequency module measures the period of the input signal using a 32-bit counter at a resolution of 50 ns.
Therefore, the minimum resolution of the counter values is 50 ns.
Computations are performed in floating point format. The data that is output from the frequency module and
written to the acquisition memory (ACQ Memory) is 16-bit binary data. The frequency module converts the data
using a weight of 1 LSB that is determined by Value/div. The data is normalized to 2400 LSB/div when displayed
on the screen.
Input: Conversion from 32-bit Counter Values to Floating Point Values
The frequency module converts the count value that it obtains using the 32-bit counter at 50 ns resolution to
floating point format, and determines period A using the following equation.
Period: A (Float) = (Count value) × 50 ns
Computation
Various computations are performed in floating point format based on the settings.
Example
Frequency: C (float) = 1/A (float)
Calculation of the 1 LSB Weight of the Output
The 1 LSB weight of the output is determined from the range (value/div).
Because 1 div = 2400 LSB,
1 LSB weight of the output = (Value/div)/2400
Computation Output: Conversion from Floating Point Values to 16-bit Binary Values (When the Offset Is 0)
When the offset value is 0, offset calculation is not performed, and C (float) = D (float).
The data is converted into 16-bit binary data and written to the acquisition memory (ACQ Memory).
16-bit binary data: E (binary) = D (float) ÷ (1 LSB weight of the output)
Offset Computation
When the offset value is not 0, the frequency module computes the offset value in floating point format using the
following equation and then converts the value to 16-bit binary data.
D (float) = C (float) − offset value (float)
In offset computation, if the computed result C (float) is equal to the offset value, the output is 0. If the computed
result C (float) is less than the offset value, E (binary) is negative.