6247G/6247C DC Voltage Current Source/Monitor Operation Manual
9.5.2 How to Calculate Accuracy for Resistance Measurement
9-15
Voltage measurement full-scale item digit value
The measurement resolution in the 5 V range for voltage measurement is 10
V according to the
specification.
The accuracy of the offset item is 120
V.
Thus, the voltage measurement full-scale item digit value is 12 digits.
CMV item digit value
CMV = A
Vo / 1 V
= 30 nA
3 V / 1 V (*1)
= 90 nA
The source resolution in the 3 mA range for current source is 100 nA according to the specification.
Thus, the CMV item digit value is 0.90 digits.
From ,
and
Full-scale (FS) item for current source = 15 12 0.9 digits
=
27.9 digits
From the above results,
Accuracy =
0.05%
27.9 digits
Next, the accuracy in current source of 3 mA and voltage measurement of 3 V is calculated.
Measurement resistance value R = V / I = 3 V / 3 mA = 1 k
(Display: 1.0000 k
)
Accuracy =
(1 k
0.05%)
27.9 digits
=
0.5
27.9 digits
Thus, the accuracy is specified between 0.9967 k
to 1.0033 k
.
*1: In accuracy calculation for resistance measurement, the measurement voltage value is used as output
voltage Vo.
2
3
1
2
3
