6240B DC Voltage Current Source/Monitor Operation Manual
9.5.2 How to Calculate Accuracy for Resistance Measurement
9-14
9.5.2
How to Calculate Accuracy for Resistance Measurement
Example) Calculating the accuracy in current source of 3 mA (3 mA range) and voltage measurement of
3 V (3 V range)
The accuracies for resistance measurement must be calculated separately.
The accuracy formula for resistance measurement is as follows:
The accuracies in the 3 mA range for current source and in the 3 V range for voltage measurement are
calculated from the specifications as follows:
Accuracy in the 3 mA range for current source:
0.03 + 800 nA + 30 nA
Accuracy in the 3 V range for voltage measurement: 0.02 + 120
V
Here, the reading item for current source is as follows:
Reading item for current source
= Current source setting item + Voltage measurement reading item
= 0.03 + 0.02 =
0.05 %
Next, the full-scale (FS) item for current source is as follows:
Full-scale item for current source =Current source full-scale item digit value- - - - - - - - -
+ Voltage measurement full-scale item digit value - - -
+ CMV item digit value- - - - - - - - - - - - - - - - - - - - -
Current source full-scale item digit value
The source resolution in the 3 mA range for current source is 100 nA according to the specification.
The offset item accuracy is 800 nA.
Thus, the current source full-scale item digit value is 8 digits.
Voltage measurement full-scale item digit value
The measurement resolution in the 3 V range for voltage measurement is 10
V according to the
specification.
The offset item accuracy is 120
V.
Thus, the voltage measurement full-scale item digit value is 12 digits.
Condition
Accuracy
(% of reading)
( digits)
Voltage source
Reading item:
(Voltage source setting item + Current measurement reading item)
Full-scale item: (Voltage source full-scale item digit value + current measurement full-scale item
digit value + CMV item digit value)
Current source
Reading item:
(Current source setting item + Voltage measurement reading item)
Full-scale item: (Current source full-scale item digit value + Voltage measurement full-scale
item digit value + CMV item digit value)
1
2
3
1
2