ACCURACY
3-4
R + Q ACCURACY
Accuracy of R = ± [A x Ki x Kv + (Kh + Kl) x 100] %
A
=
Basic Resistance Accuracy Factor
from graph.
For |Q| > 0.1, multiply the basic
accuracy factor by (1 + |Q|).
If the unit is in the constant voltage
(CV) mode, double the basic
accuracy factor.
Ki = Integration Time Factor (see Ki
Table below)
Kv
=
Voltage Error Factor (see Kv table
below).
Kh, Kl
=
Extreme Range Error Term (see
Kh, Kl table below).
Accuracy of Q = ± [(Ar / 100) x (1 + Q2)]
Ar
=
the accuracy of the Resistance
measurement (above)
Note that the accuracy of Q is specified as a
magnitude, NOT as a percent.
Ki Table
Meas Rate Frequency Zm
Ki
slow, med all
1
fast
100 Hz-1 kHz 6.25
Ω
<Zm<400 k
Ω
3
fast
all other
2
Zm = the impedance (resistance) of the device
being measured.
Kv Table
Vout (V rms)
Kv
1.0 - 055
1 / Vout
0.5 - 0.3
0.5 / Vout
0.25 - 0.15
0.25 / Vout
0.10
0.11/Vout
Note that Kv equals one for the primary drive
voltages (1.0, 0.5 and 0.25 Vrms).
Kh & Kl Table
Frequency
Kl
Kh
100,120,1k Hz (1 m
Ω
/ Rm)
(Rm / 2 G
Ω
)
10 kHz
(1 m
Ω
/ Rm)
(Rm / 1.5 G
Ω
)
100 kHz
(4 m
Ω
/ Rm)
(Rm / 50 M
Ω
)
Rm = the resistance of the device being
measured.
Note that Kl is negligible for resistances above
100
Ω
and Kh is negligible for resistances below
1 k
Ω
at all frequencies.
