Technical data

Using Auto Scale, Cursors, Measurements, and Waveform Math 2
Agilent InfiniiVision 7000B Series Oscilloscopes Evaluation Guide 23
The value of the “X1” cursor should now be measuring the carrier
frequency of this signal, which is 2.0 MHz. Without “Precision” turned
on, all automatic parametric measurements and waveform math provide
approximately 0.1% measurement resolution based on the oscilloscope’s
full-screen timebase setting. When “Precision” is turned on, at the cost
of waveform update rate, measurement resolution increases to
approximately 0.001%. The FFT measurement, which we used in this
demonstration, is the slowest math function.