Calculating Effective Resolution for Data Converters


It is often the case that an application uses a part of the analog range of a data converter. Calculating the effective resolution when it uses half or quarter is easy. This tutorial explains how to calculate the effective resolution when we use any fraction of the range.


A similar Chinese version of this article appeared on ED China, October 19, 2012.

Voltage Overhead as a Fraction of a Converter's Range

Analog systems normally have some overhead to adjust for gain errors, drifts, design tolerances, or poorly aligned equipment. When converting between the analog world and the digital world, we need to allow overhead in the digital world too. Consider an industrial control voltage of 0 to 10V. If we only allow an analog-to-digital converter (ADC) to digitize 10V maximum, then any downstream equipment must be limited to 10V or we lose information. Therefore, in industrial control it is common to allow a 5% or even 20% overhead capability.

Other systems such as video systems often add synchronization signals to the video signals. A 1VP-P video signal could easily consist of 700mV of useful video information and 300mV of sync pulse. If a 12-bit ADC is used to digitize such a signal, the video itself will only use 70% of the available range, or 2867 codes from the 4096 available codes. Now if we consider 5% overhead, we reduce the range used even further.

Therefore when converting between analog and digital worlds, we must ensure that our digital world can cope with the overhead. Fine. This makes sense, but the downside of coping with overhead is that effective resolution decreases.

Calculating the Effective Resolution for Any Fraction of the Analog Range

We start with a child's math exercise—or is it that simple?

My son recently asked me for help with a school math question. It went like this. I have a huge sheet of paper and cut it in half. I lay the two halves on top of each other so the total thickness is doubled. I now cut the halves in half again and lay them, once again, on top of each other. Now the thickness of the pad is four times as thick as a single sheet. I repeat the process again and again. How many times do I have to repeat the process before the height of the stack reaches the moon?

The formula that he needed to derive was very similar to that used to calculate effective resolution. It uses logarithms.

Consider an industrial control example where we need a voltage between 0 and 10V with 20% overhead. This is 0 to 12V. If we use a 16-bit digital-to-analog converter (DAC) for this, what is the effective resolution of the 0 to 10V signal?

We know that for a DAC with R bits of resolution, we have 2R levels. So, defining N as the number of levels:

N = 2R

We need to solve this for R, and this is where we need to use logs. We take the log of both sides:

Log(N) = R × Log(2)

Now, it is easy:

R = Log(N)/Log(2)

Returning to our industrial control example, we actually only use 10/12 = 0.833 of our available levels for the 0 to 10V range. In a 16-bit system, this is 54613. So putting the numbers back in, we can calculate the effective resolution:

R = Log(54613)/Log(2) = 15.7

Therefore, by allowing 20% overhead, we have only reduced our effective resolution by around 0.3 bits.

In fact, if we think in terms of bits only, the number of bits that we reduce is independent of the original resolution. We can simply use the ratio of codes used to codes available and derive the reduction in bits.

Δr = Log(r)/Log(2)

Thus, in the video example where we have 700mV video and 300mV sync, we use 0.7 of the available codes:

Δr = Log(0.7)/Log(2) = -0.51

We lose 0.51 bits. So in a 12-bit system, the effective resolution is 11.49 bits, and in a 16-bit system it is 15.49 bits.

For those who are wondering about connecting the earth to the moon with paper, here goes. The thickness of the stack, T = p × 2C, where p is the thickness of a sheet of paper and C is the number of cuts that I make. Notice the similarity? In the same way, we can solve this for C such that C = Log(T/p)/Log(2).

Where did this get me and my son's homework? I measured a piece of printer paper at 0.11mm thick. The moon is around 300,000km from earth. So, we need Log(3 × 1011/0.11)/Log(2) = 42 cuts. Off we go then...that will not take too long. It is an amazingly small number of cuts.


In any system that converts between the analog and digital worlds, we have to account for overhead. This often reduces the effective resolution in the system. An equation has been derived to calculate effective resolution, given the fraction of digital range used for the normally scaled analog signal. It has been shown that, in fact, even using a moderately large overhead only reduces effective resolution by a fraction of a bit.