Understanding Latency versus Throughput

The purpose of this post is to clarify two "systems" terms that are usually confused and sometimes used interchangeably: latency and throughput.

Definition of terms

Let us attempt to define those two terms:

Latency is the time required to perform some action or to PRoduce some result. Latency is measured in units of time -- hours, minutes, seconds, nanoseconds or clock periods.

Throughput is the number of such actions executed or results produced per unit of time. This is measured in units of whatever is being produced (cars, motorcycles, I/O samples, memory Words, iterations) per unit of time. The term "memory bandwidth" is sometimes used to specify the throughput of memory systems.

A simple example

The following manufacturing example should clarify these two concepts:

An assembly line is manufacturing cars. It takes eight hours to manufacture a car and that the factory produces one hundred and twenty cars per day.

The latency is: 8 hours.

The throughput is: 120 cars / day or 5 cars / hour.

A design example

Now that these two concepts are clear, let us apply these concepts to a problem "closer to home." 

Clock frequency:                                          100MHz

Time available to perform the computation:     1000ns

Throughput of the device:                              640 Mbits / second

Word width of each output:                            64 bits

Let us translate these requirements into latency and throughput measurements that are more meaningful from the point of view of the hardware designer.

Latency: 1000 ns = 1000 ns * (1 s / 10^9 ns ) * ( 100 * 10^6 clock periods/ 1s) = 10^11/10^9 = 100 clock periods.

Throughput = 640 Mbits / s = (640 * 10^6 bits/s) * (1 word / 64 bits) * ( 1 s / 100 * 10^6 clock periods) =  640 * 10^6 / 64 * 100 * 10^6 = 10 * 10 / 100 = 1 / 10 = 0.1 words / clock period.

The throughput could be read more conveniently as follows: "one word every 10 clock periods"

Latency expressed in clock periods, and throughput expressed in number of available clock cycles between words, are parameters that a designer can use to create the desired hardware according to the performance specifications.

A final clarification

Some tools do not express the throughput in units per unit of time but in clock periods. This is incorrect but commonly used because of convenience. Therefore some tools would report the throughput of our communication algorithm as 10.

This Team ESL posting is provided by Dr. Sergio Ramirez, Sr Staff Product Engineer for the C-to-Silicon Compiler high level synthesis product.

- See more at: https://community.cadence.com/cadence_blogs_8/b/sd/archive/2010/09/13/understanding-latency-vs-throughput#sthash.Jsb88dob.dpuf