Math-MateSchool maths, made clearer

Search the complete textbook

Find a Math-Mate lesson

Type at least two letters to search 240+ lessons.

Topic 34 · lesson 2 of 4

Precision In Numbers

Clear explanations and worked examples from the Math-Mate school maths textbook.

Free lessonNo sign-upPrint friendly

Precision in Numbers

The precision of a number is a similar concept to significant figures.  For instance, if I was measuring the length of my hand, I might be able to measure it to the nearest millimetre and get a length of 18 mm.  I would have measured my hand’s length to a precision of 1 mm.

If I had more precise equipment, I might be able to measure the length of my hand and get a length of 18.4 mm.  In this case, the precision of my measurement would be 0.1 mm, not 1 mm.

Like for significant figures, there can be confusion about precision when you get a measurement like this:

                                                              300 m

Was this measurement precise to 100 metres or 1 metre?  Strictly speaking, since two zeroes have been written down in the ‘10’s and ‘1’s part of the number, I would read this number as being 300 metres with a precision of 1 metre.

Using scientific notation avoids this potential confusion.  If I wanted to tell the reader the measurement was 300 metres with a precision of 100 metres, I would write:

                                                           Mathematical diagram

If I wanted to tell the reader I meant 300 metres with a precision of 1 metre I’d write:

                                                         Mathematical diagram

Precision can also be talked about in terms of significant figures.  For instance, Mathematical diagram is precise to three significant figures, whereas Mathematical diagram is only precise to one significant figure.