Data is never objective. Sometimes we must also give free rein to experience and intuition

Many Norwegian businesses want to be data-driven. They want the decisions they take, on everything from product development to operations and maintenance, to be based primarily on the data they gather.

But a Cambridge professor has now published a scientific article demonstrating that data are never objective.

Taken together with my own research as a basis, I would like to offer some advice to all those with ambitions to become data-driven. Sometimes we must also give free rein to experience and intuition.

From sensor data to questionnaires

Data are very diverse. One example are the readings of CO2 levels in a meeting room, measured by sensors. Or the responses you may receive in a questionnaire.

Many companies make key decisions on the basis of such data, based on an assumption that the information they gather is an accurate reflection of reality. Perhaps, where you work, you have been involved in discussions about the upgrade of the ventilation system, which has been decided on the basis of sensor readings.

Even more thought-provoking is the recent news from Professor Matthew Jones at the Judge Business School at the  University of Cambridge.

Three limitations to data

Jones points out that all data sets have three limitations, all of which make it unwise uncritically to trust what the data are telling you.

• There is a limit to the types of data that can be collected. No technology can measure everything. Data linked to fields such as the law and personal privacy may exhibit similar limitations.
• Pragmatic considerations, such as economics, make it necessary to be selective about the ways in which we gather data, leading to information that is of somewhat less quality that we would ideally like.
• It is difficult to know whether the data you have commissioned has actually been gathered in the ways you intended and supposed.

According to some recent findings that we have made at SINTEF, it is possible to mitigate the third of these limitations very significantly, thus making Professor Jones’ bleak scenario perhaps a little brighter. 

Human and technical factors

But first of all, a little more about what Professor Jones has to say about his third source of data error. According to the Professor, there are many reasons why you cannot entirely trust that the data gathered is the same as that which you commissioned.

In the first place, there are human factors. Perhaps those who construct questionnaires are not fully familiar with the fields in which the data will be applied? As a result, they may set questions that are imprecise or even irrelevant.

The second factor is technical. Perhaps a sensor is taking inaccurate readings?

Combinations of human and technical factors may also lead to errors. For example, the installer of a sensor may not have known how to install the instrument in order to ensure adequate accuracy of the readings.

But such misunderstandings can be prevented. A recent study carried out by our team at SINTEF, looking into the day-to-day routines of building caretakers at three Norwegian property companies, demonstrates exactly how.

Caretaker study findings offer a solution

Our study showed that communication between the users of sensor results and those who actually gather the data is invaluable.

Data quality increased when building caretakers found out who was getting the results and what the readings were to be used for. The reason is quite straightforward. If you know the people who will be using the data you are gathering, you can also ask them why the information is being collected.

This offered the caretakers greater awareness and enabled them to ensure that the sensors in the ventilation system were suitably located. At the same time, they received sufficient know-how to be able to remove possible sources of error in the readings.

Invisible data collectors

The problem has been that those who gather data have all too often been invisible to the users of the information.

The engineer responsible for the HVAC systems did not communicate with the caretaker who took readings from the sensor equipment in the ventilation system. And the sales manager planning a new campaign has not spoken to the team that designed the questionnaire on which the campaign is being based.

Our findings from the caretaker study are so important that we have now, with funding from the Research Council of Norway, launched a similar study involving IT developers employed by the IT consultants Knowit, the ticketing solutions company Entur, the Norwegian Labour and Welfare Administration (NAV) and the Norwegian Mapping Authority (Kartverket).

IT developers as well

Our hypothesis, which so far appears to be correct, is that the quality of software products is improved when those who develop the programs get the opportunity to communicate with those who have been gathering the data that support the development process.

According to a Norwegian government white paper published last year, the so-called data economy, which involves the exploitation of the Internet of Things and artificial intelligence, generated wealth amounting to NOK 150 billion in Norway in 2020.

The paper outlines how the public sector intends to stimulate Norwegian businesses towards being data-driven. If we are to achieve this goal and generate greater wealth from our data, we have to learn how to use the data correctly. Don’t fool yourself into thinking that your data are always reflecting reality. And don’t forget that your gut feeling represents an expert evaluation built on many years of experience.

Your company’s success depends on combining your gut feeling with a realistic awareness of your data. Only then will you obtain a picture of the world that is close enough to reality to enable you to establish a secure foundation for your decisions.

This article was first published in the newspaper Dagens Næringsliv (DN) on 29. October 2022 and is reproduced here with the permission of the paper.