Once upon a time management was about managing budgets and people
plus possibly but not necessarily buildings and equipment. Information
management would be missed off many people’s lists. But now Big
Data is the big thing. It’s advocates are evangelical in their
enthusiasm for the benefits of bid data, not only does it tell us what
we have done in forensic detail but if used correctly it can make us
do things better in the future. It’s the way to greater efficiency,
improved performance and increased competitiveness. Like anything, it
can be used for good or for evil. It can be used appropriately given
known limitations, or stretched wantonly until its principles fray.
But data is like people - interrogate it hard enough and it will tell
you whatever you want to hear. So managing data requires specific
skills in the same way that managing budgets and managing people does.
Managing data means being aware of the limitations of data.
You’ve hear managers say ,”the numbers don’t lie” but books have
been written about how to lie with statistics! That’s why there are
three versions of average and why the average can seem quiet
reasonable but is hiding some very troubling extremes.
The use of data to produce algorithms intended to be more
efficient, more reliable and removing human bias has provided
examples of just what can go wrong. In the USA judges were
encouraged to use an algorithm to decide both the risk of
reoffending and whether to agree bail or remand in custody. The use
of the algorithm was supposed to remove the risk of unconscious
bias. Research found that rather than reducing the number of African
Americans refused bail and given custodial sentences it increased
them. The problem was that the big data and resulting algorithm
looked at a very large number of offenders and their characterises,
educational achievements, employment status, area in which they
lived , parents occupation, whether from a single parent family and
a list of other factors that repeat offenders had in common and used
this to predict the likelihood of a new offender becoming a repeat
offender. Sounds scientific and impartial but of course the
algorithm is built on the past unconscious bias decisions of judges.
Closer to home a top university experimented with an algorithm to
decide who to offer places to. This was seen as a more efficient way
of dealing with the large number of applicants and importantly
removing human bias from the selection process. The expectation was
that more women and more students from ethnic minority backgrounds
would be offer places but this was not the outcome. And for much the
same reasons. The data on which the algorithm was based contained
the unconscious bias of the previous recruiters.
On a positive note algorithms have been successfully used in
sport. Pioneered by an American football coach but now widely
adopted in professional football across Europe by clubs to inform
their recruitment and transfer decisions.Big data and algorithms are
used to uncover some undervalued players who’s contribution to the
team had gone unnoticed.
Algorithms are good at probability. The more data the greater the
accuracy. My nephew as a student used this knowledge on a number of
betting sites and made enough to fund a holiday in an exotic
location. Success was however short lived as he found it
increasingly difficult to open a new account with a bookmaker. Never
the less on graduation he found employment with a national finance
organisation largely on the bases of his understanding of the
application of big data and algorithms.
It’s important that managers can manage information, in the same
way that it’s important that managers are skilled in managing
budgets and people. Managers need to be aware of the limitations
of big data and that as in the case of the US justice system and
university admissions if the data is a result of human decisions the
algorithm will contain their unconscious bias.