Probably the best…. - Blair McPherson

Managing data means being aware of the limitations of data. You’ve hear managers say ,”the numbers don’t lie” but books have been written about how to lie with statistics! That’s why there are three versions of average and why the average can seem quiet reasonable but is hiding some very troubling extremes.

 

The use of data to produce algorithms intended to be more efficient, more reliable and removing human bias has provided examples of just what can go wrong. In the USA judges were encouraged to use an algorithm to decide both the risk of reoffending and whether to agree bail or remand in custody. The use of the algorithm was supposed to remove the risk of unconscious bias. Research found that rather than reducing the number of African Americans refused bail and given custodial sentences it increased them. The problem was that the big data and resulting algorithm looked at a very large number of offenders and their characterises, educational achievements, employment status, area in which they lived , parents occupation, whether from a single parent family and a list of other factors that repeat offenders had in common and used this to predict the likelihood of a new offender becoming a repeat offender. Sounds scientific and impartial but of course the algorithm is built on the past unconscious bias decisions of judges. 

 

Closer to home a top university experimented with an algorithm to decide who to offer places to. This was seen as a more efficient way of dealing with the large number of applicants and importantly removing human bias from the selection process. The expectation was that more women and more students from ethnic minority backgrounds would be offer places but this was not the outcome. And for much the same reasons. The data on which the algorithm was based contained the unconscious bias of the previous recruiters. 

 

On a positive note algorithms have been successfully used in sport. Pioneered by an American football coach but now widely adopted in professional football across Europe by clubs to inform their recruitment and transfer decisions.Big data and algorithms are used to uncover some undervalued players who’s contribution to the team had gone unnoticed.

 

Algorithms are good at probability. The more data the greater the accuracy. My nephew as a student used this knowledge on a number of betting sites and made enough to fund a holiday in an exotic location. Success was however short lived as he found it increasingly difficult to open a new account with a bookmaker. Never the less on graduation he found employment with a national finance organisation largely on the bases of his understanding of the application of big data and algorithms. 

 

It’s important that managers can manage information, in the same way that it’s important that managers are skilled in managing budgets and people.  Managers need to be aware  of the limitations of big data and that as in the case of the US justice system and university admissions if the data is a result of human decisions the algorithm will contain their unconscious bias.