Instead of focusing on bettering your 401K, big banks turned their efforts towards predicting who would win the World Cup. Goldman Sachs, UBS, and Dutch Bank were amongst the notable names. Some running as many as 1 million simulations and they all were wrong – none had the winner, France, even making it to the finals. Why were these financial institutions with robust machine learning tech so horribly off?
Two areas of failure: defining the “problem” and setting proper requirements.
Goldman Sachs analyzed data points about both teams and individual players. UBS based their assessment on a team’s custom “object skill level measurement” rating. They had Germany pegged as the winner with a 24% certainty. Dutch came at it a bit differently by judging a team’s market value, presuming that a team’s market worth is closely correlated to its success.
“It is difficult to assess how much faith one should have in these predictions… But the forecasts remain highly uncertain, even with the fanciest statistical techniques, simply because football is quite an unpredictable game. This is, of course, precisely why the World Cup will be so exciting to watch,” wrote Manav Chaudhary, Nicholas Fawcett and Sven Jari Stehn, the three economists behind Goldman Sachs’ project.
Do you think machine learning could have predicted this more closely? If so, how?
Does this type of fail make you second guess using tech for major life decisions? I.e. Investing, health decisions, etc.
Source and more info:
http://observer.com/2018/07/goldman-sachs-ubs-predict-world-cup/
https://medium.com/@bakhtiyari/artificial-intelligence-failed-in-world-cup-2018-6af10602206a
Since the World Cup is only every 4 years (with a drastically different squad each occurrence) was it possible that they didn’t have enough data and the right kind of data for this tournament? I’m not a soccer fan, but I do enjoy betting on NFL games via Draftkings where algorithms are everything. However, Rookie players and weather conditions make algorithms go haywire because they don’t have the right kind of historical data 🙂
This post took me back to last semester and our discussion about the Money Ball case. While data is great and can have incredible value, there are too many variables in sport to predict an outcome. In addition, the players on national teams play against each other and together as a squad so infrequently, as Emily mentions, that the data has to have flaws.
I think that if they had defined things a bit differently, they might have gotten closer to having France as the answer. It seems like they focused too heavily and put weight in one thing (players stats vs. skill level). Tying this back to the Watson case, there were many different variables that needed to be at play in order to get an answer correctly and their methods may not have weighed the categories correctly. But at the end of the day, I think in sports and in Jeopardy and in the stock market, it doesn’t hurt to have a little human luck.