Predictive "skill" refers to the ability to make forecasts that outperform some naive baseline. A seasonal climate forecast has skill if it improves upon expectations derived from long-term climatology, a managed mutual fund has skill if it outperforms an index fund, a World Cup prediction has skill if it improves upon a simple way of generating forecasts.
In this competition I am going to employ two naive methodologies. The first is simply to take the FIFA ranking as the basis for deciding the winner of each game. This results in Brazil as the champion, with Spain as runner up. In the ESPN bracket, the rankings are conveniently provided next to the team names. Were you to come from another planet and have no knowledge of soccer, you should do no worse than a forecast generated naively by looking at the rankings. After all, the rankings are supposed to say something meaningful about relative team strengths. This entry is called Naive.2-FIFAWorldRanking in RogersBlogGroup.
I am also using a second naive methodology, which is based on the estimated player value of each national team.
| GROUP A |
| GROUP B |
| GROUP C |
| GROUP D |
|
|---|---|---|---|---|---|---|---|
| South Africa | 35 M€ | Argentina | 390 M€ | England | 440 M€ | Germany | 305 M€ |
| Mexico | 95 M€ | Nigéria | 115 M€ | USA | 55 M€ | Australia | 40 M€ |
| Uruguay | 145 M€ | South Korea
| 50 M€ | Algeria | 55 M€ | Serbia | 185 M€ |
| France | 450 M€ | Greece | 100 M€ | Slovenia | 45 M€ | Ghana | 115 M€ |
| TOTAL | 725 M€ | TOTAL | 655 M€ | TOTAL | 595 M€ | TOTAL | 645 M€ |
|
|
|
|
|
|
|
|
|
| GROUP E |
| GROUP F |
| GROUP G |
| GROUP H |
|
| Holland | 280 M€ | Italy | 400 M€ | Brazil | 515 M€ | Spain | 565 M€ |
| Denmark | 85 M€ | Paraguay | 90 M€ | North Korea | 15 M€ | Switzerland | 115 M€ |
| Japan | 70 M€ | New Zealand | 15 M€ | Ivory Coast | 180 M€ | Honduras | 45 M€ |
| Cameroon | 140 M€ | Slovokia | 70 M€ | Portugal | 340 M€ | Chile | 85 M€ |
| TOTAL | 575 M€ | TOTAL | 575 M€ | TOTAL | 1050 M€ | TOTAL | 810 M€ |
|
|
|
|
|
|
|
|
|
|
| |||||||
| Values in M€ (millions of Euros) | |||||||
|
|
The forecast based on team worth is named Naive.1-TeamWorth, and it has Spain over Brazil in the finals. It is a bit more sophisticated than the FIFA World Ranking to be sure, but it is still a fairly naive metric for forecasting.
I have also entered into the mix forecasts generated by three big financial firms: Goldman Sachs (Brazil), JP Morgan (England) and UBS (Spain). (That previous sentence may provide you with all you need to know to remove all your investments handled by JP Morgan;-) These forecasts can be found from links here. As you can see, they spent considerable effort in making these forecasts, using fairly sophisticated methods akin to ones used to guide their investment decisions. These sophisticated methods should outperform the naive methods, if they are to any value.
Do you think you have skill? Can you beat the big investment banks? Sign up here!
9 comments:
Should be Ghana in Group D instead of Hungary
-1-mowate
Thanks much, teams are always trying to sneak into the competition ;-) Now fixed.
You can consider my entry has the non-expert validation.
We will see if expert have better predictive skill.
How do you factor match-fixing potential into the odds?
-4-Reiner
This of course depends upon who you think will host in 2018, my bets are on UK, however, if you think Russia, you may wish to overweight Spain ;-)
Do any of the matches in District 9 figure into the odds?
I disagree with your naïve forecast metrics because I think they have a bit too much skill. Two metrics used for weather forecast evaluation are chance and climatology. The point being if you cannot even beat a random guess or what the weather has been in the past for the forecast date then you have no skill as a forecaster.
I think your choices of FIFA rankings and estimated player value are much better estimators than the weather forecasting skill-less forecasts. OK, and I suspect that my predictions will be worse than both and don’t want to fail the test.
In any event I made an attempt to provide one of the weather forecast metrics but did not have the data to set up the second. I tried to prepare a chance metric using the excel random function (Excel Random Function). For each match I assigned a random number function to both teams and picked the higher number to go on. In the first round I picked the two highest teams. In order to replicate climatology I think you could pick the winner using the number of each country’s world cup wins. For each match the winner would be the country with more wins. Unfortunately I did not have that information so did not make a projection.
Soccer (ie, football) is really Irish line dancing done with a ball - but without the charm of being drunk.
(Fortunately, spectators can remedy that lack.)
I came across this paper while searching for something else. Worth noting, given the current discussion....
Predicting the World Cup 2002 in soccer: Performance and confidence of experts and non-experts
Patric Andersson et al.
Abstract
This paper investigates the forecasting performance and confidence of experts and non-experts. 251 participants with four different levels of knowledge of soccer (ranging between expertise and almost ignorance) took part in a survey and predicted the outcome of the first round of World Cup 2002. The participating experts (i.e., sport journalists, soccer fans, and soccer coaches) and the non-experts were found to be equally accurate and better than chance. A simple prediction rule that followed world rankings outperformed most participants. Experts overestimated their performance and tended to be overconfident, while the opposite tendency was observed for the participants with limited knowledge. Providing non-experts with information did not improve their performance, but increased their confidence.
International Journal of Forecasting
Volume 21, Issue 3, July-September 2005, Pages 565-576.
doi:10.1016/j.ijforecast.2005.03.004
Post a Comment