Fair Scoring Rules


I have been looking for this for a long time! It is a way to give someone incentives to be honest with you about their real opinions on something.

It starts with the question: if you have a spinner that comes up "red" 60% of the time and "blue" 40%, and I will pay you a dollar each time you predict the result correctly,

how should you structure your guesses? The wrong answer is: "guess red 60% of the time and blue 40%". Actually, you should guess red 100% of the time, and you will be right

60% of the time. That is the best you can do. Now, say that you happen to know how often the spinner comes up red, but I don't, and I want to extract that information

from you - how could I do it? just asking you what you would bet doesn't tell me anything except that you think that there's a greater than 50% chance it will come up red. So

how can I get the rest of the information out of you?

Another example - when we ask a weather reporter whether it'll rain tomorrow, we're really just asking them if they think there's a more than 50% chance of it raining; cause they have

nothing to gain by letting you know the real probability is somewhere between 50 and 100% - so they just act confidently and act as if it's 0 or 100% no matter what.

A proper scoring rule fixes that problem - the person should now just tell you what they really think.

So if you are talking to an expert, you can cut through his own attempts to manipulate the answers - the scoring rule lets you give him odds that will reward him for

giving you his real prediction.



Related Pages

3/22/2013 6:52:13 PM
Contact: akasatanahamayarawan@gmail.com