|
Evaluation encore une fois |
I don't know, I turn my back for a few days and a huge thread appears on systems for scoring predictions and evaluating them. I can't follow it all because some of the posts seem to have got chewed up. You can argue for years about how big the fall-off margin should be and how rapidly your points should decrease if you don't hit the parameters on the nail, and you'll never come to a final agreement because the result is arbitrary whatever it is. The only approach that makes sense is to use a system in which the precise details of the scoring system are not important. You do this by running a random prediction sequence in parallel with the real predictions. Score both of them the same way. The degree of success of the predictor is measured by the extent to which he/she does better than the dummy predictions. If you make the scoring system really generous to the predictor, it's generous to the dummy predictions as well and so the predictor has to do really well to be better than chance. In that case any scoring system will do, and you can save a lot of argument. After all, at the end of the day, what you want to know is not whether the predictor scores 100 points or 1000 points, but whether the results are better than chance. THAT is the thing that has to be evaluated. Follow Ups: ● Re: Evaluation encore une fois - Canie 13:56:54 - 4/20/2001 (6883) (1) ● Re: Evaluation encore une fois - michael 15:27:35 - 4/20/2001 (6893) (0) ● Re: Evaluation encore une fois - Roger Hunter 13:06:51 - 4/20/2001 (6880) (1) ● Re: Evaluation encore une fois - Roger Musson 07:53:32 - 4/21/2001 (6907) (0) ● Re: Evaluation encore une fois - michael 09:50:22 - 4/20/2001 (6875) (2) ● Re: Evaluation encore une fois - Roger Hunter 13:17:16 - 4/20/2001 (6882) (1) ● Earthquake Prediction Registry - Under Construction - michael 16:17:47 - 4/20/2001 (6899) (1) ● Re: Earthquake Prediction Registry - Under Construction - michael 16:19:30 - 4/20/2001 (6900) (1) ● Re: Earthquake Prediction Registry - Under Construction - Roger Hunter 20:19:49 - 4/20/2001 (6905) (0) ● Re: Evaluation encore une fois - Don in Hollister 10:29:28 - 4/20/2001 (6878) (1) ● Learning - michael 15:30:11 - 4/20/2001 (6894) (0) |
|