Addn'l Metrics ? - Re: Don's track record for the past 7 months of forecasting
Posted by 2cents on December 08, 2001 at 14:23:58:

Go Get 'Em Don!

I suggest another additional metric for evaluating the cumulative odds against the background rate.

It would be along the lines of estimating the odds of making 'n' predictions and getting both the "Cumulative Odds Metric" (COM) and also the "Cumlative Average Odds Metric" (CAOM).

By way of example, and dealing only with the "C hits" and "AAA Hits"...lets say the odds for reach prediction were as follows:

Pred # Background Odds
1-------------- 1/4
2---------------1/10
3---------------1/5.5
4---------------1/6
5---------------1/7
...
n---------------1/x

The Cumative Odds Metric (COM) is just:

Multiply from 1 to n of the Backround Odds parameter for each prediction:

COM(1 May - 6 Dec)= (1/4)*(1/10)*(1/5.5)*(1/6)*(1/7)*...(1/x)

The Cumlative Average Odds == "COM / n ".

What this does is change the location context and transforms into an "odds space" type of context. This gets people away from discussing "bulleye" type scoring rings, etc. and keeps them discussing "Background Odds"....

Also, the length of the period of the background seismicity record should be specified (and assumed to be uniform for each prediction evaluation or stated otherwise).

Anyway a magnitude 3.0 out in aseismic country would weigh in higher than a magnitude 3.0 in heavily seismic country in the cumlative score...and indicates the quality of soem of the predictions.

In this way, two predictors with the same exact AAA and C hit counts can be further analyzed for cumlative score.

* * *
As an aside, this also raises the subject of possibly always adjusting the prediction parmeters such that it is always at a chance of 1 over "y"...where "y" is chosen to be some value ...say "4".

So all the predictions have a uniform chance of happening. If done, a casual observer may understand that a longer time window for a similar magnitude and distance prediction indicates a lower likelihood area. This form might be useful for deciding what "break point" is needed to make, say, a "public Notice" for a "large quake" (e.g. The Big One...).

(My) 2cents (worth)


Follow Ups:
     ● Re: Addn'l Metrics ? - Re: Don's track record for the past 7 months of forecasting - Petra Challus  15:23:44 - 12/8/2001  (11772)  (2)
        ● Re: Addn'l Metrics ? - Re: Don's track record for the past 7 months of forecasting - 2cents  21:10:49 - 12/8/2001  (11788)  (0)
        ● Predictions? by Don on current Bay Area Activity - Lowell  15:51:29 - 12/8/2001  (11774)  (0)
     ● Re: Addn'l Metrics ? - Re: Don's track record for the past 7 months of forecasting - Lowell  15:16:36 - 12/8/2001  (11770)  (1)
        ● Re: Addn'l Metrics ? - Re: Don's track record for the past 7 months of forecasting - 2cents  16:17:18 - 12/8/2001  (11776)  (0)