Hi all;

Petra has been angry with me for some time now because her evaluation for 2007 turned out badly and she felt that it was because there were no firm rules for evaluations.

I couldn't get it across to her that there are no "set in concrete" rules but in all fairness, I could at least post MY rules.

So here they are and I'd like suggestions for improving them.

-------------------------------------------------------------------------
Principles of Earthquake Prediction Evaluation.

Prediction requirements

1) Date

The prediction must specify the date when the quake will happen. If a date range is given, the
quake must happen within that range. Near-miss dates may be specified but will receive smaller
scores based on probability of success. If times are given, time zone must be specified.

2) Location

Location must be given in an unambiguous manner. Center point and radius, lat-lon boundaries
or areas which have defined borders such as states are acceptable.

3) Magnitude

Magnitudes must be given, usually as a range such as 4.0 to 9.9. A single value such as 4 would
be taken as 4.0 to 4.9 whereas 4.5 would allow no other answer.

4) Probability

The expected probability should be the statistical chance of success, not the predictor's
subjective confidence level. It may be omitted as it will be calculated as part of the evaluation.

Practices which are not allowed.

1) Predictions must be made before the expected date

2) A series of predictions for the same location may not have consecutive dates.

3) Predictions may not overlap in both date and location.

Evaluation procedures

1) Predictions from only one source at a time will be evaluated. The exception is when a group
jointly contribute to predictions.

2) Earthquakes from only one source will be examined. The source may be selected for special
situations but not on a case by case basis. Sufficient time must elapse before evaluation begins
since quake parameters may be revised while new information is coming to the centers.

3) Predictions will be judged strictly on what is said. "Close enough" is not allowed except when
near-miss regions are defined in advance.

4) Predictions will be awarded the first quake which fits all the parameters as stated.

5) Each quake can satisfy only one prediction and will not be considered for other predictions.

6) If a quake has been given multiple magnitudes the largest value will be used.

7) Aftershocks will be accepted if the prediction was made before the main shock happened.

8) Probability is necessary in determining significance. It will be assigned in one of two ways.
When all predictions have the same probability, as when predictions are essentially identical except
for dates, the probability will be determnined by the Jones Observed Probability method and that
value will apply to all such predictions. If predictions are all different, values will be
computed by the Jones method for each prediction.

Significance calculations

There are two tests of significance which may be used, depending on the circumstances.

If all predictions have the same probability the z-binomial test may be used. This is a standard
test of significance and may be found on the VassarStats website as an interactive applet.

If each prediction has a different probability the method devised By Dr Jones will be used. This
procedure assigns a score to each prediction. If the prediction is correct the score is -ln(p)
and if it's incorrect the score is ln(1-p), where ln is the natural log and p is the probability
of success. Each prediction has an expected value which is -p*ln(p) + (1-p)*ln(1-p) and a variance
which is p*(1-p)*(ln((p)*(1-p)))^2.

These values are summed and the standard deviation is the square root of the variance. The
final answer is (score-expected)/standard deviation. This quantity is normally distributed so the
significance is determined by comparing it to a table of the normal distribution.

The Jones Observed Probability method

The problem with earthquake catalogs is that the quakes fall into clusters caused by swarms and
aftershock sequences. Thus simple probability based on the number of days covered by the predictions
can give very misleading answers. The usual procedure is to decluster the catalog by various methods.

The Jones method has the advantage of simplicity and while it is not perfect, it greatly reduces
the clustering influence, particularly if long windows are being predicted. The procedure is as
follows:

1) Select the time span of interest.

2) divide the time span into consecutive windows of the predicted length

3) Sort the quakes in that time span into the corresponding windows

4) Count the total number of windows in the time span

5) Count the number of windows containing one or more quakes

6) Divide the number of windows containing quakes by the total number of windows. The answer
is the probability that such a window will contain a quake.

The time span to be used is a matter of opinion. A long time span gives a long term average answer
whereas a shorter time can better suit changing circumstances.

-----------------------------------------------------------------------------

Roger