Hi all;

I have been criticised elsewhere for not explaining my evaluation methods understandably so let's see if this helps.

Don in Hollister has posted a series of earthquake predictions
on the Earthwaves website. Since there were a sizeable number of them
I decided to run a statistical analysis on them to see how well he's
doing.

First I had to download his predictions, which are in a series of posts
archived on the website. Each post contains explanatory text so it was
necessary for my purposes to edit this down to one line in the proper
format for my evaluation programs to read.

Once this was done the evaluation program compared all his predictions
to all the quakes in the NEIC database, looking for quakes which matched
his predictions. The program also determined the probability for each
prediction by counting how many windows of the predicted duration
contained quakes of the predicted size, using a 5 year sample from the ANSS
database. Hit windows divided by total number of windows gives
the desired probability. This procedure is called the Jones Observed
Probability after it's originator, Dr Alan Jones of Purdue University.
Dr Jones devised it to eliminate as much as possible the effect of
clustering in the database.

The output of the evaluation program was trimmed down to a list of
probabilities flagged as hit or miss in each case. This became the
input to the next program which calculates the standard deviation of
the series by a formula again devised by Dr Jones to overcome the
need for predictions to have identical probabilities. The z-binomial
test would be used for such a series but such circumstances are seldom
encountered. One exception is the predictions of Jim Berkland who
predicts for the same areas each month.

The procedure is as follows:
For each prediction calculate the score, which is -ln(p) for a hit
or ln(q) for a miss (where ln is natural log and q is 1.0 - p)
The expected value of the score is -p*ln(p) + q*ln(q)
The variance of each score is p*q(ln(pq))^2
Subtract the expected value from each score and sum.
Divide the sum by the square root of the sum of the variances.
The result is a normal variable with mean 0 and standard deviation of 1.

The final result of this calculation shows that a series of 226 of Don's
predictions had a standard deviation of 4.299 which is far higher than
the usual standards of testing require.

So, does this help or is everyone totally confused?

Roger