I think your goal is laudable, but doing it will be very time consuming, tricky, and take a lot of statistics.

The Southern California Earthquake Center is forming a Center for the Study of Earthquake Predictability (notice that it is NOT a center for earthquake prediction, which might be taken to imply that earthquakes are inherently predictable). They are doing what you suggest, but aimed at scientists who can fit their schemes into some preordained forms that facilitate rigorous testing (a good idea, in my opinion).

Dealing with non-scientists would be more difficult that dealing with scientists - a lot of the predictors here might try to argue with the laws of probability, pushing selected propitious predictions and omitting less successful ones, and not considering tests against random outcomes as valid.

Still, if someone came up with a successful method, it would be a great service. I'm just not optimistic earthquake prediction, if it is even possible, it likely to emerge from such an endeavor.