You're missing the point.

Evaluation establishes credibility.

You can't allow someone to change things in the middle of a prediction. Suppose you predict a quake this week, then Saturday you say "Never mind, cancel this" when nothing happens. You're running your score up by dropping out the misses.

Now admittedly, if you have a valid method and get new information (as the weather people do) you must be able to change things, but this is not an evaluation stage. At this point you have already established your accuracy and are now a working predictor.

The question then becomes whether the prediction is true even if changed midstream. That's a different situation.

Roger