Saturday, August 13, 2011

Arctic Ice: Prediction That Isn't

A recent story suggests, based on computer modeling, that the decline of arctic sea ice may be a good deal further in the future than various people predicted, fifty  or sixty years instead of five or ten. It struck me because I had some posts a while back pointing out that a NASA/JPL web page was misrepresenting the facts on the subject, claiming a continued decline in the face of a (perhaps temporary) reversal. 

The other thing that struck me about the post was that the author did not understand the nature of computer modeling and how you test it:
Accuracy of future predictions was checked by running simulations of the late 20th century. The Model replicated the events of the past well enough to suggest that its forecasts of possible futures are realistic.
How are such models created? By fitting to past data. Having used that data in constructing the model, it is no longer available to test it.  Someone is said to have claimed that with ten parameters he could fit the skyline of New York. Assuming he did it, it does not follow that by keeping the same regression coefficients while increasing the range of the parameters he could predict the skyline of the rest of the country.

In order to test the predictions of a model you need to do it against actual predictions—information that didn't go into building the model.

No comments:

Post a Comment