Spread Models Are Scored by Perimeter Overlap and Arrival-Time Error
A modelled fire that looks like the real one is not evidence. Validation compares a simulated perimeter against an observed one at the same instant using set-overlap measures: the Jaccard index, the area of the intersection divided by the area of the union, and the Sorensen coefficient, twice the intersection divided by the sum of the two areas. Both run from 0 to 1, both punish a model equally for burning ground the fire missed and for missing ground the fire burned, and neither is impressed by a shape that is right but displaced. Where a full arrival-time field is observed, the sharper score is the arrival-time error at each point, in minutes, which shows whether the model is uniformly slow or wrong only in one direction. Two cautions matter. A big fire scores well on overlap almost automatically, because the two large areas must intersect, so scores are only comparable between fires of similar size. And a model tuned until it matches one fire has been fitted, not validated. After this Concept you can score a modelled perimeter against an observed one and say what the score does not tell you.
This Concept is waiting for its first lesson!
A modelled fire that looks like the real one is not evidence. Validation compares a simulated perimeter against an observed one at the same instant using set-overlap measures: the Jaccard index, the area of the intersection divided by the area of the union, and the Sorensen coefficient, twice the intersection divided by the sum of the two areas. Both run from 0 to 1, both punish a model equally for burning ground the fire missed and for missing ground the fire burned, and neither is impressed by a shape that is right but displaced. Where a full arrival-time field is observed, the sharper score is the arrival-time error at each point, in minutes, which shows whether the model is uniformly slow or wrong only in one direction. Two cautions matter. A big fire scores well on overlap almost automatically, because the two large areas must intersect, so scores are only comparable between fires of similar size. And a model tuned until it matches one fire has been fitted, not validated. After this Concept you can score a modelled perimeter against an observed one and say what the score does not tell you.
Are you a teacher? Sign in to start contributing.
Sign In