Archaeologists trained AI on fake, simulated sites to find real ones no one expected

[PulseRider]

A new study out of Kisatchie National Forest in Louisiana tackled a problem that trips up a lot of machine learning applications in archaeology, what do you do when there simply aren't enough known examples of a site type to train a reliable model. Researchers had only 12 known examples of a set of unusual circular structures that resembled historic tar kilns, structures once used to extract tar, pitch, and resin from pine trees, but their shape didn't quite match known tar kilns found elsewhere in the region.

Instead of trying to make a model work with just 12 real examples, the team generated simulated versions of the structures and digitally placed them into real lidar terrain data, essentially creating an artificial training set from scratch. They tested two versions of this approach alongside a third model trained on real tar kilns from South Carolina with the elevation data adjusted to better match the Louisiana structures. All three models ran on a Mask R-CNN architecture, trained on tiled lidar images split into training, validation, and test sets.

The results were a genuine mixed bag depending on which model you looked at. The most conservative model found 9 of the 12 known targets with relatively few false positives. The more aggressive model found all 12 known targets plus 11 additional structures worth checking out, but it also generated 686 false positives, mostly things like reservoirs, drainage features, and natural mounds that superficially resembled the target shape. Researchers had to build additional filters based on ground shape just to separate genuine candidates from the noise.

What makes this story interesting isn't really the AI methodology though, it's what happened when researchers actually went out and checked the sites in person. Auger testing at two of the structures turned up none of the charcoal, charred wood, or hard clay floors you'd expect at an actual tar kiln. Combined with the site's location near a former World War II training area, researchers now think these structures are more likely leftover howitzer emplacements from military training rather than anything related to the tar industry at all. The AI helped narrow down a huge landscape to a manageable list of targets worth checking, but it took an actual archaeologist with a shovel to figure out what those targets actually were

AI trained on simulated sites finds unknown features in lidar data | Archaeology News Online Magazinearchaeologymag.com

[SyntaxMage43]

The twist that these turned out to be howitzer emplacements instead of tar kilns is the best part of this whole story. Good reminder that AI can point you at something unusual without having any idea what that unusual thing actually is

[Jonathan]

686 false positives out of the more aggressive model's predictions sounds bad until you remember that manually surveying that same landscape on foot would have taken vastly longer than filtering through a pile of automated predictions. Even a model with a high false positive rate can be a huge net time saver if the filtering process afterward is fast enough. The real question is whether the false positive rate stays manageable as the terrain gets more complex or the target shape gets less distinctive. In this specific case it clearly worked out, but that's not guaranteed to generalize to every future application