← Research

Comparison of automated crater catalogs for Mars from Benedix et al. (2020) and Lee and Hogan (2021)

Christopher Lee

Crater mapping using neural networks and other automated methods has increased recently with automated Crater Detection Algorithms (CDAs) applied to planetary bodies throughout the solar system. A recent publication by Benedix et al. (2020) showed high performance at small scales compared to similar automated CDAs but with a net positive diameter bias in many crater candidates. I compare the publicly available catalogs from Benedix et al. (2020) and Lee & Hogan (2021) and show that the reported performance is sensitive to the metrics used to test the catalogs. I show how the more permissive comparison methods indicate a higher CDA performance by allowing worse candidate craters to match ground-truth craters. I show that the Benedix et al. (2020) catalog has a substantial performance loss with increasing latitude and identify an image projection issue that might cause this loss. Finally, I suggest future applications of neural networks in generating large scientific datasets be validated using secondary networks with independent data sources or training methods.

PDF ↗

Models

Loading
EVIDENCE / COMPARISON

Benchmarks

Explore every benchmark

Loading

These charts contain model-linked results. The charts above also include unlinked source profiles. Compare scores within each evaluation. Model quality, usage, transparency and system performance measure different things.

Install TensorTrace