The Artificial Neural Networks for Spectral Analysis (annsa) Python package applies deep learning to problems in homeland security using gamma-ray spectroscopy. This talk demonstrates how an open source machine learning research project is built and applied. We discuss how we applied annsa to challenging scenarios in nuclear material detection and demonstrate annsa’s capabilities in the context of short measurement times, low signal-to-background signals, and unreliable detector calibration.