Trust and Error in AI-Assisted History | Gitre, Backer | AI and History Conference 2026

Edward Gitre and Samuel Backer on hallucination and error in AI-assisted history research, at the AI and History Conference 2026. Edward J. K. Gitre and Samuel Backer each present a case of error in computational history and then take audience questions. Gitre describes The American Soldier in World War II, a project that reconciled about 250,000 transcription contributions from roughly 7,600 Zooniverse volunteers using a Jaccard dissimilarity method and hand review, and that found natural language processing (NLP) tools misreading race-related vocabulary. Backer argues that error has to be understood before a method is chosen, using optical character recognition (OCR) and Gensim topic modeling on the Lester S. Levy sheet music collection. He also reports work with Louis Hyman showing that historians need precision more than recall, and a failed attempt to extract ledger data from theater booking records with a large language model (LLM). Speakers: Edward J. K. Gitre (Virginia Tech) and Samuel Backer (University of Maine). 0:00 Introduction: error and hallucination in computational history 2:06 The American Soldier in World War II: 65,000 microfilmed survey pages 6:43 Prototyping in Omeka and moving transcription to Zooniverse 8:15 Choosing the best transcription with Jaccard dissimilarity 15:03 Backer's three steps for error, applied to Levy sheet music and Gensim topic modeling 22:46 Part two: quality standards, then tags and word embeddings with Virginia Tech data scientists 27:55 Network analysis of race relations and bias in NLP tools 32:16 Error in OCR work with Louis Hyman and why Levenshtein distance misleads historians 40:04 Failed ledger extraction from theater booking records 46:01 Q&A: verifying output by random sampling and quantifying error rates 51:49 Q&A: obsolete media and the future of LLMs 55:28 Q&A: AI guardrails on sensitive topics and coding agents Talk page: https://proflouishyman.github.io/ai_c... Conference program: https://proflouishyman.github.io/ai_c... All talks:    • AI and History Conference 2026   Recorded at the AI and History Conference 2026, Johns Hopkins University (SNF Agora Institute), in partnership with the American Historical Association. Hosted by the Center for Economy and Society. #DigitalHistory #AIandHistory #HistoryAndAI #AIHallucination #OCR