Should ChatGPT Code Your Text Data for a Research Paper?

Your paper's key construct is an emotion, and your texts are in a spreadsheet. Should you upload the spreadsheet to ChatGPT and have the model code that emotion? This lecture explains why the answer is no, and how to use GPT and other large language models (LLMs) for content analysis in a way that holds up in peer review. Generative AI makes it possible to code thousands of reviews, complaints, social media posts, or open-ended survey responses in hours. But a score produced by an LLM is only useful in a paper if it measures the construct your theory needs, if the procedure can be reported and rerun, and if it has been validated against the right benchmark. The lecture is aimed at researchers in marketing, consumer behavior, management, and the social sciences, including PhD students, and it uses customer anger in complaints and Yelp reviews as a running example. What the lecture covers: Part 1: Construct first. Why felt anger and perceived anger are different constructs that call for different prompts, and why the prompt should be treated as a coding protocol grounded in the measurement literature. Part 2: Running the coding. What happens when you attach a spreadsheet in a ChatGPT chat window, why chat products are not a reproducible coding procedure, how to call the model through the API instead (including getting an API key and a no-code option), and what the API does and does not fix: reproducibility, model retirement, and data permissions. Part 3: Validation. What to validate against, and what common checks establish. Agreement across runs or across models shows reliability, not validity. Human judges can serve as the ground truth for observable properties of text, but not for internal states. Predictive validity is not construct validity. The lecture closes with how to treat validation as a confirmatory exercise: freeze the prompt, pre-register the test, and collect a separate sample with self-reports when the construct is internal. This lecture is delivered by my AI avatar. The words are mine; the delivery is not. This is the October 2026 version; AI tools change quickly, so some screens may look different by the time you watch. Resources Templates, code, and a no-code coding tool: https://questionableresearch.ai Blanchard, Duani, Garvey, Netzer & Oh (2025), Journal of Marketing: https://doi.org/10.1177/0022242925134... About me: https://simonjblanchard.com Chapters 0:00 Introduction and plan 1:38 Why generative AI is attractive for content analysis 2:43 Construct first: felt vs. perceived anger 5:19 Writing the prompt as a coding protocol 7:57 Part 2: Running the coding on Yelp reviews 9:35 ChatGPT, or the model itself? 10:25 How LLMs and chat products work 13:59 What happened when I uploaded the data to ChatGPT 15:39 Calling the model through the API 16:13 Getting an API key 18:04 What the API fixes, and what it does not 19:40 Part 3: Validation and the right ground truth 22:50 Common checks and what they establish 23:19 Check 1: Reliability is not validity 25:23 Check 2: Agreement with human judges 26:46 Check 3: Predictive validity 27:40 Reusing a prompt validated elsewhere 28:36 Validation as a confirmatory exercise 29:57 Pre-registering a validation with human judges 32:34 Pre-registering a validation with a separate sample 35:29 Summary