Selling in the Wild — A Randomized Trial of an AI Sales Agent

Selling in the Wild: in a 14.8M-visitor field trial, the Javo AI sales agent lifted revenue per visitor 10% and held its own against human chat staff at 1.8% of the cost. Key takeaways: Four live arms. 100 sites (e-commerce, travel, high-consideration services incl. B2B logistics), May–Sep 2026: no agent 2.6M, Javo 5.6M, frontier agent 5.7M, human chat 0.9M. Both bots disclosed they were AI. Javo (Qwen3-32B tuned on 4.6M consented sales chats; 2,240 H100 GPU-hours) raised revenue per visitor +10.0% (95% CI +5.1 to +14.9) on 85 transactional sites.Javo vs GPT-5.2, same widget. Engaged conversion +21.6% (+16.9 to +26.3); revenue per visitor +16.9%. Identical launcher, greeting and AI line, and the visitor writes first, so the model can't change who engages: 8.07% vs 8.06% (p = 0.536).Javo vs typical human associates (20 staffed sites): engaged conversion 11.26% vs 11.20% (+0.5%, −5.0 to +6.0), non-inferior (margin −7%; −10% in the design), not superior. $0.06 vs $3.40 per engaged conversation. Widgets differed (Javo proactive; 8.07% engaged overall vs 5.80% for humans); trimming bounds are planned but not reported.Guardrails held over five months: unsupported claims 0.15% of turns (frontier 0.56%), no unauthorized offers; returns, complaints in margin.Our caveats. The sole author is Founder/CEO of AISO Boost Ltd., which operates Javo; no independent review, registration or ethics approval is claimed. Sites came partly from his client network; one frozen GPT-5.2 snapshot.Chapters: 00:00 Cold open: wind tunnel vs rush hour 01:48 The paper: Selling in the Wild 02:51 Javo: 4.6M sales chats, 2,240 GPU-hours 04:19 14.8M visitors and four studies 06:28 Revenue lift and intention-to-treat 08:23 Skeptic check: who chooses to chat? 10:14 Javo vs humans: conversion and cost 11:48 Human widgets and trimming bounds 13:21 Guardrails and novelty 14:53 Related work 17:00 Caveats: conflict of interest 19:04 Recap and closing question Sources: Selling in the Wild: Benchmark and Randomized Field Evidence for a Sales Agent Trained on Commercial Conversations (Benjamin Tannenbaum; arXiv, Oct 18, 2026)Neutral Is Not Free: Evaluating Downside Risk in Neutral Launches (Alcain et al.; arXiv, 2026)Measuring Gift Card Program Incrementality via Causal Data Fusion (Whitehouse et al.; arXiv, 2026)Enjoyed it? Follow Data Science in the Wild (one real production ML/AI system or paper per episode, hosted by Peter Liu) and rate it on Apple Podcasts or Spotify. Also from Peter: The GenAI Evolution Atlas, Wednesday AI deep dives and Friday news roundups. Made with AI-generated voices; curated by Peter Liu from the sources above.