EP 01B — Build a Bigram Language Model in Python

Build a character-level language model in pure Python, then use it to generate names and measure its predictions on unseen data. This is the coding companion to EP 01A. We take the bigram model from the theory video and implement it in a Jupyter notebook, using 29,494 unique name spellings. We build the count table, turn counts into probabilities, compare greedy decoding with weighted sampling, calculate negative log-likelihood, and choose add-k smoothing using validation data. Run the notebook: https://github.com/HussainAbuwala/llm... Download or clone the whole repository so the notebook can find its dataset and chart helpers: https://github.com/HussainAbuwala/llm... Code companion: setup, key functions, expected results, and common questions: https://github.com/HussainAbuwala/llm... Watch the theory episode first:    • EP 01A — I Built the Smallest Language Mod...   Theory handout (PDF): https://github.com/HussainAbuwala/llm... Chapters: 00:00 Names, data splits, and character tokens 10:21 Counting, probabilities, and generation 16:45 Inspecting samples, evaluation, and smoothing 25:44 Choosing k, model limits, and final test What you'll build: • A character bigram model trained by counting • Greedy and weighted-sampling name generation • Training, validation, and test evaluation • Uniform and unigram baselines • An experiment to select smoothing strength The model uses Python lists, dictionaries, math, and random. Jupyter presents the code and Matplotlib draws the charts. Basic Python knowledge is helpful. Final test NLL (lower is better): uniform 3.295837; unigram 2.818510; smoothed bigram 2.460499. All three use the same 21,053 test predictions, including END. Smoothing k = 0.3 was selected on validation data. Dataset credit: names.txt from Andrej Karpathy's makemore repository, with its MIT license preserved in our repo. The upstream README attributes the data to US Social Security name data. https://github.com/karpathy/makemore Next in the series: give the model more context and explore the limits of counting. #Python #MachineLearning #LLMFromScratch