|

From Rules to Residues: Demystifying Large Language Models via Ground-Up Structural Implementation

Mayank Kumar Sharma, IEM Newtown

Modern natural language processing is dominated by Large Language Models (LLMs) centered on the Transformer architecture. This paper systematically reduces the LLM pipeline to its baseline deterministic components: tokenization via Byte Pair Encoding (BPE), distributional semantic vector generation, and matrix-multiplication-driven self-attention. By contextualizing a 52,000-parameter toy model built completely from scratch against 80 years of computational history, we evaluate the paradigm shift from handwritten, rule-based heuristics to statistical next-token auto-regressive generation. Finally, we discuss how fine-tuning frameworks like Reinforcement Learning from Human Feedback (RLHF) and external tool-calling execution environments shape a statistical autocomplete model into an apparent conversational agent.