https://signal-and-syntax.com/tags/nlp/Recent content in NLP on Signal & SyntaxHugoen-usWed, 01 Apr 2026 06:00:00 -0700https://signal-and-syntax.com/posts/minimal-transformer/Wed, 01 Apr 2026 06:00:00 -0700https://signal-and-syntax.com/posts/minimal-transformer/In 2017, Vaswani et al. published “Attention Is All You Need,” a paper that quietly rearranged the entire landscape of machine learning. It introduced the Transformer architecture — a design that has since become the backbone of every major language model you’ve heard of: GPT, BERT, Claude, Gemini, and dozens of others. The paper’s title was a provocation. Attention mechanisms already existed. The claim was that you could throw out recurrence entirely and let attention carry the whole load.https://signal-and-syntax.com/posts/how-large-language-models-tokenize-text/Tue, 11 Nov 2025 06:00:00 -0700https://signal-and-syntax.com/posts/how-large-language-models-tokenize-text/When you type “I love programming” into ChatGPT, you might assume the model reads three words. It doesn’t. It reads somewhere between three and seven tokens, depending on how the text is split. When you ask Claude to count the letters in the word “strawberry,” it often gets it wrong. The reason is simple. Claude never saw the word “strawberry” as a complete unit. It saw tokens like "str", "aw", "berry" and tried to reason about letters it couldn’t directly access.