How a Transformer LLM works - Part 1 Residual stream and predictive distribution
ml
llm
Jun 17, 2025
My name is Kasper. My interests include machine learning, LLMs, computational biology and computational neuroscience and AI safety. I did a Master’s degree in Computer Science from ETH Zürich, focusing mostly on machine learning, and subsequently I have worked as a full-stack software developer at a large Danish software consultancy on a Danish government project. I am currently looking for a job that stokes my passion. Feel free to contact me.
Blog - Though I have a very disorganized and non-polished blog, you might get some sense of my interests. Feel free to reach out if you’d like to discuss any topic.
Selected Projects
intercebd.com - With intercebd.com you can collect and annotate and propose LLM responses. You can create SFT and DPO fine-tuning datasets, and use them with OpenAI’s fine-tuning service or push the datasets to Huggingface in ChatML format. Built with a FastAPI/Postgres backend and OpenRouter. GitHub
CS MSc Thesis: Understanding Features in Superposition in Transformer Language Models (PDF): In 2023, concurrently with Anthropic’s first SAEs trained on larger LLMs, I formalized notions of superposition that allowed for falsifiable experiments, and showed restricted kinds of superposition in realistic pre-trained Transformer LLMs using linear probes and causal interventions. I was supervised by Mor Geva.
EnigmataEval with Verifiers (Google Colab notebook): In this notebook I show how the
verifierslibrary (Github) for reasoning and agentic LLM can be used to define an environment for the reasoning benchmark Enigmata
Videos
I am exploring streaming my screen, talking about machine learning and programming.
Recent posts
No matching items
Education
Master of Computer Science, ETH Zürich
09/2021 - 11/2023
- Major in Machine Intelligence, minor in Theoretical Computer Science
- Electives focused on neuroscience and bioinformatics
- Thesis (top grade): Understanding Features in Superposition in Transformer Language Models
- GPA: 5.4/6
Bachelor of Computer Science, University of Copenhagen
09/2018 - 06/2021
- Data Science specialization
- Thesis (top grade): Unsupervised learning of objects and concepts with a focus on medical images
- GPA: 11.6/12
Work experience
AI consultant, Tiimo
09/2024 -
- Helped Tiimo use LLM in new features
- OpenAI API, structured generation, evals, FastAPI, dockerization
Software Developer, Netcompany
03/2024 - 02/2025
- Borger.dk, the most visited Danish government website.
- Full-stack, C#, Sitecore, HTML/JS, Azure DevOps, Git, agile