Hi, I'm Andreas

MSc Computer Science student at Aalborg University · Machine learning research · LLMs · Recommender systems

I'm a master's student in computer science at Aalborg University, Denmark, working on machine learning research. I care about the gap between what machine learning methods are claimed to do and what they actually do, so I analyse methods from first principles, reproduce published results under controlled conditions, and build small, well-tested research software that makes results easy to verify.

• Machine learning research: how training choices such as loss functions, reweighting and regularisation change what a model actually learns, backed by proofs and controlled experiments.
• LLMs and generative AI: parameter-efficient fine-tuning with LoRA and QLoRA, 4-bit quantisation, and open models such as Qwen.
• LLM inference optimisation: distilling large language models into smaller, faster ones, the topic of my bachelor's project.
• Recommender systems: multimodal and cold-start recommendation, graph-based recommenders, and the reproducibility of published models.
• Reproducible research: controlled experiments, sound statistics, and long training runs on GPU clusters.

Distil an LLM classification prompt into a small, fast model that runs on a CPU. Describe the task in a short YAML file and give it unlabelled texts: an instruction-tuned LLM labels them with soft targets, a small encoder learns to imitate them, and the result is exported to ONNX, optionally int8. It continues my bachelor's project on inference optimisation by distilling LLMs.

Try the live demo · Documentation · Code · pip install tinydistill

flowchart LR
    A["Task prompt +<br/>unlabelled texts"] --> B["LLM teacher(s)<br/>Qwen · Gemma · Mistral · ..."]
    T["tune: test and rewrite<br/>the task file"] -.-> B
    S["Synthetic texts<br/>(synth)"] -.-> B
    V["vLLM server ·<br/>hosted API"] -.-> B
    K["A few gold labels<br/>per class"] -.-> B
    K -.-> C
    B -- "soft labels<br/>(ensembled)" --> C["Small encoder<br/>student"]
    C --> D["ONNX / int8<br/>export"]
    D --> E["Fast CPU<br/>inference"]
    D --> W["Browser · server · Hub ·<br/>Python (Distiller)"]
    E -.-> F["Cascade: unsure<br/>texts to the LLM"]
    F -. "refresh: retrain<br/>on what it asked" .-> C
    C -. "improve: new texts<br/>for its confusions" .-> S

Results on Banking77 (77 intents, 3,080 held-out test texts; latency per text on an RTX 4090 for the teacher and 2 CPU threads for the students; the students learn from the teacher's soft labels plus 8 gold labels per class; all benchmarks):

The students are more accurate than their teacher and hundreds of times faster on a CPU than the teacher on a GPU.

An interactive, in-browser demo of knowledge distillation. A teacher network and three small students train side by side on 2D data: one from the labels alone, one from the teacher's hard answers and one from its soft targets. A live view of each neuron, the probabilities of every model at any point, and sliders for the temperature, the mixing weight, the student size and the teacher's quality show when distillation helps. Written from scratch in TypeScript, with no ML library and gradient-checked backpropagation. Try the live demo · Code

Mathematical bugs I found and reported in widely used ML libraries, each with a minimal reproduction, and fixed in a pull request where I could.

Languages

Machine learning

Methods and models

LoRA QLoRA PEFT Quantisation Knowledge distillation Inference optimisation Hashing Qwen Transformers Recommender systems Graph neural networks Multimodal learning Class imbalance Statistical testing Closed-form recommenders Cold-start recommendation Pre-registered evaluation

Research and infrastructure

Web (earlier projects)

• MSc in Computer Science, Aalborg University (ongoing)
• BSc in Computer Science, Aalborg University, with a bachelor's project on inference optimisation by distilling LLMs

• Writing up research on machine learning training objectives
• Pre-thesis project on reproducibility and cold-start ranking in multimodal recommender systems
• Auditing ML libraries for mathematical bugs (contributions)
• Open to collaborations on machine learning research and tooling