Writing
Notes on small-model agents
Technical writeups and opinions on running capable agents on local, open-weight models. Also on Medium and syndicated to Dev.to. RSS.
-
An SRE's guide to deploying Large Language Models, Part 2: from a base model to a served one
How a raw next-token predictor is turned into a model that follows instructions, reasons, and stays in bounds, and the GPU, memory, and quantization mechanics that decide whether serving it is fast or ruinously slow.
#llms#sre#fine-tuning#rlhf#inference#quantization -
An SRE's guide to deploying Large Language Models, Part 1: understand the workload
Before you can deploy and serve LLMs reliably, you have to understand them as a workload. A ground-up tour of the transformer, from tokens to attention to the feed-forward step, for SREs.
#llms#sre#transformers#inference -
I created Primer, a context optimized, loop engineering AI platform so I can get meaningful work done on my 16GB VRAM
I bought a 16GB gaming GPU, tried to get real work out of small local models, and ended up building Primer: a platform for running lots of tiny, focused agents instead of prompting one big one.
#local-llms#agents#primer#context-engineering