#llm

Ultimately my point is, no matter how much I'm certain something is AI, I would not say it out loud, because I always have a capacity to hurt a real artist in doing so. I just want people to stop going "uuhhh guys I think this is AI look at the tells" because it only hurts real people and is simply just really annoying.

I also think getting a bunch of humans in alignment is just a thing that takes time. It should be a bottleneck. I’ll forever think of Dave’s “Slow, like brisket.” Some things becomes good because they are done slowly, and it’s OK if software is one of them.

An LLM-as-a-judge HTTP proxy to secure agents in production

Open Brain — The infrastructure layer for your thinking. One database, one AI gateway, one chat channel — any AI plugs in. No middleware, no SaaS.

When you let an LLM author content into a knowledge base, and that content gets indexed alongside the original sources, you’ve just introduced unverifiable information into your source of truth. And not obviously wrong information — that would be easy to catch. I mean plausibly correct, well-structured, confident-sounding prose that happens to be a subtle interpolation of what the model thought the document said.

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Wade Foster: the agent is actually choosing what products to buy on behalf of a human, and so you are no longer advertising to the human. You are trying to get the agent to say, pick me, pick me. Pick me.

Fully autonomous AI hacker to find actual exploits in your web apps. Shannon has achieved a 96.15% success rate on the hint-free, source-aware XBOW Benchmark.

There's definitely a perception I've heard from folks who haven't spent a lot of time with the tools that the output of coding agents is always going to be a classical big ball of mud -- a horrible monstrosity with no clear architecture...just a jumbled mess of code that kind of somehow does the thing. It's not true, but that's what many folks think. So why not lean into it? I find myself engaging in software Dorodango pretty much every day.

In previous posts, Davide Eynard demonstrated how to run agentic frameworks in the browser, while Baris Guler showed how to extend them with in-browser inference and support for multiple programming languages. In this post, we leverage the JVM's polyglot capabilities to create a self-contained, enterprise-optimized server-side blueprint that combines the performance benefits of WebAssembly with the reliability and maturity of Java's ecosystem.

New tools always make it easier to code with a shallower understanding of the system. Claude Code, Cursor and their ilk are bringing programming closer to natural language just like C and Java and Python once did. This is a good thing! It’ll lead to the continued democratization of programming and ever more people in conversation with computers. But I don’t think this changes the fundamental reality that the best programmers aren’t the ones that make the widest use of the highest abstractions. That’ll continue to be those who dig down and understand what’s happening at a deeper level — that understanding will always lead to more deft use of any tools that programmers have at their disposal.

I found out about my death the way everybody finds out everything: from Google.

Generative and Agentic AI are poised to bring about a new wave of change. For the first time, there are tools developers can use to translate common-sense descriptions of desired program behavior into real, working systems. Many developers are using these tools already. Prompt-by-prompt, step-by step, they describe the system they want to build to AI code generators. This vibe coding approach to development has already been proven to be a powerful tool. But its power is limited because it is missing one important piece: a full vision of the bigger picture of what a program should do, and why it should do that. Enter specification.

We conduct a randomized controlled trial (RCT) to understand how early-2025 AI tools affect the productivity of experienced open-source developers working on their own repositories. Surprisingly, we find that when developers use AI tools, they take 19% longer than without—AI makes them slower.

Like many engineers, I’m still figuring out where AI fits best. But what’s clear is that it works best when it complements the way I already like to build software. It helps me move faster, stay in flow, and focus more on the parts of the work I enjoy, without losing the parts that matter most.

I decided to figure out how to use the official Playwright MCP server with Claude Code. It turns out it's easy: claude mcp add playwright npx '@playwright/mcp@latest' claude

I keep several claude code windows open, each on its own git-worktree. o3 and sonnet 4 create plans, sonnet 3.7 or sonnet 4 execute the plan, and o3 checks the results against the original ask. Any issues found are fed back into the plan template and the code is regenerated. The factory improves itself.

Vibe coding is what happens when you build software without understanding how anything works, relying entirely on tools to fill in the gaps. You feed ChatGPT a vague prompt, drag in some randomly generated code blocks, wire up a few buttons, and hope it works. You ship without testing. You copy code you don’t understand. You build with vibes, not logic. AI-assisted coding, on the other hand, is intentional. You still plan your features. You still understand how your system fits together. You just move faster because you’ve got an AI pair programmer who helps with boilerplate, explains confusing concepts, and suggests things you were going to write anyway.

tl:dr; Brainstorm spec, then plan a plan, then execute using LLM codegen. Discrete loops. Then magic. ✩₊˚.⋆☾⋆⁺₊✧

I feel like this is obvious but I’m not hearing it shouted from the rooftops so here it is: adding an untrusted middleman to your information diet and all of your personal communications will eventually become a disaster that will be obvious in hindsight.

This article is a collection of blog posts written by other senior or staff+ engineers exploring the use of LLM in their work, without the usual hype or buzzwords from the usual suspects. I hope you find them useful and inspiring.

Explore how AI-generated user interfaces can enhance human-AI interaction by dynamically creating the right interface components at the right time, addressing key limitations of text-only interactions.

I presented a three hour workshop at PyCon US yesterday titled Building software on top of Large Language Models. The goal of the workshop was to give participants everything they needed to get started writing code that makes use of LLMs.

But lets talk about the expense too, because I think there’s something bad there as well: coding agents, and especially Claude 3.7 (the backend of Claude Code), tend to write too much code, a phenomenon that ends up costing users more than it should.

VectorVFS is a lightweight Python package that transforms your Linux filesystem into a vector database by leveraging the native VFS (Virtual File System) extended attributes. Rather than maintaining a separate index or external database, VectorVFS stores vector embeddings directly alongside each file—turning your existing directory structure into an efficient and semantically searchable embedding store.

This is what AI's "killer app" will look like for many of us: teaching a computer how to do things that we don't like doing so that we can spend our time on things we do.

Diane, it’s Thursday and I’ve been figuring out how transcription fits into my everyday work. I had to make up a character to make it make sense, as I’ll say.

It turns out that LLMs can make CAD models for simple 3D mechanical parts. And, I think they’ll be extremely good at it soon.

Perhaps what we need is a new understanding of where happiness can exist in this AI-augmented world. Maybe the joy doesn’t have to disappear completely — it just shifts. Instead of finding delight in writing the perfect algorithm, perhaps we’ll discover satisfaction in the higher-level thinking about system design, in the creative process of describing exactly what we want to build, or in the human aspects of software development that AI can’t touch.

A field guide to responsible AI-assisted development

New data reveals the hidden network of African workers powering AI, as they push for transparency from the global companies that employ them indirectly.

Official code repo for the O'Reilly Book - "Hands-On Large Language Models"

This is my journey. It is largely the path I took. I think you could speed run it if you were compelled. I don’t think you need to follow every step, but I do think every step is additive.

Large Language Models (LLMs) have transformed natural language processing, but face significant challenges in widespread deployment due to their high runtime cost. In this paper, we introduce SeedLM, a novel post-training compression method that uses seeds of a pseudo-random generator to encode and compress model weights.

Blindspots in LLMs I’ve noticed while AI coding. Sonnet family emphasis. Maybe I will eventually suggest Cursor rules for these problems.

OSINT used to be a thinking game. Now it’s becoming a trusting game and that should terrify you.

Why did we fall in love with chat interfaces for human-ai interaction? Let us leave them behind once and for all.

OK, so AI doesn’t think the same way that people do. I’m fine with that. What’s important to me is that it can do some work for me, work that could also be done by people thinking. Cars (“horseless carriages”) do work that used to be done by horses running. No one now complains that cars work differently than horses.

By emphasizing documentation as the primary source of truth, establishing explicit linkages between development artifacts, and employing iterative, human-guided prompting, the whole team can harness the power of LLMs while mitigating their current limitations.

Model Context Protocol on steroids ;)

Right now, I’m not overly excited by MCP over “standard” tool calling. I much prefer agents.json and the concepts around endpoint discovery, which feel much more natural if you are working with APIs.

"Vibe Coding" might get you 80% the way to a functioning concept. But to produce something reliable, secure, and worth spending money on, you’ll need experienced humans to do the hard work not possible with today’s models.

One consistent pattern I’ve observed in the past year, since I published "The death of the junior developer", is that junior developers have actually been far more eager to adopt AI than senior devs. It’s not always true; a few folks have told us that their juniors are scared to use it because they think, somewhat irrationally, that it will take their jobs. (See: Behavioral regret theory. Thanks for the pointer Dr. Daniel Rock!)

I now, although tentatively, believe that LLMs can indeed succeed in all domains with perfect oracles.

Using LLMs to write code is difficult and unintuitive. It takes significant effort to figure out the sharp and soft edges of using them in this way, and there’s precious little guidance to help people figure out how best to apply them. If someone tells you that coding with LLMs is easy they are (probably unintentionally) misleading you. They may well have stumbled on to patterns that work, but those patterns do not come naturally to everyone. I’ve been getting great results out of LLMs for code for over two years now. Here’s my attempt at transferring some of that experience and intution to you.

A native macOS app that allows users to chat with a local LLM that can respond with information from files, folders and websites on your Mac without installing any other software. Powered by llama.cpp.

What if super-intelligent AI1 arrives but it can be run by anyone, basically for free?

The first company to get this will own the next phase of AI development tools. They’ll build tools for real software instead of toys. They’ll make everything available today look like primitive experiments.

Welcome to Boomer Prompts—an affectionate trip down memory lane of the elaborate, quirky, and sometimes overkill techniques used to guide earlier Large Language Models. These “relics” showcase just how much LLMs have evolved—where once you needed to triple your instructions and role-play as a wise oracle, now simpler, more direct prompts suffice. Read on for a chuckle, and discover how far we’ve come!

We’re shipping a new API in Firefox Nightly that will let you use our Firefox AI runtime to run offline machine learning tasks in your web extension.

Concluding, I think this is a very interesting way of working with AI inference in the browser. The obvious downside is that you need to convince your users to download an extension, but the obvious upside is that you possibly can save them from having to download a model they may already have downloaded and stored on their disk.

Build Your Own Timeline Algorithm

Timeline algorithms should be useful for people, not for companies. Their quality should not be evaluated in terms of how much more time people spend on a platform, but rather in terms of how well they serve their users’ purposes

By focusing on the answer correctness as a key success metric, and designing our datasets and metrics carefully, we’ve managed to build a reliable evaluation process which has helped us increase confidence in our system’s quality.

Transformer Lab is a free, open-source LLM workspace that you can run on your own computer. It is designed to go beyond what most modern open LLM applications allow. Using Transformer Lab you can easily finetune, evaluate, export and test LLMs across different inference engines and platforms.

Run your own AI cluster at home with everyday devices

The home of the Web Applets spec, demo and SDK

The /llms.txt file, helping language models use your website

llm-mlx is a brand new plugin for my LLM Python Library and CLI utility which builds on top of Apple’s excellent MLX array framework library and mlx-lm package. If you’re a terminal user or Python developer with a Mac this may be the new easiest way to start exploring local Large Language Models.

To me, all signs point towards software engineering changing radically as a profession to be much more oriented around the what and why of software, and much less around the how. This will cause disruption at a massive scale in the long run. But in the short run, it's just a lot of fun to play with these tools and see what they can do.

It is surprisingly straightforward to increase the VRAM of your Mac (Apple Silicone M1/M2/M3 chips) computer and use it to load large language models. Here’s the rundown of my experiments. ... I found a way to bypass this limitation. To allocate more of your Mac’s system RAM to VRAM – in this case, up to 28 GB – the following command can be used in the terminal window: sudo sysctl iogpu.wired_limit_mb=27536

my attempt to build such a capable AI computer without spending too much. I ended up with a workstation with 48GB of VRAM that cost me around 1700 euros

You wrote an interesting comment about getting your work into the LLM training corpus: "there has never been a more vital hinge-y time to write." Do you mean that in the sense that you will be this drop in the bucket that’s steering the Shoggoth one way or the other? Or do you mean it in the sense of making sure your values and persona persist somewhere in latent space?

Personally, I feel like I get a lot of value from AI. I think many of the people who don’t feel this way are “holding it wrong”: i.e. they’re not using language models in the most helpful ways. In this post, I’m going to list a bunch of ways I regularly use AI in my day-to-day as a staff engineer.

Undoubtedly, the sloppification of the internet will likely get worse over the next few years. And as such, the returns to curating quality sources of content will only increase. My advice? Use an RSS feed reader, read Twitter lists instead of feeds, and find spaces where real discussion still happens (e.g. LessWrong and Lobsters still both seem slop-free).

At the same time, there should be some humility about the fact that earlier iterations of the chip ban seem to have directly led to DeepSeek’s innovations. Those innovations, moreover, would extend to not just smuggled Nvidia chips or nerfed ones like the H800, but to Huawei’s Ascend chips as well. Indeed, you can very much make the case that the primary outcome of the chip ban is today’s crash in Nvidia’s stock price.

People with less knowledge about AI are actually more open to using the technology. We call this difference in adoption propensity the “lower literacy-higher receptivity” link.

I feel like half of my social media feed is composed of AI grifters saying software developers are not going to make it. Combine that sentiment with some economic headwinds and it's easy to feel like we're all screwed. I think that's bullshit. The best days of our industry lie ahead.

Just as Midas discovered that turning everything to gold wasn't always helpful, we'll see that blindly applying cosine similarity to vectors can lead us astray. While embeddings do capture similarities, they often reflect the wrong kind - matching questions to questions rather than questions to answers, or getting distracted by superficial patterns like writing style and typos rather than meaning. This post shows you how to be more intentional about similarity and get better results.

The Switch runs an off-the-shelf Nvidia Tegra X1 with 4GB of RAM. It was Nvidias second desktop GPU architecture in a mobile chip, after the Nvidia Tegra K1, containing 256 Maxwell CUDA cores. Announced ten years ago now (On January 5, 2015) it was originally intended for tablets and the automotive industry, before Nintendo picked it up. The Jetson TX1 dev kit was released and it was also present in the somewhat popular and well-known Jetson Nano (though with only half the CUDA cores enabled.) A fully functional (albeit largely proprietary, in traditional Nvidia fashion) Linux4Tegra distribution was shipped, with a working CUDA development environment. CUDA? The compute platform that’s driving the AI revolution ✨? In my Nintendo Switch? Surely you see where I’m going with this.

This document is a summary of my personal experiences using generative models while programming over the past year. It has not been a passive process. I have intentionally sought ways to use LLMs while programming to learn about them. The result has been that I now regularly use LLMs while working and I consider their benefits net-positive on my productivity. (My attempts to go back to programming without them are unpleasant.)

Welcome to OpenHands (formerly OpenDevin), a platform for software development agents powered by AI. OpenHands agents can do anything a human developer can: modify code, run commands, browse the web, call APIs, and yes—even copy code snippets from StackOverflow.

AI isn't making our software dramatically better because software quality was (perhaps) never primarily limited by coding speed. The hard parts of software development – understanding requirements, designing maintainable systems, handling edge cases, ensuring security and performance – still require human judgment. What AI does do is let us iterate and experiment faster, potentially leading to better solutions through more rapid exploration. But only if we maintain our engineering discipline and use AI as a tool, not a replacement for good software practices. Remember: The goal isn't to write more code faster. It's to build better software. Used wisely, AI can help us do that. But it's still up to us to know what "better" means and how to achieve it.

Use llamafile to generate inline code completions in react / next.js apps.

New, more powerful chips require entirely new methods to rack-mount, operate and cool them, and all of these parts must operate in sync, as overheating GPUs will die. While these units are big, some of their internal components are microscopic in size, and unless properly cooled, their circuits will start to crumble when roasted by a guy typing "Garfield with Gun" into ChatGPT.

I hope that you now have a sense for word embeddings and the word2vec algorithm. I also hope that now when you read a paper mentioning “skip gram with negative sampling” (SGNS) (like the recommendation system papers at the top), that you have a better sense for these concepts.

So the problem Bluesky is dealing with is not so much a problem with Bluesky itself or its architecture, but one that’s inherent to the web itself and the nature of building these training datasets based on publicly-available data. Van Strien’s original act clearly showed the difference in culture between AI and open social web communities: on the former it’s commonplace to grab data if it can be read publicly (or even sometimes if it’s not), regardless of licensing or author consent, while on open social networks consent and authors’ rights are central community norms.

So, in summary: maybe people shy away from copilots because they’re tired of complexity, they’re tired of accelerating productivity without improving hours, they’re afraid of forgetting rote skills and basic knowledge, and they want to feel like writers, not managers. Maybe some or none of these things are true - they’re emotional responses and gut feelings based on predictions - but they matter nonetheless.

A newsletter from author Steven Johnson exploring where good ideas come from—and how to keep them from turning against us. Click to read Adjacent Possible, by Steven Johnson, a Substack publication with tens of thousands of subscribers.

The current state-of-the-art Gemini model can fit roughly 1.5 million words in its context. That’s enough for me to upload the full text of all fourteen of my books, plus every article, blog post, or interview I’ve ever published—and the entirety of my collection of research notes that I’ve compiled over the years. The Gemini team has announced plans for a model that could hold more than 7 million words in its short-term memory. That’s enough to fit everything I’ve ever written, plus the hundred books and articles that most profoundly shaped my thinking over the years. An advanced model capable of holding in focus all that information would have a profound familiarity with all the words and ideas that have shaped my personal mindset. Certainly its ability to provide accurate and properly-cited answers to questions about my worldview (or my intellectual worldview, at least) would exceed that of any other human. In some ways it would exceed my own knowledge, thanks to its ability to instantly recall facts from books I read twenty years ago, or make new associations between ideas that I have long since forgotten. It would lack any information about my personal or emotional history—though I suppose if I had maintained a private journal over the past decades it would be able to approximate that part of my mindset as well. But as reconstruction of my intellectual grounding, it would be unrivaled. If that is not considered material progress in AI, there is something wrong with our metrics.

In this guide we'll discuss: Where and when fine-tuning can be useful. Alternative approaches to extending the capabilities and behavior of pre-trained models. The importance of data preparation. How to fine-tune Mistral 7B using your own custom dataset with Axolotl. The many hyperparameters and their effect on training. Additional resources to help you fine-tune your models faster and more efficiently.

If, as Marx argued, capital is dead labor, then the products of large language models might best be understood as dead speech. Just as factory workers produce, with their “living labor,” machines and other forms of physical capital that are then used, as “dead labor,” to produce more physical commodities, so human expressions of thought and creativity—“living speech” in the forms of writing, art, photography, and music—become raw materials used to produce “dead speech” in those same forms. LLMs, to continue with Marx’s horror-story metaphor, feed “vampire-like” on human culture. Without our words and pictures and songs, they would cease to function. They would become as silent as a corpse in a casket.

Over the past month I’ve been exploring the rapidly evolving world of Large Language Models (LLM). It’s now accessible enough to run a LLM on a Raspberry Pi smarter than the original ChatGPT (November 2022). A modest desktop or laptop supports even smarter AI. It’s also private, offline, unlimited, and registration-free. The technology is improving at breakneck speed, and information is outdated in a matter of months. This article snapshots my practical, hands-on knowledge and experiences — information I wish I had when starting. Keep in mind that I’m a LLM layman, I have no novel insights to share, and it’s likely I’ve misunderstood certain aspects. In a year this article will mostly be a historical footnote, which is simultaneously exciting and scary.

Large language models (LLMs) have shown remarkable proficiency in generating text, benefiting from extensive training on vast textual corpora. However, LLMs may also acquire unwanted behaviors from the diverse and sensitive nature of their training data, which can include copyrighted and private content. Machine unlearning has been introduced as a viable solution to remove the influence of such problematic content without the need for costly and time-consuming retraining. This process aims to erase specific knowledge from LLMs while preserving as much model utility as possible. Despite the effectiveness of current unlearning methods, little attention has been given to whether existing unlearning methods for LLMs truly achieve forgetting or merely hide the knowledge, which current unlearning benchmarks fail to detect. This paper reveals that applying quantization to models that have undergone unlearning can restore the "forgotten" information. To thoroughly evaluate this phenomenon, we conduct comprehensive experiments using various quantization techniques across multiple precision levels. We find that for unlearning methods with utility constraints, the unlearned model retains an average of 21\% of the intended forgotten knowledge in full precision, which significantly increases to 83\% after 4-bit quantization. Based on our empirical findings, we provide a theoretical explanation for the observed phenomenon and propose a quantization-robust unlearning strategy to mitigate this intricate issue...

It is nightmarish to me to read reports of how reliant on ChatGPT students have become, even outsourcing to the machines the ideally very personal assignment "briefly introduce yourself and say what you're hoping to get out of this class." It is depressing to me to read defenses of those students, particularly this one that compares an AI-written essay to using a washing machine in that it reduces the time required for the labor. This makes sense only if the purpose of a student writing an essay is "to have written an essay," which it is not. The teacher did not assign it as busywork. The purpose of an essay is to learn and practice communication skills, critical thinking, organization of one's own thoughts. These are useful skills to develop, even (especially!) if you do not go into a writing career.

I released LLM 0.17 last night, the latest version of my combined CLI tool and Python library for interacting with hundreds of different Large Language Models such as GPT-4o, Llama, Claude and Gemini.

We have entered an era of LLM democratization. By showing that smaller models can be highly effective, enabling easy experimentation, diversifying control, and providing incentives that are not profit motivated, open-source initiatives are moving us into a more dynamic and inclusive AI landscape. This doesn’t mean that some of these models won’t be biased, or wrong, or used to generate disinformation or abuse. But it does mean that controlling this technology is going to take an entirely different approach than regulating the large players.

But in calling these programs “artificial intelligence” we grant them a claim to authorship that is simply untrue. Each of those tokens used by programs like ChatGPT—the “language” in their “large language model”—represents a tiny, tiny piece of material that someone else created. And those authors are not credited for it, paid for it or asked permission for its use. In a sense, these machine-learning bots are actually the most advanced form of a chop shop: They steal material from creators (that is, they use it without permission), cut that material into parts so small that no one can trace them and then repurpose them to form new products.