Nat Knight

#project #selfhosting #llms #agents

One of the ways that folks are leveraging LLM-powered agents to do software development is the “software factory” (or “software dark factory”) pattern, where the LLMs write all the code and the humans aren't allowed to even review it.

Apparently this works, though you need a good way to verify your product. I'm a little skeptical that it's the best way to use these tools, but it's an interesting idea, so I'm going to try to build one for myself.

The Project

I'm cheating a little bit by picking a project that's embarrassingly easy to verify: a new interpreter for an extant programming language.

The programming language is called EYG and has many interesting and desirable properties:

• functional style + a sound type system
• immutable dependencies
• deterministic execution
• the editor tooling and compiler are independent of the runtime (so I don't have to implement them!)

Its only annoying drawback is that its interpreter is in JavaScript (compiled from Gleam, but running on Node). I'd really prefer a native interpreter, or better still, a Rust library that can execute EYG code.

Lucky for me, I have:

• access to LLMs that can cheaply churn out code, and
• an “oracle” that can see if my project is working properly, i.e. the existing EYG interpreter.

So that's the project: a Rust library + easily deployable binary that can interpret EYG code.

The Rules

The project will be split into two repos, each with different rules:

eyg-rs is the actual interpreter/library:

• I'm not allowed to write or review code here. I can peek in occasionally for quality control, but that should be the exception.
• The work needs to be mediated by an automated system, not me manually driving an agent.
• The code needs to end up correct, efficient, and clean.
• All agent-driven work must happen in a sandbox on my own hardware.

I'm trying to learn about the factory pattern, so I actually have to build one. I could almost certainly just point Claude Code at the EYG spec and have it converge on a working interpreter, but that wouldn't teach me anything about the system that's managing the work.

Keeping it sandboxed is a precaution against an agent hosing my home lab.

Figuring out how to get it clean and efficient is, of course, part of the challenge.

eyg-rs-factory is the system that tracks work, manages the agent, and validates the final product:

• I can read and write code here, including using agents to do it for me.
• I can use my tools in whatever mode gets the job done.
• This code can be arbitrarily janky as long as it works.

This is where the learning happens, so it gets to be hands-on and sloppy if that's what it takes.

I've started building it, but it's not working yet. I'll let you know how it goes once it's closing tickets!