How a System Architect Uses Skill Files
A Philosophical Approach to the other EA: Enterprise Architecture
Intelligence is a high-density compression of knowledge into small spaces. A principle that fits in a tweet and correctly predicts 95% of cases is more intelligent than an answer that merely describes them.
I think Enterprise Architecture is misunderstood as the latter, (i.e. just a job description at SAP) when it’s actually a distinct branch of epistemology.
Doing it well requires you to distinguish between knowing what exists and what works.
An organisation's systems, processes, and data structures are its ontology: the entities it has decided are real, the relationships it has decided are stable and the categories it has chosen to cut the world at.
Enterprise Architecture is the epistemology layered above that ontology. It is the practice of figuring out which of those commitments are fundamental and which are derivative.
Or plainly:
- The architect who can tell you what the system is has described it.
- The architect who can tell you what questions the system was built to stop asking (that is, what debates it crystallised, what optionality it traded away and for what) has understood it.
In learning the practice of Enterprise Architecture myself, I could feel how appealing it is to save brainpower and just use someone else’s map. Creative thinking and deep focus requires a specific culture in organisations, and I can see how easily EA as a function can flatten into simply maintaining the system. You need the structured thinking to determine the map of the territory and a theory of navigation in a world filled with salesmen and information viruses.
This is a fundamentally different cognitive posture than system management. It requires holding the present state and the reasoning that produced it simultaneously. Enterprise Architecture is a lot closer to theology and jurisprudence than engineering.
One of the ways that architecture is distinguished from the artefacts that it describes (a heck of a lot of documentation) is that documentation expands to fill the complexity it describes, whereas architecture compresses complexity into a form that generates correct answers. Or as my old manager used to put it: “I have opinions, and I make PowerPoints.”
The moment you separate a decision from the argument that generated it, you have preserved the answer and concealed the intelligence. Good architectural knowledge preserves its own dissent for those with the eyes to see it.
Running the Experiment
I’ve been enjoying chatting with LLMs since ChatGPT came out at the end of 2022. Like everyone else, I first started using them as an auto-fill tool, LARPing, summarisation and research. As models got better, the requests got bigger.
During my day-job I was working in the Office of the Chief Architect at an AI health-tech startup1; during my evenings I was messing about with agents. The questions my day job asked about systems had the same answers for getting more out my agents.
A skill.md file is the closest thing many organisations now have to their epistemology written down! The schema of what kinds of things exist here, what the words mean, what the rules are, what a competent actor does when the situation is ambiguous.
I started paying closer attention to figuring out how to get LLMs to run loose on whatever execution problem I had after I scoped out the enterprise of objects that I was working on.
My personal KPI is being able to bang out “fix!!” in the chat window without any other context attached. Based on my thesis that intelligence is compression, “fix!!” carries no information about what's broken, what good looks like, or what I'd accept as done. For it to work, all of that meaning must already live somewhere the agent can crunch.
Greg Isenberg has a neat metaphor for the LLM stack: a context error is a bug, a skill file is a program, the harness is an operating system, your token budget is RAM. The problem is that programs run on computers and organisations run in the world.
I founded a new hobby project titled the Innovation Commons2, a weekly newsletter and WhatsApp community that discusses social, political and technological challenges Australia is facing. I separately built a diagnostic tool for businesses to help them get unstuck when dealing with context rot. Point it at your own business for US$2 — you only pay the tokens.
I used all of this as a guinea pig to see I could apply TOGAF’s practices and metamodels to optimising how I use LLMs, so let’s get into what I’ve learned.
Lesson one: TOGAF3 is a metamodel, not a manual
This is an obvious one to practitioners, but I want to talk about TOGAF to those who aren’t architects. TOGAF has a reputation for being extremely dry document. It’s only dry if you treat it as something you read, and not as a metaphysical map!
TOGAF doesn’t tell you what to do. It shows you how to think about what you need, when what you need isn’t clear, even if you have a clear vision.
Unfortunately, no one teaches you how to consume a standard. Standards are Chesterton Fences4. The way to use them is to decompress the metamodel into your context, ‘compile’ it into your grammar and trim whatever else is overkill.
I applied the Architecture Landscape to think about my continuum levels: strategic, segment and capability. These help me think about how long I expect the output of this initiative to remain valid and keep serving the enterprise purpose.
I segmented Jira across four Architectural Domains: business, application, data and technology.
Then I crossed these two closed sets together to track my birds-eye view of the entire enterprise: every ticket carries exactly one continuum level and at least one layer, so the board is a queryable matrix rather than a list.
After three months the heaviest cell was capability × application at over a hundred issues (the signature of a build sprint) while the strategic ring stayed under fifty.
As an additional example. consider the general entities box (Figure 6). Say you’re running a marketing campaign, and you need to keep track of 500 different ad variants, and you also have to talk to product and legal and the executive team. It useful to have a framework to categorise your ads but can also be used everywhere else in the enterprise!
In this way, companies get increasingly computable.
In the case of this project, I built a simple AI diagnostic for mid-market operators and wanted visibility on the whole architecture before I upgraded to a new version. In architecture parlance, this is a foundational change initiative.
These are situations where there are ‘known objectives but are not strictly scoped or bounded by a shared vision’ → I start by describing the current state fully using an adaption of the metamodel (Figure 6) to generate a baseline architecture.
Baseline architecture is the operating word here. If you ask an LLM to do that without enough context, you get a pass through Business, Data, Application and Technology as they actually exist, closing with an opportunities-and-feasibility section, and with every inference the model couldn't verify against running systems flagged as an explicit, numbered assumption.
Lesson two: A skill file only holds rules5, not facts.
Skills must only teach agents what kinds of things exist and what laws govern them. It must never state which things exist right now. The moment you write a current fact into a skill, you have created a non-evolving copy of your enterprise’s reality which leads to SLOP.
This is an application of description logic used semantic web languages6. All skills are split into terminology and assertations.
Three errors I watch for are:
- Drift: The system contradicts the skill.
- Bloat: Every fact mutation forces an edit of the rules.
- Frozen: It should not be a narrative, since that leads to hallucinations.
You shouldn’t be editing a skill file more than every few weeks. It’s a skill file, not a database. My central governance skill originally carried the gap register, the decision-record index, and the building-block catalogue inside itself, as tables. It felt efficient but I was adding hundreds of words a week before I purged it.
Lesson three: Give every fact a register of record7
The skill is never edited to register an instance; the skill is edited only to change a rule.
So, what did I do with all my facts? I store them in Jira personally. I’ve seen a lot of cool organisation templates that use Obsidian, Linear, Notion, etc. You can use anything! It’s substrate independent.
I minted four registers of record as ordinary Jira tickets: one for architectural gaps, one for decision records, one for building blocks, one for lead-generation models. Each register is a living document with hyperlinks to every evidencing ticket, plus its own append-only change ledger. The skills were then stripped back to pure grammar, and the constitutional rule was written down where the agent reads it.
Rules in the skills, facts on the board. The board is my source of enterprise truth. If it’s not recorded, I’m going to forget.
Lesson four: make the agent run a loop, not a lookup
This process ensures that my skills never go stale.
Once rules and facts live apart, the skill's most important content becomes the work loop, the procedure that binds them back together at runtime. Mine has six steps, and the agent runs them on every task that touches the architecture:
- Load my terminology schema (within my skills). The agent now knows the rules but holds no facts.
- Read the assertations of each relevant register. The register, not the skill, answers "what exists?"
- Reconcile the register against the live system (for me, a label sweep across the boards), because the register can be stale and the ground truth it indexes cannot.
- Applying only vocabulary the skill sanctions, stamping provenance on everything touched.
- If the work touches a governed family of concepts, fire that family's validation check before anything closes.
- Write the assertations. Any new fact goes into the register and its ledger. Never, under any circumstances, into the skill.
Lesson five: protect vocabulary from amnesia
Complex organisations don't just accumulate facts; they change what things are called.8 I maintain two disciplines to keep vocabulary honest:
The first is a rule for when a word deserves governance. The heuristic is: specialisation is free until it carries analytical weight; the moment a query is run against a label, that label must have a registry definition.
Every namespace in my system has one of three postures:
- Closed: A fixed set only a rules change can extend: TOGAF layers and continuum levels.
- Governed-open: Freely extendable, but new members must be registered first. These are my building block families
- Free-leaf: Free to improvise on everything as long as it’s never queried.
This gives your people, and your agents, room to specialise without the taxonomy dissolving into nicely presented SLOP.
The second is that vocabulary changes are append-only events, never rewrites. Every rename, retirement, or split in my system is an entry in a mutation ledger. Even my own overrides are recorded as overrides.
Lesson six: Weekly checks & monthly reports is the right cadence
Governance that depends on remembering is not governance! One the last business day of every month, a cron job generate retrospective tickets against a fixed template. Each produces a delta report against the same template: summary, new work entered, work completed, stalled work, gap-register changes, decision surfaces triggered (every fork in the road the month forced, and how it resolved), architecture node status changes, and proposed actions for next month.
The retrospective is also where measurement itself gets audited. What did I actually get done this past month? The retrospective then never relabels anything, the query for "everything touched in June" already returns a complete set, because the tax was paid continuously in micro-instalments (my snapshot:month* tags) instead of as a month-end audit. The benefit of this is that this project’s past isn’t a matter of memory any more, rather a matter of query.
That's the whole system, and it compresses further than you'd expect! The standard supplies the metamodel; the skills hold the grammar; the board holds the facts; ABBs hold the requirements while SBBs churn beneath them; the crons enforce weekly what the retrospective formalises monthly; and the reports fall out of the labels for free.
While all of this ran, the products it governs shipped. A newsletter pipeline from spike to production in nine days (and I’ve never missed a week!) a paid audit product live with my first customers in June, because the governance was never instead of the work.
Bonus one: My skill files and ontologies
Bonus two: My interactive architecture report from June
- Its aspiration: use nothing more than your smart phone sensors to detect all preventable disease ↩
- After Potts & Allen’s concept of the institutional structures that exist to help generate new ideas in conditions that are too uncertain to commercialise. See Jason Potts’ book ↩
- The Open Group’s Architecture Framework ↩
- https://www.lesswrong.com/w/chesterton-s-fence ↩
- i.e. a t-box ↩
- A semantic web language is a computational logic-based language such that knowledge expressed in it can be exploited by computer programs, e.g., to verify the consistency of that knowledge or to make implicit knowledge explicit. – OWL ↩
- The A-box of my ontology ↩
- So much enterprise value in making up compelling proprietary names! ↩