Quick answer (TL;DR): Most friction with AI tools comes from assuming a capability that isn’t there, or not knowing about one that is. The fix is a capability audit: sit down and actually test what your AI assistant can and can’t do in your environment — not what the marketing claims — and write the answers into a file. Then design your workflow around the real constraints rather than fighting them. An hour spent auditing capabilities saves ten spent assuming them.
If you’re building a serious AI working method, this is the unglamorous step that quietly determines whether the whole thing feels smooth or maddening.
Why you can’t trust the marketing
Every AI tool’s marketing describes the happy path: “connects to your calendar,” “edits your documents,” “works with your files.” What the marketing rarely tells you is the texture of those capabilities in practice:
- It “connects to your documents” — but can it edit them in place, or only read and create new ones?
- It “works with files” — up to what size? In which formats? Through which channels?
- It “integrates with your tools” — but is that integration actually authorized in your specific account, or silently broken at the permissions layer?
These distinctions are invisible until you test them, and each one can derail a workflow you’ve built assuming the happy path. The single most common source of AI frustration isn’t the model being dumb. It’s a mismatch between assumed and actual capability.
What a capability audit is
A capability audit is exactly what it sounds like: you deliberately probe what your assistant can do in your real environment, and you write down the answers. Not in theory — in practice. You ask it to do the thing and watch what actually happens.
The output is a capabilities file in your second brain: a living document listing, for each tool and integration, what works, what doesn’t, and what the workaround is. This single file will save you hours, because it converts every painful surprise into a known constraint you can design around.
How to run one
Go through each capability you might rely on and actually test it. A practical checklist:
For file and document handling:
- Can it read my files? Which formats?
- Can it create new files? In what formats?
- Can it edit existing files in place, or only create new versions?
- Is there a file size limit? What happens when I exceed it?
- Where do created files actually land, and can I access them?
For integrations (email, calendar, trackers, chat tools):
- Is this integration actually connected and authorized in my account?
- Can it read? Can it write? Both?
- Does it require an approval step each time, or does it act freely?
- Are there parts that look connected but silently fail?
For the assistant’s actions:
- Can it run scheduled or autonomous tasks, or only act when I’m present?
- Can it control live interfaces (clicking around apps), or only work through data/APIs?
- What does it do when it hits a wall — tell me clearly, or fail silently?
Write the real answer next to each. Be specific. “Can create and read files but not edit in place; updates require recreating the file” is a useful entry. “Files: yes” is not.
What my own audit turned up (and why it mattered)
When I audited my setup, I found a revealing mix of delights and hard limits:
- Some tools could read and create but not edit in place. This single finding reshaped my entire file convention — I adopted a “recreate with a version stamp” approach precisely because in-place editing wasn’t available.
- Some files were too large to move through certain channels. So I learned which path to use for big files and stopped wasting time on the one that would fail.
- One whole integration was silently broken at the permissions layer — it looked connected but couldn’t actually do anything. Knowing this, I routed around it and stopped assuming it would work.
None of this was visible until I tested it. And once written down, every one of these became something to design around rather than trip over repeatedly.
The real point: design around limits, don’t fight them
This is the mindset shift. The constraints are fixed — you usually can’t change what a tool can do. So a good working method treats the constraints as fixed and builds accordingly:
- When a tool can create but not edit → stop trying to edit; recreate with a version stamp.
- When files are too big for one channel → use a different channel, and note which.
- When an integration is broken → route around it, and record the workaround so the next session doesn’t rediscover the wall.
This is what “working with the grain of the tools” means. You’re not trying to force the tool to be something it isn’t. You’re shaping your method to fit what it genuinely is. The goal isn’t a perfect toolchain — it’s a reliable one whose edges you’ve mapped.
Re-audit periodically
Capabilities change. Tools ship updates, integrations get fixed or break, limits move. Re-run a lightweight version of the audit every so often, and update your capabilities file. A constraint you designed around six months ago may be gone — and a new one may have appeared. The file should reflect reality, not history.
Why this is worth the hour
It’s tempting to skip the audit and just start working. But every assumption you make about an untested capability is a future frustration waiting to happen — and worse, some of those failures are silent, meaning you won’t even know why your workflow isn’t producing what you expected.
An hour of deliberate testing, written down, turns the entire category of “why isn’t this working” frustration into a set of known facts you build on. It’s one of the highest-return hours you’ll spend setting up an AI working method.
Write down what’s real. Build on that. Stop assuming the happy path.
Frequently asked questions
What is an AI capability audit? It’s the practice of deliberately testing what your AI assistant can and can’t actually do in your specific environment — file editing, integrations, size limits, autonomous actions — and recording the real answers in a file so you design your workflow around true constraints instead of assumed ones.
Why does my AI workflow keep breaking unexpectedly? Usually because of a mismatch between assumed and actual capability — you assumed the tool could do something it can’t (like edit a file in place) or didn’t know about a limit (like a file-size cap). A capability audit surfaces these before they derail your work.
How often should I re-run a capability audit? Periodically — tools update, integrations break or get fixed, and limits change. A lightweight re-audit every few months keeps your capabilities file accurate.
What should I do when an AI tool can’t do something I need? Design around it. If it can’t edit files, recreate them with version stamps. If a channel can’t handle large files, use another. Record the workaround so you (and future sessions) don’t rediscover the limit.
This is Part 5 of a series on building a working method with AI. Next: the reconcile — how to catch the work that falls through the cracks when requests arrive from everywhere at once.