Building an AI agent skill sounds straightforward. Write a few instructions, drop them in a folder, and let the agent handle the rest. But most people get this wrong.
The gap between what tutorials claim and what actually works is brutal. You write a vague description like “helps with research” — the agent ignores the skill entirely. You copy a generic prompt from a forum — the output is flaky. You spend hours tweaking — but the agent still makes the same mistakes.
The gap between what tutorials claim and what actually works is brutal.
There is a free tool that closes this gap. It’s called Ordain (ordain.host), and it writes the two files your agent needs to work consistently: the soul file (who the agent is) and the skill file (how to do the job).
I am not saying Ordain is magic. But the problems it solves are very specific — and very common. Let’s walk through the three biggest gaps between theory and practice, and how this tool fixes each one.
Gap #1: The Soul File is an Afterthought
Standard advice tells you to “define your agent’s personality.” People paste in lines like “you are helpful, thorough, and friendly.”
That tells the agent nothing it didn’t already assume.
An agent straight out of the box already assumes it should be helpful. What matters is how it behaves while working: does it ask before acting, or just go? Does it show reasoning first, or the answer? Does it pad lists to hit a round number?
Without specific rules, every session starts from scratch. The same request run twice gives different formats, different tone, different information.
In Hermes (the open-source agent at 236,000 GitHub stars), the soul file is a single markdown document called soul.mdcodecodecodecode. It goes into every system prompt. Whatever is in that file is what the agent believes about itself.
What actually works: Ordain’s soul builder asks you to check exactly what the agent handles (messages, email, research, reminders) and uncheck everything else. It asks for personality — but not “professional and friendly.” It gives you a choice between direct, warm, playful, and lets you pick one. The real work happens in the text box at the bottom. That’s where you write specific instructions like:
“Works in British English. Gives the answer first and the reasoning underneath. Does not pad lists to hit a round number.”
blockquoteblockquoteblockquoteblockquote
That single block of text changes the agent’s output more than a page of fluffy adjectives ever could.
Gap #2: Skills Are Too Vague to Trigger
A skill file has a YAML front matter section with a name and a description. That description is the only thing the agent reads when deciding whether to open the skill.
Most people write something like “Use this skill to research topics.” The agent then ignores it because it doesn’t know when to apply it. It can do research without a skill. The description must be precise enough that the agent says “yes, this matches the job in front of me.”
Example of failure: I watched someone build a “research brief” skill and write the description as “for creating research summaries.” Later, when they asked the agent to “find the latest pricing on AWS,” the agent did not open the skill. It just answered from training data. The description was too broad.
What actually works: The Ordain skill builder asks for the name and then a plain-English box labeled “What should it do?” What you write in that box determines whether the skill is worth having.
The source material includes a full version that names the sources it starts from, the order it works through them, what must be in the finished brief, and what to do when sources contradict. That is the difference between vague and precise.
And the requirements box — “every claim carries a link” and “nothing goes in that it hasn’t opened itself” — those are the guardrails that stop the agent from hallucinating numbers because it “sounds right.”
Gap #3: No Verification Cycle
Standard advice: write a skill, test it, done.
The problem is that a skill that reads perfectly can still produce garbage. The agent follows the procedure but formats the answer however it wants. Or it fills a gap with something plausible because nothing told it to stop.
What actually works: Every skill file should include a verification section: how to check the work, what “finished” looks like, and a pitfalls list.
Ordain includes this automatically. The generated skill.mdcodecodecodecode has sections for “how to check the work at the end” and “pitfalls.” This last part is the one most people skip — and it’s where the real value lives.
When the agent returns a result, it runs the verification loop internally. The second run comes back in the right structure. The third run identifies missing citations. The skill improves because the pitfalls section grows every time you catch a mistake.
The source video showed a case where the first run ignored the output structure entirely. The fix was adding one line: “the output goes back in the exact printed format.” That line went into the pitfalls section. The second run was correct.
One Specific Example: The Research Brief Skill
Here is a concrete comparison.
Theory: Tell the agent “research topics and give me a summary.” The agent will read a few websites and write a paragraph. Fine for one-off questions, but worthless for a recurring job where you need the same format every week.
Practice: Use Ordain to generate a skill that specifies: start from official docs and change logs, work through forum threads last, include everything that changed in the last 90 days with a link next to each number, list places where sources disagreed (write both positions, do not pick one), and at the bottom list anything it could not confirm as “not found” with exactly where it looked.
When I ran this skill twice on the same topic, the second run came back with the same headings, same order, same sources. The only difference was the content, because the research found new information. That is the whole point of writing the thing down.
The free plan on Ordain handles this. It generates the file, you download the zip, drop the folder into your agent’s skills directory, and it’s live. No coding. No manual markdown.
The Thing Nobody Talks About
Even a perfectly generated skill needs one more step: run it on a job you already know the answer to. Check whether it followed the procedure or worked around it.
Most people skip this. They trust the output because it looks right. But the agent may have taken a shortcut — ignoring the source order, filling a gap, skipping the verification step. The generated skill is just a text file. The model still decides what to do. The pitfall section is how you close that loop over time.
The more skills you add, the more your agent’s working style is stored in files instead of in your head. That is the point where it stops being a chat window you visit and becomes a system that runs while you sleep.
[
- Why do most AI agent skills fail?Most AI agent skills fail because of vague instructions that the agent ignores, leading to inconsistent output.
- What is the soul file in an AI agent?The soul file is a markdown document that defines the agent's identity and behavior rules, included in every system prompt.
- How does Ordain help create better skills?Ordain generates precise soul and skill files based on your specific choices, eliminating vague instructions.
- What is the pitfall section in a skill file?The pitfall section documents common mistakes and how to avoid them, helping close the loop over time.
- Why should you test a generated skill on a known job?Testing ensures the agent follows the procedure instead of taking shortcuts, building trust in the output.
