Version, Govern, and Retire Skills
A skill library becomes part of the business operating model. I treat changes to important skills like changes to product behaviour: reviewed, versioned, measurable, and reversible.
- Skill ownership
- Versioning and release notes
- Staged rollout
- Tenant-specific configuration
- Deprecation and migration
- Rollback and emergency disablement
The situation
A useful skill does not stay static. Policies change, tools change, models change, data sources change, and teams learn from production failures. Eventually several people are editing capabilities that affect customers or internal operations.
The naive solution
Edit the live prompt or skill file whenever someone has a better instruction. The next run automatically gets the new behaviour.
The hidden constraint
Unversioned behaviour makes incidents difficult to reproduce. A customer result generated yesterday may no longer be reproducible today. One team may depend on an old output shape while another changes it. A global update may improve one tenant and break another. Without ownership, nobody knows who has authority to approve a behavioural change.
The decision
Treat meaningful skills as versioned product assets. Give them owners, change history, evaluation gates, rollout controls, and rollback paths.
The implementation
I would keep skill definitions in version control, assign an owner or owning team, require tests for important changes, and record a semantic or monotonic version in every invocation trace. High-impact updates can be rolled out to internal users or a subset of tenants first. If output contracts change, I treat that as an interface migration rather than a silent prompt edit. I also want an emergency disable switch for dangerous capabilities.
The tradeoff
This can feel heavy for early prototypes. I would apply governance in proportion to impact: a private drafting helper does not need the same process as a skill that modifies financial records. The principle is proportional control, not bureaucracy for its own sake.
The evidence I would look for
When an incident occurs, the team can identify the exact skill version involved, reproduce it against a test case, compare with the previous release, and roll back or disable it without shutting down the entire agent.
The reusable lesson
Reusable agent behaviour is software behaviour. Once the business depends on it, change management becomes part of reliability.
The consultant's questions
- Who owns the behaviour of each high-impact skill?
- Can you tell which version produced a historical action?
- Can a risky skill be disabled without disabling the entire agent?
- What happens when a skill changes its output contract?