Make Skill Selection a System, Not a Guess
A skill library is only useful if the right capability is selected at the right time. I treat descriptions, eligibility rules, confidence, and fallback behaviour as part of the architecture.
- Explicit invocation versus automatic routing
- Skill descriptions as routing interfaces
- Eligibility and preconditions
- Confidence and ambiguity
- Fallback to clarification or general reasoning
- Routing evaluation
The situation
With three skills, selection feels trivial. With thirty, similar capabilities begin to overlap: customer-summary versus renewal-summary, prospect-research versus account-research, support-triage versus incident-triage. A model may choose a plausible skill that is subtly wrong for the business context.
The naive solution
List every skill name and description in the system prompt and ask the model to pick the best one.
The hidden constraint
Routing quality depends on more than semantic similarity. Permissions, tenant configuration, available data, user role, workflow stage, and preconditions may make a skill ineligible even if the request sounds like a perfect match. Ambiguous selection can also be expensive if the wrong skill performs side effects before anyone notices.
The decision
Treat skill routing as its own decision layer. Some skills can be explicitly chosen by product UI or deterministic workflow. Automatic routing should consider capability description, preconditions, permissions, context availability, and the cost of being wrong.
The implementation
I would give each skill a concise routing description, positive examples, negative examples, eligibility checks, and an explicit when-not-to-use section. For high-risk capabilities, I prefer deterministic entry points or confirmation before invocation. For low-risk analytical skills, model-based routing may be completely appropriate. I also log which alternatives were considered so misroutes can be diagnosed rather than treated as mysterious model behaviour.
The tradeoff
Dedicated routing adds latency and implementation work. It can also make the product feel less magical than one universal agent. I prefer reliable capability selection to magic that is difficult to inspect.
The evidence I would look for
I would maintain a routing test set containing real and deliberately ambiguous requests. The important metric is not only whether the correct skill was selected, but whether unsafe or ineligible skills were avoided and whether the system asked for clarification when it genuinely could not know.
The reusable lesson
Skill selection is part of product logic. As the library grows, routing deserves the same design attention as the skills themselves.
The consultant's questions
- How does the agent choose between two semantically similar skills?
- What makes a skill ineligible even when its description matches the request?
- What happens when routing confidence is low?
- Do you have examples that specifically test the boundaries between neighbouring skills?