jev-skill-router · recorded evaluation

source abgregs/jev-skill-router

Which skills should the agent invoke?

Prompt

Just a coding agent · fable 

● Skill(better-accessibility)
  ⎿  Launching skill: better-accessibility

● Bash(git ls-files | head -200 …)

Recorded stock runs, same prompt, four models. Three found the accessibility skill, Haiku never loaded one, and none loaded the debugging skill. Skill use rides on which model you picked.

With jev-skill-router · four models


            
Recorded routed runs, same prompt, four models. The
              same invoke verdict on all four. Every model loaded the accessibility skill
              as its first step, and two also loaded the debugging skill. The judgment
              lands before the model starts thinking.

          

        

      

    


    

    

      

        
Mechanism

        recorded 
      

      
One real turn over a real catalog of  installed
        skills. The chart replays the full judgment at its captured speed.

      

        

        

          

        

        

          
          
          
        

        

        
      

    


    

      

        
Catalog

        recorded 
      

      

      

      

      

        

      

      
      
      

      
    


    

      

        
Scale

        recorded 
      

      

        The same judgment at org scale, the whole catalog judged every turn. Each skill is
        judged on its own, so the sweep splits across parallel requests, and a catalog twenty
        times larger came back in about twice the time of the runs above, not twenty times it.
        The catalog is generated and packed with near-identical siblings to stress the sweep,
        so the picks below are illustrative, not an accuracy claim.
      

      

        

      

      

      

      

        

      

      
      
    


    

      

        
Doctor

        recorded 
      

      

        By convention a skill declares what it does and when to run in its SKILL.md
        description, and that field is everything the router and the doctor see. Each skill’s
        own description is routed against the whole catalog, so co-invocation on its home
        territory is measured confusion. Every finding names its evidence and exactly one
        action. This is the control surface for the co-invocation Catalog reports.
      

      

      
Same-author suites are exempt from overlap findings, and self-miss
        probes contribute no conclusions.