MMODELYST
Papers/Toward User Comprehension Supports for LLM Agent Skill Specifications
PAP

Toward User Comprehension Supports for LLM Agent Skill Specifications

May 19, 2026

arXiv
Abstract

Users often interpret and select agent skills through their SKILL markdown specifications. To protect users, existing audits mainly focus on malicious or unsafe skills. We study the complementary question of whether specifications help users form bounded expectations about what a skill consumes, produces, and covers. Across 878 cybersecurity skills, we used rule-based coding to measure textual cues for four comprehension anchors, namely operational basis, output contract, boundary disclosure, and example capability demonstration. Cues for operational basis were common, but only 19.0% of specifications exhibited cues for an example task, sample, or expected outcome, and only 2.3% exhibited cues for all four anchors. We further examined a small DNS/C2 telemetry subset (n$=$6) to illustrate why missing examples may matter. Examples appeared to make first local checks easier to construct, while no-example skills typically required helper code inspection to recover command arguments or output fields. We argue that agent-skill evaluation should treat specifications as user-facing capability disclosures, not merely as containers for executable instructions.

Select text to highlight · click a highlight to remove · saved in this browser only
Authors
Zikai Alex Wen
Your notes (browser-local)
saved
arXiv:2605.19362