Update an AI skill without losing the version you tested

Update a skill by naming the behavior you want to change and keeping the previous version. Compare the instructions and repeat the affected evaluations. A new filename or version number is useful only when it points to a meaningful record of what changed.

Write the reason first

“Improve the skill” is vague. “Ask for a source location before treating a quotation as verified” gives the update a specific purpose. Identify which examples should now behave differently and which behavior should remain the same.

A fictional change record

Version: 1.1
Reason: Previous workflow treated a source URL as sufficient quote evidence.
Change: Require an exact passage and location for direct quotations.
Expected improvement: Missing-source case is flagged before revision.
Regression check: Complete-source case still produces a usable edit.
Host/model tested: [actual values after testing]
Observed results: [recorded output, not an invented example]

Compare the files before testing

Look for accidental changes to scope, tools, permissions, and referenced paths. If the update adds a script, inspect its behavior separately from the prose. Preserve required notices and licenses when modifying third-party material. Your change note should distinguish upstream work from your additions.

Retest the actual host

Repeat the missing-input case that motivated the change and at least one complete-input case. If the host or model also changed, record that difference; otherwise you cannot attribute a behavior change solely to the skill.

Keep the old package available until you are satisfied with the new behavior. A rollback should restore the known files, not reconstruct them from memory. When sharing the update, state which behavior was evaluated and which remains unverified. File-format validation, download tests, native activation, and model-output checks are different layers and deserve separate records.