Problem
The classifier's GTD rules are a fixed system prompt. It can be corrected the same way a hundred
times and will make the same call the hundred and first, because nothing it is shown at call time
says anything about the corrections.
The corrections are real, specific and repeated: a sender whose mail always ends up in reference
rather than someday, a class of pull request that is always review rather than next_action, a
kind of item the user consistently leaves in the inbox on purpose. Spec 04 anticipated exactly this
and named the way out while listing it as a v1 non-goal: "Learning or fine-tuning from corrections
in v1. The classifications table exists so that few-shot examples from real corrections can be
added later."
Expected outcome
The classifier is shown a small set of the user's own past corrections, and stops repeating the
ones it has been corrected on.
- A correction the user made is capable of changing a later classification of a similar item. This
is the criterion the issue exists for, and it is demonstrable: the same input classified before
and after a correction on a similar item can differ.
- A human decision still beats a machine one absolutely (spec 00). Nothing here lets an example
override a status the user set, or re-open a task that has left the inbox.
- What was shown to the model is recoverable for any given classification, alongside the model and
prompt version already recorded. A behaviour change traceable to the prompt today stays traceable
when the examples are what changed.
- The set is bounded and the bound is stated, so a growing history cannot grow the payload without
limit or crowd out the rules themselves.
- Selection is deterministic. The same history and the same task select the same examples, so a
classification is reproducible and a wrong lesson is findable rather than intermittent.
- Nothing is sent that spec 09's content policy would not already allow to be sent for the item it
came from. A correction on an item is not a licence to send more of that item than the policy
permits.
- A correction the user reverses stops being taught.
- An install with no corrections yet classifies exactly as it does today.
Notes
Depends on the decision log, which is what makes a correction a durable fact rather than an
inference from current task state. Spec 04's non-goal and acceptance criteria move with this.
Depends on the classification measurement too, and for a reason worth stating: the characteristic
failure of few-shot examples is a confident regression on the cases the examples do not cover.
Without a baseline agreement rate there is no way to tell afterwards whether this helped, hurt, or
did nothing.
Problem
The classifier's GTD rules are a fixed system prompt. It can be corrected the same way a hundred
times and will make the same call the hundred and first, because nothing it is shown at call time
says anything about the corrections.
The corrections are real, specific and repeated: a sender whose mail always ends up in
referencerather than
someday, a class of pull request that is alwaysreviewrather thannext_action, akind of item the user consistently leaves in the inbox on purpose. Spec 04 anticipated exactly this
and named the way out while listing it as a v1 non-goal: "Learning or fine-tuning from corrections
in v1. The
classificationstable exists so that few-shot examples from real corrections can beadded later."
Expected outcome
The classifier is shown a small set of the user's own past corrections, and stops repeating the
ones it has been corrected on.
is the criterion the issue exists for, and it is demonstrable: the same input classified before
and after a correction on a similar item can differ.
override a status the user set, or re-open a task that has left the inbox.
prompt version already recorded. A behaviour change traceable to the prompt today stays traceable
when the examples are what changed.
limit or crowd out the rules themselves.
classification is reproducible and a wrong lesson is findable rather than intermittent.
came from. A correction on an item is not a licence to send more of that item than the policy
permits.
Notes
Depends on the decision log, which is what makes a correction a durable fact rather than an
inference from current task state. Spec 04's non-goal and acceptance criteria move with this.
Depends on the classification measurement too, and for a reason worth stating: the characteristic
failure of few-shot examples is a confident regression on the cases the examples do not cover.
Without a baseline agreement rate there is no way to tell afterwards whether this helped, hurt, or
did nothing.