Ada Security
Request Demo

11.6 Training

11.6.1 Starting a training

  1. Make sure every type you want trained has at least 5 examples.
  2. Click Train under the table. The button changes to Training… and the hint shows Training since The page polls every 2 seconds.
  3. When training finishes, the status line shows the summary (section 11.6.3) and the type table fills Precision and Recall.

Training usually takes seconds; it runs in a background thread on the server and does not slow the console or agent traffic. It is stopped after 15 minutes.

Train is disabled while a training runs, while the AI service or model is missing, or when no type has 5 examples.

Training always covers all types at once:

  • Types with fewer than 5 examples are skipped and listed under Not trained (under 5 examples).
  • If a type that a rule uses has fewer than 5 examples, training refuses to start: “X” is used by rule “Y” but has only N example(s); add at least 5. (Otherwise that rule would silently stop working.)
  • Counter-examples are always included.
Training error Cause
Training is already running. Another administrator started one. Wait.
Each document type needs at least 5 examples before training. No type is trainable.
The AI encoder is not installed on this server. No model package.
This server has no reference documents for encoder … Release package incomplete. Reinstall/upgrade the server release.
The examples of “X” are near-copies of one document … All examples of X are practically the same text.
Training was interrupted (the API restarted). Start it again. ada-api restarted mid-training.
Training took longer than 15 minutes and was stopped. Very large example set; contact support.

11.6.2 What the server does

  1. Decrypts the examples; examples stored for an older model are re-read with the current one.
  2. Adds about 2,000 built-in reference documents (a varied Turkish/English corpus shipped with the server) as “none of your types” — otherwise every document would be pushed toward the nearest of your types. Reference groups of the same kind as one of your types (your “CV” type versus reference CVs) are left out automatically.
  3. Adds counter-examples as “none of your types”, with their own weight.
  4. Measures quality by cross-validation (up to 5 rounds): each example is scored by a layer trained without it; near-copies stay in the same round.
  5. Calibrates probabilities so that “90 %” means roughly “right nine times out of ten” on your data, with a floor that prevents a small example set from producing 100 %-sure scores.
  6. Trains the final layer on everything and re-signs the policy. Agents receive it at their next policy sync — no Publish is needed for training results.

The method is a regularised logistic regression tuned for small example sets (5–30 per type). There are no training settings to change.

11.6.3 Reading the training results

The status line after a successful training:

Figure Meaning What good looks like
N examples Examples used (trained types only, counter-examples excluded). ≥ 20 per type.
type right Share of your examples whose most likely type, scored by a layer that never saw them, was the correct type. ≥ 90 %. Lower means types overlap or have too few/too uniform examples.
unrelated documents recognized Share of the reference documents classified as “none of your types”. ≥ 95 %. Lower means your types are too broad and will produce false alarms.
counter-examples left alone Share of counter-examples that matched none of your types at 70 %. As high as possible; the ⓘ lists types that still match some (Still matching: X (3)).
· examples changed since — train again to use them Examples were added or removed after the last training. Train again.

The ⓘ How training measured quality next to the status line adds: the number of rounds, the number of reference documents used, which reference groups were left out because your examples are that kind of document, which types were not trained, and the counter-example result.

Precision and Recall (type table and rule editor):

Measure Question it answers Example
Precision at a threshold Of the documents this type flags, how many really are this type? 95 % → about 1 in 20 flagged documents is wrong.
Recall at a threshold Of the real documents of this type, how many does it catch? 80 % → 1 in 5 is missed.

They are measured at eight thresholds (30, 40, 50, 60, 70, 80, 90 and 95 %). The table shows 70 %, the threshold new rules start with. Raising the threshold generally raises precision and lowers recall.

Interpreting typical patterns:

Pattern Likely cause Action
High precision, low recall Too few or too uniform examples; real documents vary more. Add varied examples (other languages, templates, years).
Low precision The type is broad, overlaps another type, or reference documents of the same kind look alike. Add counter-examples of the wrong matches; split or merge types; raise the threshold.
few examples badge 5–19 examples. Aim for 20+.
Precision — The type never reached 70 % in cross-validation. More examples; do not build a rule on it yet.
Good figures, wrong results in test mode Examples not representative of what users actually write. Upload the real misses as examples and the false matches as counter-examples, then retrain.