Skip to main content

Confidence vs Probability

Purpose​

Explain why confidence and probability are different analytic concepts.

Practitioner-Level Explanation​

Probability estimates likelihood. Confidence describes the strength and reliability of the analytic basis. A judgment may be assessed likely but low confidence if the evidence is thin. Another judgment may be unlikely but high confidence if strong evidence rules it out.

Most CTI products should use confidence language rather than precise percentages unless the organization has a calibrated probability model.

CTI Relevance​

This distinction prevents decision-makers from misunderstanding how much trust to place in a CTI assessment.

Common Mistakes​

  • Using high confidence to mean high likelihood.
  • Adding percentages without calibration.
  • Failing to explain confidence reasons.
  • Using low confidence as a reason to avoid action when impact is high.

Practical Workflow​

  1. Write the judgment.
  2. Decide whether the output needs likelihood, confidence, or both.
  3. Explain evidence quality.
  4. Explain uncertainty.
  5. State decision impact.

Example / Mini Case​

"We assess it is possible that this activity is related to Actor X, but confidence is low because the only overlap is commodity tooling and broad victimology."

Analyst Checklist​

  • Does the product confuse confidence and likelihood?
  • Is a probability number necessary?
  • Is confidence explained?
  • Does the recommendation match risk and evidence?

Output Artifact​

Judgment:
Likelihood Term:
Confidence Level:
Evidence Quality:
Uncertainty:
Decision Impact:

References​