Affinda provides confidence scores for data extracted to help you assess the reliability of data extracted from your documents. Rather than using a confidence value from the model that does not strongly reference the data from the documents validated by users, our approach to confidence is to provide significantly higher weight to validated documents of a similar format to the uploaded document. Confidence Scores, when available, are shown in the Affinda app when you hover over the yellow dot next to an unconfirmed field: Confidence Score Example

​
Benefits

  • Confidence is not calculated without the context of other validated documents. Instead, it pays particular attention to the data from very relevant documents
  • The ceiling for how confident the model can be in the predictions is much higher (up to 99%)

​
Limitations

  • While our method of calculating confidence delivers strong results when using Affinda’s platform at scale, it will take 2-3 examples of the same document format before confidence will be returned on fields

​
How Confidence is Calculated

1

Fingerprint Matching

When users upload a document, our Fingerprinting algorithm identifies suitable reference document(s) from Model Memory that are provided to the model to help guide the extraction.
2

User Validation

Whenever someone validates data from a processed document, we store the data results as our “ground truth” for confidence calculation. Accuracy is measured by comparing the model’s predictions with validated annotations.
3

Confidence Determination

For each new document, we look at the accuracy results from up to the last five validated documents that used the same reference document. An overall accuracy score is calculated for each extracted field.

​
Setting Confidence Thresholds

Affinda can support both field-level and global confidence thresholds. Field-level: Users can configure a specific confidence threshold by creating a validation rule in the app. For example, “@Date of Establishment confidence is >80%”  Global: Affinda can support global confidence thresholds to enable auto-validation of:
  • Fields that meet the global threshold.
  • Documents that meet the global threshold.
Global thresholds are not currently supported through validation rules. To enable for your workspace, please reach out to the Affinda team.

​
FAQs