4.2 Monitoring in AWS for Model Bias, Trustworthiness, and Truthfulness

Overview of Bias and Monitoring with AWS

Biases are imbalances or disparities in a model’s performance across different groups. To measure and monitor model bias and trustworthiness, AWS provides tools like SageMaker Clarify.

SageMaker Clarify Features

1. Bias Detection:

• Detects potential bias during:
  • Data preparation
  • Model training
  • Model deployment
• Analyzes specific attributes for bias.

1. Explainability:

• Helps explain model decisions.
• Treats the model as a black box to evaluate feature importance.
• For example: Loan rejection may be explained by factors like income and outstanding debt not meeting thresholds.
• Works with models using unstructured data like computer vision or natural language processing (NLP).

1. Processing Jobs:

• Uses SageMaker Clarify processing container to interact with:
  • S3 bucket (stores input data and model).
  • SageMaker inference endpoint (deployed model).
• Sends requests to model container and retrieves predictions.
• Saves results (e.g., bias metrics, feature attributions) to an S3 bucket.
• Results are available as:
  • JSON file with bias metrics
  • Visual report
  • Local feature attribution files.

Metrics Measured by SageMaker Clarify

1. Training Data Metrics:

• Balanced Dataset: A balanced dataset avoids bias. For example:
  • A model trained primarily on middle-aged individuals may be less accurate for younger or older individuals.
• Label Imbalance: Occurs when one class is favored over another. Example:
  • Training data may show a higher loan approval rate for middle-aged individuals.
• Demographic Disparity: Indicates imbalance in outcomes. Example:
  • In college admissions, if 46% of rejected applicants are women, but only 32% of accepted applicants are women, it shows demographic disparity.

1. Model Metrics:

• Difference in Positive Proportions in Predictions: Measures whether the model predicts positive outcomes differently across classes. This helps detect if bias persists after training.
• Specificity: Measures how often the model correctly predicts a negative outcome. Lower specificity for one group indicates bias.
• Recall Difference: Measures differences in the True Positive Rate (TPR) across classes. Bias is shown if recall is high for one class but low for another.
• Accuracy Difference: The difference in model accuracy across different classes. Bias occurs if accuracy is different between classes.
• Treatment Equality: Measures the ratio of false negatives to false positives. Even if accuracy is the same, differences in errors can indicate bias.
  • Example: Loan approval decisions with different error patterns across classes show bias.

Conclusion

• SageMaker Clarify helps ensure models are:
• Unbiased
• Trustworthy
• Explainable