Feat: DeepSHAP-based model explainer with configurable visualisation outputs

[samrat-rm]

This PR implements a DeepSHAP-based model explainer for analysing feature (band) contributions in testing models. It help us have a better understanding of feature dominance and make more informed decision for model improvements.

This explainer is also intended to serve as a diagnostic tool for the Unimodal problem, helping us understand and mitigate IR band dominance.

⚠️ Scope is currently limited to testing models only.
Support for training-time explainability will be added after review and feedback.

DeepExplainer
An implementation of Deep SHAP, a faster (but approximate) method to estimate SHAP values for deep learning models. It leverages connections between SHAP and DeepLIFT to efficiently compute feature attributions.

Limitations :
DeepExplainer does not fully support certain layers:

• LayerNorm
• DropPath
• UpsamplingBilinear2d

As a result:

• SHAP uses approximations instead of exact values
• Explanations may not sum exactly to model output (additivity gap of ~5%, tolerance is <1%)

GradientExplainer
An implementation of expected gradients, combining integrated gradients with sampling over background data to approximate SHAP values. It estimates feature attributions by averaging gradients across inputs.

As a result:

• It is slow compared to DeepExplainer but is accurate.

Features

• Implements DeepSHAP explainer for model interpretability
• It has 2 type of explainers
  • Deep Explainer is fast but less accurate (default)
  • Gradient Explainer is slow but more accurate
• Supports multiple output formats (image attached) :
  • waterfall → SHAP-style explanation (default)
  • pie_and_bar → magnitude + signed contribution view
  • Prints aligned band contribution table in terminal

Current Limitations

1. Background Sampling Sensitivity

• SHAP requires a representative background dataset
• Current implementation may produce varying results due to:
  • Random sampling
  • Potential imbalance in selected background images

Possible Solutions :

• Use a more balanced representative dataset sample.
• Use GradientExplainer which is slower but recommended for large models.

Possible Improvements / Next step

• Use a more balanced and representative background dataset
• Switch to GradientExplainer:
  • More stable for large / complex models (recommended by shap)
  • Handles unsupported layers better (LayerNorm, DropPath, UpsamplingBilinear2d)
  • Trade-off: slower and expensive execution
• Export results (CSV / JSON)
• Batch explainability support (If required)
• Extend support to training pipeline (After feedback)
• Make code modular (a separate utils file)

Usage :

python model_explainer.py -e <dataset> -f <format> -m <explainer method>

Example :

python models/model_explainer.py -e landset -f waterfall -m deep
python models/model_explainer.py -e landset -f pie_and_bar -m gradient

AI Disclosure :

Used AI assistance to refine code structure, improve readability, and format documentation.
All logic, implementation decisions, and validations were reviewed and verified manually.