AI Driven Credit Scoring Models: Enhancing Accuracy and Fairness with Explainable Machine Learning

Abstract

Financial institutions increasingly rely on data driven risk scoring to flag potentially illicit or high-risk financial transactions. This report presents an AI driven credit scoring framework built on the Big Black Money Dataset, a transaction level dataset comprising 10,000 records across 14 attributes, including transaction amount, country, transaction type, industry, source of money and a Money Laundering Risk Score. A binary target, Risk_Class, was derived from the Money Laundering Risk Score (High Risk when the score is 7 or above), producing a class split of 59.46% Low/Medium Risk and 40.54% High Risk. A complete analytical pipeline was implemented, covering data quality checks, feature engineering (log transformed amount, shell company risk flag, large transaction flag, tax haven risk flag), class imbalance correction using SMOTE, model development (a Logistic Regression baseline versus a proposed LightGBM model) and explainability analysis using SHAP. On a held-out test set of 2,000 transactions, the proposed LightGBM model achieved an accuracy of 0.5385, precision of 0.4249, recall of 0.3909, F1 score of 0.4072 and ROC AUC of 0.5187, only marginally above the Logistic Regression baseline (accuracy 0.5260, F1 score 0.4234, ROC AUC 0.5080). SHAP analysis identified shell company involvement, transaction amount, financial institution and person involved as the leading predictors of the model's output. A fairness-oriented group analysis across country, transaction type and source of money segments showed high risk rates clustered tightly within a narrow 39%–42% band, indicating no material disparity across the groups examined. When compared against previously reported benchmark results for a similarly framed LightGBM credit scoring model (accuracy 0.987, F1 score 0.928), the reproduced results in this report are substantially lower. Correlation analysis revealed that the engineered transaction level features carry only a very weak linear association with the Money Laundering Risk Score used to define the target, suggesting that the ceiling on achievable predictive accuracy is set by the limited signal present in the dataset rather than by model choice. The report discusses this gap transparently and argues that reproducible, honestly reported benchmarking is essential when evaluating explainable AI systems for credit and financial crime risk scoring.