Stacking Ensemble Approach for Predicting Loan Approval Using Machine Learning Techniques

Summary

This study develops a stacking ensemble model integrating XGBoost, CatBoost (Gradient Boosting Model), LightGBM (Efficient Gradient Boosting Model), AdaBoost, and Extra Trees to predict loan approvals using Kaggle data. Achieving 98% accuracy, it identifies key predictors like income and credit score, promoting fair, efficient decisions on loan approval and/or rejection.

Abstract

Digital lending and fintech innovations have upended established banking systems, changing financial inclusion and credit availability in nations around the world. This study examines how peer-to-peer (P2P) and digital lending platforms are changing, emphasizing how technologies like artificial intelligence and machine learning are changing the way loans are approved. A thorough study of the literature highlights the opportunities and problems in the digital lending ecosystem, such as algorithmic risk assessment, customer trust, financial exclusion, and regulatory loopholes. This paper suggests a strong machine learning approach that uses a stacking ensemble model to accurately forecast loan approvals in order to address these issues. The data was pre-processed using train-test partitioning, exploratory analysis, and label encoding using a publicly accessible Kaggle dataset that included applicant demographics, financial characteristics, and credit histories. With XGBoost serving as the meta-learner, the ensemble incorporates the Gradient Boosting Model, Efficient Gradient Boosting, AdaBoost, and Extra Trees classifiers as base learners. With an accuracy of 98%, the model was assessed using measures including accuracy, precision, recall, F1-score, and error metrics (MAE- Mean Absolute Error, MSE- Mean Squared Error, and RMSE- Root Mean Square Error). According to correlation studies, factors including assets, income, and CIBIL scores have a significant impact on loan approvals. Outperforming conventional methods, the model showed balance and generalization across both classes. The usefulness of these models for automated, data-driven credit determinations is emphasized in the paper's conclusion.

Introduction

In the latest phase of the banking industry's technology transformation, disruptive new financial service providers from outside the established banking system have entered the market¹. BigTech (large tech companies that primarily focus on lending directly or with financial institutions) and FinTech (financial technology, including models like P2P lending and online credit alternatives to traditional banks) companies are making substantial inroads into the finance sector, posing a challenge to traditional banking despite banks' efforts to adapt to the digital landscape². This rapid evolution signals a shift in the financial ecosystem, where non-traditional players are increasingly reshaping the way financial services are accessed and delivered³. The emergence of digital lending has a negative correlation with banking credit, suggesting that as new lenders enter the market, traditional banking may give way to alternative digital credit⁴. This transition was catalyzed further by the 2008 global financial crisis (GFC), which drastically lowered client trust in financial services and helped drive the expansion of financial technology or Fintech ventures⁵. Fintech is the term for the combination of technology and finance, which refers to the application of technology to provide financial solutions⁶. As Fintech matured, one of its most transformative applications was seen in the rise of P2P lending, also referred to as online lending services⁷. P2P lending's primary innovation is the direct matching of lenders and borrowers. Borrowers submit applications for small, unsecured loans, and lending platforms are used by several investors to evaluate and finance loan requests⁸. P2P lending functions similarly to a bank, but it uses the internet and cutting-edge technology to enable online lending and debt arrangements ⁹. The success and scalability of this model became evident with the launch of ZOPA.com, the first P2P platform in history, which debuted in the UK in 2005. Since then, online lending has grown significantly, reaching over $100 billion by 2015, and is expected to reach over $1 trillion in 2025¹⁰. Digital lending, particularly in emerging economies, has further evolved with Fintech integration¹¹. Fintech integration in digital lending enhances financial inclusion, particularly in emerging markets. Mobile payments and blockchain solutions enable P2P transactions and micro-lending, reducing barriers to financial services¹². This paradigm shift is driven by the incorporation of technologies such as blockchain, artificial intelligence (AI), machine learning, and digital payment systems to create a more inclusive, efficient, and customer-centric financial environment¹³. Digital lending platforms use technology to expedite applications, save expenses, and enhance credit risk evaluation, allowing small firms and individuals to receive financing more quickly¹⁴. They use big data, blockchain, AI, and machine learning to improve borrower evaluation, lower costs, and promote financial inclusion¹⁵. Machine learning, in particular, has revolutionized risk management by leveraging alternative data sources¹⁶. It surpasses traditional credit assessment approaches by leveraging non-traditional data, enhancing borrower ratings, and forecasting economic developments¹⁷. This method reduces the risk of default by increasing the accuracy of borrower assessments and assisting in the prediction of shifts in the economy¹⁸. One of the most important effects of digital lending is its ability to address the difficulties of financial inclusion, particularly in emerging economies and marginalized areas¹⁹.

In order to forecast loan acceptance with high accuracy using a structured Kaggle dataset, this paper proposes a novel stacking ensemble model that combines the Gradient Boosting Model, the Efficient Gradient Boosting Model, the AdaBoost, the Extra Trees, and the XGBoost. To improve predictive adaptability and generalization, this method combines several advanced learners with XGBoost as a meta-classifier, in contrast to earlier research that frequently uses single models or conventional classifiers. The model performed well in both accepted and rejected loan classes, with an impressive accuracy rate of 98%. This methodological development provides a practicable and expandable way to automate loan approval decisions in digital lending settings, particularly in developing financial ecosystems.

The aim of this research is to create a strong stacking ensemble model for digital lending that accurately predicts loan acceptance by combining Gradient Boosting Model, Efficient Gradient Boosting Model, AdaBoost, Extra Trees, and XGBoost. Additionally, it seeks to examine how important demographic and financial variables (income, asset value, and CIBIL-Credit Information Bureau (India) Limited Score) affect loan choices, evaluate how well the ensemble model performs in comparison to more conventional models using classification and error metrics, and emphasize how ensemble approaches can increase efficiency, generalization, and fairness. The primary goal is to statistically analyze how applicant features influence loan approval and to evaluate the performance of ensemble learning algorithms.

P2P and digital lending continue to transform the financial landscape globally, presenting both opportunities and challenges.

Digital lending is rapidly transforming the global financial landscape, offering an alternative to traditional banking²⁰. This global outlook underscores how regional contexts uniquely shape digital lending maturity. Digital lending is expanding but remains technologically immature, while automation and predictive scoring bring efficiency, and platforms still depend heavily on third-party systems for background checks, which limits robustness²¹. Despite its rapid expansion, financial exclusion remains a major issue globally, with an estimated 44% of adults in developing countries lacking access to formal financial services, necessitating urgent reform, better infrastructure, and digital literacy initiatives. Such limitations also appear in other converging sector highlights, ongoing challenges in data handling, and system integration²². As digital integration deepens, Security vulnerabilities across the Fintech space are escalating. To address these, a security-focused framework has been proposed to safeguard digital transactions²³. Similar developments are observed in other emerging markets. In Kenya, while mobile money and digital loan apps have improved financial access, data privacy remains a persistent concern, and recent regulations have limited impact, suggesting that stronger enforcement mechanisms, formal audits, and clear development guidelines are needed²⁴. This reflects a broader trend where regulatory frameworks often lag behind fintech innovation. The regulatory landscape of fintech is different from that of traditional banking. For example, unless loans are high-risk, law enforcement has less of an effect on interest rates in fintech²⁵. Especially, there is a strong need for improved supervision, use of data analytics, and regulatory updates to curb illegal fintech growth and privacy breaches²⁶. Beyond regulation, the success of digital lending also hinges on trust, so trust plays a critical role in lending decisions. Trust in barrowers is more influential than in intermediaries²⁷.

A parallel evolution is visible in India's digital lending ecosystem²⁸. The digital lending business is expanding rapidly, due to advances in fintech, helpful regulatory measures implemented by the Reserve Bank of India (RBI), and an increase in consumer trust following the COVID-19 outbreak²⁹. However, with innovation comes risk. While unlicensed digital lending applications or platforms improve access, they pose severe consumer risks, such as harassment, high interest rates, and data misuse due to weak regulations. Strengthening consumer protection and accountability is therefore critical to promoting responsible financial inclusion³⁰. The dangers of borrower defaults and fraudulent applications are substantial for digital lending; good consumer protection measures not only protect consumers but also positively influence financial performance as data security and transparency improve profitability indicators like Return on assets (ROA) and Return on equity (ROE)³¹. Globally, there is a considerable emphasis on operational improvements, with increased emphasis on enhancing loan origination systems, encouraging the use of mobile technology, and developing clear strategies to fulfil regulatory standards and consumer expectations³². To address these risks, advanced analytical and AI are increasingly being employed to predict high-risk lenders, outlier detection using indicators like failed loans, repayment duration, and credit scoring has proven effective³³. Using the socio-technical model as a guide, we discovered that risks come from both stakeholders and the lack of interdependencies between platform design and organizational components³⁴. Adoption of dynamic models like UTAUT2 dominates in explaining user adoption, with trust emerging as a key predictor of borrowing intent³⁵. Machine learning-based fraud detection algorithms, such as Random Forest and SVM models, are also used³⁶. According to the study's findings, machine learning models can adequately evaluate personal credit information and determine the likelihood of loan default; the deep neural network performed best (accuracy: 0.94)³⁷. The study, which used Naïve Bayes with 94% accuracy, discovered that characteristics such as interest rate, repayment time, description, credit grade, loan history, gender, and credit score have a substantial impact on loan success³⁸. Meanwhile, the probabilities of both prepayment and default risks exist, important occurrences that result in loan termination and loss of profit for creditors were predicted using multivariate logistic regression, and the model's overall accuracy was 76.63%³⁹. According to the study, lending clubs' revenue can be increased with high accuracy of 68 % by utilizing an Efficient Gradient Boosting Model to forecast default risk on Digital lending platforms⁴⁰. Simultaneously, more sophisticated AI models are evolving, such as deep multiview learning, which combine various variables (such as app usage and behavioural patterns) and perform better than conventional techniques, particularly in situations where historical data is limited⁴¹. Studies from China confirm that improving default predictions and financial inclusion, with models like Gradient Boosting Model and LGBM outperforming traditional credit-based evaluations⁴², system dynamic modelling also helps simulate interest rate fluctuations on P2P platforms, offering insight into borrower investor behavior under various conditions⁴³. Efficient Gradient Boosting Model has been shown to improve default prediction and platform profitability⁴⁰, while deep neural networks also outperform traditional models when proper training³⁷, and stabilize digital markets through better risk management⁴⁴ to ensure sustainability, regulatory technology is gaining traction, such as Robotic Process Automation assists financial institutions in aligning regulatory requirements with business plans, enhancing compliance and operational efficiency⁴⁵. Table 1 summarizes key studies that explore the application of machine learning in digital lending and loan approval processes.

Protocol

Data collection

This study utilized the Loan Approval Prediction Dataset available on Kaggle. The dataset was extracted in February 2025 and consists of 4269 records aimed at evaluating loan data and forecasting loan approval outcomes. It includes 12 columns comprising detailed information on applicants' demographic profiles, such as employment status, dependents, self-employed, loan amount, loan term, CIBIL scores, financial background, and loan-specific attributes. The dataset was imported using the Pandas library and visually inspected using df.head () to understand its structure and quality.

Data pre-processing

During the data pre-processing phase, the first step involved removing the identifier column (loan_id) due to its lack of predictive value and potential to introduce noise into the model. The second step involved label encoding, where categorical variables such as education, self-employed, and loan_status were converted into numerical representations. This transformation was conducted using Label Encoder from the sklearn.preprocessing module. Specifically, education was encoded as 0 for Graduate and 1 for Not Graduate; self_employed as 0 for No and 1 for Yes, and loan_status, the target variable, as 0 for Not Approved and 1 for Approved. These conversions were necessary to ensure compatibility with machine learning models, which require numeric inputs, particularly for digital lending applications. The features were separated from the target variable using X=df.drop (["loan_status"], axis=1) and y=df ["loan_status]. This setup provided a comprehensive basis for examining the factors influencing loan approval decisions using historical loan records to train multiple ensemble machine learning models. These models were intended to improve overall accuracy and robustness by combining the predictive strengths of multiple classifiers.

The processed dataset was then split into training and testing subsets using the train_test_split function from sklearn.model_selection, with 80% of the data used for training and 20% reserved for testing. This ensured that the model was trained on a sufficiently large portion of the data while retaining a representative sample for performance evaluation. With the dataset cleansed, structured, and statistically explored, the foundation was laid for the implementation of a robust machine learning framework aimed at enhancing predictive accuracy in loan approval classification. Model development was conducted using four ensemble-based machine learning algorithms: Gradient Boosting Model, AdaBoost, Efficient Gradient Boosting Model, and Extra Trees Classifier. These were selected for their proven performance in classification tasks involving structured, tabular data. The Gradient Boosting Model Classifier, implemented from the Gradient Boosting Model library, was instantiated with default settings (iterations=1000, learning rate=0.1, depth=6, verbose=False). It was trained using. fit (x_train, y_train) and evaluated with .predict (X_test). Although the Gradient Boosting Model automatically handles categorical data encoding, this feature was not utilized as the data had already been label-encoded. The AdaBoost Classifier (Adaptive Boosting, which improves weak learners) was implemented using sklearn-ensemble. AdaBoost Classifier was configured with n_estimators=100 and learning_rate=1.0, using decision stumps as the default base estimator. It was trained and evaluated in a similar fashion, contributing robustness through iterative weighting of misclassified instances. The Efficient Gradient Boosting, implemented through the Efficient Gradient Boosting Model library (LGBMClassifier), was configured with n_estimators=100, learning_rate=0.1, and max_depth=-1 (unrestricted tree depth). This model, known for its speed and efficiency, particularly excels on large datasets with high-dimensional features using optimized gradient boosting decision trees.

Finally, the ExtraTrees Classifier from sklearn.ensemble was used with n_estimators=100 and criterion="gini" as the splitting strategy. Unlike Random Forest, Extra Trees introduces further randomness by selecting cut points at random, which helps to reduce model variance and improve generalization. The ensemble was conducted using scikit-learn's Stacking Classifier, which enhances generalization by aggregating predictions from the base learners. Each model was evaluated using standard classification metrics, including accuracy, precision, F1-score, error analysis, and the confusion matrix. These metrics were computed using functions from the sklearn.metrics module to ensure standardized performance comparison across all models.

The best-performing model (based on accuracy and F1-score) was saved for deployment using the Python library. dump(model, "best_model.pkl"), ensuring that the trained model could be reused without the need for retraining. To simulate a real-world application, a sample input array containing 11 features was created using NumPy and passed to the model .predict () function. For instance, the input vector [[0, 1, 1,4100000, 12200000, 8, 417, 2700000, 2200000, 8800000, 3300000]] returned a prediction of 1, indicating loan approval. All experimentation was conducted in a Python 3.10 environment using Google Notebook on Kaggle. Model development and evaluation were carried out using the scikit-learn (v1.3), Gradient Boosting Model, and Efficient Gradient Boosting Model libraries. All hyperparameters were documented explicitly, and defaults were clearly stated where applicable. The encoding procedures followed the approach described by Pedregosa and were implemented in scikit-learn⁴⁶. This comprehensive and transparent methodology ensures that the experimental protocol is fully reproducible and adheres to rigorous academic standards in machine learning research.

The structure of the suggested methodology, encompassing the phase of data preparation feature section, model training and evaluation, are shown in Figure 1.

This research introduces a stacking ensemble learning framework that brings together the capabilities of four powerful classifiers: Gradient Boosting Model, AdaBoost, Efficient Gradient Boosting Model, and Extra Trees to predict loan approval decisions based on historical financial records. By combining both boosting and bagging strategies within stacked model architecture⁴⁶. The approach effectively overcomes the individual shortcomings of these models, such as bias and variance, lending to enhanced prediction accuracy and model generalization. Each base learner contributes unique strengths Gradient Boosting Model is efficient with categorical variable, it is designed for handling high cordiality categorical features and internally performs target encoding using ordered boosting⁴⁷. This avoids over fitting by ensuring that only past data is used in computing statistics. In the formula

   ,

each h_t (x) represents a decision tree trained on residuals of the previous model, and n_t denotes the step-specific learning contribution. AdaBoost or Adaptive Boosting, adjusts the weight of each instance during training and focuses on previously misclassified data points⁴⁸. In the formula

α_t reflects the performance of the t-th weak learner h_t(x), placing more emphasis on previously misclassified samples. Efficient Gradient Boosting Model Incorporates gradient-based one-side sampling (GOSS) and exclusive feature bundling for faster performance. Efficient Gradient Boosting offers high speed and performance on large-scale data⁴⁹.

f_t(x_i) represents the new decision tree added to minimize the loss l(•) while Ω(f_t) is a regularization term . In contrast boosting algorithms, Extra Trees reduces variance by adding randomness in decision tree splits⁵⁰. It relies on bagging principles but injects extra randomness during node splitting into its prediction rule

Averages the outputs of M independently trained randomized trees. For each split, Extra trees selects random thresholds for features and chooses the best among them, thereby reducing variance and offering high diversity across trees, which improves generalization. These models are collectively integrated through a stacking classifier, which learns to optimally combine their outputs to decide whether a loan should be approved. The framework was evaluated with common classification metrics and tested with live input samples, demonstrating its practical relevance in digital lending environments⁵¹. These models are combined collectively using a stacking classifier, which learns to blend their outputs ideally to determine loan acceptance results. The model's performance was assessed using important classification measures such as accuracy, precision, recall, F1-score, and AUC-ROC, as well as a confusion matrix, to determine its capacity to reduce both Type I and Type II mistakes. To maintain class balance, a stratified 80:20 train-test split was utilized, with 5-fold cross-validation ensuring robustness and reducing sample variability. Furthermore, the model was evaluated on realistic loan applicant profiles that included information such as credit history, income, employment status, and loan amount, yielding binary judgments and probability ratings. This two-phase test demonstrates the model's efficacy, fairness, and practicality in real-time digital lending contexts. The novelty of this work lies in the hybrid ensemble design tailored for credit scoring, making it a robust, interpretable, and reproducible model for modern financial platforms⁵² .

Results

Feature correlation analysis

The feature correlation heatmap (Figure 2) gave useful information about the interrelationships between various attributes. Strong positive correlations were found between income, annual loan amount, and asset-related variables such as luxury assets value and bank asset value, demonstrating that an applicant's financial profile is import...

Discussion

The stacking ensemble model for loan approval prediction performs exceptionally well across various evaluation metrics, demonstrating great accuracy and reliability. The correlations heatmap revealed that financial indicators such as annual income, loan amount, and asset values are strongly interrelated, emphasizing their importance in loan evolution, whereas the CIBIL scores have a strong negative correlation with loan status, strengthening their role in creditworthiness assessment. The model's confusion matrix had a lo...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
Kaggle	https://www.kaggle.com/
Pandas	https://pandas.pydata.org/
Model library	IBM	https://www.ibm.com