Executive Summary

January 16, 2026

Comparative analysis of Machine Learning techniques for default prediction

Author: Luciane Berger da Silva — Advisor: Daniel Alvarez Firmino

Summary prepared by the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege focused on synthesis and writing.

This study analyzes and compares the performance of machine learning algorithms in credit default prediction. The research compares linear models (logistic regression) and tree-based models (decision tree, random forest), investigating the effect of preprocessing strategies, such as missing value imputation and class balancing. The objective is to identify the most suitable models, considering performance metrics and practical applicability for financial institutions, especially in imbalanced data scenarios, characteristic of credit risk.

Credit granting is fundamental for the economy, but it involves the risk of default, the non-fulfillment of contractual obligations by the borrower (Baesens et al., 2016). Effective management of this risk is crucial for the sustainability of financial institutions. Predictive analysis tools are indispensable for identifying risk patterns based on historical data, allowing for preventive measures such as adjusting limits or refusing high-risk operations (Lessmann et al., 2015; Pinto et al., 2024).

The literature on credit risk transitions from traditional statistical models to machine learning algorithms. Tree-based methods, such as random forest, demonstrate better performance in capturing non-linear interactions (Aniceto et al., 2020). However, logistic regression maintains its prominence in the financial sector due to its interpretability and regulatory acceptance (Bücker et al., 2020). The challenge is to reconcile the predictive power of advanced models with the need for transparency, motivating research into hybrid approaches (Dumitrescu et al., 2022).

Despite the advances, gaps persist in the application of these techniques. Many studies do not explore the impact of treating imbalanced data, a common characteristic in default databases, where the performing class is the majority. Imbalance can induce algorithms to a bias in favor of the majority class, resulting in models with high overall accuracy but low capacity to identify default cases. This work contributes to the literature by offering a systematic comparative analysis that evaluates different algorithms and investigates how preprocessing strategies, especially class balancing ones, influence predictive performance. By analyzing the results under multiple metrics, the study aims to provide practical insights for financial institutions on how to select and configure models that align with their objectives, whether maximizing performance or minimizing losses from false negatives.

The methodology is an applied quantitative research, which uses supervised machine learning techniques to compare predictive approaches in identifying customers at risk of default. The public database “Give me Some Credit” (Freshcorn, 2011), with 150,000 anonymized records and behavioral variables, was used. Although the North American origin of the database limits generalization to the Brazilian context, its volume and use in academic studies make it a robust environment for validating modeling techniques in imbalanced data scenarios.

Data preprocessing began with the analysis of missing values, present in the income (19.82%) and number of dependents (2.62%) variables. A chi-squared test indicated that the absence was not random. Four treatment strategies were evaluated: record exclusion; median imputation; K-Nearest Neighbors (KNN) imputation; and Multiple Imputation by Chained Equations (MICE). Each approach generated a version of the database for comparative analysis of the impact of missing data treatment on model performance.

The imbalance of the target variable, with only 7% of defaulters, was another challenge. To mitigate the risk of bias in favor of the majority class, three balancing strategies were tested: none (baseline); undersampling, which reduces the majority class; and synthetic minority oversampling technique (SMOTE), which generates synthetic observations of the minority class (He and Garcia, 2009). The comparison evaluated the impact of each on metrics sensitive to imbalance, such as sensitivity and F1-score.

The database was divided into 70% for training and 30% for testing, in a stratified manner. Three classification algorithms were implemented (Kotsiantis, 2007): logistic regression (interpretability), decision tree (non-linear patterns) (Grus, 2021), and random forest (robustness and generalization) (Breiman, 2001). Hyperparameters were optimized with RandomizedSearchCV, GridSearchCV, and 5-fold cross-validation. Performance was evaluated on the test set with a comprehensive set of metrics: Area Under the ROC Curve (AUC-ROC), Area Under the Precision-Recall Curve (AUC-PR), Gini index, accuracy, precision, recall, F1-score, Matthews Correlation Coefficient (MCC), and Brier score, ensuring a multidimensional analysis.

The logistic regression results showed that L2 regularization (Ridge) was the best configuration, which is expected in scenarios with multicollinearity (Hastie et al., 2009). Maintaining all variables was justified by the pursuit of global predictive performance (Fávero and Belfiore, 2017). The strategy of excluding missing records, without balancing or with undersampling, produced the best overall results, with an AUC-ROC of 0.803 and a sensitivity of 0.627. Imputation techniques (median, KNN, MICE) had a marginal impact. The use of SMOTE increased sensitivity to 0.670 but harmed accuracy and precision. However, the combination of MICE with SMOTE achieved the best Brier score (0.168), indicating superior calibration. This highlights a trade-off: data exclusion favors overall performance, while MICE and SMOTE may be preferable when reducing false negatives and calibration are priorities.

The decision tree model, without balancing, exhibited high accuracy (0.933) but null sensitivity, failing to identify defaulters, a documented phenomenon (Breiman et al., 1984). The use of SMOTE promoted gains, raising sensitivity to approximately 0.30. The combination of median imputation with SMOTE presented the best balance, with an AUC-ROC of 0.836 and a Brier score of 0.057. Undersampling, on the other hand, maximized sensitivity (close to 0.77), but with a significant drop in accuracy and a worsening in calibration, reinforcing the trade-off between detecting defaulters and maintaining overall robustness (Thomas et al., 2017).

The random forest was the most robust algorithm, with consistent AUC-ROC around 0.86. For global performance, the combination of median imputation without balancing stood out, with AUC-ROC of 0.861, AUC-PR of 0.384, and Brier score of 0.126. To minimize false negatives, median imputation with undersampling was the most effective, achieving the study’s highest sensitivity (0.795) and the best AUC-ROC (0.864), albeit with losses in accuracy. In contrast, strategies with SMOTE maximized accuracy but reduced sensitivity.

The final comparison, using AUC-PR as the main criterion due to the imbalance, confirmed the superiority of the random forest for overall performance. This model presented the best balance, with the highest AUC-PR (0.384), sensitivity of 0.418, and the lowest Brier score (0.126), indicating excellent discrimination and calibration. Logistic regression outperformed the decision tree in sensitivity, but with losses in other metrics, corroborating studies that point to ensembles as more stable (Isidoros and Arcozzi, 2024). In the analysis focused on minimizing false negatives (sensitivity), tree-based models were superior. The random forest led with a sensitivity of 0.795, followed by the decision tree with 0.779. Both incurred costs in precision and calibration, exemplifying the trade-off in risk management: reducing false negatives mitigates capital losses, while an increase in false positives can lead to the loss of business opportunities.

The relevance of variables and the performance of models must be contextualized. The significance of traditional variables may decrease in economic shock scenarios (Gambacorta et al., 2024), and performance can be improved with alternative data, such as psychometric information, especially for populations with limited credit history (Djeundje et al., 2021). The choice of a model should not be based solely on statistical metrics. Models with high accuracy do not always minimize the regulatory and capital costs associated with errors (Xia et al., 2022). The final decision should integrate statistical analysis with the institution’s strategic objectives, considering the financial costs of each type of error. This study provides a framework for this analysis, demonstrating how different combinations of algorithms and preprocessing align with different risk appetites.

The work demonstrated that there is no universally superior model for default prediction, but rather trade-offs between performance, interpretability, and operational impact. Logistic regression proved to be a competitive and transparent benchmark. Decision trees, sensitive to imbalance, improved with resampling techniques. Random forest was the most balanced and robust model, with the best overall performance in discrimination (AUC-ROC ≈ 0.86) and calibration. The analysis highlighted the critical role of preprocessing: undersampling was more effective in increasing sensitivity, while SMOTE, in some scenarios, favored accuracy, emphasizing the need to align the methodology with strategic objectives.

The study’s limitations include the use of a single North American database and the non-exploration of algorithms such as gradient boosting or neural networks. Future research can expand the analysis to Brazilian databases, incorporate other models, and integrate error cost metrics to quantify the financial impact. It is concluded that the objective was achieved: it was demonstrated that the choice of machine learning algorithm and data preprocessing strategies for default prediction fundamentally depends on the financial institution’s strategic objectives, highlighting a clear trade-off between overall predictive performance and the minimization of losses due to false negatives.

References:
Aniceto, G. F.; Barboza, F. L. C.; Kimura, H. 2020. Credit risk analysis using machine learning classifiers. Brazilian Review of Finance 18(4): 1–28.
Baesens, B.; Roesch, D.; Scheule, H. 2016. Credit risk analytics: measurement techniques, applications and examples in SAS. John Wiley & Sons, Hoboken, NJ, USA.
Breiman, L. 1984. Classification and regression trees. Chapman & Hall/CRC, Boca Raton, FL, USA.
Breiman, L. 2001. Random forests. Machine Learning 45(1): 5–32.
Bücker, M.; Szepannek, G.; Gosiewska, A.; Biecek, P. 2020. Transparency, auditability and explainability of machine learning models in credit scoring. Journal of the Operational Research Society 71(8): 1281–1290.
Carvalho, J. R. 2015. Credit risk analysis: fundamentals, methodologies and applications. Atlas, São Paulo, SP, Brazil.
Djeundje, V . B.; Crook, J.; Calabrese, R.; Hamid, M. 2021. Enhancing credit scoring with alternative data. Expert Systems with Applications 167: 113766.
Dumitrescu, E.; Hué, S.; Hurlin, C.; Tokpavi, S. 2022. Machine learning for credit scoring: improving logistic regression with non-linear decision-tree effects. European Journal of Operational Research 297(3): 1178–1192.
Fávero, L. P.; Belfiore, P. P. 2017. Data analysis manual: statistics and multivariate modeling with Excel®, SPSS® and Stata®. Elsevier, Rio de Janeiro, RJ, Brazil.
Fávero, L. P.; Belfiore, P. P. 2024. Data analysis: exploratory and confirmatory multivariate techniques. Elsevier, Rio de Janeiro, RJ, Brazil.
Freshcorn, B. 2011. Give Me Some Credit: 2011 Competition Data. Available at: https://www.kaggle.com/datasets/brycecf/give-me-some-credit-dataset.
Gambacorta, L.; Huang, Y.; Qiu, H.; Wang, J. 2024. How do machine learning and non-traditional data affect credit scoring? New evidence from a Chinese fintech firm. Journal of Financial Stability 73: 101284.
Grus, J. 2021. Data science from scratch: fundamental concepts with Python. 2nd ed. Alta Books, Rio de Janeiro, RJ, Brazil.
Hastie, T.; Tibshirani, R.; Friedman, J. 2009. The elements of statistical learning: data mining, inference, and prediction. 2nd ed. Springer, New York, NY, USA.
He, H.; Garcia, E. A. 2009. Learning from imbalanced data. IEEE Transactions on Knowledge and Data Engineering 21(9): 1263–1284.
Hosmer, D. W.; Lemeshow, S.; Sturdivant, R. X. 2013. Applied logistic regression. 3rd ed. John Wiley & Sons, Hoboken, NJ, USA.
ISIDOROS, I.; ARCOZZI, N. 2024. Improved convergence rates for some kernel random forest algorithms. Mathematics in Engineering 6(2): 1-22.
Kotsiantis, S. B. 2007. Supervised machine learning: a review of classification techniques. Informatica 31(3): 249–268.
Lessmann, S.; Baesens, B.; Seow, H. V.; Thomas, L. C. 2015. Benchmarking state-of-the-art classification algorithms for credit scoring: an update of research. European Journal of Operational Research 247(1): 124–136.
Pinto, R. S.; Ywata, A.; Tessmann, R. H.; Lima, F. 2024. Are machine learning models more effective than logistic regressions in predicting bank credit risk? An assessment of the Brazilian financial markets. International Journal of Monetary Economics and Finance 17(1): 1–22.
Thomas, L. C.; Crook, J. N.; Edelman, D. B. 2017. Credit scoring and its applications. 2nd ed. SIAM, Philadelphia, PA, USA.
Xia, Y.; Zhang, C.; Li, Y.; Chen, W. 2022. A comparative study of credit scoring models. Knowledge-Based Systems 235: 107629.
Zou, H.; Hastie, T. 2005. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 67(2): 301–320.


Executive summary from the Final Course Work of the Specialization in Data Science and Analytics of the MBA USP/Esalq

Learn more about the course; click here:

Who edited this article

Most recent

You may also like

November 17, 2026

Detection and Diagnosis of Diseases in Corn and Rice in Mozambique with Deep Learning

Mozambican agriculture depends on corn and rice, fundamental crops for food security, but their productivity is strongly affected by phytosanitary diseases, whose early and accurate identification remains a challenge in resource-limited contexts. The study aimed to develop and compare Deep Learning-based models for the automatic detection and diagnosis of diseases in corn and rice leaves in Mozambique. Three approaches were used: a classic Convolutional Neural Network (CNN) developed from scratch, and two transfer learning models, ResNet50 and MobileNetV2. The dataset consisted of images from public databases and collected in the field, which underwent cleaning, balancing, and artificial augmentation steps. The models were trained and evaluated with standard performance metrics, including accuracy, precision, recall, F1-score, Log Loss, and Cohen’s Kappa. The classic CNN presented the best overall performance, with 94% precision, recall, and F1-score, and 93% accuracy and Cohen’s Kappa. MobileNetV2 achieved 91% F1-score and 90% accuracy, standing out for its lower Log Loss (0.30) and training time (6h32min59s). ResNet50 reached 80% F1-score and 79% accuracy. It was concluded that Deep Learning techniques have high potential to support phytopathological diagnosis in real agricultural contexts, contributing to faster and more accurate decisions in the management of corn and rice diseases.

Keywords: Deep learning; Rice; Plant diseases; Corn; Computer vision.

November 17, 2026

Artificial intelligence in accounting operational routines and financial efficiency

The incorporation of Artificial Intelligence (AI) and automation tools has reconfigured the routines of Brazilian accounting offices, impacting productivity, error mitigation, and financial efficiency. These impacts were analyzed in a small accounting firm, specialized in serving service providers in the medical field in São Caetano do Sul, São Paulo. The research adopted a qualitative approach, with a single case study design and incorporated units of analysis. Data were collected through a structured questionnaire, applied via Google Forms between January and February 2026, to five professionals in the operational structure of the firm in a census survey. The treatment followed content analysis with a priori and emergent categories. All respondents declared the use of automated tools or tools with AI resources. The perception of reduced execution time, decreased operational errors, rework, and better use of the team’s time converged in 80% of affirmative responses and 20% partial responses. The most benefited activities were bank reconciliation, import of fiscal documents, and tax assessment, with visible gains on typing errors, duplicate entries, and incorrect classification. Human intervention remained indispensable in tax planning, scenario analysis, and verification of ancillary obligations. The overall evaluation of AI’s contribution was predominantly favorable, with moderation linked to the complexity of fiscal regulations.

Keywords: Content analysis; Process automation; Accounting 4.0; Small business; Digital transformation.

November 17, 2026

Factors influencing schedule management in sewage treatment plant “retrofit” projects

Projects involving “retrofit” in Sewage Treatment Plants (STP) are characterized by high complexity, operational restrictions, and multiple technical interfaces. The study analyzed the perception of the professionals involved regarding the management and adherence to deadlines in “retrofit” projects for sewage treatment plants, considering technical, operational, and organizational factors. The research was characterized as an applied study, with a quali-quantitative approach, designed as a field survey. Data analysis was performed using descriptive statistics and qualitative categorization, employing SWOT, risk matrix (Probability x Impact), and 5W2H action plan tools. The results indicated a high perception of clarity (77.8%) and realism of deadlines (100% agreement), evidencing consistent initial planning. However, 77.7% of respondents pointed out that rework and scope changes directly impacted deadline adherence. The integration between engineering, manufacturing, and Production Planning and Control (PPC) showed an intermediate evaluation (77.7% partial/total agreement), indicating fragility in interfunctional coordination. The risk matrix highlighted as critical: scope changes, operational limitations of the STP, and technical complexity, all with a high level. As a central improvement, the standardization of scope change control, analysis, and approval was proposed, aiming to reduce rework and increase schedule predictability.

Keywords: Performance; Integration; Planning; Rework; Risk.

November 17, 2026

B2C Acquisition in diagnostic medicine: strategic partnerships with specialized medical offices

The diagnostic medicine sector in Brazil operates with high dependence on health plan providers, which limits the financial autonomy of service providers. This study proposed and analyzed a direct patient acquisition model (B2C) for a highly complex diagnostic medicine institution, structured through partnerships with Oncology and Cardiology practices and operationalized by a personalized customer service called concierge. The investigation adopted a mixed-methods approach in three phases: a field survey with a structured questionnaire applied to 87 patients, mapping of the operational flow using “Service Blueprint”, and analysis of the conversion rate from 312 medical referrals registered in the pilot project. The results indicated that trust in medical referral achieved 92.0% agreement, establishing itself as the main factor for adherence. The convenience of performing the exam on the same day reached 78.2%, and the willingness to pay directly was 55.2%, with polarization associated with the predominance of health plan users. The “Service Blueprint” identified three critical points in the transitions between stages of the patient journey. The overall conversion rate was 51.0%, with superior performance in Oncology (58.2%) compared to Cardiology (45.0%), and the greatest loss was concentrated in the acceptance and scheduling stage (28.3%). It was concluded that the model demonstrated possible operational viability and indications of patient acceptance, conditioned by price transparency and a reduction in the interval between consultation and approach by the concierge.

Keywords: Direct payment; Patient journey; Supplementary health; Concierge service; Conversion rate.

November 17, 2026

The role of the school manager in promoting an inclusive culture: leadership and educational policies

The research analyzed the school manager’s performance in promoting an inclusive culture, considering the need to transform legal guidelines and pedagogical foundations into effective institutional practices. It was assumed that school inclusion requires planning, organization, and strategic leadership in the school’s daily operations. A qualitative approach of an applied nature was adopted, developed in three complementary stages: documentary analysis of Brazilian legislation on inclusive education, bibliographic review on school leadership and management, and data systematization using management tools such as SWOT analysis and the PDCA cycle. The results indicated that the school management’s performance directly influenced the consolidation of inclusive environments, especially in the organization of processes, continuous team training, and strengthening of a collaborative institutional culture respectful of diversity. Structural, pedagogical, and relational challenges that demanded planned interventions were also identified. Finally, a Practical Guide for School Managers was developed, with guidelines, protocols, and operational orientations for decision-making and the organization of inclusive processes. It was concluded that the construction of an inclusive culture depended on the management’s ability to articulate public policies, pedagogical practices, and institutional processes, consolidating inclusion as a permanent commitment in the school context.

Keywords: educational accessibility; educational equity; participatory management; strategic planning; institutional processes.

November 17, 2026

An AI artifact based on Design Thinking to support the elicitation of software requirements from user needs

The development of software oriented to user needs frequently faces challenges in requirements elicitation, stemming from ambiguities in natural language and communication failures. Given this, an intelligent artifact integrating Design Thinking and Artificial Intelligence was proposed and validated, focusing on the Empathy and Definition stages, to support the collection, interpretation, and structuring of user needs. The research, of an applied, qualitative, and exploratory nature, conducted a Proof of Concept in a controlled environment, employing hypothetical scenarios. The artifact operated through guided natural language interviews, organizing information into elements such as pain points, context, and impact, and generating traceable problem syntheses. The results indicated that the system organized unstructured reports, reduced ambiguities, and preserved the meaning of the original information in most cases. The ability to reorganize logically in complex scenarios was also observed, although with limitations in separating multiple problems and dependence on instructions. It was concluded that the artifact has potential as a support for requirements elicitation, improving the clarity and structuring of information, but requires supervision and contextual control.

Keywords: Design Thinking; Requirements Elicitation; Requirements Engineering; Artificial Intelligence; Natural Language Processing.

November 17, 2026

The Extinction of the Monophasic Tax System in the Pharmaceutical Industry in Light of the Tax Reform

The consumption Tax Reform, established by Constitutional Amendment No. 132/2023, introduced structural changes to the Brazilian tax system, with the replacement of the Social Integration Program (PIS) and the Social Security Financing Contribution (COFINS) by the Contribution on Goods and Services (CBS), in a dual Value Added Tax (VAT) model. The objective was to analyze the impacts of the extinction of the single-stage system on the pharmaceutical industry, in light of the new regulatory design and its economic and legal effects. A qualitative, exploratory research was conducted through a case study and documentary analysis of legislation and specialized doctrine. It was examined whether the end of the single-stage system would imply a relevant increase in the tax burden for the sector. The results indicated that, although the formal overcoming of the single-stage system has occurred, the material exemption of medicines has been preserved, mainly due to the application of reduced rates of 60% and, in certain cases, a zero rate of the Tax on Goods and Services (IBS) and CBS. The change in model may cause occasional variations in revenue collection, without characterizing a structural increase in the tax burden, tending towards neutrality over time. It was preliminarily concluded that the Tax Reform reorganized the taxation technique in the pharmaceutical industry, replacing a concentrated regime with more transparent exemption mechanisms. This approach proved compatible with the principles of neutrality, full non-cumulativeness, and fiscality, maintaining tax protection for essential medicines and aligning fiscal policy with the fundamental right to health.

Keywords: CBS; COFINS; IBS; Dual VAT; PIS.