Fraud Management in Digital Business: a Survey on Practices, Metrics, and Controls

Breno Mendes Moura; Gisela Consolmagno Pelegrini

DOI: 10.22167/2675-6528-202602721

Article derived from a Final Course Work (TCC), with content based on the student’s original work and adapted to the editorial format of the E&S Magazine with the support of the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege for textual synthesis and organization.

Abstract

Frauds in digital businesses represent a growing challenge, impacting organizations financially and operationally. The study analyzed how data-driven and Machine Learning approaches can support fraud control in digital businesses, considering the impacts of defining analytical parameters on the volume of alerts and associated operational costs, based on the integration between the perception of industry professionals and the analysis of a real database of financial transactions. The research combined a survey applied to 57 professionals in technology, data, risk, and operations with the analysis of a public credit card transaction database (Credit Card Fraud, Europe/2013), containing 284,807 records and 0.17% fraud. Two predictive models were developed and compared, Logistic Regression (PR-AUC: 0.6826) and Random Forest (PR-AUC: 0.7426), focusing on the PR-AUC metric due to data imbalance. The simulation of decision thresholds demonstrated that a threshold of 0.30, applied to the Random Forest model, configured the best balance between detection and operational load, resulting in a recall of 82.4%, 166 alerts generated, and an estimated cost of R$ 1,589 in undetected fraud. The survey results indicated that 63% of respondents considered Machine Learning the most effective approach and 57.9% prioritized the balance between risk and customer experience. It was concluded that the main challenge in fraud management with Machine Learning lies in managing the trade-off between risk mitigation and the operational cost associated with the volume of alerts generated by analytical decisions.

Keywords: analytics; anomaly detection; digital fraud; risk management; machine learning.

1. Introduction

Fraud in digital operations represents a growing challenge for organizations operating in online environments, pressuring profit margins, eroding consumer trust, and negatively impacting the customer experience. In sectors such as e-commerce, marketplaces, and on-demand services, the sophistication of fraudulent practices, which include undue chargebacks, promotion abuse, massive account creation, and account takeover, demands increasingly robust controls. The need for rapid decisions on releasing, holding, or declining transactions makes erroneous choices costly in terms of revenue, reputation, and operational costs.

The continuous expansion of digital businesses has increased the volume and complexity of transactions, exposing organizations to a greater risk of fraudulent practices. Digital fraud generates significant financial, operational, and reputational impacts, in addition to undermining user confidence in digital systems. Academic literature recognizes that the growing sophistication of fraud, driven by the increase in electronic transactions, limits the effectiveness of control methods based exclusively on fixed rules.

In this context, data analysis and Machine Learning techniques have been widely investigated as more flexible alternatives for fraud detection. Their ability to identify complex patterns and adapt to new fraudulent behaviors makes them promising (Bolton and Hand, 2002; Dal Pozzo et al., 2020). These approaches offer significant potential to enhance the detection and prevention of illicit activities in real-time.

However, despite the potential of advanced analytical approaches, recent studies indicate that their adoption does not eliminate the operational challenges inherent in fraud control. A critical point refers to the trade-off between risk mitigation and operational impacts, especially in scenarios with a strong imbalance between legitimate and fraudulent transactions. Overly strict control strategies can result in an increase in false positives, raising operational costs and generating friction in the customer experience, while more permissive approaches broaden the risk of undetected fraud (Carcillo et al., 2021). It is observed that part of the academic literature tends to focus on the predictive performance of models, dedicating less attention to the practical effects of analytical decisions in real digital business environments.

Given this gap, it becomes relevant to analyze digital fraud from an applied perspective, which considers not only detection capability but also the operational impacts of analytical choices. This study, motivated by the need for a framework applicable in real digital business contexts, such as that of a Brazilian startup, sought to demonstrate how an analytical arrangement can identify fraud patterns and guide control policies. To this end, metrics suitable for rare classes were prioritized, with an emphasis on Precision-Recall to evaluate practical utility in imbalanced scenarios (Saito and Rehmsmeier, 2015), and the managerial implications of defining thresholds, allocating manual review, and trade-offs between fraud losses and false positives were discussed.

Thus, the objective of this work is to analyze how data-based approaches and Machine Learning can support fraud control in digital businesses, considering the impacts of defining analytical parameters on the volume of alerts and associated operational costs, through the integration between the perception of area professionals and the analysis of a real database of financial transactions.

2. Material and Methods

This research was characterized as an applied study, with a quantitative approach and exploratory nature. Applied research, according to Gil (2008), aims to generate knowledge for the solution of practical problems, aligning with the objective of analyzing the use of analytical approaches in fraud control in digital businesses. The quantitative approach was employed due to the use of structured data, both from questionnaires and transactional bases, allowing for measurement and statistical analysis.

Regarding the objectives, the study was framed as exploratory and descriptive, seeking to deepen the understanding of digital fraud and describe the operational impacts of analytical decisions. The methodological strategy combined survey research and documentary research. This combination is suitable for integrating individual perceptions with secondary data, as recommended by Gil (2008).

The survey research aimed to capture the perception of professionals working in or having contact with digital businesses. The focus was on the relevance of digital fraud, the use of data and Machine Learning, and the operational impacts of fraud control. The data collection instrument was a structured questionnaire, developed based on the methodological guidelines of Gil (2008), with clear and objective questions and quantitative response scales.

The survey data collection was carried out through an electronic form, made available in an online environment, which allowed for greater reach and agility. 57 professionals from the areas of technology, data, risk, and operations participated. Participation was voluntary and conditioned on reading and explicitly agreeing to the Free and Informed Consent Term (TCLE), ensuring compliance with the ethical principles of the research.

Complementarily, documentary research was adopted, using a public secondary database of financial transactions. This database, named Credit Card Fraud (Europe/2013), is widely used in academic studies on fraud detection. It contains records of legitimate and fraudulent transactions, serving as the empirical object for data analysis and the development of predictive models.

The transactional database consisted of 284,807 records, with a fraud rate of 0.17%. Although the research was motivated by the context of a Brazilian startup, “Organization Alfa” in São Paulo/SP, the database used is public and does not contain sensitive data. The predictor variables from the database were anonymized via PCA to preserve the confidentiality of the original data.

The transactional data were subjected to an initial exploratory analysis. For the development of the models, the dataset was divided into three mutually exclusive sets: training (70%), validation (15%), and testing (15%). This stratified split ensured the maintenance of the fraud proportion in each partition. Normalization of the Amount and Time variables was applied exclusively to the training set, preventing data leakage.

In the development of Machine Learning models, two algorithms were compared: Logistic Regression and Random Forest. Logistic Regression was employed as a baseline model, due to its interpretability and common use in fraud detection. Random Forest, an ensemble model, was selected for its ability to capture non-linearities and interactions between variables.

Both models were configured with class weight adjustment (class_weight=’balanced’), a technique to mitigate bias caused by data imbalance, without the need for resampling. The main evaluation metric was the Area Under the Precision-Recall Curve (PR-AUC). This metric was chosen for being more informative than the ROC curve in contexts of strong class imbalance, as demonstrated by Saito and Rehmsmeier (2015).

The focus of the modeling was not only on maximizing predictive performance, but on evaluating the effects of decision threshold variation on the volume of flagged transactions and the associated operational cost. Data processing and analysis procedures were carried out using scripts developed in Python, with the aid of the scikit-learn, pandas, and numpy libraries. The complete source code is publicly available in a GitHub repository.

3. Results and Discussion

The results of this study were obtained from an integrated approach, which combined the perception of professionals in the digital business area with the empirical analysis of a transactional database and the application of Machine Learning models. This integration allowed for a multifaceted understanding of the challenges and solutions in fraud management, connecting market expectations with technical feasibility and the operational impacts of analytical decisions. The analysis sought not only to identify fraud patterns but also to evaluate the practical implications of defining analytical parameters on the volume of alerts and operational costs, in line with the central objective of the research.

Survey Results

The structured questionnaire, answered by 57 professionals from technology, data, risk, operations, and digital business areas, revealed a unanimous perception regarding the relevance of fraud in digital environments. Participants recognized the phenomenon as a factor with significant impact on both financial results and the operational efficiency of organizations. This finding underscores the cross-cutting nature of the digital fraud problem, which affects various areas and experience levels within companies, demanding robust and integrated approaches for its control.

Regarding the most effective approaches for fraud control, the survey data indicated a clear preference for data-driven solutions and Machine Learning. Approximately 63% of respondents (36 out of 57) pointed to Machine Learning as one of the most effective approaches, followed by statistical models, with 54% of indications. Expert human review was mentioned by 40% of participants, while manual rules were cited by only 23%. This panorama suggests that industry professionals perceive the limitations of traditional approaches in the face of the growing sophistication of digital fraud, seeking more dynamic and adaptive solutions.

A crucial finding from the survey was the explicit recognition of the trade-off between fraud mitigation and operational impacts. Most respondents agreed that stricter control strategies tend to increase the incidence of false positives, generating friction in the customer experience and increasing the operational burden on analysis teams. This perception reinforced the need to empirically investigate how analytical decisions influence the volume of alerts generated by anti-fraud systems, a central point for the practical applicability of the solutions.

When asked about companies’ priority in controlling digital fraud, 57.9% of respondents indicated the search for “balance between risk and customer experience”. Minimization of financial losses was a priority for 22.8%, and minimization of false positives for 19.3%. This result corroborates the view that fraud management cannot be dissociated from customer experience, making the calibration of decision thresholds a fundamental strategic element. The prioritization of balance reflects the complexity of operating in digital environments, where security must coexist with usability and consumer satisfaction.

Exploratory Analysis of the Transactional Database

The exploratory analysis of the transactional database, composed of 284,807 credit card transaction records, confirmed the strong class imbalance, characteristic of real-world fraud scenarios. It was found that only 0.17% of the total records (492 transactions) corresponded to fraud events, while 99.83% (284,315 transactions) were legitimate. This scenario of rare events in large data volumes is a known challenge in anomaly detection and reinforces the complexity of identifying fraud without generating an excessive number of false positives.

It was observed that the average value of fraudulent transactions, R$ 122.21, was higher than the average value of legitimate transactions, which was R$ 88.29. This finding suggests that fraud tends to occur in transactions of higher relative value, which is relevant for the cost-per-decision analysis. Each undetected fraud represents an estimated financial loss of R$ 122.21, a value used as a reference for calculating the cost per false negative in threshold simulations, highlighting the importance of effective detection for the organization’s financial health.

Development and Comparison of Predictive Models

Based on the transactional base, two predictive models were developed to generate probabilistic fraud risk scores: Logistic Regression, as a baseline model, and Random Forest. Both models were configured with class weight adjustment (`class_weight=’balanced’`) to mitigate the bias of data imbalance, without the need for resampling. The main metric adopted for evaluation was the Area Under the Precision-Recall Curve (PR-AUC), considered more informative than the ROC curve in contexts of strong class imbalance, as pointed out by Saito and Rehmsmeier (2015).

In the test set, the Random Forest model presented a PR-AUC of 0.7426, surpassing the Logistic Regression model, which obtained 0.6826. Although both models demonstrated a high ROC-AUC (above 0.96), the significant difference in PR-AUC confirms the greater effectiveness of Random Forest in identifying the minority class, i.e., fraudulent transactions. The choice of PR-AUC as the primary metric was crucial, as in scenarios with only 0.17% of fraud, ROC-AUC can be misleading, indicating good performance even for models that are not very useful in practice.

Analysis of Thresholds and Operational Impact

The simulation of different decision thresholds with the Random Forest model demonstrated the direct impact of defining these parameters on the volume of transactions flagged for analysis and, consequently, on the operational load. This analysis directly connected the perception of professionals, who pointed out the trade-off between fraud mitigation and operational impact, with the empirical evidence from the transactional database, providing a practical overview of the implications of analytical choices.

By adopting a threshold of 0.50, 87 transactions were flagged as suspicious, resulting in a recall of 77.0% and an estimated cost of R$ 2,078 in undetected fraud. Reducing the threshold to 0.30 increased the alert volume to 166, raising the recall to 82.4% and decreasing the estimated cost of undetected fraud to R$ 1,589. However, a threshold of 0.10, while achieving a recall of 91.9% and a cost of only R$ 733, would generate 1,226 alerts, of which 1,158 would be false positives, negatively impacting the customer experience and the operational load of the analysis team.

This behavior empirically confirms the trade-off perceived by professionals in the survey: more rigorous strategies, with lower thresholds, reduce financial losses from undetected fraud, but increase the volume of false positives, generating operational friction and negatively impacting the customer experience. The threshold of 0.30 emerged as the most suitable balance point for the analyzed context, combining a reasonable recall of 82.4%, an estimated manageable cost of R$ 1,589, and a volume of 166 alerts compatible with human review operations, optimizing efficiency and customer experience.

Predictive Variables and Fraud Signals

The analysis of the predictive importance of the Random Forest model variables allowed the identification of the main signals associated with fraudulent behaviors. The ten variables with the highest discriminant power were identified, measured by the average impurity reduction criterion. Variable V14 accounted for 21.1% of the model’s total predictive importance, standing out as the main discriminant between legitimate and fraudulent transactions. Together, the first four variables (V14, V10, V17, and V4) accounted for 56.1% of the model’s total predictive capacity.

Although the original meaning of these variables has been anonymized for reasons of public database confidentiality, in real-world environments they would correspond to behavioral attributes, such as time-of-day patterns, transaction frequency, geographic dispersion, and deviation from the customer’s historical profile. This finding converges with the literature on fraud detection, which emphasizes the relevance of behavioral signals, rather than just static transaction attributes, as the best predictors of real-time fraud (Bolton and Hand, 2002). From a managerial perspective, this result guides organizations to prioritize the monitoring of a focused set of behavioral variables, reducing the operational complexity of anti-fraud systems without significant loss of detection capability.

In summary, the results demonstrated that data-based and Machine Learning approaches, especially the Random Forest model, are effective in detecting fraud in digital businesses, outperforming traditional methods and Logistic Regression in scenarios with strong class imbalance. The research confirmed that the main challenge lies in managing the trade-off between risk mitigation and the operational costs associated with the volume of alerts generated, highlighting that the calibration of decision thresholds, such as the 0.30 threshold, is fundamental to balancing recall, cost, and customer experience, directly addressing the study’s objective.

4. Conclusion

This study aimed to analyze how data-driven and Machine Learning approaches can support fraud control in digital businesses, considering the impacts of defining analytical parameters on alert volume and associated operational costs, through the integration of insights from industry professionals and the analysis of a real-world financial transaction database. It was found that industry professionals recognize Machine Learning as an effective approach, prioritizing the balance between risk and customer experience in fraud management. The empirical analysis of financial transactions revealed a strong class imbalance and the superiority of the Random Forest model (PR-AUC: 0.7426) over Logistic Regression (PR-AUC: 0.6826) in fraud detection. The simulation of decision thresholds demonstrated that a threshold of 0.30 configured the most suitable equilibrium point, resulting in a recall of 82.4%, 166 alerts generated, and an estimated cost of R$ 1,589 in undetected fraud, which optimizes operational efficiency and customer experience. The main contribution of this work lies in highlighting that the central challenge in fraud management with Machine Learning is not limited to detection capability, but to the administration of the trade-off between risk mitigation and the operational cost associated with the volume of alerts generated by analytical decisions, providing an applicable analytical framework to guide control policies.

However, the study presented some limitations, such as the database used, which may not fully reflect the fraud patterns of the current Brazilian market, and the anonymization of predictor variables via PCA, which restricted the direct managerial interpretability of the identified signals. Additionally, the survey sample, composed of 57 respondents, has an exploratory nature and is not statistically representative. For future studies, it is suggested to expand the respondent sample, explore additional analytical models, such as Gradient Boosting and neural networks, and evaluate explainability techniques, such as SHAP values, to increase the transparency of decisions in regulated environments, contributing to a more comprehensive and applied understanding of the digital fraud problem.

Bibliographic References

Bolton, R.J.; Hand, D.J. (2002). Statistical fraud detection: a review. Statistical Science, 17(3): 235-255.

Carcillo, F.; Bontempi, G.; Snoeck, M.; Bagheri, S. (2021). Scarff: a scalable framework for streaming credit card fraud detection with sliding windows. IEEE Transactions on Neural Networks and Learning Systems, 32(10): 1–15.

Dal Pozzo, A.; Boracchi, G.; Biganzoli, E. (2020). Adaptive machine learning for fraud detection. Journal of Financial Crime, 27(4): 1–15.

Gil, A.C. (2008). Métodos e técnicas de pesquisa social. 6. ed. Atlas, São Paulo, SP, Brasil.

Saito, T.; Rehmsmeier, M. (2015). The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets. PLOS ONE, 10(3): e0118432.

Article originating from the Final Course Work of Specialization in Digital Business from the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform