Digital Compliance and Algorithmic Biases: Case Study on Ethical Governance of Conversational AI.

Marina Saraiva Pezolito Sanchez; Carlos Alberto Santos Silva

DOI: 10.22167/2675-6528-202603258

Article derived from a Course Conclusion Work (TCC), with content based on the student’s original work and adapted to the editorial format of the E&S Magazine with the support of the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege for textual synthesis and organization.

Summary

The growing adoption of conversational Artificial Intelligence (AI) tools in financial services has increased efficiency gains, but introduced ethical risks related to algorithmic biases and the opacity of automated decision-making processes. This study investigated, through a single case study, how digital compliance mechanisms can identify, prevent, and mitigate algorithmic biases in conversational AI tools applied to the digital acquisition of financial products. A qualitative and applied approach was adopted, based on documentary analysis of an AI solution used by a financial institution with national operations. The tool was compared with international ethical governance frameworks, including the principles of Al4People, the OECD, UNESCO, the Council of Europe, the AI Act, and the LGPD. The results indicated that, although the tool presented partial traceability and human supervision, significant gaps persisted in the formalization of compliance controls aimed at mitigating algorithmic risks. These gaps included the absence of fairness metrics, algorithmic audit policies, and ethical impact assessments. It was concluded that regulatory compliance and technical controls, in isolation, are insufficient for responsible governance, requiring a digital compliance model with continuous algorithmic risk assessment, formalized accountability, and institutionalized ethical governance to deal with the complexity of these systems.

Keywords: Corporate accountability; Algorithmic auditing; Fairness in automated systems; LGPD; Personal data protection.

1. Introduction

The accelerated expansion of Artificial Intelligence (AI) has caused significant transformations in decision-making processes and business models adopted by organizations across various sectors, including financial services, digital marketing, recruitment, and customer service. AI-based tools, especially those of a conversational nature, have come to play a relevant role in automating acquisition and support journeys, promising greater operational efficiency, cost reduction, and improved user experience (Organisation for Economic Co-operation and Development, 2019a).

However, the application of machine learning models in financial services requires supervision proportional to the degree of autonomy and the impact of automated decisions, particularly when these decisions directly affect individuals’ access to products and services (Financial Stability Board, 2017; Organisation for Economic Co-operation and Development, 2019a).

Specialized doctrine discusses ethical and social risks associated with the adoption of these technologies, especially regarding algorithmic biases and the opacity of decision-making processes. Pasquale (2015) described the “black box society” phenomenon, in which automated processes affecting credit, employment, and information operate beyond the reach of public scrutiny and challenge by affected individuals, fostering a strong information asymmetry between users and technology developers.

O’Neil (2016) showed that algorithmic systems, by learning from historical data, can reproduce or intensify pre-existing inequalities, operating as what she termed “weapons of math destruction”: opaque, scalable, and harmful models, whose discriminatory effects are amplified by the very feedback logic of the data. Eubanks (2017) demonstrated how technological tools applied to public and financial services have deepened the exclusion of vulnerable populations by incorporating structural biases into their decision-making mechanisms.

In this scenario, traditional compliance programs, historically focused on regulatory conformity and financial risk mitigation, have proven insufficient to deal with the new challenges imposed by AI. Thus, the need arises to evolve towards digital compliance models, capable of incorporating principles of ethics, transparency, explainability, and algorithmic accountability.

On the international stage, multiple guidelines have reinforced the importance of these elements. Floridi et al. (2018), within the scope of the Al4People project, proposed a unified ethical framework structured around five principles: beneficence, non-maleficence, autonomy, justice, and explainability. This last principle was conceived as a condition for the possibility of the others, given that without intelligibility about how the system operates and without due accountability, it would not be feasible to ensure that AI is effectively beneficial, just, and respectful of human autonomy.

The OECD Principles on Artificial Intelligence (Organisation for Economic Co-operation and Development, 2019b) established pillars for trustworthy AI, including human-centered values and fairness, transparency and explainability, robustness and safety, and accountability. The UNESCO Recommendation on the Ethics of AI (United Nations Educational, Scientific and Cultural Organization, 2021) added the need for ethical impact assessments and risk-proportionate governance mechanisms. However, Kleinberg et al. (2016) formally demonstrated that three fairness criteria for risk scoring cannot be simultaneously satisfied, showing that bias mitigation involves normative choices that need to be explicit and justified within the scope of governance.

The Council of Europe’s study on algorithms and human rights (Council of Europe, 2018) articulated the dimensions of fair trial, privacy, prohibition of discrimination, and effective remedy, warning that automated decision-making processes can operate without sufficient transparency. The European Union’s Artificial Intelligence Act (European Union, 2024) consolidated a risk-based regulatory approach, foreseeing reinforced obligations for systems classified as high-risk. In the Brazilian context, the General Law on Personal Data Protection (Brazil, 2018) represented a relevant advancement by establishing principles of transparency and non-discrimination and rights related to automated decisions, although the legal system did not yet have general specific regulation on AI systems within the scope adopted by this research.

Given this context, regulatory compliance and technical controls, in isolation, are not sufficient for responsible conversational AI governance, necessitating the institutionalization of continuous algorithmic risk assessment, formalized accountability, and ethical governance. Thus, the present study aims to analyze how digital compliance mechanisms can identify, prevent, and mitigate algorithmic biases in a conversational AI tool applied to the digital acquisition of financial products, seeking to propose ethical and responsible governance controls applicable to the corporate context.

2. Material and Methods

The research was characterized as applied, with a qualitative approach, and adopted a single case study as the investigation method. This choice was based on the need to examine, in depth and in its real context, a contemporary governance phenomenon: the use of conversational Artificial Intelligence (AI) in financial product acquisition flows. The study did not seek statistical generalization, but rather an analytical understanding of the governance practices, controls, and gaps observed in the selected case.

The unit of analysis consisted of a conversational AI tool, developed by the company Blue Ventures (fictitious name), which was implemented in partnership with a financial institution with national operations. This tool operated as an autonomous agent in the sales and customer service journey, with the capacity to formalize proposals and resolve tickets. The analysis focused on the decision-making mechanisms and the technical controls existing in the solution.

The analyzed material was divided into two documentary sets. The first gathered technical documents referring to the studied tool, including system architecture presentations, functional descriptions of sales flows, solution development process records, and aggregated and anonymized operational performance data in a production environment. These documents were selected because they allow for an understanding of the tool’s operation and its decision-making mechanisms.

The second documentary set, of a normative-academic nature, comprised the international ethical and regulatory governance frameworks used as analytical parameters. The five ethical principles of AI4People (Floridi et al., 2018), the OECD Principles on AI (Organisation for Economic Co-operation and Development, 2019a; 2019b), the UNESCO Recommendation on the Ethics of AI (United Nations Educational, Scientific and Cultural Organization, 2021), the Council of Europe’s study on algorithms and human rights (Council of Europe, 2018), the Artificial Intelligence Act (European Union, 2024), and the General Law on Personal Data Protection (Brazil, 2018) were considered.

Complementarily, contributions from the literature on algorithmic biases (O’Neil, 2016; Eubanks, 2017; Pasquale, 2015) and on algorithmic fairness trade-offs (Kleinberg et al., 2016) were incorporated. The temporal scope of the analysis covered the period of the tool’s development and operation until August 2025. The normative frameworks considered were in effect until the research completion date.

The collection technique consisted of documentary research, through an intentional survey of available technical documents and operational data about the solution. The research did not involve primary data collection or interaction with individuals, being entirely based on secondary sources and technical documents with aggregated and anonymized information. There was no access to data that allowed for individual user identification, which was considered in defining the limits of the analysis.

The qualitative analysis was organized into five categories, derived from the reference frameworks. The categories were: (i) transparency and explainability, relating to the possibility of understanding and auditing the system’s decision-making processes; (ii) human oversight, referring to the existence of human intervention mechanisms proportional to the risk; (iii) fairness and non-discrimination, pertaining to the presence of controls for preventing algorithmic biases; (iv) accountability and governance, relating to the formalization of responsibilities and compliance policies; and (v) impact assessment, referring to the existence of ethical risk assessment processes.

The analysis technique employed was qualitative content analysis, guided by previously defined categories. The procedure occurred in three reproducible stages. Firstly, the technical documents were read and coded into the five analytical categories. Next, a systematic comparison of the findings with the criteria extracted from the theoretical and normative references was carried out. Finally, the identified risks and gaps were synthesized to derive digital compliance recommendations.

Individual demographic characteristics were not inferred, nor was statistical measurement of discrimination performed, as the available database contained aggregated and anonymized data. This limitation restricted the research to the identification of structural risks and governance gaps, rather than to the empirical proof of algorithmic discrimination. The study focused on the qualitative analysis of procedures and controls.

3. Results and Discussion

The analysis of the conversational Artificial Intelligence tool, applied to the digital acquisition of financial products, revealed a complex scenario where operational efficiency and scalability are contrasted with significant gaps in ethical governance and the mitigation of algorithmic biases. The results were examined in light of the research objectives, interpreting each finding as evidence of the presence, absence, or maturity of digital compliance mechanisms. This approach allowed for the distinction of existing controls, the identification of governance gaps, and the derivation of applicable recommendations, confronting the evidence with the literature and the previously selected international frameworks for ethical and regulatory governance.

The studied tool consisted of a conversational AI system designed to act as an autonomous agent in the sales and customer service journey, offering continuous availability and integration with internal systems. Its technical architecture was based on the combination of “prompting”, “Retrieval Augmented Generation” (RAG), and “fine tuning”. “Prompting” was crucial for defining the AI’s behavior guidelines and user inputs, while RAG allowed for the retrieval of information from structured internal databases, increasing response consistency and reducing reliance on generic model inferences. The system conducted 21 distinct sales flows, demonstrating high customization.

In terms of commercial performance, the solution demonstrated an expressive impact. The conversational application, trained with 133 distinct materials, conducted approximately 65,000 conversations in a production environment by August 2025. This volume of interactions resulted in an approximate 53% increase in the meeting scheduling rate with potential clients in the first week of operation, compared to traditional approaches. Furthermore, the tool was associated with the capture of R$ 4,377,640.16 in financial resources and the sending of 2,566 links for account opening, evidencing its active and relevant participation in the conversion journey stages and its capacity to scale operations.

Despite the efficiency, the analysis of “fine tuning” revealed that, although its objective was to adjust the form of communication without altering factual content, the distinction between form and content is not absolute. The way information is presented can significantly influence the user’s decision. Observed examples included shorter messages and the systematic use of calls to action, persuasive techniques that guide consumer behavior. Floridi et al. (2018) highlighted that non-maleficence requires attention not only to intentional harm but also to the effects of behavioral induction mechanisms, which aligns with the OECD principles (2019b) on transparency and continuous risk management. However, no formal policies were identified to assess the persuasive impact of these communication techniques on consumer decisions, indicating a governance gap that should encompass interaction architecture and influence mechanisms, not just factual content.

The tool’s multi-agent architecture, coordinated by an orchestrator, demonstrated sophistication. The orchestrator classified the user’s stage in the conversion funnel (Presentation, Persuasion, Resolution, Conclusion, or Human Redirection) and triggered the appropriate flow, ensuring concise and targeted responses. Furthermore, a “tracking” agent collected and recorded user data, which, from the perspective of the General Law on Personal Data Protection (Brazil, 2018), constituted personal data processing and required specific attention regarding legal bases, purposes, and data subject rights. This functional segmentation favored the traceability and auditability of interactions, crucial elements for human supervision in a high financial risk context, as emphasized by the Financial Stability Board (2017).

The technical documentation revealed two modes of operation: autonomous and copilot. In autonomous mode, the AI conducted interactions entirely, without direct human intervention, while in copilot mode, the AI acted as support to a human attendant, who could review and modify suggested responses. During the analyzed period, the partnership with the financial institution operated predominantly in autonomous mode. This prevalence of autonomous mode has direct implications for ethical governance, as human supervision occurred only a posteriori, through conversation logs, without real-time intervention. This scenario contrasts with the concept of meta-autonomy by Floridi et al. (2018), which advocates for preserving the human capacity to decide which decisions to delegate to the machine. The risk-based approach, present in international frameworks, reinforces the need for proportional controls and clear oversight mechanisms for systems with greater autonomy.

The Council of Europe (2018) warned that, even with formal human participation, mere ratification of algorithmic recommendations could occur due to lack of time or competence. In the case of the tool, the absence of real-time human supervision in most autonomous sales flows reinforced the need to formalize risk criteria and overflow triggers linked to ethical issues, and not just operational failures. The European Union’s AI Act (European Union, 2024) requires risk management systems and human supervision processes for high-risk systems, a requirement that was not formally documented as a compliance policy in the analyzed tool, evidencing a significant gap.

Consolidated operational data up to August 2025 showed that the tool’s monitoring was exclusively focused on commercial performance metrics, such as conversion rates, acquisition volume, and automatic ticket resolution. However, indicators of ethical impact, distribution of results by demographic profile, or monitoring of differentiated treatments were not incorporated. This asymmetry between the sophistication of commercial metrics and the absence of ethical indicators constituted one of the study’s main findings. The OECD (2019b) guidelines emphasize accountability, robustness, security, and continuous risk management as essential components of trustworthy AI. The tool’s large-scale operation, with approximately 65,000 conversations and acquisitions exceeding R$ 4 million, was not accompanied by metrics designed to detect patterns of differentiated treatment, representing a governance gap that prevents the early identification of potentially asymmetric outcomes, even without proving discrimination.

The joint analysis of the technical architecture, sales flows, and operational data allowed for the identification of five risk vectors for algorithmic biases. The first vector referred to the RAG knowledge base, composed of 133 materials selected based on commercial and operational criteria, without documented evaluation of potential biases. Materials that reflected partial views or prioritized certain products could systematically influence the AI’s responses. The second vector was linked to the orchestration logic, where the user’s classification into conversion funnel stages implied automated decisions about commercial qualification, with the potential for differentiated treatment. Users classified as closer to conversion might receive more incisive approaches, while others could be subject to less interactional investment. If correlated with uncontrolled sociodemographic variables, these classifications could reproduce historical discriminatory patterns, as warned by O’Neil (2016).

The third risk vector involved the “tracking” agent, which collected and recorded user data. The systematic collection of behavioral and contextual data creates the potential for profile formation that can serve as “proxies” for protected categories, such as race, gender, or age, as highlighted by the Council of Europe (2018). The fourth vector related to language “fine tuning”, where standardized persuasive techniques could have a differentiated impact on population segments with different levels of financial literacy, configuring a risk of non-maleficence, according to Floridi et al. (2018). Finally, the fifth risk vector was in system feedback: optimization solely for commercial metrics could amplify pre-existing biases in the data, perpetuating historical discriminatory patterns, a mechanism characteristic of the “weapons of math destruction” described by O’Neil (2016).

The tool incorporated technical elements that favor traceability, such as functional separation between agents, flow documentation, and the use of controlled knowledge bases. These aspects represented a partial adherence to the OECD’s (2019b) principle of transparency and explainability, which advocates for responsible disclosure so that users can understand and contest the results. However, the identified traceability referred predominantly to the recording of interactions, not to the substantial explainability of the decision-making criteria. The risk of the “black box society” (Pasquale, 2015) persists when the criteria that structure relevant decisions remain inaccessible to scrutiny. Floridi et al. (2018) and the Council of Europe (2018) highlighted that explainability is essential for intelligibility, accountability, and effective contestation. In the analyzed case, no institutional policies were identified that defined levels of transparency and explanation proportional to the impact of each flow, which limits the capacity for formal auditing and contestation.

The tool’s multi-agent architecture included “human-in-the-loop” mechanisms, with the possibility of automatic overflow to human support in specific situations, demonstrating adherence to the OECD’s (2019b) principle of human supervision and UNESCO’s (2021) recommendation on human review of automated decisions with significant impact. However, the analysis revealed that the overflow triggering criteria were predominantly linked to technical failures or demands outside the operational scope, rather than triggers of ethical or discriminatory risk. Floridi et al. (2018) argued that the principle of autonomy requires a meta-autonomy model, in which humans must retain the power to decide which decisions can be delegated to the machine. In the analyzed case, no formal criteria were identified that defined which decisions in the sales journey would be considered high-risk and, therefore, subject to mandatory human review. The AI Act (European Union, 2024) requires risk management systems and human oversight processes for high-risk systems, a requirement that was not formally documented as a compliance policy for the tool.

The analysis did not identify explicit metrics for evaluating ethical impacts, monitoring algorithmic biases, or tracking discriminatory effects. This absence constituted the most relevant finding from the perspective of digital compliance, especially considering the system’s large-scale operation and its direct influence on access to financial products. O’Neil (2016) warned that algorithms often encode opinions and prejudices, perpetuating inequalities. Eubanks (2017) demonstrated how the automation of financial decisions can deepen the exclusion of vulnerable populations. The Council of Europe (2018) emphasized that algorithms can contain intrinsic biases that are difficult to detect, especially when individual variables function as “proxies” for protected categories such as race, gender, or age. In the analyzed case, the combination of multiple sales flows, a large volume of interactions, and the use of historical data created a scenario conducive to the occurrence of indirect differentiated treatment.

From a technical perspective, Kleinberg et al. (2016) formally demonstrated that three algorithmic fairness criteria – group calibration, balance for the positive class, and balance for the negative class – are inherently incompatible when base rates differ across groups. This result formalized a dilemma that transcends engineering, showing that the choice of which fairness criteria to prioritize is a normative decision requiring the participation of corporate leadership, the compliance and governance team, and ideally, those affected. In the context of the analyzed tool, the absence of any formal definition regarding which equity criteria should be pursued highlighted a critical governance gap. The General Law for Personal Data Protection (Brazil, 2018) established, among its principles, non-discrimination and transparency, while the AI Act (European Union, 2024) adopted a risk-based regulatory approach, foreseeing reinforced obligations for systems classified as high-risk. Together with the analyzed ethical frameworks, these norms reinforce the need for organizations to document criteria, monitor risks, and establish controls proportional to the system’s impact. In the studied tool, the non-existence of equity metrics and formalized fairness criteria therefore constituted a relevant point of attention for governance.

The analysis revealed that the tool incorporated technical elements aligned with governance principles, such as traceability, human oversight, and knowledge base control. However, these mechanisms were predominantly oriented towards operational and commercial efficiency gains, without formalization as compliance policies. The OECD (2019b) attributed accountability to organizations and individuals who develop, deploy, or operate AI systems, requiring responsibility compatible with the principles of trustworthy AI. The Council of Europe (2018) highlighted the complexity of allocating responsibilities in algorithmic systems, especially when control is fragmented among different actors. In the analyzed case, no formal algorithmic accountability policies were identified: there was no specific designation of responsibilities for the ethical functioning of the system, documented procedures for periodic algorithmic auditing, or specific channels for contesting automated decisions. This fragmentation of responsibilities among the developing company, the partner financial institution, and the AI infrastructure providers is a challenge for algorithmic accountability, as no single actor has a complete view of the risks, and the absence of formal responsibility designation can result in governance gaps.

Floridi et al. (2018) associated explainability not only with the intelligibility of systems but also with accountability for their outcomes. From this perspective, technical records are necessary, but insufficient if not accompanied by processes capable of identifying responsible parties, investigating occurrences, and defining corrective measures. The analysis identified that the log visualization system (“Overwatch”), although allowing access to the interaction history and prompt configuration, operated as an operational monitoring tool, rather than a formal instrument for algorithmic auditing. The distinction is relevant: while operational monitoring seeks efficiency and identification of technical failures, ethical auditing systematically investigates patterns of differential treatment, adherence to governance principles, and adequacy of decision-making criteria. The evolution from the former to the latter would require, at a minimum, audit protocols, documented periodic reviews, definition of responsible parties, and recording of findings and corrective actions, which was not observed.

The analysis of the findings from the perspective of Brazilian regulation revealed that, even without a specific general framework for AI in the scope considered by the research, provisions of the LGPD (Brazil, 2018) were already relevant for the operation of tools like the one studied. Article 20 provides for the right to request a review of decisions made solely based on automated processing of personal data that affect the data subject’s interests, including decisions aimed at defining personal, professional, consumption, and credit profiles. In the analyzed context, the autonomous conduct of sales flows, the classification of users by the orchestrator, and the direction towards account opening stages highlighted the need to evaluate, in each flow, the incidence of these guarantees. Article 6, IX, of the LGPD enshrined the principle of non-discrimination, while Article 6, VI, established the principle of transparency. In the analyzed case, the combination of automated data collection by the tracking agent, user classification by the orchestrator, and autonomous conduct of sales flows demanded specific attention to the purpose of processing, transparency towards data subjects, and prevention of discriminatory outcomes. These elements bring data protection closer to algorithmic governance and demonstrate that digital compliance must integrate legal, technical, and ethical controls.

The joint reading of the LGPD and the analyzed international benchmarks indicated that the absence of a general specific AI law does not eliminate the need for internal controls. On the contrary, principles of transparency, non-discrimination, accountability, human supervision, and impact assessment offer parameters for compliance structures to translate normative obligations and expectations into concrete governance procedures. This analysis reinforced the central thesis of the work: digital compliance must act proactively, anticipating risks and incorporating ethical governance as a structural component of the tool’s lifecycle, rather than functioning solely as a subsequent response to incidents or regulatory inquiries. Regulatory compliance and technical controls, in isolation, are not sufficient for responsible conversational AI governance, necessitating the institutionalization of continuous algorithmic risk assessment, accountability, and ethical governance.

Finally, the analysis highlighted the relevance of periodic impact assessments to verify the tool’s adherence to adopted ethical principles and identify unintended consequences, especially in automated flows capable of influencing consumer decisions. UNESCO (2021) recommended that organizations conduct ethical impact assessments prior to implementation and throughout the lifecycle of AI systems, with proportionality to the risk involved. The AI Act (European Union, 2024) required high-risk systems to implement risk management systems and comprehensive technical documentation. The Council of Europe (2018) suggested that companies exercise human rights due diligence through impact assessments that investigate concrete and potential effects on individuals. In the analyzed tool, no formal processes for ethical or algorithmic impact assessment were identified, either prior to the introduction of new sales flows or as a routine for continuous monitoring. This gap proved especially relevant given the scale of the system’s use and its participation in stages of the financial product acquisition journey.

The absence of impact assessment acquired special gravity when considered in conjunction with the system’s evolutionary dynamics. Technical documentation indicated that prompts, RAG bases, and “fine tuning” parameters were subject to continuous adjustments throughout the operational period, motivated by commercial optimization objectives. Each alteration to these components had the potential to modify the system’s decision-making behavior, introducing new risks or amplifying existing ones. The absence of a formal impact assessment process prior to these changes meant that modifications with the potential to affect consumers’ access to financial products were implemented without documented analysis of their effects on equity and non-discrimination. Floridi et al. (2018) recommended that impact assessment be conducted not only as a one-off exercise but as a continuous process integrated into the system’s lifecycle, which the authors termed an “ethics by design” approach. This perspective converged with the AI Act’s requirement (European Union, 2024) that high-risk systems maintain updated risk management systems throughout their entire lifecycle. In the analyzed case, the implementation of periodic impact assessments could, for example, have proactively identified the five risk vectors mapped in the analysis, allowing for the adoption of mitigation measures before any discriminatory effects could consolidate on a large scale.

The confrontation of the case study findings with the theoretical and normative references allowed for the suggestion of four recommendation axes for the improvement of the tool’s ethical governance. The first axis consisted of the creation of metrics for monitoring equity and non-discrimination, with explicit definition of the prioritized fairness criteria and accepted trade-offs, according to the contribution of Kleinberg et al. (2016). This recommendation stemmed directly from the absence of ethical indicators identified in the case and seeks to enhance the capacity for detecting potentially differentiated treatments. The second axis comprised the formalization of periodic algorithmic audit policies, with documentation of decision-making criteria, record of changes, and definition of responsibilities, in line with the OECD’s (2019b) accountability principle. The third addressed the implementation of algorithmic impact assessments before the introduction of new workflows or substantial changes to the knowledge base and prompts, aligning with the impact assessment and risk management recommendations present in the references of UNESCO (2021), the Council of Europe (2018), and the AI Act (European Union, 2024). The fourth axis involved the designation of specific algorithmic compliance responsibilities within the corporate governance structure, incorporating channels for contestation and effective remedy for affected users, in line with the Council of Europe (2018) and the explicability principle of Al4People (Floridi et al., 2018).

Based on the findings of this research, a digital compliance model structured in three complementary layers for conversational AI was proposed. The first layer, called design controls, involved the incorporation of ethical requirements from the system’s conception phase. This included the documented definition of prioritized fairness criteria, with explicit articulation of accepted trade-offs, assessment of biases in the materials comprising the RAG base, analysis of the persuasive impact of communication techniques incorporated via fine-tuning, and risk classification of sales flows according to the degree of autonomy and the impact on consumer access to financial products. These controls aim to prevent risks at the origin of development.

The second layer, called operational controls, encompassed continuous monitoring mechanisms throughout the system’s production lifecycle. This included fairness metrics segmented by user profile, tracking the distribution of results, conversion rates, and redirection patterns by segment, as well as periodic auditing of the orchestrator’s decisions to detect discriminatory patterns. It also included the review of changes to the knowledge base and prompts with impact logging, and the definition of overflow triggers linked to ethical risks, not just operational ones, ensuring that high-impact decisions about the consumer can be reviewed by a human. These controls are essential for identifying deviations and ensuring timely intervention.

The third layer, named accountability controls, referred to the institutional structure of responsibility. This involved the designation of an algorithmic compliance officer within the corporate governance structure, algorithmic impact assessment procedures prior to the introduction of new workflows, channels for dispute and effective remedy for users who consider themselves affected by automated decisions, and periodic algorithmic governance reports directed to senior management. This three-layer model sought to respond to the need, identified throughout the research, for digital compliance to go beyond the dimension of regulatory compliance to achieve substantial ethical governance, capable of dealing with the complexity and scale of contemporary conversational AI systems. The results showed that the studied tool incorporated technical solutions favorable to control and traceability, such as functional separation of agents, controlled knowledge bases, interaction logs, and possibilities for human supervision. However, relevant gaps were identified: absence of fairness and non-discrimination metrics, non-existence of formal algorithmic audit policies, lack of periodic ethical impact assessments, and absence of specific algorithmic accountability responsibilities. Due to the limitations of the available data, these findings do not demonstrate the occurrence of discrimination, but they do show an insufficiency of mechanisms designed to detect and prevent it systematically. The analysis concluded that regulatory compliance and technical controls, in isolation, are not sufficient for responsible governance of conversational AI. The contribution of Kleinberg et al. (2016) demonstrated that different fairness criteria can involve incompatible trade-offs, making it necessary to explicitly state the normative choices adopted. Floridi et al. (2018), in turn, reinforced the centrality of explainability and accountability, while OECD, UNESCO, and the Council of Europe converged on the need for transparency, risk management, supervision, and accountability. The confrontation of these references with the case supported the proposal of a three-layer digital compliance model: design controls, operational controls, and accountability controls. In this scenario, digital compliance proved to be an instrument for mediation between technological innovation, rights protection, and corporate governance. The connection with the ESG agenda also proved relevant: in the social dimension, the prevention of discrimination and consumer protection are directly affected by how automated systems interact with users; in the governance dimension, transparency, accountability, and internal controls need to be adapted to the complexity of AI systems. Thus, the three-layer model proposed in this work represents a practical contribution to incorporating algorithmic risks into compliance and governance structures, without aiming to constitute a universal model or to prove, by itself, the existence of biases in the studied tool.

In summary, the research demonstrated that, although the conversational AI tool analyzed presented relevant technical advances for traceability and partial human supervision, significant gaps persisted in the formalization of digital compliance controls aimed at mitigating algorithmic risks. The absence of fairness metrics, algorithmic auditing policies, and ethical impact assessments, coupled with the lack of formalized accountability, indicated that regulatory compliance and technical controls alone are insufficient for responsible governance. The proposed digital compliance model, with its three layers of design, operational, and accountability controls, offers a path to institutionalize continuous algorithmic risk assessment, formalized accountability, and ethical governance that responds to the complexity and impact of these systems in the digital acquisition of financial products, aligning technological innovation with the protection of rights and corporate sustainability.

4. Conclusion

This study analyzed how digital compliance mechanisms can identify, prevent, and mitigate algorithmic biases in a conversational Artificial Intelligence tool applied to the digital acquisition of financial products. It was found that, although the technical solution presented favorable elements for traceability and partial human supervision, significant gaps persisted in the formalization of digital compliance controls aimed at mitigating algorithmic risks. The absence of fairness and non-discrimination metrics, the non-existence of formal algorithmic audit policies, the lack of periodic ethical impact assessments, and the non-formalization of specific algorithmic accountability responsibilities were identified. Operational data revealed that the tool’s monitoring was predominantly oriented towards commercial metrics, without the incorporation of ethical indicators. Five risk vectors for algorithmic biases were mapped, arising from the knowledge base, orchestration logic, tracking agent, language fine-tuning, and system feedback. These findings, although not proving the occurrence of discrimination, highlighted the insufficiency of systematic mechanisms for its detection and prevention. It was concluded that regulatory compliance and technical controls, in isolation, are not sufficient for responsible governance of conversational AI.

Based on this analysis, the work proposed a digital compliance model structured in three complementary layers: design controls, operational controls, and accountability controls. This model aims to institutionalize the continuous evaluation of algorithmic risk, formalized accountability, and ethical governance that responds to the complexity and impact of these systems in the digital acquisition of financial products, aligning technological innovation with the protection of rights and corporate sustainability. The research, however, was limited by having been conducted exclusively from documentary and technical sources, without interviews, direct algorithm testing, or access to disaggregated training data, which prevented the quantitative measurement of disparities and restricted the generalization of findings to a single case. For future studies, it is suggested to conduct quantitative algorithmic audits, with disparity tests between user profiles, and comparative studies between conversational AI solutions in financial services. Investigations into algorithmic impact assessment models in the Brazilian regulatory context and research that incorporate consumer perspectives on transparency and the exercise of rights provided for in the LGPD are also relevant, in order to test and improve the proposed digital compliance model.

Bibliographic References

Brasil. 2018. Lei nº 13.709, de 14 de agosto de 2018. Dispõe sobre a proteção de dados pessoais e altera a Lei nº 12.965, de 23 de abril de 2014 (Marco Civil da Internet). Diário Oficial da União, Brasília, 15 ago. 2018. Seção 1, p. 59.

Council of Europe. 2018. Algorithms and Human Rights: Study on the Human Rights Dimensions of Automated Data Processing Techniques and Possible Regulatory Implications. DGI(2017)12. Council of Europe, Strasbourg, France.

Eubanks, V. 2017. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. St. Martin’s Press, New York, USA.

Financial Stability Board [FSB]. 2017. Artificial Intelligence and Machine Learning in Financial Services: Market Developments and Financial Stability Implications. FSB, Basel, Switzerland. Disponível em: <https://www.fsb.org/2017/11/artificial-intelligence-and-machine-learning-in-financial-service/>. Acesso em: 03 fev. 2026.

Floridi, L.; Cowls, J.; Beltrametti, M.; Chatila, R.; Chazerand, P.; Dignum, V.; Luetge, C.; Madelin, R.; Pagallo, U.; Rossi, F.; Schafer, B.; Valcke, P.; Vayena, E. 2018. Al4People: An ethical framework for a good Al society: Opportunities, risks, principles, and recommendations. Minds and Machines 28(4): 689-707.

Kleinberg, J.; Mullainathan, S.; Raghavan, M. 2016. Inherent Trade-Offs in the Fair Determination of Risk Scores. arXiv:1609.05807v2 [cs.LG]. Cornell University, Ithaca, USA.

O’Neil, C. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Crown Publishing Group, New York, USA.

Organisation for Economic Co-operation and Development [OECD]. 2019a. Artificial Intelligence in Society. OECD Publishing, Paris, France. DOI: https://doi.org/10.1787/eedfee77-en.

Organisation for Economic Co-operation and Development [OECD]. 2019b. OECD Principles on Artificial Intelligence. OECD, Paris, France. Disponível em: <https://legalinstruments.oecd.org/en/instruments/OECD-LEGAL-0449>. Acesso em: 03 fev. 2026.

Pasquale, F. 2015. The Black Box Society: The Secret Algorithms That Control Money and Information. Harvard University Press, Cambridge, USA.

United Nations Educational, Scientific and Cultural Organization [UNESCO]. 2021. Recommendation on the Ethics of Artificial Intelligence. UNESCO, Paris, France.

European Union. 2024. Artificial Intelligence Act (AI Act). European Parliament and Council of the European Union, Brussels, Belgium.

Article originating from the Final Course Work of the Specialization in Compliance and ESG of the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform