AI and ethics in sensitive sectors

Column

November 11, 2025

AI and ethics in sensitive sectors

Transformative potential, risks, and guidelines for the responsible use of intelligent systems in health, security, and other critical areas

Artificial intelligence (AI) has already established itself as one of the most significant forces of transformation of our time. With the ability to process immense volumes of data, identify patterns, and generate real-time predictions, it has been reshaping production chains, strategic decisions, and even entire business models. However, this advance is not without risks.

Imagine a nurse selection system that consistently approves only white women. Or a modern skin cancer diagnostic test that frequently fails on Black patients. Or even an algorithm that assigns low-income individuals to long-term inpatient units, while wealthier patients are discharged home. Cases like these show how AI, when unsupervised, can crystallize and even amplify existing prejudices (Nelson, 2019).

In this sense, the application of this technology in sensitive areas requires increased attention and a solid foundation of ethical principles and clear rules. Topics such as algorithmic bias, transparency, and social impact need to be at the center of the discussion so that technological advancement goes hand in hand with justice and responsibility (Kim; Kim; Lee, 2025). Understanding both the potential and the risks of AI is essential for governments, companies, and society to jointly build the limits and conditions for its use.

AI in healthcare

In the field of health, AI already performs functions ranging from the analysis of imaging exams to the support of complex clinical diagnoses. Deep learning-based tools can identify diseases in early stages, increasing the effectiveness of treatments and reducing hospital costs. However, the integration of these systems requires attention to patient privacy, the reliability of databases, and the explainability of the algorithms used. Failures or biases in predictive models can result in incorrect diagnoses or inadequate treatments, with relevant ethical and legal implications. It is necessary for innovation to be accompanied by rigorous clinical validation protocols and continuous auditing mechanisms.

One of the most emblematic cases of ethical and technical risks in the application of computational systems in sensitive sectors occurred in the 1980s, with the series of accidents involving the Therac-25 radiotherapy machine. The equipment, developed to treat cancer patients using electron and radiation beams, had a software flaw that, under certain conditions, caused extremely high doses to be applied. Unlike incidents caused by mechanical failure, in the Therac-25 the error was in the control code, which lacked adequate safeguards and only manifested in very specific operational situations, making its detection difficult. As a result, several patients suffered severe burns, irreversible neurological damage, and even death (Leveon; Turner, 1993).

The subsequent analysis revealed not only programming problems and lack of rigorous testing, but also failures in the software engineering process and in product governance, such as lack of clear documentation, underestimated risks, and absence of external audit protocols. The case became a landmark in the history of critical systems security, demonstrating that, in environments such as healthcare, transportation, and energy, technological errors are not merely technical failures, but events with profound ethical, legal, and social implications, reinforcing the need for rigorous validation, continuous supervision, and shared responsibility among developers, manufacturers, and operators (Leveon; Turner, 1993).

AI in public safety

If in health the risks involve life and physical integrity, in public security the debate turns to fundamental rights and equal treatment. In security, AI is applied in facial recognition, risk area monitoring, and predictive crime analysis. These solutions promise greater efficiency in the deployment of police resources and prevention of offenses. However, the use of large-scale surveillance technologies raises concerns about the right to privacy and the risk of algorithmic discrimination. Studies show that facial recognition systems can present higher error rates for certain racial or gender groups, reinforcing inequalities. Thus, it is essential to have clear legal frameworks, ensuring that the application of these tools respects fundamental rights and is subjected to independent audits.

In 2016, an investigation conducted by the website ProPublica revealed that the Compas (Correctional Offender Management Profiling for Alternative Sanctions) system, used in several U.S. states to assess the risk of criminal recidivism, presented significant racial bias. The algorithm assigned higher risk scores to Black defendants compared to White defendants in similar situations, overestimating the probability of recidivism for the former and underestimating it for the latter. As Compas was employed by judges to support decisions on bail, parole, and sentencing, this bias had a direct impact on the freedom and rights of individuals, perpetuating inequalities in the criminal justice system. The analysis revealed not only technical flaws in the model and the choice of variables, but also a lack of transparency—as the algorithm was proprietary and could not be publicly audited. The case became emblematic in the debate on ethics in artificial intelligence in public safety, highlighting the need for independent audits, model explainability, and constant supervision to prevent systemic discrimination (Angwin et al., 2016).

AI in justice

In the legal field, the dilemmas take on another dimension: the preservation of impartiality, autonomy, and procedural guarantees. In addition to health and safety, sectors such as the legal field also face ethical dilemmas in the use of AI. An emblematic example of artificial intelligence use is the Chinese system known as Smart Court, implemented in some provinces since 2017. Integrating natural language processing, data analysis, and voice recognition, the platform assists judges in drafting sentences, screening cases, and managing digital evidence. Although it offers gains in speed and standardization, the system raises concerns about the autonomy of judicial decision-making and the possibility of reinforcing biases already present in the history of judgments.

Critics point out that, by training the algorithm with data from past decisions, there is a risk of perpetuating discriminatory patterns or restrictive interpretations of the law. Furthermore, the opacity of the criteria used and the absence of robust external auditing mechanisms make it difficult to challenge decisions suggested or influenced by AI. This case illustrates that, in the Justice system, the challenge is not merely technical, but involves the preservation of procedural guarantees and the maintenance of human control over the final decision (Wang et al., 2021).

Responsible use

The ethical and responsible adoption of AI requires not only general principles but also concrete practices that ensure the reconciliation between technological innovation and social protection. International experiences demonstrate that the indiscriminate or poorly regulated use of AI can generate significant impacts on fundamental rights, reinforcing inequalities and undermining public trust. Therefore, organizations such as Unesco, the OECD, and the European Union have published governance frameworks that emphasize values such as transparency, fairness, security, privacy, and accountability. These guidelines are based on the assumption that technology should be designed in a human-centered manner, ensuring that critical decisions remain under effective human control and that any negative impact is identified and mitigated in a timely manner. Data protection and cybersecurity, in turn, must be incorporated from the design stage (privacy by design and security by design), preventing them from being mere afterthoughts in response to incidents.

In this context, detailed guidelines are fundamental to operationalize these principles. Transparency implies that both decision criteria and AI limitations are clearly communicated, allowing for external auditing and social participation. Accountability demands that developers, suppliers, and users assume specific obligations, with mechanisms to assign and collect responsibilities in case of failures or abuses. Non-discrimination requires the detection and mitigation of biases in data and algorithms, especially in applications affecting rights, such as health, justice, and public safety.

Continuous human oversight is essential to reverse unfair or incorrect automated decisions, preserving due process. Furthermore, digital education and algorithmic literacy need to be fostered so that citizens, professionals, and authorities understand the functioning, limitations, and risks of AI, enabling them to critically interact with these tools. Finally, the security and resilience of systems must be prioritized, ensuring robustness against cyberattacks and operational failures, so that social trust in technology is preserved. Some suggestions can help guide the ethical and responsible use of artificial intelligence:

  • Transparency and explainability: making decision logic and system limitations visible;
  • Effective human oversight: keep people able to intervene and reverse automated decisions;
  • Periodic independent audits: verify biases, performance, and regulatory compliance;
  • Clear accountability: assign duties and consequences to all actors involved in the AI lifecycle;
  • Data protection and security by design (by design): prevent leaks and attacks;
  • Active bias mitigation: ensuring vulnerable groups are not harmed;
  • Digital education and algorithmic literacy: empowering society to understand and question technology;
  • Operational robustness and resilience: ensure that systems function reliably, even in adverse scenarios.

AI represents a watershed moment in global technological evolution, with the potential to radically transform sectors such as health, security, justice, education, and economy. Its ability to process large volumes of data, identify complex patterns, and offer real-time solutions already translates into significant gains in efficiency, precision, and quality. However, the same technology that can generate unprecedented advances also carries the risk of widening structural inequalities, compromising fundamental rights, and causing social and economic impacts of great magnitude, especially when used indiscriminately or without adequate regulation.

For AI to be an ally of progress and not a vector of exclusion or injustice, it is essential that its development and application be anchored in a solid ethical commitment, with clear rules, independent oversight mechanisms, and participatory governance. This implies creating institutional structures capable of auditing algorithms, preventing biases, ensuring data protection, and guaranteeing that decision-making ultimately remains under human control. Active societal participation—through public consultation, digital education, and open debate—is fundamental for technological solutions to reflect democratic values and serve the collective interest. Only through this combination of responsible innovation, continuous supervision, and social involvement will it be possible to equitably and sustainably reap the benefits of this technological revolution. The future of AI will not be defined solely by its technical capabilities, but by how societies, governments, and companies choose to use it, balancing efficiency and fairness, speed and prudence, transformative potential and ethical responsibility.

To access the references of this text click here.

Who wrote this column

Renato Máximo Sátiro

Doutor em Administração pela UFG, professor e orientador no curso de Data Science, Inteligência Artificial e Analytics, do MBA USP/Esalq. Administrador de Empresas na Saneago e pesquisador em grupos de pesquisa da UFG e da UnB, com foco em IA, políticas públicas e acesso à Justiça. Desenvolve projetos em machine learning, deep learning, modelos estatísticos, algoritmos e ética na IA, domínio de ferramentas R, Python, Gretl, SPSS e Stata.

You may also like

November 17, 2026

Detection and Diagnosis of Diseases in Corn and Rice in Mozambique with Deep Learning

Mozambican agriculture depends on corn and rice, fundamental crops for food security, but their productivity is strongly affected by phytosanitary diseases, whose early and accurate identification remains a challenge in resource-limited contexts. The study aimed to develop and compare Deep Learning-based models for the automatic detection and diagnosis of diseases in corn and rice leaves in Mozambique. Three approaches were used: a classic Convolutional Neural Network (CNN) developed from scratch, and two transfer learning models, ResNet50 and MobileNetV2. The dataset consisted of images from public databases and collected in the field, which underwent cleaning, balancing, and artificial augmentation steps. The models were trained and evaluated with standard performance metrics, including accuracy, precision, recall, F1-score, Log Loss, and Cohen’s Kappa. The classic CNN presented the best overall performance, with 94% precision, recall, and F1-score, and 93% accuracy and Cohen’s Kappa. MobileNetV2 achieved 91% F1-score and 90% accuracy, standing out for its lower Log Loss (0.30) and training time (6h32min59s). ResNet50 reached 80% F1-score and 79% accuracy. It was concluded that Deep Learning techniques have high potential to support phytopathological diagnosis in real agricultural contexts, contributing to faster and more accurate decisions in the management of corn and rice diseases.

Keywords: Deep learning; Rice; Plant diseases; Corn; Computer vision.

November 17, 2026

Artificial intelligence in accounting operational routines and financial efficiency

The incorporation of Artificial Intelligence (AI) and automation tools has reconfigured the routines of Brazilian accounting offices, impacting productivity, error mitigation, and financial efficiency. These impacts were analyzed in a small accounting firm, specialized in serving service providers in the medical field in São Caetano do Sul, São Paulo. The research adopted a qualitative approach, with a single case study design and incorporated units of analysis. Data were collected through a structured questionnaire, applied via Google Forms between January and February 2026, to five professionals in the operational structure of the firm in a census survey. The treatment followed content analysis with a priori and emergent categories. All respondents declared the use of automated tools or tools with AI resources. The perception of reduced execution time, decreased operational errors, rework, and better use of the team’s time converged in 80% of affirmative responses and 20% partial responses. The most benefited activities were bank reconciliation, import of fiscal documents, and tax assessment, with visible gains on typing errors, duplicate entries, and incorrect classification. Human intervention remained indispensable in tax planning, scenario analysis, and verification of ancillary obligations. The overall evaluation of AI’s contribution was predominantly favorable, with moderation linked to the complexity of fiscal regulations.

Keywords: Content analysis; Process automation; Accounting 4.0; Small business; Digital transformation.

November 17, 2026

Factors influencing schedule management in sewage treatment plant “retrofit” projects

Projects involving “retrofit” in Sewage Treatment Plants (STP) are characterized by high complexity, operational restrictions, and multiple technical interfaces. The study analyzed the perception of the professionals involved regarding the management and adherence to deadlines in “retrofit” projects for sewage treatment plants, considering technical, operational, and organizational factors. The research was characterized as an applied study, with a quali-quantitative approach, designed as a field survey. Data analysis was performed using descriptive statistics and qualitative categorization, employing SWOT, risk matrix (Probability x Impact), and 5W2H action plan tools. The results indicated a high perception of clarity (77.8%) and realism of deadlines (100% agreement), evidencing consistent initial planning. However, 77.7% of respondents pointed out that rework and scope changes directly impacted deadline adherence. The integration between engineering, manufacturing, and Production Planning and Control (PPC) showed an intermediate evaluation (77.7% partial/total agreement), indicating fragility in interfunctional coordination. The risk matrix highlighted as critical: scope changes, operational limitations of the STP, and technical complexity, all with a high level. As a central improvement, the standardization of scope change control, analysis, and approval was proposed, aiming to reduce rework and increase schedule predictability.

Keywords: Performance; Integration; Planning; Rework; Risk.

November 17, 2026

B2C Acquisition in diagnostic medicine: strategic partnerships with specialized medical offices

The diagnostic medicine sector in Brazil operates with high dependence on health plan providers, which limits the financial autonomy of service providers. This study proposed and analyzed a direct patient acquisition model (B2C) for a highly complex diagnostic medicine institution, structured through partnerships with Oncology and Cardiology practices and operationalized by a personalized customer service called concierge. The investigation adopted a mixed-methods approach in three phases: a field survey with a structured questionnaire applied to 87 patients, mapping of the operational flow using “Service Blueprint”, and analysis of the conversion rate from 312 medical referrals registered in the pilot project. The results indicated that trust in medical referral achieved 92.0% agreement, establishing itself as the main factor for adherence. The convenience of performing the exam on the same day reached 78.2%, and the willingness to pay directly was 55.2%, with polarization associated with the predominance of health plan users. The “Service Blueprint” identified three critical points in the transitions between stages of the patient journey. The overall conversion rate was 51.0%, with superior performance in Oncology (58.2%) compared to Cardiology (45.0%), and the greatest loss was concentrated in the acceptance and scheduling stage (28.3%). It was concluded that the model demonstrated possible operational viability and indications of patient acceptance, conditioned by price transparency and a reduction in the interval between consultation and approach by the concierge.

Keywords: Direct payment; Patient journey; Supplementary health; Concierge service; Conversion rate.

November 17, 2026

The role of the school manager in promoting an inclusive culture: leadership and educational policies

The research analyzed the school manager’s performance in promoting an inclusive culture, considering the need to transform legal guidelines and pedagogical foundations into effective institutional practices. It was assumed that school inclusion requires planning, organization, and strategic leadership in the school’s daily operations. A qualitative approach of an applied nature was adopted, developed in three complementary stages: documentary analysis of Brazilian legislation on inclusive education, bibliographic review on school leadership and management, and data systematization using management tools such as SWOT analysis and the PDCA cycle. The results indicated that the school management’s performance directly influenced the consolidation of inclusive environments, especially in the organization of processes, continuous team training, and strengthening of a collaborative institutional culture respectful of diversity. Structural, pedagogical, and relational challenges that demanded planned interventions were also identified. Finally, a Practical Guide for School Managers was developed, with guidelines, protocols, and operational orientations for decision-making and the organization of inclusive processes. It was concluded that the construction of an inclusive culture depended on the management’s ability to articulate public policies, pedagogical practices, and institutional processes, consolidating inclusion as a permanent commitment in the school context.

Keywords: educational accessibility; educational equity; participatory management; strategic planning; institutional processes.

November 17, 2026

An AI artifact based on Design Thinking to support the elicitation of software requirements from user needs

The development of software oriented to user needs frequently faces challenges in requirements elicitation, stemming from ambiguities in natural language and communication failures. Given this, an intelligent artifact integrating Design Thinking and Artificial Intelligence was proposed and validated, focusing on the Empathy and Definition stages, to support the collection, interpretation, and structuring of user needs. The research, of an applied, qualitative, and exploratory nature, conducted a Proof of Concept in a controlled environment, employing hypothetical scenarios. The artifact operated through guided natural language interviews, organizing information into elements such as pain points, context, and impact, and generating traceable problem syntheses. The results indicated that the system organized unstructured reports, reduced ambiguities, and preserved the meaning of the original information in most cases. The ability to reorganize logically in complex scenarios was also observed, although with limitations in separating multiple problems and dependence on instructions. It was concluded that the artifact has potential as a support for requirements elicitation, improving the clarity and structuring of information, but requires supervision and contextual control.

Keywords: Design Thinking; Requirements Elicitation; Requirements Engineering; Artificial Intelligence; Natural Language Processing.

November 17, 2026

The Extinction of the Monophasic Tax System in the Pharmaceutical Industry in Light of the Tax Reform

The consumption Tax Reform, established by Constitutional Amendment No. 132/2023, introduced structural changes to the Brazilian tax system, with the replacement of the Social Integration Program (PIS) and the Social Security Financing Contribution (COFINS) by the Contribution on Goods and Services (CBS), in a dual Value Added Tax (VAT) model. The objective was to analyze the impacts of the extinction of the single-stage system on the pharmaceutical industry, in light of the new regulatory design and its economic and legal effects. A qualitative, exploratory research was conducted through a case study and documentary analysis of legislation and specialized doctrine. It was examined whether the end of the single-stage system would imply a relevant increase in the tax burden for the sector. The results indicated that, although the formal overcoming of the single-stage system has occurred, the material exemption of medicines has been preserved, mainly due to the application of reduced rates of 60% and, in certain cases, a zero rate of the Tax on Goods and Services (IBS) and CBS. The change in model may cause occasional variations in revenue collection, without characterizing a structural increase in the tax burden, tending towards neutrality over time. It was preliminarily concluded that the Tax Reform reorganized the taxation technique in the pharmaceutical industry, replacing a concentrated regime with more transparent exemption mechanisms. This approach proved compatible with the principles of neutrality, full non-cumulativeness, and fiscality, maintaining tax protection for essential medicines and aligning fiscal policy with the fundamental right to health.

Keywords: CBS; COFINS; IBS; Dual VAT; PIS.