November 11, 2025
AI and ethics in sensitive sectors
Transformative potential, risks, and guidelines for the responsible use of intelligent systems in health, security, and other critical areas
Artificial intelligence (AI) has already established itself as one of the most significant forces of transformation of our time. With the ability to process immense volumes of data, identify patterns, and generate real-time predictions, it has been reshaping production chains, strategic decisions, and even entire business models. However, this advance is not without risks.
Imagine a nurse selection system that consistently approves only white women. Or a modern skin cancer diagnostic test that frequently fails on Black patients. Or even an algorithm that assigns low-income individuals to long-term inpatient units, while wealthier patients are discharged home. Cases like these show how AI, when unsupervised, can crystallize and even amplify existing prejudices (Nelson, 2019).
In this sense, the application of this technology in sensitive areas requires increased attention and a solid foundation of ethical principles and clear rules. Topics such as algorithmic bias, transparency, and social impact need to be at the center of the discussion so that technological advancement goes hand in hand with justice and responsibility (Kim; Kim; Lee, 2025). Understanding both the potential and the risks of AI is essential for governments, companies, and society to jointly build the limits and conditions for its use.
AI in healthcare
In the field of health, AI already performs functions ranging from the analysis of imaging exams to the support of complex clinical diagnoses. Deep learning-based tools can identify diseases in early stages, increasing the effectiveness of treatments and reducing hospital costs. However, the integration of these systems requires attention to patient privacy, the reliability of databases, and the explainability of the algorithms used. Failures or biases in predictive models can result in incorrect diagnoses or inadequate treatments, with relevant ethical and legal implications. It is necessary for innovation to be accompanied by rigorous clinical validation protocols and continuous auditing mechanisms.
One of the most emblematic cases of ethical and technical risks in the application of computational systems in sensitive sectors occurred in the 1980s, with the series of accidents involving the Therac-25 radiotherapy machine. The equipment, developed to treat cancer patients using electron and radiation beams, had a software flaw that, under certain conditions, caused extremely high doses to be applied. Unlike incidents caused by mechanical failure, in the Therac-25 the error was in the control code, which lacked adequate safeguards and only manifested in very specific operational situations, making its detection difficult. As a result, several patients suffered severe burns, irreversible neurological damage, and even death (Leveon; Turner, 1993).
The subsequent analysis revealed not only programming problems and lack of rigorous testing, but also failures in the software engineering process and in product governance, such as lack of clear documentation, underestimated risks, and absence of external audit protocols. The case became a landmark in the history of critical systems security, demonstrating that, in environments such as healthcare, transportation, and energy, technological errors are not merely technical failures, but events with profound ethical, legal, and social implications, reinforcing the need for rigorous validation, continuous supervision, and shared responsibility among developers, manufacturers, and operators (Leveon; Turner, 1993).
AI in public safety
If in health the risks involve life and physical integrity, in public security the debate turns to fundamental rights and equal treatment. In security, AI is applied in facial recognition, risk area monitoring, and predictive crime analysis. These solutions promise greater efficiency in the deployment of police resources and prevention of offenses. However, the use of large-scale surveillance technologies raises concerns about the right to privacy and the risk of algorithmic discrimination. Studies show that facial recognition systems can present higher error rates for certain racial or gender groups, reinforcing inequalities. Thus, it is essential to have clear legal frameworks, ensuring that the application of these tools respects fundamental rights and is subjected to independent audits.
In 2016, an investigation conducted by the website ProPublica revealed that the Compas (Correctional Offender Management Profiling for Alternative Sanctions) system, used in several U.S. states to assess the risk of criminal recidivism, presented significant racial bias. The algorithm assigned higher risk scores to Black defendants compared to White defendants in similar situations, overestimating the probability of recidivism for the former and underestimating it for the latter. As Compas was employed by judges to support decisions on bail, parole, and sentencing, this bias had a direct impact on the freedom and rights of individuals, perpetuating inequalities in the criminal justice system. The analysis revealed not only technical flaws in the model and the choice of variables, but also a lack of transparency—as the algorithm was proprietary and could not be publicly audited. The case became emblematic in the debate on ethics in artificial intelligence in public safety, highlighting the need for independent audits, model explainability, and constant supervision to prevent systemic discrimination (Angwin et al., 2016).
AI in justice
In the legal field, the dilemmas take on another dimension: the preservation of impartiality, autonomy, and procedural guarantees. In addition to health and safety, sectors such as the legal field also face ethical dilemmas in the use of AI. An emblematic example of artificial intelligence use is the Chinese system known as Smart Court, implemented in some provinces since 2017. Integrating natural language processing, data analysis, and voice recognition, the platform assists judges in drafting sentences, screening cases, and managing digital evidence. Although it offers gains in speed and standardization, the system raises concerns about the autonomy of judicial decision-making and the possibility of reinforcing biases already present in the history of judgments.
Critics point out that, by training the algorithm with data from past decisions, there is a risk of perpetuating discriminatory patterns or restrictive interpretations of the law. Furthermore, the opacity of the criteria used and the absence of robust external auditing mechanisms make it difficult to challenge decisions suggested or influenced by AI. This case illustrates that, in the Justice system, the challenge is not merely technical, but involves the preservation of procedural guarantees and the maintenance of human control over the final decision (Wang et al., 2021).
Responsible use
The ethical and responsible adoption of AI requires not only general principles but also concrete practices that ensure the reconciliation between technological innovation and social protection. International experiences demonstrate that the indiscriminate or poorly regulated use of AI can generate significant impacts on fundamental rights, reinforcing inequalities and undermining public trust. Therefore, organizations such as Unesco, the OECD, and the European Union have published governance frameworks that emphasize values such as transparency, fairness, security, privacy, and accountability. These guidelines are based on the assumption that technology should be designed in a human-centered manner, ensuring that critical decisions remain under effective human control and that any negative impact is identified and mitigated in a timely manner. Data protection and cybersecurity, in turn, must be incorporated from the design stage (privacy by design and security by design), preventing them from being mere afterthoughts in response to incidents.
In this context, detailed guidelines are fundamental to operationalize these principles. Transparency implies that both decision criteria and AI limitations are clearly communicated, allowing for external auditing and social participation. Accountability demands that developers, suppliers, and users assume specific obligations, with mechanisms to assign and collect responsibilities in case of failures or abuses. Non-discrimination requires the detection and mitigation of biases in data and algorithms, especially in applications affecting rights, such as health, justice, and public safety.
Continuous human oversight is essential to reverse unfair or incorrect automated decisions, preserving due process. Furthermore, digital education and algorithmic literacy need to be fostered so that citizens, professionals, and authorities understand the functioning, limitations, and risks of AI, enabling them to critically interact with these tools. Finally, the security and resilience of systems must be prioritized, ensuring robustness against cyberattacks and operational failures, so that social trust in technology is preserved. Some suggestions can help guide the ethical and responsible use of artificial intelligence:
- Transparency and explainability: making decision logic and system limitations visible;
- Effective human oversight: keep people able to intervene and reverse automated decisions;
- Periodic independent audits: verify biases, performance, and regulatory compliance;
- Clear accountability: assign duties and consequences to all actors involved in the AI lifecycle;
- Data protection and security by design (by design): prevent leaks and attacks;
- Active bias mitigation: ensuring vulnerable groups are not harmed;
- Digital education and algorithmic literacy: empowering society to understand and question technology;
- Operational robustness and resilience: ensure that systems function reliably, even in adverse scenarios.
AI represents a watershed moment in global technological evolution, with the potential to radically transform sectors such as health, security, justice, education, and economy. Its ability to process large volumes of data, identify complex patterns, and offer real-time solutions already translates into significant gains in efficiency, precision, and quality. However, the same technology that can generate unprecedented advances also carries the risk of widening structural inequalities, compromising fundamental rights, and causing social and economic impacts of great magnitude, especially when used indiscriminately or without adequate regulation.
For AI to be an ally of progress and not a vector of exclusion or injustice, it is essential that its development and application be anchored in a solid ethical commitment, with clear rules, independent oversight mechanisms, and participatory governance. This implies creating institutional structures capable of auditing algorithms, preventing biases, ensuring data protection, and guaranteeing that decision-making ultimately remains under human control. Active societal participation—through public consultation, digital education, and open debate—is fundamental for technological solutions to reflect democratic values and serve the collective interest. Only through this combination of responsible innovation, continuous supervision, and social involvement will it be possible to equitably and sustainably reap the benefits of this technological revolution. The future of AI will not be defined solely by its technical capabilities, but by how societies, governments, and companies choose to use it, balancing efficiency and fairness, speed and prudence, transformative potential and ethical responsibility.
| To access the references of this text click here. |
Who wrote this column
Renato Máximo Sátiro
