Article

October 02, 2026

Time series predictive modeling for filter maintenance in water treatment plants

Predictive Modeling by Time Series for Filter Maintenance in Water Treatment Plants

Eduardo Antonio Ribeiro Heinisch; Gustavo Nazato Furlan

DOI: 10.22167/2675-6528-202602860

Article derived from a Course Conclusion Work (TCC), with content based on the student’s original work and adapted to the editorial format of the E&S Magazine with the support of the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege for textual synthesis and organization.

Summary

The maintenance of filters in water treatment plants (WTPs) is crucial for water quality, and anticipating turbidity peaks is fundamental to optimizing backwashes. The research proposed the application of predictive algorithms to anticipate these peaks in filtered water, providing decision-making support for filter maintenance. A robust historical database (2009 to 2025) of operational data from the WTP’s industrial management corporate system and rainfall data from INMET was used. The study compared models based on trees (Random Forest and XGBoost) and recurrent neural networks (LSTM). After preprocessing and feature engineering, which consisted of creating temporal lags and moving averages, the results demonstrated that the LSTM model showed a greater capacity for time series generalization. From a practical filter operation standpoint, by converting regression predictions into binary classification alerts, adhering to the 0.30 NTU limit, the model demonstrated a 55% preventive anticipation capacity of critical events. It was concluded that the data-driven approach proved viable and acted as a proactive tool, capable of ensuring water treatment quality and assisting operators in minimizing waste from unnecessary backwashes.

Keywords: Deep Learning; LSTM; Machine Learning; Water Treatment; Turbidity.

1. Introduction

With the advancement of technology and the growing availability of data, the field of Data Science has evolved at a rapid pace, consolidating itself as a promising solution for organizations seeking to make predictions about the future based on past data (Gujarati and Porter, 2011; Montgomery and Runger, 2016). In this scenario, the high capacity for data storage, generation, and availability, characterized by the dimensions of volume, velocity, variety, variability, and complexity, has driven Machine Learning as a tool capable of learning through the association of different data and pattern recognition (Géron, 2021; Hastie et al., 2009).

The application of this predictive context in the sanitation sector, focusing on predicting the maintenance of filters in water treatment plants (WTPs), is of utmost importance. The quality, efficiency, and frequency of filter washing in a WTP are crucial factors that directly impact the quality of water supplied to the population and operational costs (Libânio, 2010; Sarmento, 2023; Abraão, 2025). Optimizing this stage means significantly reducing the waste of treated water, ensuring the safety and efficiency of the treatment (Libânio, 2010; Batista, 2024; Maciel et al., 2024).

However, filter washing can consume a significant volume of the total production of a WTP. Inefficient washing, whether due to excessive time, inadequate frequency, or improper technique, represents a great waste of potable water and, consequently, a financial and environmental loss (Libânio, 2010; Freires, 2023; Nominato, 2024). The unpredictability of the ideal washing moment is aggravated by seasonal variations in raw water quality, especially in Brazil, where the rainfall regime directly and immediately influences surface water sources. During periods of intense rain, the transport of solid particles drastically increases turbidity, while in dry months the values tend to stabilize at low levels (Libânio, 2010).

This cyclical and high-variance behavior imposes continuous challenges on the operation of Water Treatment Plants (ETAs) (Libânio, 2010; AESBE, 2023; Instituto Trata Brasil, 2024; Mesquita, 2025). Furthermore, the management of water treatment plants often encounters data governance limitations, with the recording of turbidity measurements and the control of filter run times still relying on manual annotations and human imputations. This operational dependence makes the process vulnerable to typing errors, reading failures, and, mainly, human bias, where the decision to wash a filter may be based more on the operator’s subjective interpretation than on the actual water condition. In this context, data-driven automation emerges as a fundamental mechanism to mitigate human errors and standardize decision-making.

To mitigate the complexity imposed by these climatic, operational, and data governance variations, the transition from empirical and purely reactive methods to predictive approaches has been consolidated as a critical necessity in the sanitation sector. Recent literature shows that the integration of data-driven analytical models in WTPs not only enhances water security but also provides operational resilience against sudden turbidity peaks (Kamyab et al., 2023; Dalibard et al., 2024; He et al., 2024; Kwarko-Kyei et al., 2025). Given this latent difficulty in predicting filter run time and the need to define the optimal backwash time to avoid waste (Libânio, 2010; BRK Ambiental, 2022), the present work is justified by the application of knowledge and tools acquired in Data Science to solve a real problem. The main objective of this study was to develop and compare the empirical performance of different Machine Learning predictive models — Random Forest, XGBoost, and Long Short-Term Memory (LSTM) — to predict water turbidity and, consequently, the ideal filter maintenance window.

2. Material and Methods

This study was characterized as a quantitative research, with a methodological approach focused on predictive modeling using time series. The main objective was to develop and compare the performance of different Machine Learning algorithms to predict the turbidity of filtered water in water treatment plants (WTPs) and, consequently, identify the ideal window for filter maintenance. The research was based on the proposition that there is a dependency relationship between operational and climatic variables and the need for filter maintenance (Fávero and Belfiore, 2017).

The unit of analysis consisted of historical operational data from a WTP and rainfall data. Data collection covered the period from 2009 to 2025, totaling approximately 500,000 hourly records. The information was extracted from two main sources: the WTP’s corporate industrial management system, which provided turbidity measurements, schedules, washing time, and water consumption volume, and stations from the National Institute of Meteorology (INMET, 2026), located in the same catchment basin, which provided daily precipitation data.

The study’s development was implemented in the Python language, using consolidated libraries for data analysis and Machine Learning. Among the tools employed, Pandas and NumPy stand out for data manipulation, Scikit-Learn for preprocessing and modeling, and TensorFlow/Keras for building neural networks (Fávero and Belfiore, 2017; McKinney, 2018; Géron, 2021).

The preprocessing stage of pluviometric data involved the identification and correction of anomalies, such as recording failures or corrupted values, through the linear interpolation technique. Subsequently, the two databases, operational and meteorological, were unified using date and time as a temporal index, ensuring information synchronization.

In the feature engineering phase, temporal variables were created to capture the inertia of the hydrological system. Lag variables were included for precipitation, measuring rainfall that occurred in intervals of 1 hour, 2 hours, and up to 48 hours prior. Rolling windows were also generated for the sum of accumulated precipitation over multiple periods, such as 12, 24, 48, and 72 hours (Libânio, 2010; Kuhn and Johnson, 2019).

Additionally, to allow the algorithms to identify cyclical patterns, seasonality extraction was performed. The temporal index was broken down into independent columns for ‘Hour’ and ‘Month’. The predictive target variable, water turbidity, was temporally shifted by 6 hours (“lead time”) so that the model could make estimates with sufficient advance for operational decision-making.

Outlier treatment was performed to correct data distortions, which could arise from typing errors or reading failures. Negative turbidity readings were adjusted to the minimum physically admissible. For anomalous values exceeding the upper limit, the capping/winsorizing technique based on the Interquartile Range (IQR) method was applied, preserving the continuity of the time series (Fávero and Belfiore, 2017; McKinney, 2018).

For model validation, the dataset was chronologically split, with the first 80% allocated for training and the final 20% reserved for testing, preventing data leakage. The predictor variables were normalized using StandardScaler for tree-based models and MinMaxScaler for the LSTM network, with standardization applied exclusively to the training data (Baouab and Cherif, 2017).

Four algorithms were selected for comparison: Random Forest, XGBoost, and Long Short-Term Memory (LSTM) as predictive models, and SHAP (SHapley Additive exPlanations) for interpretability. The Random Forest was configured with 100 decision trees and a random state (random_state=42). The XGBoost used 100 sequential trees, a learning rate of 0.1, and a random state (random_state=42). The LSTM network was built with a layer of 50 neurons, ReLU activation, 20% Dropout, 24-hour lookback, Adam optimizer, and Early Stopping with 5 epochs (Géron, 2021; Goodfellow et al., 2016; Chollet, 2017).

The validation protocol was structured in two phases. In Phase 1, a continuous predictive evaluation (regression) was performed, where the models were compared using statistical metrics such as Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and Coefficient of Determination (R²). In Phase 2, an operational decision evaluation (binary classification) was carried out, converting regression predictions into binary classification alerts, with a cutoff threshold of 0.30 NTU for critical events. The evaluation of this phase used the confusion matrix, focusing on the Recall metric, to minimize false negatives (Libânio, 2010; Kuhn and Johnson, 2019; Géron, 2021).

3. Results and Discussion

Exploratory data analysis (EDA) revealed crucial patterns and interrelationships between variables before applying predictive algorithms. The seasonality of turbidity in Brazilian water sources, for example, showed a strong correlation with the rainfall regime. It was observed that during months of intense rainfall, typically from November to March, turbidity rises drastically due to the transport of solid particles. In contrast, during dry months, from May to September, turbidity levels tend to stabilize at lower levels, as previously identified by Libânio (2010).

The evaluation of Pearson’s linear correlation between the variables, including temporal lags and accumulated precipitation variables, indicated that, although the direct linear correlation with final turbidity was low, the hydrological system’s inertia and input turbidity are important immediate predictors. Cross-correlation analysis was fundamental to quantitatively validate the watershed’s response time to precipitation. An ideal “lag” was identified, where accumulated rainfall reaches its maximum correlation with turbidity at the WTP, reinforcing the need to incorporate temporal windows in modeling to capture this delayed effect (Libânio, 2010; Kuhn and Johnson, 2019).

The importance of variables (Feature Importance) was extracted using the Random Forest algorithm, confirming the data relevance raised in the preprocessing. Variables related to the duration and volume of the last wash, the turbidity of the result and the filter turbidity, along with accumulated precipitation in different time windows, showed the highest importance indices. This finding corroborates previous studies, such as that of Kwarko-Kyei et al. (2025), which highlight water quality in the stage immediately before filtration as the attribute with the greatest weight in reducing predictive error.

The predictive modeling stage compared the performance of three algorithms: Random Forest, XGBoost, and Long Short-Term Memory (LSTM). The tree-based models, Random Forest and XGBoost, encountered difficulties in handling the predictability of continuous turbidity, resulting in negative determination coefficients R² of -0.313333 and -0.252568, respectively. These values indicate that these models failed to explain the variability of the time series satisfactorily, treating turbidity peaks as isolated noises rather than continuous patterns (Hewamalage et al., 2021; Géron, 2021).

In contrast, the LSTM model demonstrated superior and more stable performance, with a Mean Absolute Error (MAE) of 0.062854, a Root Mean Squared Error (RMSE) of 0.080221, and a positive R² of 0.152867. Although the R² is still modest, reflecting the complexity and stochasticity of the hydrological series, the LSTM was the only model capable of capturing temporal patterns and the directional dynamics of turbidity, successfully anticipating upward and downward movements. This generalization capability is crucial for dynamic time series, as pointed out by He et al. (2024).

The architecture of the LSTM model was configured with a layer of 50 neurons and ReLU activation, followed by a 20% Dropout layer to prevent overfitting, a common problem in time series modeling (Goodfellow et al., 2016). Training used the Adam optimizer to minimize the Mean Squared Error (MSE), and Early Stopping was implemented to halt the process if the validation loss did not improve for five consecutive epochs, ensuring the restoration of the best model weights (Chollet, 2017).

The convergence of the mean squared error (MSE) during the LSTM model training demonstrated the neural network’s stability. It was observed that both training and validation errors stabilized jointly, which proves the effectiveness of Dropout and Early Stopping techniques in preventing overfitting. This behavior is an indication that the model learned the data patterns robustly, without memorizing noise specific to the training set.

For practical application purposes in the operation of the Water Treatment Plant (WTP), the study’s main contribution was directed towards the identification of “critical events”, rather than the exact prediction of continuous turbidity. Predicting the exact turbidity value at each instant proved less relevant than the system’s ability to detect, in advance, moments of risk. Thus, the continuous regression model was adapted for a binary classification problem, aiming to generate operational alerts for filter maintenance.

The business rule established for binary classification was as follows: if the predicted turbidity exceeded the threshold of 0.30 NTU, an intervention alert (filter washing) should be issued. Predictions with a probability above 50% generated a maintenance/washing “Alert”. This approach aims to act preventively, minimizing false negatives, i.e., the failure to predict high turbidity, which is more critical for water treatment safety than false positives (Libânio, 2010; Kuhn and Johnson, 2019; Géron, 2021).

The evaluation of the binary classifier, using the confusion matrix, revealed a Global Accuracy of 77%, indicating the model’s total hit rate. The Sensitivity (Recall) metric, which measures the ability to detect real turbidity peaks (critical events), reached 55%. This value demonstrates that the model acts as a conservative barrier, capable of anticipating the event and alerting the operator most of the times when the filter presents a real risk, prioritizing the safety of treated water (Hastie et al., 2009; Fávero and Belfiore, 2017).

The model’s precision was 60%, indicating that, of the times the model issued a “Warning” of backwash, the turbidity actually rose in 60% of the cases. Although precision is not as high as accuracy, the operational priority in water treatment safety justifies the preference for a higher Recall, even if this implies a slightly higher number of false positives, i.e., preventive backwashes that could be considered unnecessary in a lower-risk scenario.

The ROC (Receiver Operating Characteristic) Curve of the LSTM classifier presented an Area Under the Curve (AUC) of 0.797. This value attests to the model’s general robustness in correctly discriminating between the states of “normal operation” and “high turbidity risk”, regardless of the probabilistic threshold chosen by the operator. Obtaining an AUC score far from the randomness line (AUC = 0.5) on a test set with over 12,000 independent samples significantly rejects the null hypothesis, conferring robust statistical significance (p-value < 0.001) to the findings (Hastie et al., 2009; Fávero and Belfiore, 2017; Géron, 2021).

The interpretability of the LSTM model was addressed through SHAP (SHapley Additive exPlanations) analysis, a technique that allows quantifying the influence of each variable on the prediction of future turbidity. Unlike the static importance of Random Forest, SHAP provided a global importance ranking for the LSTM network, highlighting `tuf_resultado`, `Hour`, `Month`, `precipitacao`, and `ultima_lavagem_volume` as the most crucial variables for accurately capturing the dynamic water oscillations (Kuhn and Johnson, 2019; Maußner et al., 2025).

SHAP analysis also allowed visualizing the directional impact of variables on turbidity prediction. Red dots in the swarm plot indicated that high values of a variable pushed the turbidity prediction upwards (increasing the risk), while blue dots indicated that low values pushed the prediction downwards. This transparency, characteristic of Explainable AI, increases confidence in the tool, as it proves that the LSTM algorithm’s predictions are based on the real physics of the filtration process, and not on “guesses” (Dalibard et al., 2024; Kamyab et al., 2023; Maußner et al., 2025).

To consolidate the practical utility of the predictive model, a Python function was developed that simulates a decision support tool for the operator. This function allows the manual insertion of a punctual turbidity reading, which is then integrated into the recent historical context of the WTP (rainfall, previous washes, and system inertia) to calculate and display the immediate probability of a critical event, recommending the appropriate action. This tool aims to mitigate human errors and standardize decision-making, elevating the plant’s technological maturity.

In summary, the results demonstrated that the application of predictive algorithms, especially the LSTM model, is viable for anticipating turbidity peaks in filtered water at treatment plants. Although continuous regression has limitations in explaining the entirety of the variability, the conversion to a binary operational alert classifier allowed the model to act as a proactive tool, with a 55% preventive anticipation capacity of critical events. This ensures the quality of water treatment and assists operators in optimizing filter maintenance, minimizing waste from unnecessary backwashes, thus responding to the study’s main objective.

4. Conclusion

This study aimed to develop and compare the performance of Machine Learning predictive models to anticipate water turbidity and optimize filter maintenance in treatment plants. It was found that the non-stationary nature of hydrological time series imposed significant challenges on the algorithms. Tree-based models, such as Random Forest and XGBoost, showed notable limitations, with negative R² coefficients of determination, indicating difficulty in capturing the variability and continuous patterns of turbidity. In contrast, the Long Short-Term Memory (LSTM) model demonstrated superior generalization capability, with a positive R² and more stable error metrics, successfully capturing the directional dynamics of turbidity. The practical application of the model was consolidated by converting its regression predictions into binary classification alerts, establishing a threshold of 0.30 NTU for critical events. This approach resulted in a sensitivity (Recall) of 55% in anticipating these events, prioritizing operational safety. SHAP analysis complemented the findings, providing transparency to the LSTM model and confirming that its predictions are based on relevant operational and environmental attributes, such as previous turbidity, hour, month, and accumulated precipitation.

The study’s main contribution lies in demonstrating the feasibility of a data-driven predictive approach as a proactive tool for water treatment plant management. By providing decision support for filter maintenance, the model assists operators in standardizing procedures, mitigating human bias, and minimizing the waste of potable water associated with unnecessary backwashes, thereby enhancing the plant’s technological maturity. However, it is acknowledged that the modest R² in the continuous regression reflects the inherent complexity and stochasticity of hydrological series. For future studies, it is suggested to optimize the hyperparameters of the LSTM architecture, include additional real-time variables such as pH and temperature, and stratify the database for developing specific models for different filter categories and typologies, aiming to further refine accuracy and reduce false alarm rates.

Bibliographic References

Abraão, R.P. 2025. Análise de viabilidade do uso de coagulantes naturais no tratamento de águas de abastecimento público. Dissertação de

Gujarati, D.N.; Porter, D.C. 2011. Econometria Básica. 5ed. AMGH Editora, Porto Alegre, RS, Brasil.

Géron, A. 2021. Mão à Obra: Aprendizado de Máquina com Scikit-Learn, Keras & TensorFlow. 2ed. Alta Books, Rio de Janeiro, RJ, Brasil.

Hastie, T.; Tibshirani, R.; Friedman, J. 2009. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. 2ed. Springer, New York, NY, EUA.

Libânio, M. 2010. Fundamentos de Qualidade e Tratamento de Água. 3ed. Editora Átomo, Campinas, SP, Brasil.

Montgomery, D.C.; Runger, G.C. 2016. Estatística Aplicada e Probabilidade para Engenheiros. 6ed. LTC, Rio de Janeiro, RJ, Brasil.

Sarmento, P.V.F. 2023. Avaliação do sistema de lavagem dos filtros de uma ETA de filtração direta descendente. Trabalho de Conclusão de Curso em Engenharia Civil. Universidade Federal do Rio Grande do Norte, Natal, RN, Brasil.

Article originating from the Final Course Project of the Specialization in Data Science and Analytics of the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform

You may also like

Economy, Investments And Banking

October 23, 2026

Financial Education, Behaviors and Biases in Financial Decision Making

Personal financial management represents a growing challenge, influenced by cognitive biases that affect decision-making, even in individuals with high knowledge. The present study analyzed the relationship between the level of financial education and the occurrence of behavioral biases in the financial decision-making of adults residing in Brazil. An applied, quantitative, and descriptive-correlational field survey was conducted by administering a structured questionnaire to 163 respondents. The instrument included sociodemographic variables, financial knowledge and habits, and hypothetical situations to identify cognitive biases. The results indicated that higher levels of financial education were associated with better financial control, saving, and investment practices, with significant correlations with income and schooling. However, the presence of behavioral biases (anchoring, loss aversion, scarcity, optimism/negativity, present bias, status quo, overconfidence, and mental accounting) persisted even among individuals with higher schooling and income, with no statistically significant association with the analyzed sociodemographic variables. It was concluded that financial education, although essential for financial organization and control, was not sufficient to neutralize the influence of cognitive biases. The incorporation of behavioral approaches into financial education programs is suggested.

Keywords: Financial behavior; Behavioral economics; Personal finance; Cognitive heuristics; Cognitive biases.

October 22, 2026

Consumer Journey in the Decision to Purchase Infoproducts in the Digital Environment

The expansion of e-commerce and technological transformation have driven the need to understand consumer behavior regarding infoproducts, whose perceived experience has become crucial for professional development. The study aimed to analyze the factors that influenced the purchase decision of infoproducts in the digital environment, considering the stages of the consumer journey. An applied, quantitative, and descriptive research was conducted through a field survey with 100 consumers from Serra Gaúcha. Data collection occurred via a structured questionnaire, using non-probabilistic sampling. The results revealed a qualified audience with strong “mobile-first” behavior. It was found that the availability of detailed information and social approval were the main attraction triggers in the pre-purchase phase. In the transactional and subsequent stages, the minimization of operational friction, the agility of immediate access, and system security proved crucial for conversion and satisfaction, although gaps in human support efficiency were identified. Retention was based on the trust generated by positive past experiences, overcoming the price factor. It was concluded that the infoproduct consumer journey operates as an integrated system, requiring alignment between technological infrastructure, content curation, and support humanization to ensure engagement.

Keywords: E-commerce; Consumer behavior; Consumer experience; Loyalty; Virtual business.

October 22, 2026

Forecast of Aggregate Default in the Brazilian Financial System by Sarimax Model with Macroeconomic Variables

Aggregate non-payment in the Brazilian financial system is a crucial indicator of credit risk, the prediction of which is regulated by CMN Resolutions n.° 4.557/2017 and 4.966/2021 for the prospective measurement of expected losses. Aggregate non-payment was modeled and predicted using the SARIMAX model, projecting its trajectory until December 2026 under optimistic, base, and pessimistic scenarios. 164 monthly observations (July/2012 to February/2026) and macroeconomic variables such as unemployment, Selic, IPCA, IBC-Br, and household debt were used. The specification with contemporaneous variables was selected based on information criteria and coefficient stability, and residual diagnostics confirmed statistical adequacy, with a MAPE of 3.12%. Historical non-payment ranged between 4.33% and 7.64%. The estimation identified unemployment (β = 0.2534; p < 0.001) and Selic (β = 0.2128; p < 0.001) as the main positive determinants, with strong temporal persistence (AR(1) = 0.8965). Selic showed increasing correlation with non-payment (r = 0.62 at lag 12), while unemployment demonstrated a contemporaneous effect (r = 0.33). Projections indicated non-payment between 6.30% and 7.98% in December 2026, with a range of 1.68 percentage points, highlighting direct implications for provisioning in financial institutions.

Keywords: Unemployment; Predictive modeling; Monetary policy; Loss provisioning; Time series.