Orchestration of weather forecasts via API in layered architecture: paulista municipalities of agricultural relevance

doi:10.22167/2675-6528-202602686

Article

September 30, 2026

Orchestration of weather forecasts via API in layered architecture: paulista municipalities of agricultural relevance

Orchestration of Weather Forecasts Via API in Layered Architecture: Paulista Municipalities of Agricultural Relevance

Arnaldo Patrício Pinhoni Silva; Jorge Costa Silva Filho

DOI: 10.22167/2675-6528-202602686

Article derived from a Course Conclusion Work (TCC), with content based on the student’s original work and adapted to the editorial format of the E&S Magazine with the support of the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege for textual synthesis and organization.

Summary

An automated data pipeline for weather forecasts, obtained via Open-Meteo, was developed and evaluated in five São Paulo municipalities of agricultural relevance, from March 14 to April 01, 2026. The solution was structured on the Extract, Load, Transform (ELT) approach, using PostgreSQL for raw storage, dbt for transformations, and Apache Airflow for daily orchestration, in a containerized environment with Docker and dependency management by uv. The analytical architecture was organized into bronze, silver, and gold layers, preserving raw data, structuring hourly records, and generating daily aggregations. The model maintained the history of forecast rounds and deduplicated views for meteorological characterization. 17 rounds per city were preserved, totaling 14,280 hourly records and 595 daily aggregations. The comparison between rounds revealed variations of up to 3.02 °C in average temperature and 25.7 mm in precipitation for the same day and location. The validation of the deduplicated base against ERA5 reanalysis indicated a mean absolute error of 0.56 °C in daily average temperature and 2.38 mm in daily precipitation, with distinct performance between variables. The results demonstrated the viability of the approach in generating versioned time series, with traceability and automated quality tests, and provided a basis for future studies.

Keywords: Apache Airflow; Data automation; dbt; Data engineering.

1. Introduction

Brazilian agriculture plays a strategic role in the national economy, showing high sensitivity to climate variability. Extreme events, irregular precipitation, and thermal oscillations directly affect the productivity of regionally relevant crops, especially in the state of São Paulo. Contemporary agriculture faces growing hydrometeorological uncertainties, demanding analytical solutions capable of transforming raw data into useful operational information. Consistently organized meteorological forecasts, in regional contexts, can support environmental monitoring, risk tracking, and operational planning, becoming a critical input for decision-making in the field (IPCC, 2023; El Hachimi et al., 2022).

Despite the growing availability of meteorological data through public application programming interfaces (APIs), its effective analytical exploitation is not guaranteed. The utility of information depends on attributes such as consistency, adequate representation, and fitness for use, dimensions recognized as central in the data quality literature in environments of high volume and high ingestion velocity (Taleb et al., 2021). The absence of structured validation and transformation processes compromises the reliability of the derived analyses, making it necessary to adopt architectures that ensure quality from collection to analytical consumption.

In the specific scenario of weather forecasts obtained via APIs, additional challenges arise. These include the recurring consumption of semi-structured data, temporal standardization, the selection of pertinent variables, and the conversion of JSON responses into reusable tabular structures. The Open-Meteo platform, for example, provides hourly forecasts parameterizable by geographic coordinates, weather variables, and time zone, which favors programmatic integration but requires technical organization to convert raw data into a consistent analytical base (Open-Meteo, n.d.).

To face these challenges, layered architectures have been widely employed in data engineering. They serve as a strategy to organize data evolution, from its raw form to analytical consumption. Recent literature, addressing concepts such as lakehouse and medallion architecture, describes a progressive refinement flow in bronze, silver, and gold layers. In this model, the initial layer preserves the source data, the intermediate layer performs cleaning and structuring, and the final layer consolidates data sets oriented towards analytical consumption (Armbrust et al., 2021; Microsoft, n.d.a). Tools like Apache Airflow and dbt strengthen this design, enabling declarative workflow definition, SQL transformations, and automated quality tests (Apache Airflow, n.d.; dbt Labs, n.d.).

In Brazil, the adoption of data engineering practices for agricultural monitoring is still recent, especially regarding the automation of ingestion flows and the versioning of time series of climate forecasts. The structuring of reproducible and auditable pipelines for this purpose represents a relevant contribution. This applies both from a technical point of view and from the perspective of information management in agribusiness, by enabling comparative analyses between forecast rounds and tracking data quality over time (Taleb et al., 2021).

Given this context and the need for robust systems for managing meteorological data in agribusiness, this work aimed to develop and evaluate an automated pipeline for ingestion, transformation, validation, and analytical organization of meteorological forecasts obtained via public API, structured in a layered architecture and applied to five paulista municipalities of agricultural relevance.

2. Material and Methods

This research, of an applied nature, aimed at the development and evaluation of an automated system for the ingestion, transformation, and analytical organization of meteorological forecasts. It was classified as descriptive in terms of objectives, seeking to characterize the functioning of the computational artifact without manipulating variables or establishing causal relationships (Gil, 2022). The methodological approach was quantitative, with results expressed in measurable numerical variables, such as temperature and precipitation (Creswell and Creswell, 2021). The design was configured as an applied case study, focused on five municipalities in São Paulo of agricultural relevance.

The spatial delimitation of the study comprised the municipalities of Ribeirão Preto, Piracicaba, Campinas, São José do Rio Preto, and Presidente Prudente, in the state of São Paulo. The selection of these locations was based on their notable socioeconomic relevance and significant regional insertion in agribusiness, as evidenced by indicators of Gross Domestic Product (GDP) and Gross Value Added (GVA) from agriculture (IBGE, n.d.). The monitoring and data collection period extended from March 14 to April 01, 2026, with forecasts extending to April 07 of the same year.

The data used were secondary, obtained exclusively from the public API of the Open-Meteo platform, without the need for direct field collection or proprietary instrumentation (Marconi and Lakatos, 2022). The weather forecasting service was accessed through the /v1/forecast endpoint (Open-Meteo, n.d.), which provides hourly forecasts parameterizable by geographic coordinates, meteorological variables, and time zone. The forecast horizon was maintained at seven days, resulting in 168 hourly values per request and per municipality.

API queries were performed in hourly resolution, with JSON format return. Collected variables included air temperature at 2 m, air relative humidity at 2 m, precipitation, dew point at 2 m, shortwave solar radiation, wind speed at 10 m, vapor pressure deficit, and reference evapotranspiration (ETo). ETo was obtained directly from the API, calculated based on the FAO-56 standard (Valle Júnior et al., 2021; Open-Meteo, n.d.).

The work development followed four sequential stages. Initially, the problem was identified and the scope of the solution for the ingestion and analytical organization of weather forecasts was defined. Next, the data pipeline was designed and developed in a layered architecture. The third stage consisted of the system’s operation, accumulating 17 ingestion rounds. Finally, the artifact was evaluated through quantitative metrics of volume, integrity, divergence between rounds, and accuracy (Munappy et al., 2020).

The operationalization of data collection adopted a metadata-driven pipeline, using an external configuration via the `OPEN_METEO_CITIES_JSON` variable. This strategy allowed for the expansion or replacement of interest locations without direct changes to the source code, conferring scalability and technical maturity to the solution (Microsoft, n.d.b). Such an approach mitigated structural rigidity and favored process reuse, ensuring low coupling between ingestion logic and parameter definitions.

Data modeling adopted the Extract, Load, Transform (ELT) approach, structured in bronze, silver, and gold layers, aligning with the medallion architecture and lakehouse references (Armbrust et al., 2021; Microsoft, n.d.a). The bronze layer preserved the raw JSON responses from Open-Meteo entirely, along with the temporal ingestion record, fulfilling an audit and traceability function without semantic transformations.

In the silver layer, the raw responses were expanded into hourly atomic records by city, ingestion round, and hour, through JSON array denormalization and conversion to explicitly typed columns. The models were materialized as views on top of PostgreSQL. A historical view kept all rounds, while a current view derived from the historical for operational use, and a deduplicated view retained a single forecast per municipality and target date (Firmani et al., 2016; Merino et al., 2016).

The gold layer consolidated hourly data into daily aggregations by city, ingestion round, and day, producing nine meteorological indicators. These included average, maximum, and minimum daily temperature, total precipitation, average dew point, average shortwave solar radiation, average 10 m wind speed, average vapor pressure deficit, and accumulated reference evapotranspiration, calculated by the FAO-56 method (Ruiz-Ortega et al., 2024). Historical and current views were maintained for different analytical purposes.

The analytical transformations and data modeling were executed through the dbt (data build tool) framework, employing SQL language. Automated quality tests were implemented in dbt, verifying null values and uniqueness of records by city and temporal record after each ingestion round (Firmani et al., 2016; Merino et al., 2016). This procedure ensured reliability in continuous processing contexts.

The entire operational flow was orchestrated by Apache Airflow 2.9.1, executed in LocalExecutor mode on Python 3.10, through a DAG (Directed Acyclic Graph) with daily scheduling (Apache Airflow, n.d.). The DAG coordinated the sequential execution of ingestion tasks in the bronze layer (`bronze_ingest`), transformation via dbt (`dbt_run`), and quality test validation (`dbt_test`). Docker containerization ensured environment portability and consistency.

Python dependency management was performed with `uv` and fixed by the lock file `uv.lock`, ensuring the computational reproducibility of the environment (Alston & Rick, 2021). The DAG was configured with automatic retry in case of failure, with a five-minute interval, and the `catchup` parameter was kept disabled, so that runs not performed on the scheduled date were not recovered later (Munappy et al., 2020).

To evaluate the accuracy of the stored predictions, the deduplicated dataset was compared with the ERA5 reanalysis, produced by the European Centre for Medium-Range Weather Forecasts (Hersbach et al., 2020). The reanalysis was accessed through the Open-Meteo archive service for the same coordinates and dates. The comparison was conducted on a daily basis for average temperature and accumulated precipitation, employing bias, mean absolute error (MAE), and root mean square error (RMSE) as verification measures (Jolliffe and Stephenson, 2012).

The pipeline’s source code is available in a public repository under an MIT license, accessible at `https://github.com/arnaldopinhoni/tcc-climate-data-pipeline`. The repository includes the dbt models for the silver and gold layers, automated quality tests, the queries that reproduce the presented tables, and the validation script against the ERA5 reanalysis. For environment recreation, the container composition file, the dependency lock file, and the dbt environment variable and connection profile configuration models are versioned.

3. Results and Discussion

The automated pipeline for ingestion, transformation, validation, and analytical organization of weather forecasts operated as expected, covering the period from March 14 to April 01, 2026. During this 19-day interval, 17 ingestion rounds per city were accumulated, distributed over 14 distinct days, with forecasts extending to April 07, 2026. The historical silver layer consolidated a total of 14,280 hourly records, while the historical gold layer generated 595 daily aggregations, of which 125 composed the deduplicated base used for meteorological characterization. The uniformity in record count across the five monitored municipalities demonstrated that all initiated rounds were fully processed by the subsequent layers, without any partial data loss, confirming the robustness of the flow.

Adequacy of the layered architecture

The layered architecture, composed of bronze, silver, and gold, proved highly effective for the ingestion and transformation of weather forecasts, aligning with the references of medallion architecture and lakehouse (Armbrust et al., 2021; Microsoft, n.d.a). The bronze layer acted as a faithful repository of raw JSON responses from the Open-Meteo API, preserving entirely 85 raw records, corresponding to 17 rounds for each of the five cities, with a total volume of approximately 385 kB. This approach ensured complete traceability and the ability to reprocess from the original data, a significant advantage over methods that transform data directly upon ingestion (Armbrust et al., 2021).

In the silver layer, the raw responses were expanded into 14,280 explicitly typed hourly records per municipality, ingestion round, and hour, with no variations or gaps between locations. This relational structuring is crucial for identifying patterns and trends (Firmani et al., 2016; Merino et al., 2016). Maintaining historical, current, and deduplicated views in the silver layer, with 14,280, 840, and 3,000 records respectively, allowed for different analytical purposes to be met. The gold layer, in turn, consolidated the hourly data into 595 daily aggregations, producing nine essential meteorological indicators for analytical consumption, such as average temperature, total precipitation, and reference evapotranspiration.

Data quality and completeness

The completeness analysis of records in the silver layer confirmed absolute uniformity among the five municipalities, with each city accumulating 2,856 hourly records. The verification of null values in the eight monitored meteorological variables, including air temperature, relative humidity, and precipitation, revealed complete filling in all analyzed rounds. This result validates the correct parameterization of requests to the Open-Meteo API and the adequacy of the transformation process in the silver layer (Garbanzo León et al., 2025). The absence of gaps is attributed to the API’s reliability, precise parameterization, and automated quality tests implemented in dbt, which interrupt the pipeline in case of integrity failures.

The uniformity in the count of records between layers and municipalities, with 2,856 hourly records in the silver layer and 119 aggregations in the gold layer, demonstrated the operational consistency and integrity of the transformation process. The metadata-driven approach, which parameterized cities via an external configuration variable, eliminated coupling between business logic and operational parameters, allowing for the scalability and maintainability of the solution (Microsoft, n.d.b; Munappy et al., 2020). Full traceability, guaranteed by recording the round identifier in all layers, allows auditing the transformation path of any daily aggregation to the original JSON response, a fundamental requirement for decision support systems in critical domains (Taleb et al., 2021).

Orchestration and computational reproducibility

The orchestration of the operational flow by Apache Airflow 2.9.1, executed in LocalExecutor mode on Python 3.10, ensured the ordered execution of ingestion, transformation, and quality testing tasks without manual intervention in all 17 rounds performed. Docker containerization ensured environment portability and consistency, while uv dependency management strengthened computational reproducibility, aligning with good computational research practices (Alston and Rick, 2021). The automatic retry mechanism configured in the Airflow DAG, with re-execution at five-minute intervals in case of failure, reinforced resilience to transient external API interruptions, an essential requirement in production data pipelines (Munappy et al., 2020).

Despite the system’s robustness, the monitoring period registered 17 ingestion rounds in 19 calendar days, with five days without execution due to operation on a personal workstation and the catchup parameter disabled. This configuration, although suitable for demonstration, implied that executions not performed on the scheduled date were not recovered. In contrast, March 31st concentrated four executions, including three manual re-executions. These observations highlight that automatic retry acts on failures within a scheduled execution but does not replace the continuous availability of the execution environment, which would be achieved on a dedicated server with catchup enabled.

Meteorological characterization of municipalities and agricultural implications

The comparative analysis of meteorological variables, performed on the deduplicated base of 25 dates per municipality, revealed patterns consistent with the regional climatic characterization. Presidente Prudente registered the highest average temperature (25.62 °C) and absolute maximum (34.30 °C), while Campinas presented the lowest values (23.58 °C and 31.40 °C, respectively). The difference of 2.04 °C in average temperature between the extremes is compatible with the thermal gradient expected due to latitude and altitude, with Campinas being the locality of higher altitude. These data, although descriptive of the period, do not constitute cross-verification of the Köppen climate classification, which is based on long-term averages (Alvares et al., 2022).

Regarding the accumulated precipitation forecast for the period, Presidente Prudente (85.4 mm) and Ribeirão Preto (80.1 mm) showed the highest volumes, while Campinas registered the lowest (53.0 mm). The difference of 32.4 mm between the extremes represents a variation of 61.1%, evidencing spatial heterogeneity in precipitation forecasts. Accumulated reference evapotranspiration (ETo) ranged between 101.9 mm in São José do Rio Preto and 107.4 mm in Piracicaba, a range of only 5.5 mm (5.4%). This relative homogeneity in evapotranspirative demand contrasts with the variability of precipitation, suggesting that the water balance of crops in the analyzed period is more sensitive to water supply from rain than to atmospheric demand.

The average vapor pressure deficit (VPD) ranged from 0.846 kPa in Campinas to 0.979 kPa in Presidente Prudente. These values indicate predicted conditions within favorable ranges for plant development in most municipalities, although Presidente Prudente approaches the 1.0 kPa threshold, often associated with atmospheric water stress for sensitive crops (Grossiord et al., 2020). The average daily thermal amplitude ranged from 8.85 °C in Presidente Prudente to 11.38 °C in Piracicaba, a difference of 2.53 °C. This variation is relevant for planning agricultural activities sensitive to temperature fluctuations, such as the application of pesticides and the harvest of perishable products (Antuniassi and Boller, 2019).

Relationships between meteorological variables

Pearson correlation analysis revealed patterns of covariation between meteorological variables. The correlation between solar radiation and reference evapotranspiration (ETo) was very high, ranging between 0.978 and 0.988 in all municipalities. This result is not an empirical finding, but a direct consequence of the formulation of the FAO-56 method, which uses solar radiation as the main input variable for ETo calculation (Valle Júnior et al., 2021; Ruiz-Ortega et al., 2024). The correlation between temperature and ETo ranged between 0.562 and 0.831, indicating that warmer days tend to be associated with a higher evaporative demand, a behavior compatible with the Penman-Monteith formulation.

The correlation between precipitation and relative humidity was low, ranging between 0.079 and 0.173. This result suggests a not very direct relationship between these variables in the analyzed set, which can be explained by the temporal scale of the data and atmospheric dynamics, where precipitation events do not always translate into proportional increases in average humidity (Valle Júnior et al., 2021). These coefficients are descriptive of the analyzed period and should not be treated as stable estimates of the relationship between the variables, given the dispersion among localities and the reduced sample size of 25 days per municipality.

Aggregation and gold layer

The gold layer consolidated hourly records into daily aggregations by city and ingestion round, producing nine daily meteorological indicators, including average, maximum, and minimum temperature, total precipitation, average dew point, shortwave solar radiation, average wind speed, vapor pressure deficit, and reference evapotranspiration by the FAO-56 method. Each municipality presented 119 daily aggregations, resulting from 17 rounds and 25 distinct target dates, with an average of 4.76 forecasts per target date. Average precipitation per aggregation ranged from 2.7 mm in Campinas to 3.3 mm in Piracicaba and Ribeirão Preto, and 2.9 mm in Presidente Prudente and 2.8 mm in São José do Rio Preto.

Distribution of reference evapotranspiration

The distribution of daily ETo by magnitude ranges, calculated on the deduplicated base, revealed that the moderate range (3 to 5 mm per day) concentrated most of the dates in all municipalities, representing between 72% and 88% of the analyzed days. No date reached the very high ETo range (equal to or greater than 7 mm per day). This predominance of the moderate range indicates an intermediate atmospheric demand, favorable for supplemental irrigation management in perennial crops and for the implementation of ratoon crops (Garbanzo León et al., 2025). The availability of this already processed and aggregated variable in the gold layer eliminates the need for intermediate calculations by the end-user, reducing adoption barriers in agricultural decision support systems (El Hachimi et al., 2022).

Occurrence of significant precipitation

The identification of days with predicted precipitation above 5 mm is relevant for planning field operations that require dry soil (Antuniassi and Boller, 2019). Presidente Prudente and Ribeirão Preto presented the highest number of dates with significant precipitation, with 7 days out of 25, while Campinas registered the lowest, with 3 days. The average precipitation on these days ranged between 9.5 mm and 16.0 mm, indicating moderate to heavy intensity events, compatible with the pattern of convective rainfall typical of the transition period between summer and autumn in the region. These values describe what the model anticipated for the period, not observed rainfall.

Variation between forecast rounds and their impacts

The comparison between forecasts issued in different rounds for the same day and locality revealed significant variations, especially in temperature and precipitation. The largest average temperature divergence between rounds was observed in Campinas on March 18, 2026, with a variation of 3.02 °C among the four available rounds. In precipitation, the largest divergence for the entire period was 25.7 mm, recorded in Ribeirão Preto. The average temperature variation between rounds ranged from 0.91 °C to 1.37 °C, with extreme cases reaching 3.02 °C. The average precipitation variation ranged from 4.2 mm in Campinas to 6.0 mm in Ribeirão Preto, with the maximum of 25.7 mm also in Ribeirão Preto.

The case of the forecast for Campinas on March 18, 2026, illustrates the impact of this uncertainty. The forecast collected four days earlier predicted an average temperature of 22.27 °C and no precipitation. The forecast from March 16 revised the prediction to 25.29 °C, a variation of 3.02 °C. Precipitation, predicted as zero until the March 16 forecast, appeared for the first time in the forecast from March 17 (0.2 mm) and was revised to 7.5 mm in the forecast from March 18 itself. This magnitude of successive revisions reinforces the need for climate data management systems that provide not only the current forecast but also indicators of information reliability (El Hachimi et al., 2022).

Accuracy of predictions and usage limits

The validation of the deduplicated database against the ERA5 reanalysis, for 125 comparison pairs on a daily basis, revealed distinct performance among the variables. For the daily average temperature, the ensemble bias was +0.17 °C, the mean absolute error (MAE) was 0.56 °C, and the root mean square error (RMSE) was 0.73 °C, with 85.6% of days showing an error of up to 1.0 °C. The bias per municipality remained between -0.27 °C and +0.63 °C, with no common systematic trend. This performance indicates that the temperature forecast is sufficiently accurate to support operational decisions within the analyzed horizon.

For daily precipitation, the ensemble mean absolute error was 2.38 mm and the root mean square error was 4.43 mm, with only 56.8% of days within 1.0 mm and a maximum error of 20.6 mm on a single day. The bias varied between municipalities, with Campinas underestimating the accumulated amount for the period (53.0 mm predicted against 92.0 mm in reanalysis), while Piracicaba and Presidente Prudente overestimated it (73.7 mm against 40.1 mm and 85.4 mm against 41.8 mm, respectively). This asymmetry is consistent with the literature on numerical forecasting, which points to precipitation as the most difficult variable to predict due to its discontinuous nature (Bauer et al., 2021). Therefore, precipitation forecasts should be treated as an indication of trend, rather than a reliable point value.

Applicability for agricultural planning

The distribution of reference evapotranspiration by magnitude ranges provides direct support for the sizing of irrigation systems and the planning of irrigation shifts. The predominance of the moderate range (3 to 5 mm per day) indicates intermediate atmospheric demand conditions, compatible with supplemental irrigation management in perennial crops. The absence of dates in the very high ETo range (≥7 mm per day) in the analyzed period, corresponding to the end of summer and beginning of autumn, showed no forecast of extreme evapotranspirative demand. The identification of 3 to 7 days with predicted precipitation greater than 5 mm per municipality allows for advance planning of field operations sensitive to soil moisture conditions, such as mechanized harvesting and herbicide application (Antuniassi and Boller, 2019).

The correlation between temperature and reference evapotranspiration, between 0.562 and 0.831, suggests that air temperature can serve as an approximate indicator of evapotranspirative demand for simplified management purposes, although the observed dispersion between locations advises against generalizing this substitution. The spatial heterogeneity observed in precipitation forecasts, with a difference of 32.4 mm between the municipality with the highest and the one with the lowest predicted accumulation, reinforces the need for monitoring systems that operate at a municipal granularity or higher. The developed pipeline met this requirement by allowing the incorporation of new locations through external parameterization, without changes to the source code, ensuring the usefulness of the information for agricultural planning on specific properties.

In summary, the pipeline demonstrated the viability of a layered architecture for managing weather forecasts, with traceability and automated quality tests. The results indicated that the daily average temperature has high accuracy, being useful for operational decisions, while precipitation should be interpreted as an indicator of trend. The expressive variation between forecast rounds and the spatial heterogeneity of precipitation reinforce the importance of systems that preserve the forecast history and operate at a municipal granularity, providing valuable information for agricultural planning.

4. Conclusion

This work aimed to develop and evaluate an automated pipeline for the ingestion, transformation, validation, and analytical organization of meteorological forecasts obtained via a public API, structured in a layered architecture and applied to five São Paulo municipalities of agricultural relevance. It was found that the layered architecture, composed of bronze, silver, and gold, proved effective in the progressive refinement of data, ensuring full traceability and the ability to reprocess from the raw API responses. Orchestration with Apache Airflow and containerization with Docker ensured computational reproducibility and consistency of executions. Automated quality tests implemented in dbt confirmed the completeness and integrity of records in all layers. It was observed that daily average temperature forecasts showed high accuracy, with a mean absolute error of 0.56 °C, while daily precipitation revealed greater uncertainty, with a mean absolute error of 2.38 mm, indicating its use as a trend indicator. The comparative analysis between forecast rounds showed significant variations, up to 3.02 °C in temperature and 25.7 mm in precipitation for the same day and locality, reinforcing the importance of systems that preserve forecast history. The main contribution of the study lies in the viability of a data engineering solution for generating versioned time series of meteorological forecasts, with traceability and automated quality tests, providing a robust foundation for agricultural planning.

As limitations, the monitoring period of 17 rounds in 19 days, although sufficient to demonstrate the system’s operability, proved restrictive for robust seasonal analyses. The discontinuity of the series, with five days without execution due to operation on a personal workstation and the catchup parameter deactivated, was also a limiting factor. Furthermore, validation against the ERA5 reanalysis, which is a modeling product, implies that the reported errors represent the discrepancy between two estimates and not a deviation from direct measurement. For future studies, it is suggested to extend the temporal horizon of data collection, operate the pipeline in a continuous execution environment on a dedicated server, incorporate observed data from meteorological stations for more direct validation, and extend the geographical coverage to other agricultural regions of the country, in order to strengthen the applicability and robustness of the solution.

Bibliographic References

Armbrust, M.; Ghodsi, A.; Xin, R.; Zaharia, M. 2021. Lakehouse: a new generation of open platforms that unify data warehousing and advanced analytics. In: Conference on Innovative Data Systems Research (CIDR), 2021, Online. Anais…

El Hachimi, C.; Belaqziz, S.; Khabba, S.; Sebbar, B.E.; Dhiba, D.; Chehbouni, A. 2022. Smart weather data management based on artificial intelligence and big data analytics for precision agriculture. Agriculture 13(1): 95.

Intergovernmental Panel on Climate Change [IPCC]. 2023. Climate Change 2023: Synthesis Report. IPCC, Geneva, Switzerland.

Microsoft. n.d.a. Medallion architecture. Disponível em: <https://learn.microsoft.com/en-us/azure/databricks/lakehouse/med

Open-Meteo. n.d. Open-Meteo weather forecast API documentation. Disponível em: <https://open-meteo.com/en/docs>. Acesso em: 19 mar. 2026.

Taleb, I.; Serhani, M.A.; Bouhaddioui, C.; Dssouli, R. 2021. Big data quality framework: a holistic approach to continuous quality management. Journal of Big Data 8(1): 76.

Article originating from the Final Course Project of the Specialization in Data Science and Analytics of the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform

Most recent

October 22, 2026

Challenges and Possibilities of the First School Year of Children with ASD for Teaching Work

October 22, 2026

Consumer Journey in the Decision to Purchase Infoproducts in the Digital Environment

October 22, 2026

Psychosocial Risks and Mental Health at Work in a Banking Institution

October 22, 2026

Digital Observatory of the Amazon: Data Science Applied to the Analysis of Social, Economic, Environmental, and Health Indicators

October 22, 2026

Continuous Learning as a Factor of Organizational Growth in the Perception of Collaborators

October 22, 2026

Statistical Regression in Microgrids for Operational Performance Estimation: Campusgrid Case

October 22, 2026

Forecast of Aggregate Default in the Brazilian Financial System by Sarimax Model with Macroeconomic Variables

October 22, 2026

Stakeholder Engagement Factors in International Technology Transfer to a Public Hospital

October 22, 2026

Combating Commercial Electric Energy Losses Using a Time Series Model

The inclusion of children with Autism Spectrum Disorder (ASD) in Early Childhood Education presents significant challenges, given the difficulty of adapting to new environments and the intensity of stimuli. The objective was to analyze the teaching performance in Early Childhood Education in the initial schooling process of children with ASD. A case study was conducted, using narrative interview as a data production technique. The results revealed that the teaching performance was characterized by mediation and affection practices, with the flexibility of routine and the use of playful resources as essential strategies for the child’s development and sensory organization. The importance of the school support professional for the success of the pedagogical work was also highlighted. The child’s integral development was promoted through intentional teaching actions that prioritized environmental adaptation, strengthening of bonds, and attention to the specific needs of ASD. As a contribution, a guide of pedagogical strategies aimed at welcoming children with ASD in public daycare centers was developed.

Keywords: Early Childhood Education; School Inclusion; Teacher Narratives; Inclusive Pedagogical Practices; Autism Spectrum Disorder.

October 22, 2026

Consumer Journey in the Decision to Purchase Infoproducts in the Digital Environment

The expansion of e-commerce and technological transformation have driven the need to understand consumer behavior regarding infoproducts, whose perceived experience has become crucial for professional development. The study aimed to analyze the factors that influenced the purchase decision of infoproducts in the digital environment, considering the stages of the consumer journey. An applied, quantitative, and descriptive research was conducted through a field survey with 100 consumers from Serra Gaúcha. Data collection occurred via a structured questionnaire, using non-probabilistic sampling. The results revealed a qualified audience with strong “mobile-first” behavior. It was found that the availability of detailed information and social approval were the main attraction triggers in the pre-purchase phase. In the transactional and subsequent stages, the minimization of operational friction, the agility of immediate access, and system security proved crucial for conversion and satisfaction, although gaps in human support efficiency were identified. Retention was based on the trust generated by positive past experiences, overcoming the price factor. It was concluded that the infoproduct consumer journey operates as an integrated system, requiring alignment between technological infrastructure, content curation, and support humanization to ensure engagement.

Keywords: E-commerce; Consumer behavior; Consumer experience; Loyalty; Virtual business.

October 22, 2026

Psychosocial Risks and Mental Health at Work in a Banking Institution

Mental health in the workplace is a topic of growing relevance, impacting professional development and employee well-being. A study was conducted to identify aspects and situations that can influence absenteeism levels in a banking institution. The research involved 108 participants and used a structured questionnaire, based on the short version of COPSOQ II by Gonçalves (2021), which contained 45 closed Likert scale questions and two open-ended questions. Data were collected via an online platform and analyzed with a descriptive and quantitative approach. The results revealed a scenario that indicated the need for adjustments in the distribution of activities and in managers’ perception of behaviors and signs of disorders. Important evidence was identified in the participants’ open-ended responses. It was concluded that the study contributed to promoting the perception of a safe environment for mental health in the organization.

Keywords: Absences due to disorders; Professional development; Operational leadership; Medical leave; Mental health.

October 22, 2026

Digital Observatory of the Amazon: Data Science Applied to the Analysis of Social, Economic, Environmental, and Health Indicators

The state of Amazonas faced historical challenges in consolidating the Unified Health System (SUS), stemming from geographical barriers, socioeconomic inequalities, and structural limitations. This study described the development and implementation of a health information geoportal, designed to support monitoring, evaluation, and planning in the Amazonian context. A descriptive case study approach was adopted, based on documentary analysis and secondary data, covering the mapping of functionalities, technical infrastructure, and validation steps of the tool. The Portal was developed with open technologies and integrated multiple national databases, offering interactive dashboards, thematic maps, and a digital document repository. Among the results, highlighted were the georeferenced visualization of key indicators, the possibility of temporal comparative analyses, and the use of the platform in reviewing health plans and redirecting strategic actions. The portal can represent an advance in digital governance and evidence-based management, with potential for replication in other highly complex territories, provided it is accompanied by investments in technological infrastructure, professional training, and inter-institutional coordination.

Keywords: Amazon; Health Evaluation; Digital Health; Health Information Systems.

October 22, 2026

Continuous Learning as a Factor of Organizational Growth in the Perception of Collaborators

The impact of continuous learning and knowledge management on organizational growth was studied in a context of globalization and constant changes. The study’s objective was to analyze employees’ perception of learning culture and its importance for personal and professional development. Data were obtained through an anonymous survey, with a quantitative approach, using a form with “survey” type questions applied to 87 professionals from the technology, health, education, commerce, and services sectors. The results indicated that employees had a positive perception of the contribution of continuous learning to personal and organizational growth, recognizing its benefits for the development of new skills, innovation, and competitiveness. However, it was observed that the incentive for internal knowledge exchange and the offering of practical training still presented moderate levels of agreement. It was concluded that continuous learning significantly contributes to individual and organizational development, suggesting that companies should develop strategic policies for learning focused on new skills, aiming for greater competitive advantage and innovation.

Keywords: Continuous learning; Self-development; Organizational growth; Knowledge Management; Innovation.

October 22, 2026

Statistical Regression in Microgrids for Operational Performance Estimation: Campusgrid Case

A multiple statistical regression model was proposed to estimate the operational performance of UNICAMP’s CampusGrid microgrid, based on internal electrical quantities. The microgrid integrates photovoltaic generation, batteries, and a gas generator. Energy management system data were collected at five-minute intervals over one month. The Operational Reliability Index, defined as the ratio between supplied and demanded energy, was established as the dependent variable. The model was adjusted for the islanded operation mode, evaluating the impact of demand, state of charge, solar generation, and supplied energy. The results revealed high explanatory power, indicating that operational reliability depends on the interaction between supply and demand. It was observed that the nighttime period concentrated the highest rate of partial supply, attributed to the reduction in solar generation. It was concluded that multiple regression offers an interpretable and useful framework for microgrid monitoring and control strategy support.

Keywords: Battery storage; Photovoltaic generation; Reliability Index; Islanded operation; Energy Management System.

October 22, 2026

Forecast of Aggregate Default in the Brazilian Financial System by Sarimax Model with Macroeconomic Variables

Aggregate non-payment in the Brazilian financial system is a crucial indicator of credit risk, the prediction of which is regulated by CMN Resolutions n.° 4.557/2017 and 4.966/2021 for the prospective measurement of expected losses. Aggregate non-payment was modeled and predicted using the SARIMAX model, projecting its trajectory until December 2026 under optimistic, base, and pessimistic scenarios. 164 monthly observations (July/2012 to February/2026) and macroeconomic variables such as unemployment, Selic, IPCA, IBC-Br, and household debt were used. The specification with contemporaneous variables was selected based on information criteria and coefficient stability, and residual diagnostics confirmed statistical adequacy, with a MAPE of 3.12%. Historical non-payment ranged between 4.33% and 7.64%. The estimation identified unemployment (β = 0.2534; p < 0.001) and Selic (β = 0.2128; p < 0.001) as the main positive determinants, with strong temporal persistence (AR(1) = 0.8965). Selic showed increasing correlation with non-payment (r = 0.62 at lag 12), while unemployment demonstrated a contemporaneous effect (r = 0.33). Projections indicated non-payment between 6.30% and 7.98% in December 2026, with a range of 1.68 percentage points, highlighting direct implications for provisioning in financial institutions.

Keywords: Unemployment; Predictive modeling; Monetary policy; Loss provisioning; Time series.

October 22, 2026

Stakeholder Engagement Factors in International Technology Transfer to a Public Hospital

This study described the critical factors perceived for stakeholder engagement in international technology transfer in a public hospital, with a focus on planning. The objective was to describe these conditioning factors, according to the perception of professionals. A single case study was conducted, applying a questionnaire to 25 professionals and using content analysis, complemented by descriptive tabulations. The main obstacles identified were resistance to change (72%), funding (64%), lack of communication (60%), and bureaucracy (40%). Only 28% of respondents recognized the existence of adequate tools for stakeholder management, while 48% denied it and 24% did not know. Exploratory cross-tabulations indicated variations in the assessment of influence according to time of practice, but did not establish causal relationships. Lack of engagement was associated with delays, low adherence, waste of resources, communication failures, compromised knowledge absorption, and risks to patient safety. The conclusions were limited to the perceptions of the researched group and supported preliminary recommendations for intersectoral communication and coordination.

Keywords: Case study; Project Management; Health innovation; Stakeholders; Public sector.

October 22, 2026

Combating Commercial Electric Energy Losses Using a Time Series Model

Commercial losses of electric energy represented one of the main challenges for distributors, directly impacting economic sustainability and supply quality. The efficiency of time series models, focusing on exponential smoothing techniques, was evaluated in forecasting billed consumption of consumer units, to support the identification of deviations associated with non-technical losses. Historical data of monthly consumption from an electric energy distributor in Brazil were used, segmented by behavior profiles (stationarity, trend, and seasonality). Simple exponential smoothing models, Holt, and Holt-Winters (additive and multiplicative) were applied, compared to a traditional moving average. The Mean Absolute Percentage Error (MAPE) was the evaluation criterion. Additionally, the ETS approach was employed for automatic model selection and residual diagnostic tests, such as Ljung-Box and autocorrelation function analysis. The results demonstrated that more robust models showed a greater capacity for adaptation to the series’ characteristics and lower error, with a highlight on superior performance in scenarios with trend. For seasonal series, the ETS model showed a greater capacity for adaptation to transitions. It was concluded that the use of more structured time series models contributed significantly to the improvement of forecast accuracy and offered relevant analytical support for the identification of commercial losses and the optimization of operational and commercial processes in the electric sector.

Keywords: Energy distribution; MAPE; Optimization; Consumption forecasting; Exponential smoothing.

Economy

October 19, 2026

End of the 6×1 scale: what do the numbers allow us to conclude?

Following the promulgation of the end of the 6×1 shift system, the debate on the shorter workday moves to implementation and requires distinguishing social benefits from economic costs. The end of the 6×1 shift system transitioned from legislative discussion to a constitutional change in Brazil. On October 16, 2026, the National Congress promulgated Constitutional Amendment No. 141/2026, originating from Proposal […]

Artificial Intelligence

October 16, 2026

Why study if AI exists?

When answers become easy, learning comes to mean more than just arriving at them. Why do I need to learn this if I can ask an artificial intelligence? This type of question has accompanied education for decades. Why learn an operation if a calculator exists? Why memorize information if it can be found […]

Tax Management

October 15, 2026

Pink tax and the gender of price

How different marketing treatments for men and women are incorporated by consumption taxation Imagine a shelf on which two razors of the same brand and intended for the same purpose are displayed side by side: one has dark packaging and promises strength, precision, and efficiency; the other comes in shades of pink or […]

Health

October 13, 2026

The missing link in health transformation

Why is innovation not enough to drive changes in organizations and health systems Digital transformation has become a strategic priority for health systems worldwide. Electronic health records, artificial intelligence, telemedicine, remote monitoring, and data analysis are now part of the strategic agenda of governments and health organizations. Paradoxically, […]

Marketing

October 09, 2026

Bitcoin or pets: what Artificial Intelligence would not see

A real-life consumer intelligence case shows what gets left out of the brief A few years ago, an American insurance company hired a specialized consumer intelligence firm to answer three questions: The insurer was also considering launching paid insurance in bitcoin. At the time, the consultancy’s workflow was as follows: the analyst created a query […]

School Management

October 06, 2026

The school is not a casino: when trading cards become currency in the playground

Why, for certain students, does school manage to offer so few experiences of achievement? The World Cup is over. Brazil left the competition, the school holidays came, the final was played and the football calendar moved on. It would be reasonable to imagine that, with the end of the tournament and the return to classes, […] would also begin

Energy Transition

September 30, 2026

Renewable energies: the Brazilian advantage

Brazil needs to invest in the capacity to transform natural resources into innovation, productivity, and development. Talking about energy is talking about development. Behind industry, transportation, services, cities, and increasingly artificial intelligence, there is an energy infrastructure that sustains the economy. Understanding a country’s energy matrix is understanding […]

Data Analysis

September 30, 2026

When the data changes, has reality changed?

An analysis of almost 20 million teaching positions-year between 2020 and 2025 shows how changes in population composition and record production can alter the interpretation of historical series Between 2020 and 2025, the number of formal teaching employment positions identified in RAIS (Annual Relation of Social Information), from the Ministry of Labor, […]

Artificial Intelligence

September 28, 2026

Artificial intelligence and the trends

Why do so many people join in on the fun with ChatGPT? I was never the trend person. I still am not. Some pass through my timeline, I see them, I find them interesting but not to the point of trying them out. This was the case, inclusive, with some recent waves of images made by artificial intelligence, like the 80s trend, the one that […]

Tax Management

September 01, 2026

Tax reform and estate planning in the countryside

Efficiency in managing taxes facilitates governance and allows for the organization of family succession. Rural wealth organization has become a strategic measure in light of the tax reform in Brazil, which came into effect this year. The planning of activities and rural wealth management, through tools such as the transition from an Individual to […]

Renata de Gaspari Valdejão Almeida

Artificial Intelligence

August 05, 2026

Inside the company and off the radar: the risks of Shadow AI

According to Microsoft, 75% of professionals already use artificial intelligence at work without company approval. The use of generative artificial intelligence is already part of the corporate routine. Tools like ChatGPT, Copilot, and other platforms help professionals produce texts, analyze data, create presentations, and accelerate tasks in practically all sectors. But this advance […]

Renata de Gaspari Valdejão Almeida

Strategy

July 02, 2026

The future as a competence

Academic interest reinforces the projection of scenarios as an essential tool for decision-making. While some companies are still trying to get the next step right, others are already working with multiple possible futures — and preparing for all of them. But after all, what does it mean to “think about the future”? Is it a trend, a strategy, or a new competence that […]

Renata de Gaspari Valdejão Almeida

Innovation

Technology

May 21, 2026

Agro SP Corridor: innovation, institutional articulation and future

Pecege commands the executive operation of the 400 km ecosystem activities that became known as the Brazilian agro Silicon Valley In 2025, in addition to taking over the management of the Piracicaba Technology Park, the Pecege Institute was also designated to command the executive operation of the State Agribusiness Innovation Corridor activities, the so-called […]

Education

April 27, 2026

How to turn a TCC into a business

New management of the Piracicaba Technology Park offers consulting to MBA graduates The new project director of the Piracicaba Technology Park (PTP), Alexandre Barreto, often says that one of the greatest rewards he has ever had in life was seeing his MBA USP/Esalq students structure companies from their final projects […]

Technology

March 20, 2026

The human repertoire in the age of AI

Critical thinking and creativity are irreplaceable in the work of a marketing professional Artificial Intelligence (AI) is capable of incredible feats, but it does not yet possess the judgment, empathy, intuition, and strategic creativity necessary to do the job of a good marketing professional. In this area, it is important to know how to use the tool to automate […]

Renata de Gaspari Valdejão Almeida

Agribusiness

Innovation

February 23, 2026

ESG as a growth strategy in a cooperative in RS

Santa Clara balances social and environmental responsibility with innovation Founded in 1911 by Italian immigrants in Carlos Barbosa, in the Serra Gaúcha region, Cooperativa Santa Clara is an example that it is possible to balance economic growth with social and environmental responsibility. From a small cheese production company, the organization has evolved into a […]

Renata de Gaspari Valdejão Almeida

Agribusiness

Innovation

January 22, 2026

Brazil free of Foot-and-Mouth Disease: what changes for Brazilian beef?

With a herd of over 200 million head of cattle, the country is currently the leader in exports. Brazil recently received, for the first time, the certification of country free of foot -and-mouth disease without vaccination, conferred by the World Organisation for Animal Health (WOAH). The seal strengthens the country’s position as a reliable supplier of meat for the […]

Renata de Gaspari Valdejão Almeida

Innovation

December 09, 2025

Innovation: the new cycle of the Piracicaba Technology Park

Instituto Pecege assumes management, betting on the creation of new businesses Starting in 2026, Instituto Pecege will assume the management of the Piracicaba Technology Park (PTP). Among its responsibilities will be the administration, operation, maintenance, and expansion of the park, in addition to fostering activities in innovation, entrepreneurship, and science and technology. The choice was made […]

Renata de Gaspari Valdejão Almeida

July 06, 2026

Large-Scale Architecture: DSR-SCRUM Change Management

Mônica Alessandra Guerios; Maria do Carmo Assis Todorov Abstract generated by ResumeAI, an artificial intelligence solution developed by Instituto Pecege focused on synthesis and writing. Architecture and urbanism are configured as essential bases in an increasingly complex and challenging urban space, where the intrinsic connection between social, environmental, and technological issues […]