Academic Performance of Students: Longitudinal Multilevel Modeling of Associated Factors

Cleverson Pinheiro; Daniel Alvarez Firmino

DOI: 10.22167/2675-6528-202602796

Article derived from a Final Course Work (TCC), with content based on the student’s original work and adapted to the editorial format of the E&S Magazine with the support of the ResumeAI tool, an artificial intelligence solution developed by Instituto Pecege for textual synthesis and organization.

Summary

The academic performance of higher education students represents a crucial topic for educational management, allowing the identification of factors associated with student success and retention. The determinants of the Academic Performance Index (API) over time were analyzed, considering the heterogeneity among students. Multilevel modeling (HLM2) was used with longitudinal data from 1,928 observations of 205 students, which allowed for the simultaneous evaluation of individual effects and trajectories over the semesters. The methodology included exploratory analysis, estimation of hierarchical models with fixed and random effects, model comparison via maximum likelihood, and residual diagnostics. The results revealed that variables such as attendance, number of failed courses, gender, type of origin school, and year of entry significantly influenced academic performance, with attendance having the greatest positive impact. High variability among students was observed, with an intraclass correlation coefficient greater than 90%, highlighting the importance of individual differences. Distinct academic evolution trajectories were also identified, including recovery profiles and persistent low performance. It was concluded that multilevel modeling proved adequate for capturing the complexity of the phenomenon, contributing to the understanding of factors affecting academic performance and supporting educational management actions.

Keywords: Statistical analysis; Longitudinal data; Academic performance; Higher education; Multilevel modeling.

1. Introduction

The academic performance of students in higher education is a widely investigated topic, as it results from the interaction of multiple factors, including individual characteristics, socioeconomic conditions, and institutional aspects. Variables such as family income, parental education, gender, age, and academic trajectory influence not only students’ performance but also their retention and academic success. Brandt et al. (2020) highlight that performance is determined by a set of interrelated factors, which transcend individual characteristics. Understanding these determinants is especially relevant in light of recent transformations in higher education, marked by the expansion of access and the diversification of teaching modalities. Although this process has broadened educational opportunities, it does not, in itself, guarantee the quality of training, reinforcing the need to investigate the factors that influence academic performance in heterogeneous contexts (Araujo, 2021).

Despite advances in the literature, many analyses still use statistical approaches that do not adequately capture the complexity of the relationships involved, especially due to heterogeneity between groups. In this context, multilevel models have stood out for enabling the analysis of data with hierarchical structure, simultaneously considering variations between individuals and between groups (Fávero and Belfiore, 2024). Recent studies, such as Santos et al. (2025), highlight the relevance of this approach by demonstrating that individual and contextual factors significantly influence academic performance. Sales, Hiejmans and Silva (2017) also reinforce that academic performance is the result of multiple interrelated factors, involving individual characteristics and institutional and contextual elements, which requires analytical approaches capable of capturing this complexity.

Additionally, the literature emphasizes the importance of analyzing not only access to higher education, but also academic success and student retention, considering the influence of factors such as educational background and institutional conditions (Silva, 2018). Multilevel modeling, therefore, presents itself as an adequate tool to capture the complexity of these relationships, allowing for a more robust analysis of the determinants of academic performance.

Given this context, the present work aims to apply multilevel modeling to the analysis of the academic performance of students from eight classes formed between 2015 and 2025, in the Bachelor of Business Administration course at a public higher education institution in the state of São Paulo. It seeks to investigate the impact, over the semesters, of variables such as high school of origin, time between high school and university admission, and number of failures on the Academic Performance Index (API). It is expected that the results will contribute to the understanding of the factors that influence academic performance, supporting the formulation of public policies aimed at improving success and reducing dropout rates in higher education.

2. Material and Methods

This study was characterized as a quantitative research, with a methodological approach based on linear multilevel modeling, also known as hierarchical linear models (HLM). This approach was selected for its suitability for analyzing data with nested structure and repeated measures over time, allowing for the investigation of heterogeneity among students and the evolution of academic performance at different levels of analysis. The consideration of the clustering effect proved fundamental, as analyses based solely on individual units may disregard heterogeneity among students (Santos et al., 2025).

The unit of analysis consisted of the academic performance of students in a Bachelor of Business Administration program, offered by a public higher education institution located in the state of São Paulo. The data were collected directly from the Unified Public Administration System (SUAP) of the aforementioned institution, covering eight classes formed between the years 2015 and 2025.

The study population comprised 205 students, totaling 1,928 longitudinal observations. Each observation represented a student’s performance in a given semester. The Business Administration program lasts eight semesters, and all participants were subjected to the same Pedagogical Course Plan (PPC), with a semester-based organization of curricular components.

The data structure adopted for multilevel modeling was two-level (HLM2). At level 1, observations regarding the semesters were nested, while at level 2, students were considered. This organization allowed for the simultaneous evaluation of individual effects and performance trajectories over the semesters taken.

The dependent variable analyzed was the Academic Performance Index (API), measured over the semesters. Level 1 explanatory variables, which vary over time, included the semester taken, attendance obtained in the semester, and the number of failures per semester. These variables were considered to capture the dynamics of academic performance.

At level 2, the explanatory variables of an individual nature and invariant over time were: the type of origin school (public or private), the time in years between the completion of high school and entry into undergraduate studies, the student’s entry year into undergraduate studies (cohort), and the declared sex of the students. Initially, the database contained thirteen general variables, which were standardized and selected, resulting in nine variables for the final modeling.

The multilevel modeling considered fixed effects to represent the average relationship between the explanatory variables and the GPA, and random effects to capture variations among students. Random effects were applied to both the intercept, indicating initial performance, and the slope associated with the semester, reflecting performance evolution over time.

The estimation of the models was performed using the maximum likelihood (ML) and restricted maximum likelihood (REML) methods. The ML method was employed for the comparison between nested models, while REML was used in the final estimation of parameters, for offering less biased estimates of the variance components (Fávero and Belfiore, 2024).

The selection of the final model followed an incremental strategy, known as the *step up strategy*. It started with a null model and, progressively, level 1 and level 2 variables were incorporated. The contribution of each set of variables to the model fit was evaluated through the likelihood ratio test and the Akaike information criterion (AIC).

To verify the adequacy of the estimated model, residual diagnostics were performed. Graphical analyses of normality, homoscedasticity, and adherence between observed and fitted values were included. These procedures aimed to ensure the validity of the obtained statistical inferences.

The data treatment and analysis steps were executed using scripts developed in the Python language, in the Spyder development environment. For this, specific libraries for manipulation, visualization, and statistical modeling were used, such as pandas, numpy, matplotlib, seaborn, scipy, statsmodels, and pymer4, the latter in association with the R software.

3. Results and Discussion

The initial exploratory data analysis revealed important characteristics about the Academic Performance Index (API) and the explanatory variables, providing an overview of student performance over time. It was found that the distribution of the API presented an approximately unimodal shape, with a predominance of high values and a slight left skew, indicating that most students maintained good performance, although with some lower extreme values. This initial observation is crucial for understanding the variability that would be subsequently investigated by multilevel modeling, confirming the suitability of the data for the proposed statistical approach.

The composition of the sample of 205 students, distributed by type of origin school in high school, showed that 142 students attended public schools and 63 attended private schools. Regarding declared sex, 84 students were male and 121 were female. This demographic distribution, with greater representation of students from public schools and of the female sex, was considered in the interpretation of the results, as the heterogeneity of the sample is one of the pillars of multilevel analysis. Global descriptive statistics indicated an average GPA of 7.94, an average frequency of 0.90, and an average number of failures of 0.23, providing a quantitative basis for subsequent analyses.

Over the years of class entry, a trend of improvement in the average GPA was observed, accompanied by a reduction in its variability, suggesting greater homogeneity in the academic performance of more recent classes. In parallel, there was an expressive reduction in the average number of failures, indicating an improvement in the academic flow of students over time. The average attendance, in turn, remained at high levels, generally above 0.86, with low dispersion and a slight upward trend, suggesting consistent student engagement with academic activities.

The analysis of the evolution of the average GPA by sex revealed that female students showed superior initial performance, although both groups showed a tendency to reduce GPA over time, with the averages approaching each other in the final periods. Despite a slightly higher median for females, greater dispersion was observed among male students, indicating more pronounced heterogeneity in this group. However, the overlap of the distributions suggested that the effects of sex were moderate, a finding that would be corroborated by multilevel models, where sex showed a significant effect only on the intercept.

Regarding the type of origin school, students from private schools showed a slightly higher IRA median. However, public school students exhibited greater dispersion and a higher occurrence of low values, indicating more pronounced heterogeneity in this group. Despite these differences, the significant overlap between the distributions suggested that the distinctions between the groups were moderate. The evolution of the average IRA over the semesters for both groups showed similar trajectories, with a tendency for reduction in the more advanced periods, although private school students maintained a slightly higher average performance.

Average attendance showed a gradual decrease throughout the course, with a more pronounced reduction in the final semesters. The joint analysis of attendance and academic performance revealed a positive association, indicating that more diligent students tended to perform better. This relationship was consistent with the results of the multilevel model, in which attendance demonstrated a positive and statistically significant effect on the GPA. The higher concentration of observations at high attendance levels, accompanied by high GPA values, reinforced this association, although with some dispersion at intermediate levels and the presence of atypical cases.

The relationship between GPA and the number of failed courses in the semester showed a clear negative association: an increase in the number of failed courses was directly related to a reduction in academic performance. Students without failed courses concentrated at higher GPA levels, while those with a higher number of failed courses presented lower values and greater dispersion. This pattern justified the inclusion of the failed courses variable in the multilevel model, with the expectation of a negative and statistically significant effect on GPA, which was confirmed in the modeling stages.

The correlation matrix between the main study variables reinforced the exploratory observations. The Academic Performance Index (API) showed a moderate positive correlation with attendance (r ≈ 0.53) and a moderate negative correlation with the number of failures (r ≈ -0.40). The semester variable exhibited a weak negative correlation with the API (r ≈ -0.19), suggesting a tendency for performance to decrease over time. No high correlations were observed between the explanatory variables, which indicated the absence of severe multicollinearity problems, allowing their joint inclusion in the multilevel models.

Multilevel modeling was conducted incrementally, starting with the null model to decompose the total variance of IRA into intra- and inter-individual components. The null model results indicated a mean intercept of approximately 8.04 (p < 0.001), representing the overall average performance of the students. The between-student variance was estimated at 0.609, and the residual variance at 0.133, resulting in an intraclass correlation coefficient (ICC) of approximately 82.05%. This high ICC value demonstrated that most of the variability in academic performance was associated with differences between students, fully justifying the use of multilevel modeling to capture this heterogeneity.

The inclusion of the semester variable in the model allowed for the analysis of the average GPA trajectory over time. The coefficient associated with the semester was positive and statistically significant (β ≈ 0.028; p < 0.001), indicating that, on average, academic performance tends to gradually increase throughout the course. Despite this inclusion, most of the GPA variability remained associated with differences between students, with the ICC still high at approximately 93.9%, demonstrating that time explains the average trend but does not substantially reduce individual heterogeneity.

In the next step, the frequency variable was incorporated into the model, given the strong indication of a positive association with IRA observed in the exploratory analysis. The results indicated that frequency has a positive and statistically significant effect on academic performance (β ≈ 1.24; p < 0.001). The effect of time (semester) remained positive and significant, confirming the continuity in performance evolution. There was an improvement in model fit, evidenced by the reduction in variances, but the ICC remained high at around 94.0%, reinforcing that most of the variability in IRA was still due to differences between students.

The inclusion of the number of failures variable, representing the student’s academic history, revealed a negative and statistically significant effect on academic performance (β ≈ -0.060; p < 0.001). This showed that higher levels of failure are associated with lower GPA values. Frequency remained positive and significant, although with a slight reduction in its magnitude, suggesting a partial overlap between its effects and those of failures. The ICC remained high at approximately 93.9%, indicating that individual differences continue to be the main factor of variability in performance.

After controlling for level 1 variables, the sex variable was included in the model, representing individual student characteristics. The results indicated a statistically significant effect (β ≈ -0.295; p < 0.01), suggesting that male students, on average, had lower GPA values than female students. This variable contributed to explaining part of the individual heterogeneity in academic performance, although the ICC remained high at approximately 93.7%, indicating the presence of other relevant unobserved factors influencing variability.

In the final adjusted model, after excluding the variable time between high school and undergraduate admission for not presenting statistical significance, the results confirmed that the semester (β ≈ 0.0312), attendance (β ≈ 0.9873), and year of admission (β ≈ 0.1167) positively influenced the GPA. In contrast, the number of failures (β ≈ -0.0605), being male (β ≈ -0.2035), and coming from a public school (β ≈ -0.2656) showed negative and statistically significant effects on academic performance. The final ICC of 92.23% reinforced the persistence of high variability among students, even after the inclusion of multiple explanatory variables.

The analysis of random effects showed significant heterogeneity among students, both in initial performance and in evolution over the semesters. Different academic trajectory profiles were identified, including students with high initial performance and lower evolution, as well as those with lower initial performance but with a recovery trajectory. This diversity of patterns reinforces the capacity of multilevel modeling to simultaneously capture variations between individuals and their trajectories over time, contributing to a deeper understanding of the phenomenon of academic performance.

The diagnostics of the final model’s residuals confirmed its adequacy. The distribution of residuals appeared approximately symmetric and centered around zero, with relatively light tails, indicating good adherence to the normality assumption. Although there were few extreme values, no severe deviations that compromised the model’s fit were verified. The strong adherence of the fitted values to the observed values, with high predictive capacity, reinforced the importance of considering the hierarchical structure of educational data, aligning with literature that highlights the influence of multiple levels in explaining performance (Barbosa and Fernandes, 1999; Araujo, 2021).

In summary, the results of the multilevel modeling demonstrated that academic performance is a complex phenomenon, determined by an interaction of individual, behavioral, and structural factors. Attendance and year of entry were positive predictors, while the number of failures, male sex, and origin from public school were associated with lower performance. The high variability between students, captured by the high intraclass correlation coefficient, underscores the predominance of individual differences and the need for educational approaches that consider this heterogeneity to promote success and retention in higher education.

4. Conclusion

This study sought to apply multilevel modeling to analyze the academic performance of students in a Bachelor of Business Administration program, investigating the impact of individual and temporal variables on the Academic Performance Index (API) over the semesters. It was found that attendance and year of entry positively influenced the API, with attendance having the greatest impact. Conversely, the number of failures, male gender, and origin from a public school were associated with lower academic performance. Multilevel modeling proved particularly suitable for capturing the complexity of the phenomenon, revealing high variability among students, with an intraclass correlation coefficient exceeding ninety percent, which underscored the predominance of individual differences in performance.

Additionally, distinct academic evolution trajectories were identified, including profiles of recovery and persistent low performance, which reinforced the need for differentiated educational strategies. The main contribution of this work lies in the in-depth understanding of the factors that determine academic performance in a longitudinal and heterogeneous context, supporting the formulation of more effective public policies and educational management actions, which consider both individual characteristics and institutional conditions. However, the analysis faced the limitation of the reduction in the number of observations in the final semesters, which required caution in interpreting the observed instability. For future studies, it is suggested to investigate other unobserved factors that may contribute to performance variability, as well as to apply multilevel modeling in different institutional contexts to validate and expand the findings.

Bibliographic References

ARAÚJO, Elisabeth Freitas de; PEREIRA, Antonio Gualberto; FÁVERO, Luiz Paulo Lopes. Relação entre modalidade de ensino e desempenho acadêmico: Análise multinível do Enade em Ciências Contábeis. Revista Contabilidade Vista & Revista, Belo Horizonte, v. 34, n. 2, p. 179-203, maio/ago. 2023.

BARBOSA, Maria Ligia de Oliveira; FERNANDES, Cristiano. A escola brasileira faz diferença? Uma investigação dos efeitos da escola na proficiência em matemática dos alunos da 4ª série. Revista Brasileira de Estudos de População, v. 16, n. 1-2, p. 155–172, 1999.

BRANDT, Jaqueline Zermiani; TEJEDO-ROMERO, Francisca; ARAUJO, Joaquim Filipe Ferraz Esteves. Fatores influenciadores do desempenho acadêmico na graduação em administração pública. Scielo: Educação e Pesquisa, São Paulo, v. 46, n. 2020, p. 1-20, 2020.

FÁVERO, Luiz Paulo Lopes; BELFIORE, Patrícia. Análise de dados: Estatística e machine learning com com Excel®, SPSS®, STATA®, R® PYTHON®. 2. ed. Rio de Janeiro: LTC, 2024. 1255 р.

SALES, Paula Elizabeth Nogueira; HEIJMANS, Rosemary Dore; SILVA, Carlos Eduardo Guerra. Análise multinível da transição estudantil do curso técnico para o ensino superior. Estudos em Avaliação Educacional, São Paulo, v. 28, n. 69, p. 896–925, set./dez. 2017.

SANTOS, Priscila Soares dos; OLIVEIRA, Sibele Vasconcelos de; BECKER, Kalinca Leia. Planejamento e Políticas Públicas (PPP): Determinants of academic performance in brazilian public higher education: a multilevel analysis. Planejamento e Políticas Públicas, [S.L.], v. 1, n. 68, p. 189-212, 10 mar. 2025. Trimestral. Instituto de Pesquisa Econômica Aplicada – IPEA.

SILVA, Maria Eugénia Neto Ferrão da. Contexto e trajetórias do aluno na explicação do sucesso no ensino superior: a contribuição da modelação multinível. 2018. Tese (Doutorado) – Instituto de Educação, Universidade do Minho, Lisboa, 2018.

Article originating from the Final Course Work of the Specialization in Data Science and Analytics of the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform