Article

October 26, 2026

Interpretable machine learning for story points prediction in Scrum projects

Interpretable Machine Learning for Story Point Prediction in Scrum Projects

Matheus de Oliveira Avila Sales; Manoel Flavio Leal

DOI: 10.22167/2675-6528-202601221

Article prepared by the ResumeAI tool, an artificial intelligence solution developed by the Pecege Institute focused on synthesis and writing.

Summary

The estimation of story points in Scrum projects is susceptible to cognitive biases, compromising sprint predictability. Machine Learning (ML) models have been applied to automate these estimates, but they often prioritize accuracy over interpretability, which is essential for adoption by agile managers. The research evaluated the viability of interpretable ML models as support for decision-making in agile planning, combining story point prediction with explainability via SHAP. Three supervised learning algorithms (Linear Regression, Random Forest, and XGBoost) were implemented to predict story points from nine interpretable features, extracted from 23,313 issues from 16 open-source projects. XGBoost achieved the lowest Mean Absolute Error (4.64 story points), but the three models showed similar performance, indicating that the predictive ceiling stemmed from the feature space. The SHAP technique quantified the contribution of each feature, identifying the amount of code blocks, the title length, and the frequency of technical terms as the attributes with the greatest impact. The combination of prediction and explainability configured a decision support tool in agile planning, by making explicit the textual attributes that influence estimates.

Keywords: Supervised learning; Effort estimation; Agile management; Planning poker; SHAP.

1. Introduction

Scrum is one of the most widely adopted agile frameworks in the software development industry (Schwaber and Sutherland, 2020). Within this context, sprint planning, fixed-duration work cycles, crucially depends on the estimation of story points. Story points represent a relative measure of effort that teams assign to backlog tasks, reflecting the complexity, risk, and volume of work involved. The accuracy of these estimates is fundamental to the team’s ability to plan realistic deliveries, calculate its development velocity, and effectively communicate deadlines to stakeholders.

Traditionally, story points estimation is performed through consensus techniques, such as planning poker. This approach, although it promotes egalitarian participation by requiring all members to reveal their estimates simultaneously, is susceptible to cognitive biases (Mahnič and Hovelja, 2012). Group dynamics can, for example, amplify the optimism bias, especially in teams with limited experience, compromising sprint predictability and generating significant deviations between the estimate and the actual effort. In fact, Moløkken and Jørgensen (2003) documented that deviations of 30 to 40% from the planned effort are recurrent in the industry. Pasuksmit et al. (2024) identified information quality and team factors as frequent contributors to inaccuracy in these estimates.

Faced with these challenges, Machine Learning (ML) has been applied to automate the estimation of story points. The evolution of these predictive approaches has demonstrated consistent gains in predictive accuracy. Initially, techniques such as TF-IDF were combined with classifiers (Porru et al., 2016). Subsequently, deep learning models, such as Deep-SE, emerged to capture complex semantic relationships (Choetkiertikul et al., 2019). More recently, pre-trained language architectures, such as GPT2SP (Fu and Tantithamthavorn, 2023) and SBERT (Yalçıner et al., 2024), have been employed, reporting competitive Mean Absolute Errors. However, a persistent limitation of these approaches lies in the fact that they function as “black boxes”. The agile management professional receives a numerical estimate without understanding which characteristics of the issue influenced it, which hinders trust in the prediction and the incorporation of the results into planning discussions.

This scenario reveals a significant gap in the literature: the absence of studies that combine story point prediction with explainability techniques oriented towards agile management. For predictive models to be effectively adopted by Scrum Masters and Product Owners, it is essential that the results are understandable and actionable, allowing them to question and validate the generated predictions (Fernández-Diego et al., 2020). Trust in the tool directly depends on the ability of its users to understand the “why” behind each estimate. The SHapley Additive exPlanations (SHAP) technique, proposed by Lundberg and Lee (2017), offers a promising solution by quantifying the marginal contribution of each feature to the individual prediction, based on cooperative game theory and satisfying axiomatic properties that guarantee a fair and transparent attribution.

Unlike approaches that use numerical text representations with thousands of dimensions, this work prioritized interpretability. Three supervised learning algorithms – Linear Regression, Random Forest, and XGBoost – were implemented using nine structural and interpretable features, extracted directly from the textual fields (title and description) of 23,313 issues from 16 open-source projects of the Deep-SE Story Point Dataset (Choetkiertikul et al., 2019). This choice allowed the SHAP technique to provide accessible explanations, such as “the prediction increased because the issue contains six code blocks,” instead of complex interpretations of high-dimensional vectors. Thus, the objective of this work is to evaluate the viability of interpretable machine learning models as decision support in agile planning, combining story point prediction with explainability via SHAP.

2. Material and Methods

The research was characterized as a study of an applied nature, with a quantitative approach and an experimental-computational design. The central objective was to evaluate the viability of interpretable machine learning models as decision support in agile planning, combining story points prediction with explainability via SHAP. To this end, a data processing and regression model training pipeline was implemented.

The implementation pipeline was structured in four sequential stages: acquisition and exploratory analysis of the dataset, engineering of interpretable features from the textual fields of the issues, preprocessing and partitioning of the data, and training with optimization of the regression models. The entire implementation was conducted in Python, using libraries such as scikit-learn for regression models and cross-validation, XGBoost for gradient boosting, SHAP for calculating Shapley values via TreeExplainer, pandas for tabular data manipulation, matplotlib for figure generation, and textstat for readability indices.

The reproducibility of the analyses was ensured by fixing the seed `random_state=42` in all operations involving randomness. The complete source code of the project was made available in a public repository, ensuring transparency and the possibility of verifying the adopted procedures.

The dataset used was the Deep-SE Story Point Dataset, publicly available from Choetkiertikul et al. (2019). This public repository served as a secondary data source, originating from tracking systems such as Jira and Bugzilla, from 16 open-source software projects. The adoption of this dataset allowed for direct comparability with previous studies (Choetkiertikul et al., 2019; Fu and Tantithamthavorn, 2023).

The dataset gathered 23,313 issues, with each record containing the fields `issuekey`, `title`, `description`, and `storypoint`. The inclusion criteria defined by the original authors of the dataset excluded projects with fewer than 300 estimated issues and removed records with story points equal to zero, negative, or greater than 100. This filtering affected approximately 2.66% of the total records.

Regarding data handling, the `description` field presented 2,231 missing values, corresponding to 9.57% of the total. These cases were handled by replacing them with an empty string, preserving the corresponding records without artificial imputation. The other fields (`issuekey`, `title`, `storypoint`, and `project`) showed no missing values.

Instead of high-dimensional vector representations, nine interpretable features were extracted directly from the textual fields (`title` and `description`) of each issue. The features included: `desc_length` (description length in characters), `title_length` (title length in characters), `is_user_story` (presence of the canonical user story pattern), `has_acceptance_criteria` (presence of acceptance criteria), `code_snippet_count` (code blocks or method calls), `url_count` (URLs referenced in the description), `description_readability` (Flesch-Kincaid Grade Level index), `keyword_complexity` (frequency of technical terms), and `title_desc_ratio` (title length/description length ratio).

The binary features (`is_user_story` and `has_acceptance_criteria`) were detected by regular expressions. The `description_readability` feature employed the Flesch-Kincaid Grade Level index, calculated by the textstat library. The `keyword_complexity` feature counted the occurrence of 20 technical terms associated with high-complexity tasks, such as ‘refactor’, ‘migrate’, ‘integration’, ‘API’, ‘database’, ‘security’, ‘performance’, ‘architecture’, ‘infrastructure’, ‘scalability’, ‘deployment’, ‘authentication’, ‘authorization’, ‘configuration’, ‘optimization’, ‘upgrade’, ‘redesign’, ‘overhaul’, and ‘rewrite’.

The set of 23,313 issues was partitioned into 80% for training (18,650 issues) and 20% for testing (4,663 issues). Random shuffling was controlled by the seed `random_state=42`. Normalization was applied to continuous numerical features by StandardScaler, fitted exclusively on the training set to avoid data leakage. Binary features were kept on the {0, 1} scale, dispensing with normalization.

Three supervised learning algorithms were implemented to predict story points: Linear Regression, Random Forest, and XGBoost. Linear Regression served as the baseline model, with coefficients determined by the ordinary least squares method on the nine features, without hyperparameter optimization.

Random Forest, an ensemble method by bagging, built multiple independent decision trees on random subsamples of instances and features, aggregating individual predictions by averaging. Hyperparameters were selected by grid search with five-fold cross-validation (cv=5), minimizing Mean Absolute Error (MAE) as the criterion. The grid evaluated 36 combinations, totaling 180 adjustments.

XGBoost implemented gradient boosting over decision trees, adjusting each subsequent tree to minimize the residual of the previous iteration by gradient descent, with L1/L2 regularization to control complexity. Grid search evaluated 81 combinations, totaling 405 adjustments with five-fold cross-validation.

The explainability analysis employed the SHapley Additive exPlanations (SHAP) technique, proposed by Lundberg and Lee (2017). This technique assigns a value to each feature that quantifies its marginal contribution to the individual prediction of each observation, based on cooperative game theory. The TreeExplainer, an algorithm optimized for tree-based models, calculated the exact Shapley values for Random Forest and XGBoost without resorting to stochastic approximations.

The results of the SHAP analysis were organized into four types of visualization: global importance (bar plot), directional distribution (beeswarm), local explanation (waterfall), and feature dependency (dependence plot). These visualizations were used to interpret the contribution of each feature to the model predictions, without presenting the empirical findings in this section.

3. Results and Discussion

This research assessed the viability of interpretable machine learning models as support for decision-making in agile planning, combining story point prediction with the explainability provided by the SHAP technique. The initial results of the exploratory analysis of the Deep-SE dataset, composed of 23,313 issues from 16 open-source projects, revealed a distribution of story points with a pronounced positive skew (6.00) and high kurtosis (45.69). This characteristic indicated that the majority of issues were concentrated in the lower values of the Fibonacci scale (1, 2, 3, and 5 story points), while values exceeding 20 story points were rare occurrences, a pattern that directly influenced the predictive behavior of the models.

The variability of story points across the analyzed projects proved to be significant, with the median values ranging from 2 to 8 story points and a substantially different interquartile range among the teams. This pattern reflected the existence of distinct estimation conventions, a finding consistent with what was documented by Tawosi et al. (2022a) in their analysis of the TAWOS dataset, which includes the same projects. This heterogeneity motivated the decision to adopt a cross-project model, trained and evaluated on the entirety of the data, without specific project separation, seeking a generalization of the observed patterns.

The relationship between the length of textual fields and the assigned story points proved to be weak, without a significant linear correlation. Both short and long issues received low and high values, indicating that raw textual characteristics, without further transformation, capture a limited predictive signal. This result corroborates findings by Porru et al. (2016), who reported an analogous pattern when investigating story point estimates from raw term counts. The concentration of the most frequent values in the canonical points of the Fibonacci scale (1, 2, 3, 5, and 8 story points) was confirmed, with the presence of extreme values that produced the heavy tail observed in the distribution.

Feature engineering prioritized interpretability, extracting nine attributes directly from the textual fields of the issues, such as description length, title length, presence of canonical user story, presence of acceptance criteria, code block count, URL count, description readability, keyword complexity, and title/description ratio. This choice was deliberate to ensure that the explanations would be understandable by agile management professionals, without the need for familiarity with language models or advanced algebra. The trade-off between accuracy and interpretability was, therefore, intentional, and the resulting loss of predictive capability was quantified in the model evaluation.

The execution of the feature extraction pipeline on the 23,313 issues revealed that only 0.4% of the issues followed the canonical user story format and only 7.1% contained explicit acceptance criteria. The average description length was 670 characters, with some records exceeding 50,000 characters due to the inclusion of stack traces or full logs. The average Flesch-Kincaid readability score was 11.45, consistent with university-level technical texts. These structural data were pre-processed, with continuous numerical features normalized and binary features kept at their original scale, before partitioning into training and testing sets.

Comparative performance of the models

The three supervised learning algorithms – Linear Regression, Random Forest, and XGBoost – were evaluated on the test set (4,663 issues) using the Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and coefficient of determination (R²) metrics. XGBoost achieved the lowest MAE, with 4.64 story points, closely followed by Random Forest, with 4.66 story points, and Linear Regression, with 4.68 story points. Random Forest presented the highest R² (0.026) and the lowest RMSE (9.15). The absolute differences between the models were marginal, with a variation of only 0.04 story points in MAE and 0.020 in R², indicating a notable performance proximity.

This convergence of performance between algorithms of distinct natures (a linear model, a bagging ensemble, and a gradient boosting ensemble) suggested that the predictive ceiling was imposed by the feature space, rather than by the intrinsic capacity of the algorithms. Linear Regression achieved nearly identical performance to the tree-based models, which implies that most of the predictive signal available in the nine structural features has already been extracted by a weighted linear combination. This finding is relevant for scenarios with computational cost constraints, where a simpler model may be preferable.

The hyperparameters selected by the grid search for the tree-based models were conservative, with a maximum depth (max_depth) of 10 for both, aiming to avoid overfitting. XGBoost used a low learning rate (0.01) and a smaller number of trees (100), while Random Forest compensated for the absence of sequential boosting with a larger number of estimators (500). Column subsampling (colsample_bytree = 0.7) in XGBoost acted as additional regularization. These configurations reinforced the hypothesis that the feature space offered limited predictive signal for the regression task, and that the search for more aggressive parameters resulted in overfitting on the validation set.

Analysis of individual predictions

The three models exhibited a regression to the mean pattern, with predictions concentrated around approximately six story points, regardless of the actual value assigned to the issue. For issues with story points greater than 20, the models systematically underestimated the estimates, rarely exceeding ten story points. This behavior stems from the strong asymmetry of the story point distribution in the dataset, where approximately 75% of issues received up to 8 story points. The models learned to predict values close to this region of higher density, which limited the usefulness of point predictions for high-complexity issues, precisely where decision support is most critical.

The residual distributions (difference between the actual value and the predicted value) of the three models were approximately symmetric around zero, with a high concentration of small errors (in the range of -5 to +5 story points) and a heavy right tail, indicating underestimation in issues with high values. This characteristic is expected in distributions with pronounced positive skewness and high kurtosis, such as that of story points in this study. The proximity of the residual distributions between the algorithms reinforced the evidence that predictive capability was limited by the feature space, not by the algorithm choice, a finding similar to that documented by Tawosi et al. (2022a).

Importance of features in models

The analysis of feature importance by the models’ internal mechanisms (mean decrease in impurity – MDI for Random Forest and gain of information for XGBoost) revealed that, in Random Forest, four features concentrated 90.6% of the total importance: the title/description ratio (29.4%), description readability (24.5%), description length (21.7%), and title length (15.1%). These variables captured complementary dimensions of the issue text, such as the proportion between title and description, linguistic complexity, absolute length, and conciseness of the statement, respectively.

In XGBoost, feature importance was distributed more uniformly. A notable finding was the jump of the `keyword_complexity` feature, which moved from seventh position in Random Forest (4.7%) to second position in XGBoost (17.4%). This difference can be attributed to the gradient boosting mechanism, which adjusts sequential trees on the residuals of the previous iteration, allowing it to capture residual signals in the frequency of technical terms that Random Forest, based on bagging, did not prioritize in its splits. The binary features `is_user_story` and `has_acceptance_criteria` were the least relevant in both models, with importance below 0.1% and 0.2% respectively, due to the extreme imbalance of these variables in the dataset.

Explainability via SHAP

The explainability analysis via SHAP (SHapley Additive exPlanations) revealed significant divergences compared to the MDI/gain importance metrics, especially for the `code_snippet_count` feature. In Random Forest, this feature ranked seventh in MDI (0.022), but rose to first place in average SHAP (0.80). This inversion occurred because SHAP, based on cooperative game theory, captures the marginal effect of each feature on individual prediction, while MDI accounts for usage frequency weighted by impurity reduction. The `code_snippet_count` was used in few tree splits, but with substantial impact on individual predictions, an effect that MDI underestimated. This discrepancy reinforced the need for explainability techniques with axiomatic foundations.

Despite the magnitude differences in SHAP values between Random Forest and XGBoost (Random Forest values were approximately 2.5 times larger, reflecting the model’s higher variance), both algorithms converged on the five most important features: `code_snippet_count`, `title_length`, `desc_length`, `description_readability`, and `title_desc_ratio`. This convergence between models with distinct learning mechanisms provided evidence that these variables possess a genuine predictive signal and are stable indicators for agile management professionals, regardless of the algorithm employed.

Influence Direction

The analysis of the direction of influence, through the beeswarm graph, demonstrated that the `keyword_complexity` feature exhibited a strong positive correlation between its value and the SHAP value. Issues with a high frequency of technical terms, such as “refactor”, “migrate”, “integration”, “API”, and “database”, received high positive SHAP values, indicating that the model increased the story points estimate for these issues. This pattern is interpretable in the context of agile management, as tasks involving these terms tend to be more complex and demand greater effort. For a Scrum Master, this information is actionable, allowing for a re-examination of estimates that were driven by these terms.

In contrast, the `desc_length` feature showed a weak directional pattern. Long descriptions were associated with both positive and negative SHAP values, suggesting that description length, in isolation, is not a reliable indicator of effort. A description can be long due to detailed requirements or, alternatively, by including redundant information or complete logs, which does not necessarily imply greater effective complexity. This result challenges the common intuition that “long description equals complex task,” and the same pattern was observed in Random Forest, indicating that teams relying on this informal heuristic may be basing their decisions on a weak signal.

The `title_length` and `title_desc_ratio` features showed non-linear relationships with SHAP. For `title_length`, short titles were associated with predominantly negative SHAP values, while long titles did not produce a consistent direction. For `title_desc_ratio`, extreme values (very high or very low) generated greater impacts than intermediate values. This suggests that the disproportion between the title and the description, in either direction, is more informative than the proportion itself, indicating that an imbalance in textual communication may signal complexity or lack of clarity.

Local Analysis

The waterfall visualization decomposed the prediction of an individual observation into contributions from each feature, starting from the expected value (average of predictions) and reaching the predicted value. This individual traceability allowed us to identify which features pulled the prediction up or down and by what magnitude. This transparency is crucial for Scrum Masters and Product Owners, as it allows them to compare the model’s estimate with the team’s estimate and identify discrepancies attributable to specific textual characteristics of the issue. For example, if the model estimates a substantially higher value than the team, the waterfall decomposition can reveal that a high count of technical terms or the presence of code blocks inflated the prediction, providing actionable insights to re-examine the human estimate and mitigate anchoring bias, as described by Mahnič and Hovelja (2012).

Interactions between features

The dependency plot revealed an interaction between `keyword_complexity` and `desc_length`. For issues with a high frequency of technical terms, the effect on prediction was amplified when the description was also long. A description combining dense technical vocabulary with considerable length signaled a task involving multiple dimensions of complexity (architectural, integration, refactoring), justifying higher estimates. This interaction would not be captured by a univariate importance analysis (MDI or gain), highlighting the value of SHAP dependency plots for identifying multivariate patterns in the data. The Random Forest results for the four SHAP visualizations were qualitatively similar to those of XGBoost, with the same predominant features and the same directional patterns, and the five features with the highest impact coincided entirely between the two models.

Contextualization with the literature

The MAE values obtained, between 4.64 and 4.68 story points, were higher than those reported by studies that employed high-dimensionality textual representations. Choetkiertikul et al. (2019) achieved an MAE of 2.08 with Long Short-Term Memory (LSTM) networks, and Yalçıner et al. (2024) reported an MAE of 2.15 with Sentence-BERT combined with gradient boosted trees. The difference in predictive accuracy reflected a deliberate design decision, prioritizing the interpretability of the nine features over maximum predictive capability. High-dimensional semantic representations, while capturing complex linguistic nuances, remain opaque to Scrum Masters and Product Owners, hindering the understanding of the “why” behind the estimates.

The robustness of the metrics reported in the literature was questioned by Tawosi et al. (2024), who, upon replicating Fu and Tantithamthavorn’s (2023) GPT2SP, identified an error in the original MAE calculation and demonstrated that a predictor based on the median of story points per project achieved competitive performance with the transformer model after correction. This finding partially explained the R² values close to zero obtained in this work, as the median already constituted a strong predictor, and the residual space for learning was narrow. Even with R² close to zero, the implemented models produced differentiated predictions per issue and allowed, via SHAP, to track which textual attributes influenced each individual prediction, a traceability that a median predictor did not offer. Alexander and Jayadi (2024) reached an analogous conclusion in a direct comparison, in which traditional ML models maintained competitive performance with Large Language Models in story point prediction, reinforcing that the adoption of interpretable models did not imply a significant loss of accuracy compared to state-of-the-art models.

The distribution of story points in Deep-SE followed the Fibonacci scale (1, 2, 3, 5, 8, 13, 21…), used to reflect increasing uncertainty in larger tasks. This scale is ordinal, not interval, and the continuous treatment of the variable, shared by all reviewed studies (Choetkiertikul et al., 2019; Fu and Tantithamthavorn, 2023; Yalçıner et al., 2024), may have contributed to the models’ difficulty with higher values on the scale. Tawosi et al. (2022b) documented that the correlation between story points and actual effort was strong in only 7% of projects, with imprecise estimates in 78% of cases, indicating that story points capture perceptions of relative complexity and not absolute effort. SHAP analysis therefore revealed the textual factors that influenced these perceptions, rather than the actual implementation time.

Fernández-Diego et al. (2020) found that subjective techniques, such as planning poker and expert judgment, remain dominant in the industry, despite evidence of automated approaches. The gap was attributed to the lack of transparency of ML models, as managers resist tools whose predictions cannot be interpreted. Pasuksmit et al. (2024) identified the quality of information about issues as the most frequent reason for inaccuracy in estimates, a dimension that SHAP analysis addressed by quantifying the influence of attributes such as textual length, technical vocabulary, and code blocks. Priya Varshini et al. (2025) validated the Random Forest + SHAP combination in software effort estimation, reporting improved manager confidence, a methodological convergence that reinforced the relevance of the interpretable approach adopted in this work.

Limitations of the study

The study’s central limitation resided in the restricted space of nine structural and textual features. The exclusion of high-dimensional semantic representations was a deliberate choice to ensure the interpretability of SHAP results for Scrum Masters and Product Owners. However, this decision resulted in a limited predictive signal, evidenced by R² values close to zero. In practice, a coefficient of this magnitude implies that point prediction should not be adopted in isolation for critical decisions, as the captured variance was insufficient for capacity planning or resource allocation. The value of the approach resided in the SHAP decomposition, which transformed individually imprecise predictions into actionable diagnostics about the textual attributes that influenced each estimate.

Story points are inherently noisy as a target variable, as documented by Tawosi et al. (2022b), who observed a strong correlation with actual effort in only 7% of projects. This intrinsic noise sets a ceiling on predictability regardless of the model employed. Furthermore, the 16 Deep-SE projects were open-source, with heterogeneous estimation practices and agile maturity. Training in cross-project mode, rather than project-specific, diluted team-specific patterns, which may have limited performance. Generalization to proprietary projects with stable teams and standardized practices remains an open question, depending on access to confidential data.

Treating story points as a continuous variable was a simplification, given that the Fibonacci scale is ordinal and traditional regression did not ideally capture its discrete structure. Khan et al. (2025) proposed a comparative learning framework that infers relative effort, which may be more consistent with the planning poker decision-making process. Another limitation was the absence of temporal features, such as position in the backlog, team historical velocity, or sprint phase. Tawosi et al. (2022b) indicated that these features could complement structural variables and expand the predictive power of the models, as the correlations between story points and actual effort vary throughout the project lifecycle.

Finally, the computational cost of SHAP in production environments represents a limitation. Although TreeExplainer was feasible for the test set of this work (4,663 observations), calculating Shapley values for real-time inference or sets with millions of issues may require strategies such as caching, sampling, or stochastic approximations (KernelSHAP). These considerations are important for the scalability of the solution in high-volume data industrial contexts.

Practical implications for agile management

SHAP analysis translated the predictive results into practical recommendations for Product Owners and Scrum Masters. For Product Owners, the `keyword_complexity` feature emerged as a consistent indicator that issues mentioning terms like “refactor”, “migrate”, “integration”, “API”, and “database” receive systematically higher estimates. This information is crucial for mitigating anchoring bias during planning poker sessions, where superficial familiarity with these terms can mask the task’s real complexity. SHAP decomposition offers a counterpoint to bias, flagging issues that deserve in-depth discussion before voting, allowing for a more informed estimate.

For Scrum Masters, the waterfall visualization functions as an individual diagnostic tool. It allows comparing the model’s estimate with the team’s and identifying the features responsible for the divergence. If the model assigns a high estimate to an issue evaluated with a low value by the team, and the decomposition reveals `code_snippet_count` and `keyword_complexity` as drivers, the Scrum Master can reopen the discussion about technical complexity. Planning poker generates alignment through discussion, and SHAP generates alignment through the transparency of predictive factors, acting as complementary and non-competing approaches, according to the perspective of Moløkken-Østvold and Haugen (2007).

The study’s approach reinforces that the model’s role is to provide an additional perspective, based on patterns extracted from a large volume of data, enriching the discussion without replacing the team’s judgment. The story point scale is a social convention, and SHAP made explicit the structural reasons behind the prediction, allowing the team to assess whether the same factors were considered in their estimate, in line with Schwaber and Sutherland’s (2020) view on the responsibility for sizing by developers. The low prevalence of formal specification practices in the dataset (0.4% of canonical user stories and 7.1% of explicit acceptance criteria) constituted an applicable finding. Teams that systematically adopt acceptance criteria and a standardized user story format can improve both the predictability of estimates and the richness of features available for predictive models, as Pasuksmit et al. (2024) identified information quality as a critical factor for accuracy.

In summary, the research demonstrated that, although predictive models based on interpretable features showed modest performance in terms of point accuracy (MAE of 4.64 story points and R² close to zero), the combination with the SHAP technique transformed them into a valuable decision support tool in agile planning. The ability to explain the “why” of each estimate, identifying the contribution of textual attributes such as the number of code blocks, title length, and frequency of technical terms, allowed Scrum Masters and Product Owners to understand and validate the predictions, mitigating biases and enriching planning discussions, which is fundamental for the effective adoption of Machine Learning models in agile management contexts.

4. Conclusion

This study evaluated the feasibility of interpretable machine learning models as decision support in agile planning, combining story point prediction with explainability via SHAP. Three supervised learning algorithms – Linear Regression, Random Forest, and XGBoost – were implemented to predict story points from nine structural and textual features, extracted from 23,313 issues across 16 open-source projects. It was found that XGBoost achieved the lowest Mean Absolute Error (4.64 story points), but the three models exhibited similar predictive performance, indicating that the predictability ceiling was imposed by the feature space, rather than the algorithms’ intrinsic capabilities. SHAP analysis quantified the contribution of each feature, identifying the number of code blocks, title length, and frequency of technical terms as the attributes with the greatest impact on estimates. The main contribution of this work lies in configuring a decision support tool that, by making explicit the textual attributes influencing estimates, allows Scrum Masters and Product Owners to understand and validate predictions, mitigating biases and enriching agile planning discussions, overcoming the “black box” limitation of previous predictive approaches.

However, the study presented important limitations, such as the restricted space of nine structural and textual features, a deliberate choice that resulted in a limited predictive signal and R² values close to zero. Furthermore, story points are inherently noisy as a target variable, and training in cross-project mode on open-source project data introduced heterogeneity in estimation practices. Treating the Fibonacci scale as a continuous variable and the absence of temporal features also constituted simplifications. For future studies, it is suggested to explore hybrid architectures that combine interpretable features with low-dimensionality semantic representations to increase accuracy without sacrificing transparency. Empirical validation of the approach with Product Owners and Scrum Masters in qualitative studies is also recommended, in order to measure the effect of SHAP visualizations on the calibration of estimates and on the dynamics of planning poker sessions, bringing ML models closer to effective adoption in agile management environments.

Bibliographic References

Choetkiertikul, M.; Dam, H.K.; Tran, T.; Pham, T.; Ghose, A

Mahnič, V.; Hovelja, T. 2012. On using planning poker for estimating user stories. Journal of Systems and Software 85(9): 2086-2095.

Moløkken, K.; Jørgensen, M. 2003. A review of surveys on software effort estimation. IEEE Transactions on Software Engineering 29(10): 1104-1115.

Pasuksmit, J.; Thongtanunam, P.; Karunasekera, S. 2024. A systematic literature review on reasons and approaches for accurate effort estimations in agile. ACM Computing Surveys 56(11): 1-37.

Porru, S.; Murgia, A.; Demeyer, S.; Marchesi, M.; Tonelli, R. 2016. Estimating story points from issue reports. In: International Conference on Predictive Models and Data Analytics in Software Engineering, 12., 2016, Ciudad Real, Espanha. Proceedings… p. 1-10.

Schwaber, K.; Sutherland, J. 2020. The Scrum Guide: the definitive guide to Scrum, the rules of the game. Disponível em: <https://scrumguides.org/>. Acesso em: 30 mar. 2026.

Article originating from the Final Course Work of the Specialization in Software Engineering of the MBA USP/Esalq

To learn more about the course, click here and access the MBX Academy platform

You may also like

November 17, 2026

Detection and Diagnosis of Diseases in Corn and Rice in Mozambique with Deep Learning

Mozambican agriculture depends on corn and rice, fundamental crops for food security, but their productivity is strongly affected by phytosanitary diseases, whose early and accurate identification remains a challenge in resource-limited contexts. The study aimed to develop and compare Deep Learning-based models for the automatic detection and diagnosis of diseases in corn and rice leaves in Mozambique. Three approaches were used: a classic Convolutional Neural Network (CNN) developed from scratch, and two transfer learning models, ResNet50 and MobileNetV2. The dataset consisted of images from public databases and collected in the field, which underwent cleaning, balancing, and artificial augmentation steps. The models were trained and evaluated with standard performance metrics, including accuracy, precision, recall, F1-score, Log Loss, and Cohen’s Kappa. The classic CNN presented the best overall performance, with 94% precision, recall, and F1-score, and 93% accuracy and Cohen’s Kappa. MobileNetV2 achieved 91% F1-score and 90% accuracy, standing out for its lower Log Loss (0.30) and training time (6h32min59s). ResNet50 reached 80% F1-score and 79% accuracy. It was concluded that Deep Learning techniques have high potential to support phytopathological diagnosis in real agricultural contexts, contributing to faster and more accurate decisions in the management of corn and rice diseases.

Keywords: Deep learning; Rice; Plant diseases; Corn; Computer vision.

November 17, 2026

Artificial intelligence in accounting operational routines and financial efficiency

The incorporation of Artificial Intelligence (AI) and automation tools has reconfigured the routines of Brazilian accounting offices, impacting productivity, error mitigation, and financial efficiency. These impacts were analyzed in a small accounting firm, specialized in serving service providers in the medical field in São Caetano do Sul, São Paulo. The research adopted a qualitative approach, with a single case study design and incorporated units of analysis. Data were collected through a structured questionnaire, applied via Google Forms between January and February 2026, to five professionals in the operational structure of the firm in a census survey. The treatment followed content analysis with a priori and emergent categories. All respondents declared the use of automated tools or tools with AI resources. The perception of reduced execution time, decreased operational errors, rework, and better use of the team’s time converged in 80% of affirmative responses and 20% partial responses. The most benefited activities were bank reconciliation, import of fiscal documents, and tax assessment, with visible gains on typing errors, duplicate entries, and incorrect classification. Human intervention remained indispensable in tax planning, scenario analysis, and verification of ancillary obligations. The overall evaluation of AI’s contribution was predominantly favorable, with moderation linked to the complexity of fiscal regulations.

Keywords: Content analysis; Process automation; Accounting 4.0; Small business; Digital transformation.

November 17, 2026

Factors influencing schedule management in sewage treatment plant “retrofit” projects

Projects involving “retrofit” in Sewage Treatment Plants (STP) are characterized by high complexity, operational restrictions, and multiple technical interfaces. The study analyzed the perception of the professionals involved regarding the management and adherence to deadlines in “retrofit” projects for sewage treatment plants, considering technical, operational, and organizational factors. The research was characterized as an applied study, with a quali-quantitative approach, designed as a field survey. Data analysis was performed using descriptive statistics and qualitative categorization, employing SWOT, risk matrix (Probability x Impact), and 5W2H action plan tools. The results indicated a high perception of clarity (77.8%) and realism of deadlines (100% agreement), evidencing consistent initial planning. However, 77.7% of respondents pointed out that rework and scope changes directly impacted deadline adherence. The integration between engineering, manufacturing, and Production Planning and Control (PPC) showed an intermediate evaluation (77.7% partial/total agreement), indicating fragility in interfunctional coordination. The risk matrix highlighted as critical: scope changes, operational limitations of the STP, and technical complexity, all with a high level. As a central improvement, the standardization of scope change control, analysis, and approval was proposed, aiming to reduce rework and increase schedule predictability.

Keywords: Performance; Integration; Planning; Rework; Risk.

November 17, 2026

B2C Acquisition in diagnostic medicine: strategic partnerships with specialized medical offices

The diagnostic medicine sector in Brazil operates with high dependence on health plan providers, which limits the financial autonomy of service providers. This study proposed and analyzed a direct patient acquisition model (B2C) for a highly complex diagnostic medicine institution, structured through partnerships with Oncology and Cardiology practices and operationalized by a personalized customer service called concierge. The investigation adopted a mixed-methods approach in three phases: a field survey with a structured questionnaire applied to 87 patients, mapping of the operational flow using “Service Blueprint”, and analysis of the conversion rate from 312 medical referrals registered in the pilot project. The results indicated that trust in medical referral achieved 92.0% agreement, establishing itself as the main factor for adherence. The convenience of performing the exam on the same day reached 78.2%, and the willingness to pay directly was 55.2%, with polarization associated with the predominance of health plan users. The “Service Blueprint” identified three critical points in the transitions between stages of the patient journey. The overall conversion rate was 51.0%, with superior performance in Oncology (58.2%) compared to Cardiology (45.0%), and the greatest loss was concentrated in the acceptance and scheduling stage (28.3%). It was concluded that the model demonstrated possible operational viability and indications of patient acceptance, conditioned by price transparency and a reduction in the interval between consultation and approach by the concierge.

Keywords: Direct payment; Patient journey; Supplementary health; Concierge service; Conversion rate.

November 17, 2026

The role of the school manager in promoting an inclusive culture: leadership and educational policies

The research analyzed the school manager’s performance in promoting an inclusive culture, considering the need to transform legal guidelines and pedagogical foundations into effective institutional practices. It was assumed that school inclusion requires planning, organization, and strategic leadership in the school’s daily operations. A qualitative approach of an applied nature was adopted, developed in three complementary stages: documentary analysis of Brazilian legislation on inclusive education, bibliographic review on school leadership and management, and data systematization using management tools such as SWOT analysis and the PDCA cycle. The results indicated that the school management’s performance directly influenced the consolidation of inclusive environments, especially in the organization of processes, continuous team training, and strengthening of a collaborative institutional culture respectful of diversity. Structural, pedagogical, and relational challenges that demanded planned interventions were also identified. Finally, a Practical Guide for School Managers was developed, with guidelines, protocols, and operational orientations for decision-making and the organization of inclusive processes. It was concluded that the construction of an inclusive culture depended on the management’s ability to articulate public policies, pedagogical practices, and institutional processes, consolidating inclusion as a permanent commitment in the school context.

Keywords: educational accessibility; educational equity; participatory management; strategic planning; institutional processes.

November 17, 2026

An AI artifact based on Design Thinking to support the elicitation of software requirements from user needs

The development of software oriented to user needs frequently faces challenges in requirements elicitation, stemming from ambiguities in natural language and communication failures. Given this, an intelligent artifact integrating Design Thinking and Artificial Intelligence was proposed and validated, focusing on the Empathy and Definition stages, to support the collection, interpretation, and structuring of user needs. The research, of an applied, qualitative, and exploratory nature, conducted a Proof of Concept in a controlled environment, employing hypothetical scenarios. The artifact operated through guided natural language interviews, organizing information into elements such as pain points, context, and impact, and generating traceable problem syntheses. The results indicated that the system organized unstructured reports, reduced ambiguities, and preserved the meaning of the original information in most cases. The ability to reorganize logically in complex scenarios was also observed, although with limitations in separating multiple problems and dependence on instructions. It was concluded that the artifact has potential as a support for requirements elicitation, improving the clarity and structuring of information, but requires supervision and contextual control.

Keywords: Design Thinking; Requirements Elicitation; Requirements Engineering; Artificial Intelligence; Natural Language Processing.

November 17, 2026

The Extinction of the Monophasic Tax System in the Pharmaceutical Industry in Light of the Tax Reform

The consumption Tax Reform, established by Constitutional Amendment No. 132/2023, introduced structural changes to the Brazilian tax system, with the replacement of the Social Integration Program (PIS) and the Social Security Financing Contribution (COFINS) by the Contribution on Goods and Services (CBS), in a dual Value Added Tax (VAT) model. The objective was to analyze the impacts of the extinction of the single-stage system on the pharmaceutical industry, in light of the new regulatory design and its economic and legal effects. A qualitative, exploratory research was conducted through a case study and documentary analysis of legislation and specialized doctrine. It was examined whether the end of the single-stage system would imply a relevant increase in the tax burden for the sector. The results indicated that, although the formal overcoming of the single-stage system has occurred, the material exemption of medicines has been preserved, mainly due to the application of reduced rates of 60% and, in certain cases, a zero rate of the Tax on Goods and Services (IBS) and CBS. The change in model may cause occasional variations in revenue collection, without characterizing a structural increase in the tax burden, tending towards neutrality over time. It was preliminarily concluded that the Tax Reform reorganized the taxation technique in the pharmaceutical industry, replacing a concentrated regime with more transparent exemption mechanisms. This approach proved compatible with the principles of neutrality, full non-cumulativeness, and fiscality, maintaining tax protection for essential medicines and aligning fiscal policy with the fundamental right to health.

Keywords: CBS; COFINS; IBS; Dual VAT; PIS.