Zoe Hoy
Mark Xu
Portsmouth Business School, University of Portsmouth, Portsmouth PO1 3DE, UK Author to whom correspondence should be addressed. Information 2023, 14(6), 322; https://doi.org/10.3390/info14060322Submission received: 25 April 2023 / Revised: 28 May 2023 / Accepted: 31 May 2023 / Published: 6 June 2023
(This article belongs to the Special Issue Optimization and Methodology in Software Engineering)Agile software requirements engineering processes enable quick responses to reflect changes in the client’s software requirements. However, there are challenges associated with agile requirements engineering processes, which hinder fast, sustainable software development. Research addressing the challenges with available solutions is patchy, diverse and inclusive. In this study, we use a systematic literature review coupled with thematic classification and gap mapping analysis to examine extant solutions against challenges; the typologies/classifications of challenges faced with agile software development in general and specifically in requirements engineering and how the solutions address the challenges. Our study covers the period from 2009 to 2023. Scopus—the largest database for credible academic publications was searched. Using the exclusion criteria to filter the articles, a total of 78 valid papers were selected and reviewed. Following our investigation, we develop a framework that takes a three-dimensional view of agile requirements engineering solutions and suggest an orchestrated approach balancing the focus between the business context, project management and agile techniques. This study contributes to the theoretical frontier of agile software requirement engineering approaches and guidelines for practice.
Agile methodology, since its introduction about 20 years ago, has been widely implemented for software development [1]. Agile approaches help organisations to act with speed [2] and flexibility to quickly adapt and react to changing situations [3]. There are many agile variants; Scrum, for example, is one of the most used approaches in industry [4], as it can have short software development cycles of typically 2–4 weeks, namely “sprints” [5], and identify user requirements just in time [6]. It is reported that agile projects are twice as likely to succeed when compared to traditional projects [7]. However, there are contradictory reports that show 8% of agile projects failed and 50% were challenged for being over-budget, over the time estimate, or with smaller scope than initially specified [7].
Using agile is not without challenges, and the challenges tend to vary along the software development process. For example, Rehman [8] asserts that there is no proper separate phase of testing in agile methodology, and taking out time for comprehensive testing is difficult, thus keeping hold of the testing documents, test data, test cases, and scenarios during agile rapid development is a challenge that is to be worked upon. Agile is a popular methodology used for requirements engineering (RE)—a vital process to gather requirements for software development [9,10]. Agile for RE also faces challenges due to multiple stakeholders and changing dynamics of the business context. The process encompasses eliciting the requirements, analysing, specifying and validating, which requires interaction with stakeholders, prioritising the requirements and managing conflicting requirements [11]. RE produces specifications including both functional (FR) and non-functional requirements (NFR). Since customer requirements are continuously changing, agile teams are more focused on delivering the product rather than focusing on the documentation [8] which can weaken functional and non-functional design and testing. Some of the most frequently quoted ARE (agile requirement engineering) challenges concern non-functional requirements (NFRs) [12]. Weak requirements engineering is known to be a main reason for project failures [13].
The above challenges tend to be specific to the agile approach. Research also reveals challenges rooted from wide organisational settings. For example, Neto [14] noted that many challenges in the application of large-scale agile development relate to either RE or software testing, and in particular to the alignment between these areas. Alignment refers to coordination of practices, artifacts, and roles. The specific context here is the large organisation’s agile project, and the challenges all point to dynamic changes and misalignment in documentation, understanding, and coordination for actions. Along similar lines, Luong et al. [1] suggest that anxiety, motivation, mutual trust, and communication competence were found to be significantly correlated with human-related challenges in agile teams. Solutions to these challenges require interaction with stakeholders, prioritising the requirements and managing conflicting requirements, using negotiation, collaboration and communication skills [11].
Some systematic literature up to 2020 on agile systems challenges shows consistent interest in describing the challenges, but lack of focus on ascertaining the problems in order to find solutions. In particular, it is not clear whether the key challenges are rooted in the agile methodology along, or in non-agile methodological issues such as the context of the business, the nature of the project, the fit/alignment between the chosen agile method, and the specific context of the project, and also the execution and management of the agile projects. An early study [15] found agile challenges are disproportionate and significantly mismatched when compared to the solutions. This raises two main research questions:
RQ1: What are the typologies/classifications of challenges faced with agile software development in general and specifically in requirements engineering?
RQ2: How do the solutions address the challenges?Answering the questions can contribute to the literature debate on agile software development challenges with the insight drawn from the latest literature from 2020–2023, and for practice, it will help refocus on finding suitable solutions in order to cope with the challenges.
To achieve the aim as laid down in the research questions, we need to place a few boundary conditions on the scope of this paper. Firstly, given the vast and diverse literature on agile software development, we restrict the enquiry to agile requirements engineering. As aforementioned, RE is the most vital process in the software development cycle. Secondly, we assume other stages, e.g., agile software testing, face the same organisational and project conditions; hence, the non-agile methodology challenges are posited to be the same as that of RE. Thirdly, we include both RE challenges and solutions discovered from our own systematic literature review covering the period from 2009 to 2023.
The rest of the paper is structured as follows: Section 2 discusses the research method by explicating the research strategy and data extraction and analysis processes. Section 3 presents the results and discussions. Section 4 presents a typology/classification of the key challenges-solutions framework. Section 5 concludes the study with discussions of research contributions.
This study took a two-step approach to address the research questions. First, a systematic literature review (SLR) was conducted to identify and classify all the challenges and solutions related to agile requirement engineering using the guidelines of Webster and Watson [16]; second, a three-dimensional framework addressing agile development challenges was conceptualised and then validated by using the latest empirical study data by Pertti Karhapää et al. [17].
Following the SLR approach suggested by Webster and Watson [16], we conducted a wider search for studies published in the last decade to gain a holistic view of agile requirements engineering challenges and solutions. Studies were selected with the following inclusion criteria: peer-reviewed journal articles published in English between 2009 and 2023. Studies that did not discuss agile requirements engineering challenges and studies that only mentioned agile methodology as an example without further discussion in the main article were excluded. Scopus—the largest database for credible academic publications was used primarily for this research. We also extended the search to other databases including Wiley Online Library, ProQuest, IEEE Xplore, Emerald, EBSCOHost, and Web of Science. The majority of the articles found in these other databases were also contained in the Scopus results, except one paper from EBSCOHost which was included in the final corpus. Duplicated papers from these multiple sources were removed. The final corpus was made up of 78 valid papers from 38 journals (see Appendix A Table A1).
The search string used for Scopus was (TITLE-ABS-KEY(“agile methodology” or “agile method” or “agile approach” or “agile development” or “agile practices” or “agile requirements engineering practices” or “agile way” or “agile software development”)) AND (TITLE-ABS-KEY(“requirements engineering” or “user story” or “user stories” or “feature” or “task” or “requirement”)) AND (TITLE-ABS-KEY(“challenge” or “problem” or “issue” or “obstacle” or “limitation”)). Search Terms commonly used to describe agile, requirements engineering, and challenges were adopted from previous studies. As a single reviewer collected the data, a protocol was not needed to ensure consistency among reviewers. Figure 1 shows the selection process followed, where Dybå and Dingsøyr’s [18] quality assessment questions were used to assess the eligibility of the included papers. The distribution of 78 journal articles by year of publication is shown in Appendix A Figure A1 and the distribution of research methods used in the included studies is shown in Appendix A Figure A2.
Webster and Watson’s [16] concept-centric approach was used to tabulate the challenges and solutions for data analysis. Concept A consisted of the agile requirements engineering challenges identified in the included studies, and concept B represented the solutions identified for the agile requirements engineering challenges in the included studies. According to Webster and Watson [16], a logical approach for analysing systematic literature is to group them and present them. Hence, a word cloud was created to visualize the reported challenges so as to identify the most prevalent terms used. The challenges associated with the most prevalent terms were organised into 11 subgroups, and likewise, the associated solutions were visualized and inductively grouped into nine subgroups. During the analysis phase, seven additional papers were found through backwards snowballing, to further evaluate some solutions cited in the included studies; these were additionally quality-assessed before use.
Next, we classify the 11 challenges and the nine solutions into three dimensions according to their thematical relevance to (1) organisational setting/business challenges, (2) project management challenges, and (3) agile methodology challenges. This classification is based on the themes that emerged with reference to previous research where one or more of these dimensions are mentioned mostly in a haphazard manner, (e.g., [20,21,22,23,24]).
As a systematic review, we are able to calculate the frequencies of the themes addressed in each study. Although the frequency does not necessarily mean the importance, it shows the commonality or prevalence of the issue in agile development and application, also the concentration of research attention, despite the variations and diversities in agile projects with different organisational contexts.
From the 78 papers reviewed, we identified 11 challenges which are briefly discussed below. The sources for these 11 challenges are listed in Table A2.
Challenge 1 (C1)—Quality Requirements (QR) are neglected. The focus remains mainly on FRs [25]. QRs are ignored during the early sprints [26]. Late integration testing to verify requirements could reveal QR defects [27]. Late testing and late QRs can cause rework [27,28]. Changes in Functional Requirements (FRs) affect QRs [29] and QRs are rarely implemented in a single piece of code [27]. Requirements from different stakeholders could conflict [30] that lead to poor QRs.
Challenge 2 (C2)—Minimal documentation. Agile is less documentation-oriented, and documentation tends to be ad hoc, inaccurate, incomplete or non-existing [31]. Agile relies on sharing tacit knowledge [6], which could lose meaning over time [32,33] and through personnel turnover [21,34]. Problems arise when there is a communication breakdown [23], e.g., user stories are difficult to validate and maintaining documentation can be a daunting task [21].
Challenge 3 (C3)—Inappropriate prioritisation method. Ordinal scales and ratings to measure priority are difficult to make objective [35]. There are scalability problems with large numbers of requirements [35] that make it difficult to select the optimal requirements [30], and hard to maintain a requirement priority list [36]. Functional requirements (FRs) are often prioritised over non-functional requirements (NFRs) [37]. ‘Business value’ overweighs other aspects leading to ignoring NFRs—for example, maintainability or security [21]. There is a reported case where customers were unwilling to prioritise nor accurately represent their requirements [34].
Challenge 4 (C4)—Managing change. Business needs evolve [20] that require rapidly changing requirements and continuous reprioritisation [38], which causes an instability issue [39]. The impact and risk of requirement changes needs to be considered [40]. Requirement changes cause existing documents to be updated [41] and can impact on the cost and schedule of the project [42]. When requirement changes are not constrained to the start of a sprint, disruption or even cancelling a sprint can happen [43].
Challenge 5 (C5)—Poorly written requirements. Requirements can be inadequate [44], poorly written [28] with quality defects [21], ambiguous [45,46], underspecified, incomplete, inconsistent, and unfeasible [38], too granular or too high-level causing differing interpretations [47]. User stories focus on describing functionality rather than quality requirements [37].
Challenge 6 (C6)—Inaccurate effort estimation. Estimates can be changed substantially by a requirement change in subsequent iterations [20,48]. Unreliable estimation happens due to the absence of historical data and experts [21,49,50], particularly for large user stories [51] and for agile teams in distributed geographical locations [52].
Challenge 7 (C7)—Customer unavailable or low availability. Customer involvement and interactions with the development team are challenging [24,27,53], particularly with offshore development where the customer is located on a different site [21]. The challenge also refers to customers lacking authority to make decisions, surrogate customers not conveying real requirements [54], a weak relationship [38], and the low availability of customers for negotiation, clarification, and feedback [27,40]. Requirements ownership is a problem when many levels exist between the real customer and the development team [55].
Challenge 8 (C8)—Customer knowledge. When customers are lacking knowledge about agile or do not believe in the agile approach, it is hard to obtain quality feedback from them [54]. The challenge also encompasses customers’ inability [24,46], lack of technical competence [54], incomplete domain knowledge [38,56], insufficient knowledge of requirements [21], and of agile software development methods [47].
Challenge 9 (C9)—Inappropriate architecture. Architecture decisions made in the early cycles can become redundant when new requirements arise [27,46,57]. There is then a cost to refactor the software to meet the customer’s needs [46].
Challenge 10 (C10)—Communication methods. It can be difficult to communicate efficiently with stakeholders [40]. There is a reported case where communication methods were too costly, requiring a technical background to use, and were not designed for agile. Face-to-face meetings are the best communication method and instant messaging is the worst communication method [58].
Challenge 11 (C11)—Maintaining a software requirements specification. This can be difficult for an overloaded team who worked on several projects simultaneously [59].
Unclassified challenges. Some challenges are not directly related to requirements engineering, but can be general to agile projects, hence they are put into this unclassified category. Inayat et al. [23] and Okesola et al. [24] reported contractual limitations as a challenge, where fixed-price contracts can prevent changes. An organisation working in a multi-disciplinary and regulated environment used traditional tools for requirements management and an agile tool ‘JIRA’ for development. The tools were disconnected [55]. Wagner et al. [38] suggest ‘insufficient support by project lead’, ‘implementation of features without requirements’, and ‘not enough time in general’ are challenges.
According to the frequency of these challenges in the papers, we depict the most prevalent (not necessarily the most important, but the commonality) challenges in Figure 2.
The reported solutions related to requirement engineering are summarised below. The papers for each solution are shown in Table A3.
Solution 1 (S1)—Provide the requirement information needed. This solution includes providing requirements in the form of test cases [13]; the RE-KOMBINE framework to specify requirements in a flexible way to support requirement change [23]; user goal-oriented stories and delivery stories [24,47]; W8 story cards [12]; creating use case diagrams [45,60]; and concise acceptance criteria in user stories [27]. The quality of user stories could be evaluated through a tool or by checking against a set of criteria [61]. A domain expert or business analyst role could help improve the quality and correctness of requirements [62]. Sprints of short duration with smaller requirements have clearer software requirements specifications [59].
Solution 2 (S2)—Manage the quality requirements (QRs). Timely integration testing of the whole system could help to identify any QR defects early. A project team could have a separate QR backlog and prioritise one QR for each sprint, or a whole sprint could be allocated to work on QRs. Some large, distributed projects give ownership of QRs to specialist teams, who ensure the QRs are implemented [27]. Elicit and manage QRs with their associated risks [63]. Maintaining a list of assumptions made about QRs by the agile team. The lightweight approach of the NORMATIC tool can achieve a balance between minimal documentation and neglecting QRs [24]; it supports FRs, NFR modelling and QRs. Follow guidelines developed for supporting quality requirements [37,64].
Solution 3 (S3)—Share knowledge about the requirements. Among global agile software development teams, Borrego et al. [32] found that instant messages and emails were preferred methods of communication and developed a method to extract architectural knowledge from these files. Other methods include a communications hub between developers and clients [58]; on-site customer representatives could improve user-developer interactions [65]. Frequent communication by means of face-to-face interactions, weekly meetings, and interview sessions [23,32,66] are recommend, using a surrogate customer if the customer is unavailable [24]. To share knowledge and synchronise development efforts, communities could be created around a shared interest [60].
Solution 4 (S4)—Manage a product backlog. This solution aims to cover different viewpoints, customer, architecture, FRs, and QRs using multiple backlogs [27,60]. Gaikwad and Joeg [47] suggest using the agile incremental-delivery model to renegotiate requirement priorities. All changes identified during a running sprint are reflected in the product backlog and considered for future sprints [43].
Solution 5 (S5)—Improve the estimation process. Estimation is a challenge for agile as the requirements are unstable at the start [24]. Solutions include eliciting rationales from each estimation to help the team achieve consensus for the estimate, setting aside regular time for estimating [43], and use of data-driven techniques to estimate effort [50].
Solution 6 (S6) Maintain requirements traceability. Firdaus et al. [29] propose a model to trace security and performance QRs. Urbieta et al. [67] propose an approach using Language Extended Lexicon to organise user stories and assist with traceability. Well-known tools such as JIRA could facilitate managing requirements [23].
Solution 7 (S7)—Identify the minimal documentation needed. A consolidated software requirements specification is recommended instead of various artefacts [59]. Architecture support is needed for agile, Gahyyur et al. [68] recommends that a sub-system model, reusable component list, architectural significant features, and a rationale are documented.
Solution 8 (S8)—Improve the method for requirements prioritisation. Consider three criteria in prioritisation: ‘stakeholder satisfaction’, ‘risk of implementing’, and ‘available resource’ [30].
Solution 9 (S9)—Share the product vision. Maintain a common product vision between the customer and development team to reduce requests for change, for a running sprint [43].
Unclassified solutions. Okesola et al. [24] recommend the adoption of legal measures for contracts to support the flexible nature of agile requirements engineering. Inayat et al. [23] suggest the use of fixed payments per release. These are not included in the classification as they are not directly RE-related.
According to the frequency of these solutions in the papers, the most prevalent (not necessarily the most important, but the commonest) solutions are shown in Figure 3.
The reported challenges and solutions as summarised in the above sections tend to be patchy, anecdotal as each one is drawn from a specific context where the study has been conducted, and diverse, as they are not all rooted in the deficiencies of agile methodology. In order to answer our research questions, a further examination was conducted by skipping the specific context and focusing on the thematic meanings of these challenges and solutions. It becomes apparent that many challenges and solutions are rooted in organisational setting/business problems and project management issues, in addition to agile methodology. Hence, we use these three dimensions to map out all the challenges and solutions. The results are shown in Figure 4. It is noted that some challenges and solutions fall into multiple dimensions.
The success of the agile approach relates to how well it fits into the organisation’s structure, resources, processes, and culture. Some of the issues in this dimension adversely affect agile practice including:
Large organisational setting —where many levels exist between the real customer and the development team, the requirement of ownership and loss of information at each level is a problem. Real customers may not be effectively engaged with the agile team for requirements, knowledge sharing, and feedback [55].
Dynamic changes in business — changes in environment, objectives, people, etc. require continual re-planning and adjusting to new requirements, which could lead to misalignment in documentation, understanding, and coordination of actions by agile teams.
Motivation and communication —anxiety, motivation, mutual trust, and communication competence were found to be significantly correlated with human-related challenges in agile teams [1].
Coordination and shared understanding —multi-functional teams in organisations may not share common conceptualisation on issues and inform changes, even with conflicting requirements. Poor coordination between these teams, particularly in distributed geographical locations [52], is potentially the root cause of agile problems.
Our analysis shows that organisational setting/business problems are more likely to create challenges in customer availability, knowledge, and communication, hence leading to poor requirement quality and specification, i.e., C5 Poorly written requirements, C7 Customer unavailable or low availability, C8 Customer knowledge, and C10 Communication methods.
Most of the agile challenges identified from this study are related to project management (PM) challenges. As seen from Figure 4, six challenges are classified into this dimension; they are C1 Quality requirements are neglected, C3 Inappropriate prioritisation method, C4 Managing change, C6 Inaccurate effort estimation, C9 Inappropriate architecture, and C11 Maintaining a software requirements specification. Large-scale agile projects would require an experienced project manager to be responsible for the whole project team and be accountable for the success or failure of the project [69]. The six challenges discovered in agile RE are actually rooted in the five PM processes: manage project execution, develop project structure plan, develop project schedule, estimate and define costs based on requirements, and develop and manage the team. An agile team under a project management matrix structure usually has its members from different functional areas and at different levels, with multiple stakeholders, external agile specialists, analysts and developers. As reviewed from this study, misunderstanding and conflicts in interest, requirements, costs and benefits, etc. can be the root causes of an agile project failure.
Three challenges are grouped under this dimension, due to direct links with agile methodology. They are C1 Quality requirements are neglected, C2 Minimal documentation and C5 Poorly written requirements. Agile is less documentation-oriented, which can cause difficulties for testing. Functional requirements (FR) are heavily reliant on user/customer involvement, whereas non-functional requirements|(NFR) cannot be proactively specified in the early sprint by customers, and hence tend to be ignored in many agile projects. It is clear that there is room for improving agile methods, but this is not the sole solution to cope with all the challenges.
Following the same approach, all the solutions are reviewed and classified into three dimensions as shown in Figure 4. It is interesting to note that seven of the identified solutions are most likely to fall into the category of agile methodology improvement, five of these are related to project management, and two are orientated to organisational setting improvement. A few solutions appear to cross the classification boundaries, i.e., S9—maintaining a common product vision, S8 on prioritisation methods, and S3—sharing knowledge are both organisational and project management solutions. This is elaborated in more detail below.
Organisation/Business Dimension has two solutions: Sharing knowledge (S3) and Maintaining a common product vision (S9). These are essential to the success of agile projects. Sharing customer knowledge among the agile project team reflects an organisation’s norm and culture, which has been ranked the third most common solution. It also requires effective coordination and communication between distributed teams and divisions. Maintaining a common project vision requires a visionary leadership to articulate clearly the goal and targets across levels and teams, which will turn into explicit requirement specifications.
Project Management Dimension contains five solutions: Manage the quality requirements (S2), Improve the estimation process (S5), Improve the method for requirements prioritisation (S8), and cross-boundary solutions S9 and S3. These solutions are orientated towards better agile project management, for example, the project team to have a separate quality requirement backlog and prioritise one quality requirement in each sprint, also improved estimation of time, costs, and risks in order to achieve consensus. As aforementioned, sharing customer knowledge and vision can be considered an effective project management attribute in the context of agile development.
Agile Methodology Dimension solutions include seven solutions drawn from the literature—the largest from the three-typology point of view. They include providing the requirement information needed (S1), managing a product backlog (S4), maintaining requirements traceability (S6), identifying the minimal documentation needed (S7), and cross-boundary solutions S2, S8, and S9 as discussed. These are solutions aimed at specific agile methodology improvements.
Mapping the prevalence (commonality) between the challenges and the solutions shows an apparent mismatch between the dimensions of challenges vs. solutions, i.e., it appears to put more attention on agile method solutions than considerations of organisational/business/project management endeavour, despite the fact that many challenges are actually rooted in organisational/business and project management dimensions. This is explored further in the next section.
In agile software development literature, there is overwhelming research interest in challenges, but disproportionately little in exploration and exploitation of solutions;
Of the solutions summarised by this study, the largest portion is aimed at finding ways to improve agile approaches, and to some extent, to improve project management. Challenges rooted in the organisational setting and business context have been neglected;
Challenges faced by agile projects (teams) are rooted in the aforementioned three dimensions, hence, improving agile methods would not solve all the problems, and any endeavour to improve agile practice has to consider the organisational/business setting and project management practice. We echo Al-Zewairi et al.’s [70] assertion that agile is ideal for projects that need to deliver in short intervals of time, with high capability of change in requirements, the capabilities of people working on the projects and technologies being used;
Organisational setting and business are complex, dynamic, and context-specific, which creates many challenges that demand different solutions. The fit (or alignment) between these complexities and agile suitability should be assessed before taking an agile approach. This reaffirms Neto et al.’s [14] assertion that organisational context is key in RE because agile methods emphasise reactivity and informal communication that are hard to guarantee across multidisciplinary teams within a large organisation;
The agile project and the team(s) are the visible entity in the centre to execute various agile tasks (e.g., RE, testing, implementation…) and deliver the desired outputs. The operations of an agile project are affected by four driving forces: (a) the organisational/business settings determine its governance, member composition, roles and responsibilities, resources and stakeholders; (b) project management competence and experience determine the effectiveness of initiation, planning, control, coordination and communication; (c) the business requirements and the expected functionalities of the software (FR and NFR); (d) the ability of the agile methods. Hence an orchestrated approach is needed.
The relationships and the driving forces, as suggested above, are depicted in Figure 5 as a framework for considering agile challenges and solutions.
This framework suggests several possible relationships that can be extended and validated by future research. We assume that the agile software functionality is the deliverables (in various versions during the development process) as the dependent variable, which is determined by the four factors as independent variables. Among these independent variables, inter-correlations exist. To give a detailed account:
Organisational setting/business context determine the functional and non-functional requirements, it also influences the governance, composition, management and operations of the project, as mentioned before, the levels of customers, the geographic locations of teams, the thin or multi business objectives, requirements, frequency of changes, can lead to different PM team structure and management mechanism. This also determines the suitability of taking agile approaches and the choice of the agile methods;
Requirements engineering is key to the agile process. Functional requirements can be identified and explicitly specified with the right customers (with knowledge and availability); however, we doubt that non-functional requirements can be accurately specified in the early sprints, because these are systems performance features that cannot be pre-specified by customers until they experience how the systems would function. Thereby, there is a backward loop to enable feedback from customers of early version functionalities, in order to stimulate NFR specification and improvement;
Agile methodology as an independent variable covers different agile approaches along the development process, e.g., Scrum, Kanban, Scrumban, eXtreme Programming [12], each with its own capabilities, conditions, and limitations, hence, the choice could affect the agile project performance;
Finally, the agile project team and management assemble and orchestrate the people, resources, skills, technologies, and methodologies to deliver the outputs.
Comparing our results to a most recent study by Gupta, Poels and Bera [46] where SLR (2009–2019) method and interviews were adopted, shows that our results confirm most of the challenges discovered by Gupta et al. [46], but significantly differentiated from that study by classifying the challenges to include organisational challenges, and mapping the solutions to develop a framework to address the gaps. Gupta et al. [46] have systematically searched for the challenges to requirements engineering with agile methods. Using 57 selected papers, they found 22 challenges, and added three from their own empirical study, which confirm eight previously found challenges [10,20].
Our study confirms all the project management and customer involvement-related challenges found by Gupta at al. [46], e.g., the team’s lack of involvement and motivation, breakdown of team communication and coordination, difficulty in managing dispersed teams, not sharing knowledge, lack of management involvement, difficulty in customer interaction, customer inability and disagreement, and difficulty in estimating time and costs. Our study partially confirms Gupta et al.’s study [46] on the agile methodology-related challenges, but not at the level of details, e.g., minimal documentation, detailed user stories not being created or integrated, difficulties in decomposing user stories, availability of testing resources, reduced testing and testing coverage, and incomplete NFR.
Some challenges identified in Gupta et al.’s study [46] are not actually challenges, but the repercussions of not dealing with the challenges—for example, incomplete and missing requirements, ambiguous requirements, and inadequate requirement verification. Although Gupta et al.’s study [46] also identified external visibility on project tasks and requirement volatility as challenges, these are not considered from an organisational perspective. Furthermore, the study proposes the use of conceptual models to address the challenges, but does not map out the gaps in solutions.
One of the implications of this framework is that the agile methodology does not cause all the challenges, and therefore it should not be expected to solve all the problems. Challenges rooted in other dimensions need to be considered for alternative solutions. As proposed in this framework, a coordinated, orchestrated approach centred around agile project management with sufficient consideration of the organisational setting and business context could deliver the intended software functionalities. For theoretical development, we propose that:
A shift from studying agile challenges to influential elements and their relationships. Studies on challenges show a sign of reaching saturation, as evidenced from a recent empirical case-based study by Pertti Karhapää et al. [17] (p. 40): “not all of the challenges found in literature were identified in our study, nor did we find any new challenges to add”. Shifting the focus would help explore more solutions relevant to their root causes;
More studies to examine effectiveness and efficiency of agile practice (including each method) in different context. Effectiveness of agile practice means satisfaction (and more sustainability) of meeting both FR and NFR requirements; efficiency means resource utilisation of the agile project—both intangible and tangible (time, costs…). This will help exploitation of existing agile methods.
For practice, we suggest the following:The business problems and the organisational setting attributes—multiple objectives, dispersed teams, distributed problem ownership and decision-making, customer resources and knowledge availability, etc.—shall be considered as the preconditions for taking an agile approach. That implies that although agile offers many advantages, it may not fit complex organisational situations where requirements vary significantly across different levels and geo-locations. We refer to this as agile eligibility evaluation;
In the case that an agile project has been approved, it remains critical for the leadership to articulate clearly the vision and goals to all the problem owners, team members, developers, agile specialists, and stakeholders. This is to ensure consistency in understanding the requirements, making changes in a dynamic manner, and developing/testing the functionalities before, during, and after the agile development process;
Once an agile project has been initiated, it is critical to ensure a competent project team has been established with adequate resources and control power. More importantly, customers as participants in the agile team shall have sufficient knowledge, time, responsibilities, and positive attitude to work effectively with agile specialists for functional requirements specification, feedback, and co-designing desirable functionalities;
With regard to NFR specifications, these are unlikely to be known explicitly before users and problem owners see the earlier sprint results in different phases. Hence, agile developers should initiate NFR in demonstratable fashion, so as to engage users and problem owners for verification and improvements. The rationale for this is that the rapid development of digital technologies in security, scalability, and architecture means that business users are not expected to foresee the potential of the advanced technologies during agile development;
Lastly, but not the least, for agile developers, programmers, and researchers in this area, there are rooms for agile methodological improvements, particularly on issues with minimal documentation, lack of traceability, difficulty in testing and prioritisation, and coping with implicit requirements and constant changes in requirements.
This study examined the challenges and solutions of agile requirement engineering based on a systematic literature review. Through thematic analysis, the reported challenges and the solutions are classified into three dimensions: organisational/business context, project management, and agile methodology. This answers research question 1 (RQ1). Figure 4 identified a mismatch between the solutions and the challenges. It showed a concentration of solutions for agile methodologies, but a lack of solutions to address challenges arising from organisational setting and business context and agile project management. This answers research question 2 (RQ2). The conclusion drawn from the insight is that agile requirement engineering challenges/problems are not all rooted in the deficiencies of agile methodologies, although there is room for improvements. Agile can only be effective when there is a supporting organisational setting coupled with appropriate project management. Organisational setting/business context shapes how agile projects shall be initiated and run, hence a coordinated approach addressing organisational/business and project challenges is required. A framework establishing the key dimensions and the inter-relationships has been suggested, which shifts attention beyond the current focus on agile methodology. This opens a new avenue for future research endeavour in seeking agile project solutions.
A number of limitations should be noted. First, the key evidence is based on a systematic literature review with limited scope, as described in the introduction, and a small sample. Second, although a structured and guided approach was followed for this study, unintentional oversight and bias in search and reading the literature can still happen. However, our results tend to be reliable, as the challenges and solutions are found to be similar in a recent empirical study conducted by Pertti Karhapää et al. [17]. Third, the classification of the challenges and the solutions into three-dimensional typologies are driven by the search results, but the analysis is subjective in nature. Fourth, after an extensive literature search, 13 Scopus indexed journal articles from the initial search results and one from the initial backwards search could not be sourced. However, this is a very small sample of the 440 articles which were analysed in our study, therefore the impact is negligible. Fifth, we have only selected papers published in English. This was not considered a limitation [71] as there were only 10 Scopus results for articles published in other languages for 2009 to 2023, meeting our search terms. Last, the framework illustrates a conception drawn from the analysis of this study. It can be a useful guide for theoretical development and for agile practice, but needs further studies to validate it. These limitations can be addressed by future empirical studies following the research avenue set out earlier.
Conceptualization, Z.H.; methodology, Z.H.; formal analysis, Z.H.; writing-original draft preparation, Z.H.; writing-review and editing, Z.H. and M.X.; supervision, M.X. All authors have read and agreed to the published version of the manuscript.
The authors have no relevant financial or non-financial interests to disclose. All authors certify that they have no affiliations with or involvement in any organisation or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.
The authors have no conflict of interest to declare that are relevant to the content of this article.
| Journal Outlets | Count | Percentage |
|---|---|---|
| ACM Transactions on Management Information Systems | 1 | 1.00 |
| Artificial Intelligence Review | 1 | 1.00 |
| CLEI Electronic Journal | 1 | 1.00 |
| Computers and Electrical Engineering | 1 | 1.00 |
| Computers and Operations Research | 1 | 1.00 |
| Computers in Human Behavior | 1 | 1.00 |
| Empirical Software Engineering | 4 | 5.00 |
| Future Computing and Informatics Journal | 1 | 1.00 |
| IAENG International Journal of Computer Science | 1 | 1.00 |
| IEEE Access | 5 | 6.00 |
| IEEE Transactions on Software Engineering | 3 | 4.00 |
| IET Software | 3 | 4.00 |
| Information (Switzerland) | 1 | 1.00 |
| Information and Software Technology | 10 | 13.00 |
| Information Systems Journal | 1 | 1.00 |
| International Journal of Advanced Computer Science and Applications | 4 | 5.00 |
| International Journal of Applied Engineering Research | 1 | 1.00 |
| International Journal of Computer Applications | 1 | 1.00 |
| International Journal of Computer Science and Applications | 1 | 1.00 |
| International Journal of Engineering and Technology (UAE) | 1 | 1.00 |
| International Journal on Advanced Science, Engineering and Information Technology | 1 | 1.00 |
| Issues in Informing Science and Information Technology | 1 | 1.00 |
| Journal of Industrial Information Integration | 2 | 3.00 |
| Journal of Information Technology Case and Application Research | 1 | 1.00 |
| Journal of Software Evolution and Process | 3 | 4.00 |
| Journal of Software Maintenance and Evolution | 1 | 1.00 |
| Journal of Systems and Software | 11 | 14.00 |
| Journal of the Brazilian Computer Society | 1 | 1.00 |
| Journal of the Chinese Institute of Engineers, Transactions of the Chinese Institute of Engineers, Series A/Chung-kuo Kung Ch’eng Hsuch K’an | 1 | 1.00 |
| Journal of Theoretical and Applied Information Technology | 1 | 1.00 |
| Jurnal Teknologi | 1 | 1.00 |
| Requirements Engineering | 5 | 6.00 |
| Scandinavian Journal of Information Systems | 1 | 1.00 |
| Soft Computing | 1 | 1.00 |
| Software Quality Journal | 1 | 1.00 |
| Tehnicki Vjesnik | 1 | 1.00 |
| TQM Journal | 1 | 1.00 |
| VINE | 1 | 1.00 |
| 78 | 100 |
Note—The distribution of the 78 included studies is shown in Figure A1. The years 2017, 2018, 2019 and 2022 yielded the highest numbers of publications, with 10, nine, 12, and 13 in total. There were no publications in 2013. The years 2009, 2010, 2011, and 2012 gave the lowest number of publications, with one, two, two, and one, respectively. The results show an increasing trend for papers published since 2014, a drop in 2020 and a second increasing trend from 2021, with the highest number of publications in 2022.
Note—Figure A2 shows the distribution of the research methods used in the included papers. The types of paper identified by Alavi and Carlson [73] for MIS research were used. Most of the papers were empirical (55 out of 78), and the results were based on data and observations; others were non-empirical and were based on ideas. Of the 55 empirical works, the majority were case studies (28), followed by surveys (13) and laboratory experiments (eight). Of the 23 non-empirical papers, the majority focused on a conceptual model (seven), followed by a conceptual overview (seven) and a conceptual framework and its applications (six). As most of these articles presented case studies, this indicated that the researchers sought to gain a rich understanding of the study context. There were only 13 surveys, meaning that there is an opportunity for future research in which more surveys are conducted, as these provide opportunities to explore possible relationships between variables and to produce models of these relationships for agile requirements engineering.
| Challenges | Sources | No. |
|---|---|---|
| C1 QRs are neglected | [6,10,12,17,20,21,23,24,25,26,27,28,29,37,38,40,46,53,54,63,64,74,75,76,77,78,79,80,81,82] | 30 |
| C2 Minimal documentation | [6,10,17,21,23,24,27,28,31,32,33,34,37,41,46,47,51,53,54,55,56,64,65,77,82] | 25 |
| C3 Inappropriate prioritisation method | [10,20,21,23,25,27,30,34,35,36,37,39,40,46,47,54,59,60,62,74,77,83,84] | 23 |
| C4 Managing change | [20,21,23,24,28,34,35,38,39,40,41,42,43,47,53,65,82,83,84,85,86] | 21 |
| C5 Poorly written requirements | [20,21,27,28,34,37,38,40,44,45,46,47,54,55,59,61,71,74,78,87,88] | 21 |
| C6 Inaccurate effort estimation | [10,20,21,23,24,27,31,40,43,46,48,49,50,51,52,53,54,62,89] | 19 |
| C7 Customer unavailable or low availability | [10,20,21,23,24,27,28,34,38,40,47,53,54,55,59,65,74,77] | 18 |
| C8 Customer knowledge | [21,23,24,31,38,46,47,54,55,56,76,77,90] | 13 |
| C9 Inappropriate Architecture | [21,23,27,34,46,54,57,76] | 8 |
| C10 Communication methods | [40,46,58] | 3 |
| C11 Maintaining a Software Requirements Specification | [59] | 1 |
| Solutions | Sources | No. |
|---|---|---|
| S1 Provide the requirement information needed | [13,17,23,24,27,28,42,45,46,47,53,59,60,61,71,74,83,91,92] | 19 |
| S2 Manage the QRs | [12,22,23,24,27,37,63,64,74,75,81,93,94] | 13 |
| S3 Share knowledge about the requirements | [22,23,24,28,32,39,47,58,60,64,65,66,95] | 13 |
| S4 Manage a product backlog | [23,27,43,47,60,65] | 6 |
| S5 Improve the estimation process | [10,24,43,50,89] | 5 |
| S6 Maintain requirements traceability | [23,29,65,67,92] | 5 |
| S7 Identify the minimal documentation | [28,46,59,68] | 4 |
| S8 Improve the method for requirements prioritisation | [25,30,84,95] | 4 |
| S9 Share the product vision | [43] | 1 |
