Results of agile project management implementation in software engineering companies

Agile project management methodologies, tools and techniques have been becoming more and more popular among Russian and foreign software companies. Though the agile software project management methodologies appear to be more flexible and sound alternative to traditional project management approaches, the practice of agile project management implementation needs more research. The article presents the research on the practical results of agile project management implementation in Russian software engineering companies. The survey-based study covers 8 companies and 35 their projects managed with the help of agile methodologies. In contrast to some optimistic researchers of agile practices, the research findings show that in the investigated projects agile project management led to the deterioration of cost and schedule performance. However, the quality, both perceived by clients and assessed by internal technical analysts, improved after the implementation of agile software project management practices.


Introduction
In the last ten years, many theorists and practitioners of software project management has been engaged in the discussion about the applicability of the traditional project management methodologies, tools and techniques to the area of software engineering and development.Nowadays, it is clear that classical project management approaches, such as those described in PMBoK [ 1] or other standards, need serious revision and should be supplemented or even replaced by more flexible and dynamic approaches.
As it was expressed by Highsmith (2009), linear thinking, prescriptive procedures and standardized practices are no longer suited to modem conditions of software development [2].One of the new directions in software project management is the agile project management methodology.Today agile tools and techniques are accepted and implemented by many companies.However, the current body of research still needs more empirical studies of the practical results of agile project management implementation in software companies.

Overview of agile project management methodologies
Basic principles of agile project management are described in the books of Highsmith (2009) [2], Chin (2004) [3], Larman (2003) [4], Abrahamsson et al. (2002) [5], Koch (2004) [6] and others.a Corresponding author: s_suetin@mti.edu.ru The main ideas of agile software project management are: • People and their face-to-face interactions are more important than processes and procedures, • Reliable software is more important than technical documentations, plans and budgets, • Cooperation with clients is more important than addiction to contracts, • Openness to changes is more important than following plans.
On the basis of these general principles many various agile methodologies were developed with different tools and techniques.The most widely known agile methodologies are 'Serum' [7], 'Crystal Clear' [8], dynamic systems development method (DSDM) [9], feature-driven software development [10], ICONIX [11] and others.Very close to agile is the methodology of extreme programming [12].

Practical results of agile project management implementation
Though agile project management methodologies has received a lot of attention from theorists and practicing project managers, the empirical research on the results of the implementation of agile tools and techniques remains quite scarce.Even fewer researchers have investigated the agile implementation results and consequences in software engineering companies.Highsmith (2009) in his overview of agile practices presented the results of the survey according to which 89% of respondents reported about the increase in productivity, 84% -decrease in the number of errors and defects, 82% -decrease in projects' duration, 66% -cut of project costs [2].Mah (2008) conducted two case studies and discovered that in one company agile management led to cost cuts on 61%, time savings on 24%, and decrease in the number of defects on 83%.In the second company he found cost cuts on 5%, decrease in project duration on 59%, and decrease in the number of defects on 11% [13].Coforto and Amaral (2008) reported about excellent results of the implementation of agile project management in two software companies.
According to their research the agile project management methodologies can increase creativity of employees, higher level of self-organization in project teams, and creation of additional value for companies and their clients [14].
Though the number of authors who reported about positive results of the agile project management implementation prevails, there are some sceptical reports that underline that agile project management can lead to additional risks, decrease in controllability and as a result to poor performance [ 15].
Thus, there is a need for further research on the possible results and consequences of the introduction of agile project management in software companies.

Research methodology
The research was organized as on-line and, partly, faceto-face survey of project managers and top managers from 8 Russian software engineering companies.The survey covered 35 software engineering projects finished within a year before the start of the research.In all projects under survey which agile project management practices were implements.The respondents were asked about the results of the projects realized with the help of agile project management methodologies in terms of cost, schedule and quality performance indices.Cost and schedule performance indices were calculated according to the traditional formulas of earned value management [16].Quality performance indices in all cases consisted oftwo different metrics: index ofthe quality perceived by clients and index of the technical solution quality assessed by the internal technical analysts.Both quality performance indices were calculated as the ratio of the number of quality features of the project final results that fitted or exceeded the expectations (of clients or technical analysts) to the total number of quality requirements formulated up to the start of projects [17][18].Hence, quality performance indices evaluated how much of the quality requirements were implemented in the final results of projects.
The values of the indices provided by the respondents were compared with the average valued of the same indices calculated on all projects of a company, including those that were realized with agile practices and those that were managed with traditional project management approaches.The variances between average values and the values of the indices from particular projects were analyzed with traditional statistical methods.
For instance, one of the respondents reported that in one of her project the cost performance index was equal to 1,1 whereas the average value of the cost performance index in all projects of the company was 0,95.Hence, the variance equal to 0,15 was taken into analysis.If the value of an index in a particular project was smaller than the average value the variance was calculated as negative.Therefore, the analysis of the variances between the values of particular projects and the average values can provides insights about the effectiveness of agile project management practices in terms of cost, schedule and quality.

Research findings
The distribution of the variance between the average cost performance index and the same index in the projects analyzed is shown in Figure 1.The basic statistics on cost variance index variance are as follows: • Mean value: -0,096; • Standard deviation: 0,120; • Minimum value: -0,430; • Maximum value: 0,150.The distribution of the variance between the average schedule performance index and the same index in the projects managed with agile project management practices is presented in Figure 2.  The basic statistics on cost variance index variance are: • Mean value: -0,131; • Standard deviation: 0,198; • Minimum value: -0,420; • Maximum value: 0,290.In Figure 3 there is shown the distribution of the variance between the average quality performance (perceived by clients) index and the same index in the projects under investigation.The basic stat1st1cs on client quality performance index variance are the following: • Mean value: 0, 178; • Standard deviation: 0,152; • Minimum value: -0,150; • Maximum value: 0,430.Finally, the distribution of the variance between the average value of internal technical quality index and the same index in the particular projects analyzed is presented in Figure 4.The basic statistics on technical quality performance index variance are the following: • Mean value: 0,090; • Standard deviation: 0,104; • Minimum value: -0,120; • Maximum value: 0,290.

Discussion on findings
From the research findings we can conclude that in the companies under analysis the implementation of the agile project management methodologies brought the notable deterioration of the cost performance in software engineering projects.The cost performance index in agile projects worse than on average in all projects of companies (the mean variance is -0,096).There were 4 agile projects in which cost performance was better than in all projects.However, the improvements were not significant.
Agile project management practices were not able to improve the time performance in projects.Even more, the introduction of agile tools and techniques was accompanied by the decrease in project time performance.On average the agile projects were less effective in terms of time performance than all projects in companies (the mean variance is negative and equal to -0,131).Hence, the agile project management practices led to the deterioration of time performance which even worse than the deterioration of cost performance.However, it is worth noting that there was the group of 6 agile projects in which time performance was better than in all projects.Though the results of agile project management for cost and time performance are not impressive, the capability of agile management to create better quality is clear.From Figure 3 and 4 we can see that agile methodologies led to significant improvement of quality index from both perspectives -client and internal.The average increase in quality performance perceived by clients is even more than the similar increase in internal technical quality performance -0,178 and 0,090 respectively.

Conclusion
In contrast to the existing research [13,14] the findings of the research presented showed the implementation of agile project management methodologies in software engineering companies led to the ambiguous results.In the project analyzed the agile project management brought about the deterioration of cost and time performance indices.However, the quality performance metrics increased after the introduction of agile tools and techniques.
We can recommend for practicing project managers to be more careful in their agile initiative and take into consideration the possible drawbacks of these initiative for the project time and cost performance.However, in the projects where clients emphasize the high priorities of quality and are able to pay for this by additional budgets and increase in project duration the agile project management can be implemented successfully, without any reservations.

DOI: 10
.1051/ C Owned by the authors, published by EDP Sciences,

Figure 3 .
Figure 3.Client quality performance index variance distribution.

Figure 4 .
Figure 4. Technical quality performance index variance distribution.