ROI Dashboard Overview

1.0 Introduction

2.0 DACS ROI Dashboard Summary
2.1 Capabilities
2.2 System Requirements

3.0 Graph Procedures
3.1 Phase 1: Accessing the DACS ROI Dashboard
3.2 Phase 2: Creating Graphs
3.3 Phase 3: Obtaining Detailed Information .
3.4 Phase 4: Generating A Timeline

4.0 References

Appendix A: Definition of Box and Bar Charts
A.1 Boxplots
A.2 Bar Graphs
A.3 Schedule Variance

1.0 Introduction

This document is the user’s manual for the Data & Analysis Center for Software (DACS) Return On Investment (ROI) Dashboard. The DACS ROI Dashboard provides the user with information on the costs and benefits of various improvements to software technology. This data was obtained from open source literature. The user obtains access to information in the database through a Web interface, the ROI Dashboard, located at http://www.thedacs.com/databases/roi/.

2.0 DACS ROI Dashboard Summary

2.1 Capabilities

The DACS ROI Dashboard provides the user with capabilities to examine the empirical distribution of the impact of software technology improvements on the following variables:

• Productivity: measured by percent improvement
• Quality: measured by

1. Percent decrease in defect density
2. Percent of defects found

• ROI: defined as:
  
  
  
  where C is the (unnormalized) cost of the improvement and B is the benefits obtained
  
• Cost of the Improvement: measured as percentage of total project cost
• Project Cost: measured as percent change in project cost (from previous baseline or calculated savings from defect detection/prevention)
• Rework: percent change in rework from previous baseline.
• Schedule Variance: percent improvement in Schedule Variance as defined in Earned Value Analysis. See appendix.
• CycleTime: The percent change in cycle time (the time from inception to product release)

Empirical distributions of the variables above for each improvement are presented by graphs showing distribution-free statistics (median, first and third quartiles, and smallest and largest values) and parametric statistics for the distribution (mean and multiples of the standard deviation). The distribution-free statistics are shown on box-plots, while the parametric statistics are shown on bar charts. These graphs are further defined in the appendix.

A timeline is also shown for each variable for each improvement. This timeline allows the user to see the periods over which the indicated organizations implemented the improvement. A further description of the timeline follows later.

Detailed information is presented on each entry in the database. This detailed information allows the user to see how the impact on the above variables was presented in the source document, background information and other observations about the organization, and the reference from which the data was obtained.

2.2 System Requirements

To access the DACS ROI Dashboard, the user is required to have a computer with an Internet connection. A web browser must be installed on the user’s computer.

3.0 Graph Procedures

3.1 Phase 1: Accessing the DACS ROI Dashboard

The user should invoke a Web browser and enter the following URL: http://www.thedacs.com/databases/roi/. The web page that should be displayed is shown below.

3.2 Phase 2: Creating Graphs

Creating graphs consists of two phases where desired inputs to the graphs are entered. The first phase consists of a scrollable menu where one or more improvements is selected. The second phase consists of two radio buttons where the graph type is selected. The user must select one or more software improvement and one graph type.

A single improvement is selected with a mouse click (select). To select multiple improvements, depress the CTRL key simultaneously with a mouse click (select) on each improvement desired. The user’s choices are highlighted after they are selected. The improvements are split into two groups: those which have extensive benefit data and those with limited data.

The user should select a graph type by checking the appropriate radio button. After selecting software improvements and the graph type, mouse click (select) on the submit button.

Figure 1: Selecting Improvements and Graph Type

3.3 Phase 3: Obtaining Detailed Information

One or more graphs should be displayed on the screen. Each graph shows the empirical distribution of a variable for the selected software improvements. Graphs are only shown for variables for which data exists in the database for the selected software improvements.

More information is available about the statistics summarizing the distributions and the data from which the distributions are constructed. Move your mouse over the horizontal lines in the graphs. If the line represents a statistic (for example, a quartile or the median), a box appears. The value of the statistic is given inside the box.

Figure 2: Graph Selection with Results

If the line represents a data point used in constructing the graph, a box appears containing information on:

• The organization that provided the data point
• The value of the variable for that data point
• How that value was reported in the literature (for example, an increase in productivity might be reported as an increase in Functions Points per month or in Source Lines Of Code per hour)

Even more detail is available. Mouse click (select) on a line that produces such a box and another Web page will appear.

At the top of each bar is the total number of data points displayed for that improvement.

This page contains:

• The organization providing the data, the time period for which data is reported for that organization, and the software improvements they implemented
• Background information on the organization
• Observed results, including empirical data characterizing the effects of the software improvements other than that graphed
• Bibliographic data for the source of the data.

Additional graphs can be generated from the page presenting graphs by repeating Phase 2.

3.4 Phase 4: Generating A Timeline

Section 3.3 describes the results of mouseovers and mouse clicks (selects) on horizontal lines in the graphs. Mouse clicks (selects) in the blue background of the graphs also produce more information, namely, a timeline. The horizontal axis of the timeline shows years, while the vertical axis shows the value of the variable in the graph from the previous page. Entries are color-coded rectangles showing the period an organization was implementing improvements and the variable value. A key lists the organizations generating the data. Mouseovers and mouse clicks (selects) on the entries in the timeline produce the same detailed information available from the previous graph in Figure 3.

Figure 4: Timeline

4.0 References

(Tukey 1977) J. W. Tukey, Exploratory Data Analysis, Addison Wesley.

Appendix A: Definition of Box and Bar Charts

A.1 Boxplots

Figure A-1 presents an example of a boxplot, also called a box and whiskers plot. A boxplot displays the empirical distribution of a single variable. Half the distribution is in the center box. Whiskers, at the top and bottom of the box, show the extent of most of the remainder of the distribution. Finally, outliers and extreme values are plotted beyond the whiskers. Tukey (1977) first described boxplots.

The boxplot shows various statistics. Boxplots on the DACS ROI Dashboard display two measures of central tendency, the mean and the median. Traditional boxplots do not display the mean, which is the quotient of the sum of the data points and the number of data points. The median is such that half the sample is less than its value. If the number of data points, n, is odd, the median is the (n + 1)/2 th point. Otherwise, the median is the mean of the (n/2)th and the (n/2 + 1)th point. The median is less sensitive to extreme values and outliers; it is also easier to interpret for non-Gaussian (non-bell-shaped) and non-symmetric distributions.

The lower and upper edges of the box, known as “hinges”, approximate the first and third quartiles of the distribution. (The median is the second quartile.) The first quartile is such that a quarter of the data points are less than its value. If n + 1 is divisible by four, where n is the number of data points, the first quartile is the (n + 1)/4 th point. The lower hinge is the median of the points less than or equal to the median. In cases where the first quartile is found by interpolating between two data points, the hinge will typically come out as a different interpolation. Tukey defined hinges for ease of calculation. The upper edge is found, similarly, as the median of the points greater than or equal to the median. The upper hinge approximates the third quartile, which is such that three quarters of the distribution is less than its value. Note that the central half of the distribution is between the two quartiles. The interquartile range is a measure of the variability of the data.

Figure A-1: A BoxPlot

A.2 Bar Graphs

Figure A-2 presents an example of a bar graph. The statistics shown on a bar graph consist of the mean and first multiple of the standard deviation. The mean is defined by the following equation:

where n is the number of data points and x 1, x 2, …, x n are the data points. The standard deviation is defined by the following equation:

For Gaussian (bell-shaped) distributions, 68.3% of the population distribution is within one standard deviation of the mean, 95.4% is within two standard deviations, and 99.7% is within three standard deviations.

Figure A-2: A Bar Graph

A.3 Schedule Variance

This appendix derives expressions for the percentage improvement in Schedule Variance, as calculated from data on:

• The percentage of work completed by the scheduled delivery date
• The percentage of schedule overrun.

Schedule Variance is defined in Earned Value Analysis as:

where SV is Schedule Variance, BCWP is Budgeted Cost of Work Performed, and BCWS is Budgeted Cost of Work Scheduled. Note that Schedule Variance is negative at the scheduled completion date for a project that runs over schedule. The percent improvement (decrease) for projects that run over schedule is:

where SV₁ is the Schedule Variance before the improvement, and SV₂ is the Schedule Variance after the improvement.

A.3.1 Percentage of Work Completed

The percent improvement in Schedule Variance can be calculated from data on the proportion of work completed by the scheduled delivery (completion) date before and after the improvement. This proportion is the Schedule Performance Indicator:

where SPI is the Schedule Performance Indicator. Let

• 100 SPI₁ be the percentage of work completed by the scheduled completion date before the improvement.
• 100 SPI₂ be the percentage of work completed by the scheduled completion date after the improvement.

Then:

Assume that the Budgeted Cost of Work Scheduled is unchanged after the improvement. This will be the case if requirements increase with productivity so that total project costs remain constant. Then the percent improvement for schedule variance is:

For example, suppose 50% of the work is completed by the scheduled completion date before the improvement, and 87% of the work is completed after the improvement (see Bib. Reference 40). Then schedule variance has decreased by 74%.

A.3.2 Percentage of Schedule Overrun

The percent improvement in Schedule Variance can be calculated from data on the percentage the schedule is overrun. Let

• 100 Y₁ be the percentage the schedule is overrun before the improvement.
• 100 Y₂ be the percentage the schedule is overrun after the improvement.

The proportion of work completed by the scheduled completion date, assuming a linear expenditure path, is:

So this case can be transformed into an already solved problem. Thus, the percent improvement in Schedule Variance is:

For example, supposes projects take 70% more time than scheduled before the improvement and 5% more time after the improvement (Bib. Reference 43). Then schedule variance has decreased by 88%.