RETURN
ON TRAINING INVESTMENT
When asked to rate the contribution of the
new innovation program to the improved functioning of the
Product Development team, participants had no difficulty
assigning a percentage to their perceptions (only five5
declined to estimate). Responses ranged from 10% to 80%
of effectiveness attributed to the new program, with an
average of 40%. Participants indicated that their estimates
were conservative, considering that other factors contributed
to the group's improved efficiency (such as experience gained
on previous projects, more familiarity with team members
etc.) making it difficult to compare across projects. Even
as a conservative estimate, a 40% perceived improvement
in group functioning as a result of one initiative is substantial.
Current methods of calculating the return
on Human Resources Development (HRD) initiatives, particularly
for "soft skills" training, suggest converting qualified
participant estimates of improved efficiency into cost savings
based on staff salaries, benefits and overhead (SBO) savings
(Phillips, 1997). The perceived 40% efficiency improvement
is calculated on the projected SBO that the project would
have incurred had the team performed at the historical efficiency
rate. Based on the team's efficiency improvement estimate,
MDS SCIEX interpreted the actual team SBO incurred as 60%
of the projected SBO, as follows:
$519 / day - Average team member salary, benefits
and overhead (SBO)
$519 x 28 staff = $14,532 / day - Total team SBO
$14,532 x 100 days = $1,453, 200 - Actual team SBO for 5
months of product development
$1,453,200 / .60 = $2,422,000 - Projected team SBO, assuming
40% efficiency improvement
PROGRAM BENEFITS
Projected SBO - Actual SBO = $2,422,000 -
$1,453, 200 = $ 968,800
The investment portion of the return-on-investment
calculation includes all costs associated with the innovation
skills program, as follows:
PROGRAM COSTS
-
"trainer training" pro-rated for two Athena
sessions: $ 600
-
trainer's daily SBO x 2 training days:
$ 1,540
-
training day cost
-
total team SBO for one day $14,532
-
cost of training materials ($125 x28)
$ 3,500
-
participant and instructor meals ($5 x
30) $ 150
-
impact study design, data collection and
interpretation $18,089
$35,261
Considering only the conservative estimate of the impact
of the program provided by the participants, the return-on-investment
within the 5 months of use of the skills on one project is
as follows:
ROI (%) = Net Program Benefits x 100 = (Benefits - Costs)
x100Program Costs
Costs= $ 968,800 - $35,261 x 100 = $ 933,539 x 100 = 2648
%$35,261 $35,261
This calculation indicates that for every dollar spent on
the innovation skills program for the Athena group, MDS SCIEX
reaped an additional $26.48, after the cost of the program
had been recovered, based on team members estimates of improved
efficiency. This is a substantial return. Furthermore, this
simplified calculation in no way encompasses the full bottom-line
impact of the program.
Consistent improvements in design review meeting efficiency
and atmosphere, see Exhibits 2 and 3, are difficult to quantify
in dollars, but no doubt had significant effects on the project
goals. There is also no direct or easy way to assign a value
to any given idea or solution that emerged from the design
review meetings as a result of the new innovation tools.
The impact study reveals that meeting agendas were better
managed during the Athena project than in previous projects,
new ideas were encouraged and explored to a greater extent,
team members participated more and behaved in a less adversarial
manner, the quality of decisions improved, and ideas were
built upon more constructively, among many other improvements.
These results indicate that the potential for innovation was
enhanced and barriers removed as a result of the application
of the tools, setting the stage for improved group effectiveness
in innumerable tangible and intangible ways.
PROJECT OUTCOMES
The design phase took one and one-half months longer than
planned while accommodating 15 change requests and being initially
under staffed. The first Athena units were delivered to Manufacturing
less than one year from the start of design. Other similarly
staffed MDS SCIEX projects would have taken well over a year
to accomplish the Athena design effort. The Athena units have
a radically different architecture, which addresses all of
the shortcomings of the Research breadboard units including
transportation, safety, and manufacturability. The performance
of the commercialized units have exceed their design specifications
and that of the top breadboard units. All agency tests were
readily met.
The Six Thinking Hats technique was instrumental in the
development of a radically different mechanical architecture
for the Athena units. This involved the development of over
36 different high-level system architectures and up to eight
different concepts for each subsystem. The bulk of the concept
work was completed in five weeks when compared to eight weeks
on less complicated subsystems on other projects. These project
outcomes corroborate the team's estimation that the Athena
product development process was 40% more efficient as a result
of the new innovation tools.
Many of the problems discussed earlier with design reviews
were avoided with the use of the Six Thinking Hats technique.
The project team was not overwhelmed when a given design was
found to have problems since the review meeting also provided
possible solutions. The general feeling at the end of most
design reviews was optimistic even when problems were raised.
The biggest endorsement of the tool is the use of the process
by other project teams in their design review meetings and
in the development of new design concepts.
CONCLUSIONS
This paper has demonstrated the following three critical
ideas:
- Innovation tools can positively impact the product development
process. Without the Six Thinking Hats technique the Athena
project would not have succeeded in meeting its aggressive
delivery schedule.
- The use of innovation tools results in new behaviors,
which can be measured; hence their effectiveness can be
assessed. The Six Thinking Hats technique has succeed in
creating new behaviors related to the design review process
and in helping to create a more supportive environment for
innovation.
- Program evaluation models provide an effective framework
for assessing the impact of innovation skills training.
The assessment models used to monitor the Athena project
have provided a quantitative approach to analyzing the qualitative
and quantitative impact of an innovation technique.
The investment in innovation tools to help improve the product
development process was a sound investment and the impacts
of these tools can be easily assessed and quantified by the
users of the new process themselves.
REFERENCES
Six Thinking Hats is a registered trade mark with Advanced
Practical Thinking Training Inc.
Kirkpatrick, D.L. (1998). Evaluation Training Programs: The
Four Levels. 2nd ed. Berret-Koehler: San Francisco.
Phillips, J.J. (1997). Return On Investment In Training And
Performance Improvement Programs. Gulf: Houston.
Senge, P. (1999). The Dance Of Change: The Challenges To
Sustaining Momentum In Learning Organizations. Doubleday/Currency:
New York.
West, K. (1999). Group Knowledge Creation In A High Technology
Setting. Doctoral dissertation in preparation, Ontario Institute
for Studies in Education of the University of Toronto. |
