Chapter 8: Application: Where Craft and Science Meet
Traditionally the issues of building applied theories and applying
the applied theories have been treated quite independently. This is
a mistake. One of the main reasons for building applied theories is to
convert research practices from craft-based practices to theory-based
practices. Thus it is critical to build applied theories based on the
real needs of the tools research community. Applied theory building must
therefore take place in the context of the expected domain of application.
Otherwise the results risk being irrelevant and impotent. It is
not adequate to take a pet theory and see how far it stretches.
In a complimentary way, the issue of craft-based practices will not
go away until the right theories are available. It does no good to
chastise tool developers for not applying psychological theories when
the psychology is in no position to offer meaningful and cost-effective
assistance.
One conclusion that can be reached from these considerations is that
it is important to evaluate applied theories in the context of the
outstanding research problems of a field. The field may be convinced
that the theories are worthwhile, and the theorists may be able to
derive sufficient requirements to ensure that their theories cover
a broad enough scope to the right depth.
Towards this end, this chapter explores the applicability and
usefulness of RODS, HASTI, and CoSTH in the context of a specific
field of tools research: reverse engineering tools. Reverse engineering
tools research is intimately concerned with cognitive support issues
(how to make system comprehension easier, for instance), it has
accumulated relatively deep craft wisdom about how to build such tools,
and yet it has a poor record of formalizing this knowledge in established
science-based theories. Two exemplar reverse engineering tools,
Rigi and
RMTool
are analyzed using RODS, HASTI, and CoSTH. The analysis reveals
the following:
- The theories concord with the beliefs within the reverse engineering
community concerning the cognitive advantages offered by the tools.
This provides a theoretical backing to the craft knowledge, and
simultaneously bolsters the confidence in the applicability
(scope, generalizability) of the theories.
- The cognitive support theories can reconstruct some of the design
ideas that were developed in subsequent design iterations of the
tools. This implies that if the theories could have been used
during development, some of the burdens of iterative redesign
might have been avoided. The suggestion to researchers is that
theory-based design methods can improve existing design methods.
- Many of the central concepts of the tools could have been derived
at or before the time the tools were being created. The main
aspects of the theories effectively predate the tools. This implies
that the tools could conceivably have been developed with the assistance
of the theories. In the case of RMTool, the needed theories were
in place more than a decade before RMTool was developed. Although
it obviously is speculatively and inconclusion, the obvious
suggestion to researchers is that theory-based methods might yield
innovations that would take years to make otherwise.
|