Performance of planning support systems: what is it, and how do we report on it?

Planning Support Systems (PSSs) are a family of computer based instruments specifically designed to support actors in their complex tasks in the field of planning. There is a gap between the high expectations that PSS developers have about the usefulness of their instruments and the instruments’ application in daily planning practice. PSS academics have proposed several ways to close this so-called implementation gap through a range of software, hardware and orgware approaches. Several of these approaches have been applied in practical planning settings. There is however a lack of consistent and structured reporting on the effectiveness of these approaches in improving PSS performance. Therefore, it is hard to distinguish between successful and less successful strategies, and it is difficult to draw overall lessons. This paper (1) proposes a comprehensive multidimensional framework that operationalizes PSS performance, and (2) analyses how recent PSS implementation studies have reported on this performance. The developed framework, based on literature from Group Model Building and group psychology, is sensitive to a wide variety of performance dimensions and therefore forms a useful guideline for assessing PSS implementation strategies. Studying these in a common framework supports the potential transfer of lessons to other PSS implementations. Most of the analyzed studies only posed hypotheses about which dimensions are improved through a specific strategy, but did not report on measuring impacts. By structurally measuring the effectiveness of a range of strategies to improve PSS implementation, lessons can be exchanged and a consistent body of knowledge can be built.