A Modified Delphi Approach to a New Card Sorting Methodology
Journal of Usability Studies, Volume 4, Issue 1, November 2008, pp. 7-30
Abstract
Open card sorting is used by information architects to gather insights from users to incorporate feedback into an information architecture. In theory, it is one of the more inexpensive, user-centered design methods available to practitioners, but hidden costs make it less likely to be conducted properly and affect the quality of results produced. The following proposes a new card sorting method called the Modified-Delphi card sort to replace the Open card sort. The Modified-Delphi card sort is based on a well-known forecasting technique called the Delphi method. Instead of producing individual models that are then analyzed as a whole, participants work with a single model that is proposed and modified throughout the study. The Modified-Delphi card sorting method produces more useful results to aid in the design of an information architecture than the Open card sorting method.
A series of studies were conducted to directly compare the Modified-Delphi and Open cart sorting methods. First, two parallel studies using both methods were conducted with the same dataset and number of participants. Then, two studies were conducted on the results of the parallel studies: a heuristic review and ranking with information design experts and an Inverse card sort with additional users of the proposed architecture. The Modified-Delphi card sorting method produced results that were found to be at least as good as the Open card sorting method results and in some cases, better.
Practitioner's Take Away
The Modified-Delphi card sort is an exciting new method that promises to replace Open card sorting as a pre-design method in information architecture. The following are some of the discussed benefits learned by this study:
- Get better results for feedback in to the design of an information architecture.
- Save time in laboratory studies by reducing the number of participants in a study and the amount of data to analyze.
- Possibly save money in laboratory studies by using fewer participants, fewer days of facilities costs, and fewer hours of analysis time.
- Number your cards; it really helps when recording results.
- A digital camera can help save time in recording data between tightly scheduled participant sessions.
Article Contents
- Introduction
- Current and Related Work
- Methodology
- Results
- Discussion
- Conclusion
- Next Steps
- References
- Appendix 1: Modified-Delphi Information Structure
- Appendix 2: Open Information Structure
Download Article
A Modified Delphi Approach to a New Card Sorting Methodology