How to use Paired Comparison in Application Planning

August 29th, 2007

For our new decision making technique of the week, we’ll look at paired comparisons . Paired Comparisons are a method for comparing options when trying to decide which problem to tackle first, or which choice is the best one. For instance, I’m writing a web app and I have several directions I could take. Which one do I focus on?

Which Problem First?

I have several problems I can tackle while building the app, all of which can be done now. Of course, some of them fall into a natural order because they will be easier to do after another problem is fixed. But it is still flexible enough that I’m not sure how to proceed, which is causing me to work a little bit on each one and not really get anywhere.

List Your Options

Begin by listing all of the options that you are considering. If an option clearly depends on another option list it anyway. This method will address those options in the process.

Scoring Logic
CRUD (Create, Read, Update, Delete) of options and tools
Interface
Design
Social network functionality
Authentication and Permissions

I could use a pareto analysis to rank these options, but they are fairly interrelated, so it might help to compare each one to the other individually.

Create A Matrix

Create a matrix with each option in both rows and columns. Block out the bottom triangle of the matrix and the center diagonal like so:

Compare Options and Calculate Rank

For each blank cell, compare the row and column option. Put the letter of the one that is more important and the score (say between 1 and 5).

Now add up the score for each cell in which an option beat the other option. Divide the total score for each option by the total number of points awarded to get a percent rank. The resulting list is a priority list for how you should handle your problems, or for which option is most important. Here are my results:

What I really like about this method is that it reveals connections between paths that we may not notice when we are considering all paths together. By considering each one individually in relation to the other, we pick up on those connections.

My original options were implicitly ranked when I wrote them down, so I’m very surprised to see that authentication and permissions make it’s way to the top. In retrospect, it makes sense that the application should be designed with attention to authentication from the beginning, but I haven’t been thinking in those terms.

Have any of you used paired comparison to make decisions? What do you think of the method?

-zot

Support The Decision Strategist.

Popularity: 8%

Vote for this post on: These icons link to social bookmarking sites where readers can share and discover new web pages.