« Ten fallacies of software development | Main | Approaches to product design »

Users versus researchers

After an exhaustive survey of users, JuxtConsult recently named Cleartrip the most user-friendly travel web site in the country. More details about the survey and JuxtConsult's measurement model are available here.

About a year ago, the astute fellows over at JuxtConsult published their User Friendliness Index for 2007, a report on "the most user friendly websites in India." At that time, we wondered just what gave JuxtConsult the right to preach usability -- after all, they don't even link their logo to their web site's home page (go here to check it out).

This is the methodology JuxtConsult followed while preparing their report last year:

[The] evaluation and classification of the top 121 highly popular websites in India was done by JuxtConsult by measuring these websites on 32 individual criteria grouped under 6 different 'usability' aspect of a website. The 6 usability aspects tested were 'branding', 'navigation structure and value added features', 'website design' 'company and contact info', 'contact responsiveness' and 'technical parameters'. All the websites were measured on the 32 individual criteria to arrive at the comprehensive usability index, or ‘User-Friendliness Index' (UFI), for each website.

In other words, their 'findings' last year were just their own opinions. Who considers 'branding', 'company and contact info' and 'contact responsiveness' as usability parameters anyway?

But the chaps at JuxtConsult got smarter this year--they decided that gathering actual user opinions makes more sense than touting their own opinions. Let's compare what they found when they asked users as opposed to imposing their judgement:

RankJuxtConsult's opinion (2007)User's opinions (2008)
1YatraCleartrip
2TravelguruMakemytrip
3MakemytripYatra
4CleartripTravelguru

Hats off to JuxtConsult for putting the users back in usability; for focusing on users instead of researchers--as we can see, user opinion is almost diametrically opposite to researcher opinion.

Posted on Tuesday, March 25, 2008 at 10:28AM by Registered CommenterHrush | Comments3 Comments

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments (3)

I introduced my parents to all these Indian travel sites (including Cleartrip) and more than usability, their biggest complaint has always been how annoying the entire experience of calling someone up in order to cancel their tickets or change travel dates is. They tell me that they had a better experience with Cleartrip but with a few others, they had to go through hell to get their refund. So I recommend working on the entire call center experience rather than making your AJAX UI more slick :-) Either that or you should seriously consider automating these other activities (why can't you just allow a user to login to the site and cancel the tickets and just get a refund immediately without any fuss?).

Unfortunately as a result of this, my dad is back to using the local travel agent (not that they offer any better customer service but at least there's this option of dropping by their office and demanding your refund!).
March 25, 2008 | Unregistered CommenterHarish Mallipeddi
Anybody who bothers to follow the link to their measurement model will find exactly what I did - nothing! It's a page full of marketing gook that provides no information at all about the methodology and criteria adopted for the study.

What makes this company think that I would be interested in buying a report and spend INR 480,000 every year with them without knowing even the basics - who all did the study cover, how large is the representative sample, what is the criteria for measurement, etc.

One look around the JuxtConsult website tells you what they really know about usability - not much.
- none of their links are distinguished from text content
- their right column is common across the site so it's not context-sensitive
- they use images instead of text for no apparent reason (funnily enough the one text item you would want to mask behind an image - an email address to prevent spam - is not masked)
- text on the diagram used is really too small to be any use at all
- they use light grey text for body content that will print terribly on a monochrome printer (most corporates - their target audience - use monochrome lasers for everyday printing)
- they don't offer a separate style-sheet for print
- the column widths change from page to page
- the border style on the right column changes from page to page
- their navigation is such that if you didn't click on one of the reports on the home page, you're not likely to find it from the navigation available on any of the other pages of the site. Goto their Products and Services page, and suddenly you can't find any of their three main featured reports - India Online 2008, NRI Online 2008 and the Website User Friendliness Study.

JuxtConsult - take a look at yourself before pointing fingers around the Internet. Seriously, what gives you the right to preach usability?
March 25, 2008 | Unregistered CommenterDhruv Chopra
Harish--we take cancellations and refunds extremely seriously at Cleartrip. And we do that because we've heard so many of our users complain about it.

You can cancel your booking online:
http://www.cleartrip.com/forums/40/topics/55

Unfortunately, immediate refunds are simply not possible at the moment. We'd love to be able to do immediate refunds, maybe someday...
March 25, 2008 | Unregistered CommenterHrush

PostPost a New Comment

Enter your information below to add a new comment.
Author Email (optional):
Author URL (optional):
Post:
 
All HTML will be escaped. Hyperlinks will be created for URLs automatically.