At Facebook we never did testing or looked at analytics. At Mint (read here how I quit Mint and lost $1.7million), Aaron (CEO) was very, very methodical and even flew in his dad who is a usability expert. We did surveys, user testing and psychological profiles. This was extremely useful in identifying the types of users we may have on the site and especially for seeing how people use the site. I never really did this before and was AMAZED how people use the site vs. what I expected. Most people know I am very practical or as my ex-gfs call it “cheap.” Anyways, here how our new start-up user tests and website usability testing questions.
How to get people to test for you
1- Recruit people for free on Craigslist.com. Get normal people who you think may use your site. You can also message people on facebook or yahoo groups and/or forums related to your topic. Post that you are looking for X (football, shopping, etc..) fans. The main goal is to get ideal users of your site. Asking Silicon Valley friends is relatively useless, unless you are building valleywag.
2- In cafes, go up to people who are on computers or look like a user you may want on your site. I did this and it was a nice way to meet women. Hey, not gonna lie
3- Prepare your questions ahead of time so you can compare responses instead of having random stuff from each user. Use surveymonkey.com to create surveys.
Questions to ask ahead of time
a) Background. Learn about them. what sites do they go to. how do they normally invite friends to things. who do they hear about things from. what do they do for fun. (these ?s are useful in shaping your product/experience)
b) Objective based. Figure out what you want to learn from them specifically. Then create questions to make them do ACTIONS on your site. Can you please add a friend? How does someone post a new item?
c) Ask them to speak outloud and tell you what they are thinking or confused about.
d) Follow up. If they didn’t do what you expected, afterwards ask them what they thought about it or what made them do something else.
WARNING: DO NOT TELL THEM WHERE TO CLICK OR SHOW THEM ANYTHING.
How to do the actual testing (online usability testing tools):
a) Record. Ideally you can record the audio and screen cast of the session. Snapx is a great video capture tool, also techsmith.com has some neat tools. Audacity, for sound. I didn’t pay for these but you can if you want to show / remember what happened. I just took notes on a textpad file.
b) Watch. Adobe has a great free product called acrobat.com where you can use ‘connect’ and do remote user testing for free.
c) Projector. If you have one use it. Nice to see on big screen instead of creepy style and peering over the side of a person.
d) Automated. My mentor showed me this site Usertesting.com which is AWESOME. For $19.95 a person they will get you someone in your target demographic and provide you a screencast of them using your site and show you bugs and other cool stuff.
(I also highly recommend Crazy Egg which tracks where people are looking on your site.)
Rinse wash and repeat. After you implemented the changes that you received from a few people, get fresh new people in and see if they notice the difference. Jakob Nielsen did research and you only need 5 people before you see overall site issues/trends. If you are a small start-up like ours, you don’t have the resources or time to be testing forever. Even after you do usability testing it’s important build relationships with your customers. I can say that the ROI on this type of stuff is insane, I know you may not do it or have not read this far in the post but DO it. At least 1 person, if it’s not worth it I will buy you a burrito.
Any other ways that people do user testing?