Project at a Glance

pegah.png

Problem

Pegasystems hired our group to do a qualitative usability study to get feedback on four main areas of the site and A/B test two possible navigation options.

Concept

Perform heuristic evaluation of the website to flag potential issues before usability tests. Recruit participants against three personas provided by client, and validate heuristic evaluation findings.

My Role

This was a group project with four people and we each participated across all activities. We split up website areas and compiled them into group heuristic evaluation. We all worked on creating a screener and moderator guide, recruiting participants, moderating sessions, prioritizing our findings, and presenting them to the client.

Tools/Skills

  • Moderating

  • Participant screener

  • Moderator guide

  • Usability testing

  • Remote usability testing

  • Heuristic Evaluation

  • Recruiting

  • Summarizing and prioritizing study findings

  • GoToMeeting

  • MURAL

Research

peganav2.png

  • The client provided us with three personas that represent their audiences: business architects and systems architects who are current customers, and technical professionals who are not current customers.

  • Nine usability tests, performed both in lab and remotely, that covered five different areas of the PDN website.

  • The tests consisted of participant briefing and consent, background questions, typical tasks for personas provided, follow up questions, and closing statements.

  • We presented the client with positive findings, opportunities for improvement, and behavior findings that were neutral but worth noting.

  • We prioritized the opportunities using Nielsen’s severity rating model and likert scales for overall impressions.

 

severityscale.png

Evaluation

 

Challenges/Limitations

One of our challenges was recruiting 8-10 people against three different personas. We worked with the client to change the focus from three different personas to two: existing and new users. This provided us more data to analyze the differences between users.

Next Steps

We recommended that the client would backtesting the new navigation in larger A/B test to verify our results.

Lessons Learned

Recruiting, especially with very specific requirements and internationally can at times be a challenge. It is important to stick to the moderation guide and ask the same questions in the same way, otherwise compiling results is from four different moderators is difficult. With different moderators and different levels of product expertise held by the participants, some of the findings are less insightful and need to be further researched.