Social Security Tribunal of Canada

Website user testing results – June 2021

Our website revamp project is well underway. To make sure we’re on the right track, we did more user testing over the winter to see if our proposed new website structure really is more user-friendly. To make sure the evidence supports the new structure, we compared it to baseline testing of our current website done in October 2020. The data proves that our new website structure will be much more user-centric and task-focused.

 

Method

Out of the ten tasks chosen to represent what people would come to the Social Security Tribunal (SST) website for, we gave participants five randomly assigned tasks to complete in round one, and six tasks to complete in round two.

With each task, participants got scores in three categories.

Correct – self-serve

Applies when the user can figure out the correct solution to the task

Correct – contact/help

Applies when the user chooses the ‘Contact the SST’ link or chooses pages related to getting help for the specific appeal

Wrong

Applies when the user can’t figure out how to complete the task and moves on to the next one

Results

Correct – self-serve

Correct – self-serve, Increased from 27% to 58%

Increased from 27% to 58%.

More than doubled self-serve success rate (215% increase).

Correct – contact/help

Correct – contact/help, Decreased from 17% to 14%

Decreased from 17% to 14%

Shows greater confidence in ability to self-serve because less people choose this option and more people are able to figure out the correct solution.

Wrong 

Wrong, Decreased from 56% to 28%

Decreased from 56% to 28%

Halved the number of wrong answers and identified clear fixes to most remaining ones.

Improvements by task

We measured task improvement by comparing our current website structure to the new one we’re working on.

Task Total correct
Find the list of documents you need to give the SST to appeal a decision. 40 63
58% improvement
Find out how long it will take to get a decision about your Canada Pension Plan appeal at the General Division. 62 71
15% improvement
Find out how to change representatives during the Old Age Security appeal process at the Appeal Division. 7 92
1183% improvement
Find out who can help with an Employment Insurance appeal in Victoria. 43 80
87% improvement
The General Division dismissed a Canada Pension Plan Disability appeal but you received a new medical report that could affect the decision. Find out how to ask the SST to reconsider your appeal. 27 62
132% improvement
Find out if there are any fees involved with submitting an Employment Insurance appeal. 43 27
36% decrease
Find the form you need to fill out when you file an Old Age Security appeal to the General Division. 63 88
40% improvement
Your hearing for your Canada Pension Plan (non-disability) appeal with the Appeal Division is next week. Find out who will be at the hearing. 40 79
96% improvement
Find out whether an appellant who doesn’t speak English or French can get an interpreter at their Old Age Security hearing with the General Division. 64 90
41% improvement
Find out how to send your updated contact information to the SST. 55 69
27% improvement

Who did we talk to?

pie chart of user testing results statistics
Text version
Who did we talk to Total
Appellant 64%
Non-legal representative 18%
legal representative 11%
Added party 3%
Other 4%

Feedback

During both rounds of testing, we asked participants how we could improve the current website. Here are some suggestions we heard.

Feedback
Text version
  • Include less text overall with more process diagrams, audio and video components, and infographics.
  • Index published decisions by subject.
  • Group information according to appeal type.
  • Better explanation of the benefits of having a representative.
  • Organizational structure improvements, including to menu bars and sub-options.
Date modified: