← Case studies

See Research Plan document →

The current social security website indicates low scores for usability, efficiency, and accessibility, according to published research.

Putting people first and embracing a user-centered approach is widely recognised by many governments and intergovernmental organisations as a fundamental driver for the development of electronic public services. As an example, the UK Government Service Manual advice is that “when designing a government service, always start by learning about the people who will use it. If you do not understand who they are or what they need from your service, you can’t build the right thing” (UK Government, 2017). Similarly, the OECD calls for a paradigm shift from traditional government centricity to citizen centricity (OECD, 2009). Despite the apparent unanimity, many critics claim that the users’ perspective has been overlooked, and that the decision-making process has been primarily guided by supply-side factors and technology developments (Bertot & Jaeger, 2006; Ebbers et al., 2008; Kunstelj et al., 2004).

While the Portuguese government has a reputation of embracing a user-centered approach to the development of its digital services, the scarce research published about the subject indicates gaps, critical errors, and non-compliances across the 130 websites analysed. Amongst them, the social security website obtained high scores for content (72%) and lower scores for usability (56%, including particularly low results for sub-indicators responsive design, 14%, and font size, 29%), efficiency (45%), and accessibility (14%).

This project’s research question is: to what extent is the Portuguese social security system providing a digital experience that effectively serves the users’ needs?

In the first part of the research project, a survey will help to identify the high-level themes and users’ needs. In the second part of the research, interviews will help to understand some of the survey’s findings and clarify some of the users’ needs. Finally, in the third part of the research, usability testing will discover how users’ needs are being met by existing designs. 

Surveys have a number of advantages in comparison with other data collection methods. Particularly relevant to this study are the high-level insights into the user experience that can be obtained and the themes (or users’ needs) that can be identified. As Lazar et al. (2017, p. 106) noted, "surveys can be very useful for getting an overview, or a snapshot, of a user population". Moreover, a large number of people spread throughout a relatively large geographical area, such as Portugal, can be reached.

Interviews will help to gain an understanding of, and clarify the initial findings. In addition, if new themes start emerging, it will be possible to adjust the interview guide, ask more detailed follow-up questions, and therefore generate new findings. Another advantage of an interview is data granularity. As Lazar et al. (2017, p. 106) mentioned, “A survey is very good at getting limited shallow data from a large number of people, but is not very good at getting deep, detailed data”. In contrast, interviews can discover the core problems, offering a more complete picture of the users’ needs. This does not mean, however, that the interview will have specific questions only. In fact, interviews will start with general questions and move towards more specific ones over the course of the session. As Lazar et al. (2017) pointed out, the process of gathering requirements necessitates asking “broader questions about current—possibly non-computer—practices, future goals, frustrations and concerns” (Lazar et al., 2017, p. 193).

While the focus of the first and second parts of the study (the survey and the interviews) will be on what the participants think and say, the main focus of usability testing will be on what the participants do. As McCloskey (2014) noted, “The most effective way of understanding what works and what doesn’t in an interface is to watch people use it”. Usability testing focuses on how users interact with an interface in real-time. The goal is “to improve the quality of an interface by finding flaws-areas of the interface that need improvement” (Lazar et al., 2017, p. 264). These findings are expected to be more granular than the ones previously uncovered by the survey and the interviews.

We will report the findings as proposed in the research plan document, and then strive to work in close collaboration with social security staff (senior managers, software engineers, project managers, designers, etc) and its users (citizens, business owners, employees, other government entities, etc) to prototype and test user experience design solutions. If validated, we will build and optimise using lean software development principles.

The social security website has an engagement of around 5 million visits per month. Citizens use the website to do things such as find information about retirement plans or apply for benefits. It is therefore essential to provide them with an accessible, inclusive, and user-friendly interface that will save them time and stress.

Businesses use the website to do things such as manage statements or pay bills. Good government digital services allow businesses to spend less time on administrative tasks and have more time to focus on growth.