We conducted a user study to evaluate if our data protection icon set was sufficiently comprehensible. This was necessary because the visual representation and recognition of abstract legal-technical concepts can be challenging. We also wanted to find out if other communication modalities were effective at conveying information on data protection, specifically: simplified definitions, vignettes, and classical legal terms. Some of the lessons we learned are:

1.
User-friendly label
A user-friendly label must always be associated to the visual representation of a data protection concept. It can’t be expected that users exposed for the first time to abstract or arbitrary icons will be able to recognize them easily, especially if the concepts are unfamiliar or technically difficult.

2.
Familiarity
Icons can be learned over time. The recognition rate improves as familiarity increases, even if the visual representations are arbitrary.

3.
Established visual language
The more the icons rely on an established visual language and common mental models, the more recognizable they are. We identified some elements of the shared visual language on data protection, but more must be integrated into a more recognizable (or more learnable) set of icons.

4.
Text scenarios
Text, if it exemplifies how data protection principles can affect people, has significant value. Our participants appreciated the text scenarios significantly more than the icons, thus we will conduct the test with a wider sample to conclude how generalizable this preference is.

5.
Multiple channels
We confirm our hypothesis: data protection information must be provided through multiple communication channels. Different users will use one channel or a combination of them, according to their preferences and their literacy.

6.
The real challenge
The challenge remains: what is the tradeoff between accuracy and coherence of representation and users’ desire of simplicity? Some of our icons were deemed “too complicated”, “too busy”, or “too similar to be different”. However, this need for simplification and distinctiveness conflicts with the precision of representation and elements’ coherence across the icon set, which we tried to achieve for legal purposes.

7.
Test in context
The next step will be to test the icons in context, instead of as stand-alone elements. If their role is giving a pre-understanding of what can be found in the different paragraphs of a privacy policy, rather than as the sole representation of a concept, they might be evaluated differently. Based on similar studies on contracts, we expect the inclusion of icons in the legal text to significantly enhance its skimmability and comprehensibility, compared to a text-only agreement.

 Download the transcripts of the interviews [.zip].