Doing research at 18F

Research isn’t only usability testing. Testing our assumptions by actively engaging with the world around us is at the heart of how 18F works. We do research when we meet with stakeholders to learn about a project, when we investigate and compare tools and systems, and when we find members of the public to tell us about their experiences with the federal government. Research can include questionnaires and surveys, as well as analytics.

These guidelines should be familiar to anyone who’s done research in the private sector or in academia. However, as public servants, we need to make sure that we’re following a few basic principles.

  • Respect: We honor people’s opinions and choices, only blocking actions that are obviously going to harm others. We make sure that anyone participating in our research is doing so of their own free will, and that they have enough information to make that decision responsibly.
  • Beneficence: We do our best to further people’s best interests, and we avoid actions that might hurt them. This goal extends to society at large: before doing research, we need to consider whether what we’re doing is in the best interests not just of individual participants but of the organizations and the country that we serve.
  • Justice: We do our best to ensure that we compensate research participants in some way for the time and energy that they contribute. We don’t have to pay money, but we do have to give something back to participants. Sometimes that means just saying “thank you” and letting them know how much their input has helped improve government services. Other times, it can mean making a card or handing out cookies or fruit. Participation incentives, which are common in the private sector, are more tricky here in government. So please ask one of the Research Guild leaders if you think they’re necessary for a specific study.

While only scholarly researchers are strictly obligated to follow these principles, they are widely accepted in the United States as a set of overarching values guiding all investigations involving humans.

This document explains two main tasks we do to operationalize our basic principles: informing and getting consent, and managing personally identifiable information (PII).

18F researchers have an obligation to ensure that our participants know what it means to participate in our research and how we plan to use the information we collect. Before getting consent, ensure participants understand:

  1. They cannot expect to receive compensation for their participation (unless otherwise agreed to with our agency partners — note that this is a legal issue!)
  2. The types of information we may record (this could be name, direct quotations, photos, screen captures, etc.)
  3. How we will share information we collect. Usual options include: No one ever (off the record), within the immediate project team, on a need-to-know basis (our default), and with the general public.

18F gets explicit consent from anyone who participates in our moderated research, generally by asking that our research participants sign a “design research participant agreement”. 18F maintains a template for internal use, and an example participant agreement for sharing with interested parties. Please make a copy of the participant agreement document template, put it into your project folder, and edit the highlighted text for each of your research studies.

Feel free to adapt our form as needed, but don’t change any of the legal language without asking #legalstuff first. If your participants are likely not fully literate in English, consider having the form translated to the correct language or working out a process for verbal consent using a translator.

When sending participant agreements electronically, please use our participant agreement email template. This template clearly specifies how people can opt out of a study and/or request that we do not contact them in the future.

Managing Personally Identifiable Information (PII)

We work in the open, but we need to be mindful of discussing agency partners, collecting any PII, or accidentally disclosing PII.

What is PII?

PII is information that can be used to distinguish or trace an individual’s identity, either alone or when combined with other information. Because there are many different types of information that can be used to distinguish or trace an individual’s identity, the term PII is necessarily broad. The definition of PII is not anchored to any single category of information or technology. Rather, it requires a case-by-case assessment of the specific risk that an individual can be identified using information that is linked or linkable to said individual. For that reason, Social Security Numbers, Alien Registration Numbers, and driver’s license numbers are always PII. Email addresses, phone numbers, and mailing addresses are sometimes PII, and when presented in combination with a first name, often become PII. Photos of people’s faces are almost always PII, and that’s why we always ask before taking photos and get explicit statements about if we can share them.

Sensitive PII is information which, if shared, could seriously harm or embarrass someone. Unique identifying numbers and biometric data is always sensitive: In general, combining:

  • Citizenship or immigration status
  • Ethnic or religious affiliation
  • Medical information
  • Sexual orientation
  • Criminal history

with PII always makes that PII sensitive. However, determining what is and isn’t PII, and what is or isn’t sensitive PII can often require some reflection.

Much of our research involves federal employees. When we are talking to federal employees (not contractors, and not vendors) about their work, they generally have no reasonable expectation of privacy. After all, the material may have to be disclosed pursuant to a judge’s order, a Congressional request, or a FOIA request.

As researchers, however, we need to follow the principle of beneficence. So just following the legal guidelines on PII isn’t enough if you’re collecting information that you know might harm or embarrass your participants if it became generally known, whether or not those participants are federal employees. Different agencies and organizations may have different levels of sensitivity and exposure concerns about types of personal information (identifying or not) you collect and how you use it. Attributing quotations with agency affiliation, for example, may be more sensitive than first name and photo. In those cases, the principle of beneficence demands that we think carefully about protecting participants before sharing working notes or finished reports, even if we have satisfied the law.

Storing PII and sharing research data

The best place to store PII is not to store it at all. The best place online, if you must collect it, is in a locked-down document on Google Drive that is only accessible to the immediate research team. Using that document, link each participant to a pseudonym form (eg a codename or number such “PO1”) Research reports and notes should contain only those pseudonyms. When it comes time to release reports outside of the immediate research team, refer back to your locked-down file for the correct sharing permissions. If you have any paper documents with PII, they always go into a locked file cabinet.

A good rule is to check with your project lead before sharing information outside the immediate team. What has been approved by GSA may not be approved by partners. Check the Appendix in Design page for detailed information about who can see what, when. The social media, collaboration, and security classes in GSA’s Online University can also be helpful in managing access.

Be especially mindful as you post meeting notes or sidebar conversations during interviews in Slack channels. No PII should go into Slack channels unless your participant has explicitly consented to have PII shared with the general public. And even then, be aware of the potential level of visibility any information will have once you post it in Slack.

Carefully restricting access to personally identifiable information is a matter not just of people’s right to respect but of their right to privacy. For more information, please see the Design research privacy impact assessment (PIA), or this 30-minute overview of privacy as it relates to research narrated by the GSA Privacy Office.

Bottom line: If you have questions about sharing information, just ask. If you’re not sure if you’re collecting PII, you can ask on Slack in #research to get other researchers’ perspectives, and in #legalstuff to track down relevant policy guidance. If you are not sure where the right place might be to store any given file, or what access permissions to grant, you can post a question to #infrastructure.

Join the research guild!

The Guild talks in #research and meets once a week to discuss the theory and practice of asking questions. The current guild leaders are Joanne McGovern and Ben Peterson.