The power and pitfalls of focus groups

Focus groups, which are a research methodology in the social sciences, can be useful tools for exploring the perspectives and experiences of physics-community members. In academic settings, they can be used to answer questions such as “How can teachers better engage students in physics classrooms and labs?” (see the article by Natasha Holmes and Carl Wieman, Physics Today, January 2018, page 38 ), “What would improve student and teacher experiences in a physics department?” and “What are the unique experiences of minority students in physics departments?” The results of a focus-group study can help improve classrooms, labs, departments, and university policies.

In this article, we describe how best to conduct focus groups for science departments. During a session, a discussion is held with multiple participants—ideally four to eight—to address a specific issue. Focus groups typically last for 60–120 minutes and are facilitated by a researcher who asks a series of predetermined questions; depending on a group’s goals, those questions can be structured or open-ended. The discussion format can be particularly valuable when group members build on each other’s responses and debate any differences of opinion.

The first step is to ensure that a focus group is the most appropriate method to answer the questions. In general, focus groups are most effective when the researcher wants to collect descriptive, detailed data on the experiences of numerous people. And they are best used for answering questions about the following:

► Opinions: What do participants think about a certain topic? For example, what teaching approaches do students like or dislike in physics courses?

► Perspectives: What are participants’ personal experiences with a certain topic? For example, how do students interact with teachers and staff in a physics department?

► Needs: How can the organizers of the discussion better serve a given population? For example, what do students need to succeed in physics courses?

► Evaluations: Is a program or product having the desired impact? For example, how effective is the physics program at the university?

Focus groups are not as useful for assessing research questions about actions and behaviors or controversial and sensitive topics. During discussions, people may say what they do, but the researcher has no way of knowing what they actually do. Experiments and observations are better methods of learning about actual behaviors. In addition, people may not feel comfortable discussing controversial issues among strangers and may refrain from expressing their views candidly. Extra attention would need to be paid to group composition and confidentiality. To make people feel more comfortable, it may be more appropriate to form the group around certain demographic characteristics, such as age or gender.

Focus groups are not useful for learning about the opinions of an entire population. Focus groups involve a small number of people, whose opinions might not reflect the opinions of a larger group. Surveys would be a more appropriate method for large groups.

How do you form a focus group?

It is not enough to ask the right questions. As the chair of a physics department or task force, say, you need to ask the right people to participate. Discussions should include all relevant perspectives, even if that means assembling more than one group. If all those perspectives are not represented, then the data will be incomplete. For example, when considering how to best revamp the physics curriculum at a school, you may include only faculty in the focus group sample. By not asking students their opinions, you would not have all the requisite data to effectively answer your research question. That problem often arises because the most convenient group to sample may not be the most appropriate group.

Controversial topics especially may mandate more than one focus group. But even for less contentious topics, conducting the same interview with multiple groups can provide a more robust data set for analysis. Having multiple groups will also allow for comparisons between them.

In focus groups, researchers gather narrative data, qualitative data on opinions and perspectives, and observational data. Narrative data encompass everything said during the sessions, which are typically recorded and transcribed. The transcript can be coded and analyzed for important themes related to the research questions. Every word can be a data point. Coding of qualitative data sets is a well-established research technique used across the social sciences to organize and sort vast amounts of data. Through that process, a story may start to form that offers critical insights into your research question.

Observational data are typically the notes taken during the focus group by a researcher who assists the facilitator. They include nonverbal data, such as tone of voice, facial expressions, body language, and the degree of the participants’ engagement. The response of an individual to someone else’s statement can provide valuable information that can be missed in the narrative data.

Who should facilitate a focus group?

It is vital to the integrity of the research that the facilitator should not have a stake in its outcome. Anyone with such a stake would have a personal bias that can influence participant responses and threaten the data’s validity. Participants may be less willing to be honest with a project stakeholder, or a stakeholder may cause participants, sometimes unintentionally, to respond in a particular way. Working with someone outside your research team, organization, or department will provide the most objective, high-quality results.

It is not easy to conduct a focus group, and a successful session requires a trained and qualified moderator. A facilitator has many roles. Among other tasks, he or she sets the tone for the group, leads the discussion, ensures that it stays on topic, asks questions, and maintains an environment in which everyone can participate equally. If a facilitator performs any of those tasks poorly, the data will be affected.

An effective facilitator is also aware of how his or her own actions can shape the behavior of a focus group. Just as the facilitator reads the body language and tone of participants, participants respond to the cues and tone of the facilitator. If a facilitator looks skeptical of a participant’s response to a question, that could affect how that participant shares for the rest of the session. Conversely, if a facilitator looks pleased with a response, that could influence other participants to respond in that same way.

The relationship between a facilitator and participants needs to be understood before organizing a focus group. As the head of a department, you may be a trained facilitator, but asking your students to discuss their experiences with you is not likely to work. Anyone familiar with the participants could bias the results, since participants may try to please that person; also, group members may be less honest with someone in a position of authority over them.

The focus group process, step by step

To start the process, you must first determine whether a focus group is a good fit for your overarching research question. Once you have decided to use one (or more), invite a social scientist to help plan it and design the discussion questions. You will need to think about what financial resources are available for hiring a facilitator and offering cash, gift cards, or other incentives to the participants.

Next, consider who needs to be included in the conversation and whether you have access to the appropriate people. How many individuals should be in each session? How many sessions should be conducted? How long will each session take?

Third, prepare your questions. Be sure they are appropriate for your audience and are free of bias. They should be compatible with an open-ended, freewheeling discussion. It is also important that they yield the kind of data that can be analyzed—social science expertise is especially valuable in that regard.

Fourth, conduct the focus groups, analyze the data, and share your results with others. Then, finally, you can start to take meaningful actions based on the findings.

Focus groups are a valuable tool that can be used to better understand perspectives, experiences, and opinions. However, the data they generate are easily biased and misused. When poorly designed or run by ill-trained facilitators, focus groups produce meaningless results. If you are interested in using focus groups, work closely with trained social scientists throughout the project to avoid that outcome.

Have questions about focus group research? Feel free to reach out to the Statistical Research Center at the American Institute of Physics to better understand the topic. Contact Laura Merner at lmerner@aip.org .