Sybille Reidl, Sarah Beranek, Julia Greithanner, Anke Schneider & David Sellitsch
Sybille Reidl is sociologist at the Institute for Economic, Social and Innovation Research (POLICIES) of JOANNEUM RESEARCH. Her main research areas are gender/diversity, human resources and evaluation of gender equality policies in R&I. Moreover, she supports technology development in integrating gender and diversity dimensions.
Sarah Beranek works at the Institute for Economic, Social and Innovation Research (POLICIES) of JOANNEUM RESEARCH. Her background is in Social Sciences. She works mainly on the topic of gender equality in the R&I sector, as well as user-oriented technology development.
Julia Greithanner is sociologist at the Institute for Economic, Social and Innovation Research (POLICIES) of JOANNEUM RESEARCH. Her main research areas are gender/diversity and evaluation of gender equality policies in R&I.
Anke Schneider is scientist at AIT – Austrian Institute of Technology at the Center for Technology Experience. The development of new human-centered innovations based on future business models with a focus on gender and diversity dimensions is in her research area. Another goal of her research is the analysis of user behavior with a strong focus on behavioral changes using several biofeedback measurements.
David Sellitsch is Expert Advisor at the AIT – Austrian Institute of Technology at the Center for Technology Experience. He specializes in the field of human-computer interaction and deals with topics from the areas of gamification and tangible interaction.
1. Introduction
The climate crisis and mobility constraints caused by the COVID-19 pandemic have led to a boom in online meetings, which should be maintained for sustainability and environmental reasons. However, studies show, that virtual communication can exacerbate existing inequalities; for example, women are more likely to be overlooked or ignored in online meetings (Armentor-Cota 2011; Connley 2020). In addition, age and education level may negatively influence receptivity to technology (NeXR 2020; Buchebner-Ferstl et al. 2020). In this respect, it is important to further develop online meeting technologies to facilitate inclusion and belonging in digital spaces.
For this purpose, within the FEMtech research project FairCom, we aimed to contribute to the development of inclusive online meeting solutions – in terms of both inclusive software development and the facilitation of online meetings. Our objective was to develop ideas within a participatory process and co-creatively reflect with users on how to make online meetings more inclusive through technological and social/group dynamic processes. Our goal was to develop first prototypes of solutions. Furthermore, when planning FairCom, we aimed to consider gender beyond the binary concept.
To achieve these objectives, we structured the project as follows (see Figure 1) [1]:
First, we conducted a literature review to identify relevant diversity dimensions and exclusion mechanisms to be considered in the subsequent research process. The results of the literature review were used throughout the research process (e.g. in the creation of the survey instruments). Then, we recruited seven teams from different contexts (work, education, leisure) to observe their regular meetings and conducted interviews and an online survey about experiences of online-meetings, exclusion mechanisms and needs for improvement. As we were not able to attract another team specialising in advocacy for trans- inter and non-binary people for our project, we recruited additional trans- inter and non-binary people for the interviews and the online survey. Building on the results of this user needs assessment, personas and user scenarios were developed, which were used in the subsequent co-creation workshops. In a participatory process, diverse user groups developed ideas for technological solutions and ideas for interaction concepts. Topics that participants brought forward but for which no ideas emerged in the workshops were further worked on in an internal workshop with the project team. Table 1 shows an overview of the methods used in FairCom:
In this paper, we discuss our approach to accounting for the diversity of meeting participants during the research process, the challenges we faced, and our strategies to address them. In addition, we want to determine how we can improve our practice of dealing with diversity and contribute to learnings and further development based on our practical experience. This text is therefore intended as a report on practical experiences in a research process and seeks to contribute to an open and failure-friendly exchange of approaches in research.
Firstly, we aim to examine exclusion mechanisms present in online meetings, focusing on user groups that have been identified in the literature so far, as this was our starting point for sampling and developing the survey instruments (chapter 2). We then describe our methodology and its pitfalls for identifying the user habits and needs of a diverse user group (chapter 3). Then we will focus on the major challenge of incorporating a diverse sample of potential users into the needs assessment and the co-creation of potential solutions (chapter 4 and 5). Finally, we will discuss the difficulty of shedding more light on the topic of “making diversity visible” (chapter 6). The co-creation workshops demonstrated that the participants came up with a wide range of solutions, but unfortunately, in relation to the objective of addressing the diversity of participants in online meetings, there were only a few suggestions.
2. What kind of diversity and exclusion mechanisms might play a role in online communication?
In 2022, we started the FairCom Research Project with a literature review to identify relevant diversity dimensions and exclusion mechanisms we should consider in our research process. The literature review revealed that, depending on how they are utilised, online meetings can both reduce and exacerbate existing inequalities.
The findings of the existing bodies of literature in the field of gender studies indicate that online communication in general is not a gender-neutral space. Women, BIPOC and LGBTQIA* individuals are often subject to aggressive intimidation, harassment and threats in virtual spaces (Amarasekara und Grant 2019; Nadim und Fladmoe 2021; Veletsianos et al. 2018; Frey 2020; Rubin et al. 2020; Kawsar 2021; Herring und Stoerger 2013). Both offline and online communication are characterised by exclusionary communication practices and microaggressions, such as interruptions, which disproportionately affect non-binary individuals, women, queer women, women with disabilities, and black women (Thomas et al. 2019; Mendelberg et al. 2014; da Silva Figueiredo Medeiros Ribeiro, Karen 2020). Women are more likely to be ignored and overlooked, and their voices may not be valued to the same extent as those of male colleagues, according to research based on a binary understanding of gender (Armentor-Cota 2011; Connley 2020).
There are some factors, which can influence the level of participation in online meetings: Certain user groups may feel insecure due to their lack of technological proficiency and digital literacy, which are related to gender and age (Laitinen und Valo 2018; Arellano 2020; Reidl et al. 2020; Hauk et al. 2018; ÖIAT 2014). The level of education can also be related to the openness towards digital (educational) formats (Buchebner-Ferstl et al. 2020; SPECTRA Marktforschung 2018). In virtual meetings, age (potentially related to hierarchy) can also influence active participation (NeXR 2020). Additionally, social anxiety and introversion can play a role in this regard (Luk 2021; Lowenthal et al. 2020; Callahan 2021). Moreover, communication is a challenge for individuals whose native language is different from the language used in the meeting, and online communication can further exacerbate this difficulty due to the absence of comprehensive non-verbal cues and subpar video and audio quality (Hui et al. 2021; Rini et al. 2021; Arellano 2020; Sohn 2018; Mori 2020).
In summary, we found the following exclusion mechanisms related to online and offline communication behaviour in the literature, which can reinforce existing differences in participation:
- Exclusionary communication practices and micro aggressions: These include interrupting, ignoring and overlooking people in online communication (Cullinan 2016; Connley 2020), as well as the use of technical language, monologuing and debating, which shifts the focus away from collaborative to individualised communication. Passive aggressive behaviours such as rolling the eyes or making disparaging remarks are also examples of exclusionary online communication (Arellano 2020), as are misgendering or ignoring, not acknowledging or pathologising gender identities (Scheuerman et al. 2021).
- Perception and participation: privileged individuals occupy more speaking time, while others participate less actively in discussions and their contributions are regarded as less valuable (Catalyst 2020).
- Unstructured meeting culture promotes exclusion: people who speak without waiting for others to speak have an advantage over those who wait to be requested to speak (Tannen 1995; Heath und Flynn 2014).
- Camera Use: Although a deactivated camera makes participation in communication more difficult, not using the camera is more pertinent for certain groups than others. This is especially true for women and BIPOC, according to a study of US-American students (Castelli und Sarvary 2021). In part, this is due to poor internet connection, and in part it has to do with dissatisfaction with the current design (Meyer 2020).
These exclusion mechanisms can occur in online communication and lead to certain individuals being disadvantaged and having less influence. However, much of the literature primarily originates from the US context and often considered one diversity dimension rather than including several dimensions. Therefore, we wanted to collect more data on online meetings from our Central European cultural context and consider several diversity dimensions like gender, age, language, ethnicity and education. In the next chapter we will discuss our methodological approach and the difficulties in this attempt.
3. What was our methodology for the needs assessment?
In the development of the various survey instruments for the needs assessment (quantitative online survey, interview guideline, observation protocol), we attempted to take the exclusion criteria and diversity dimensions identified in the literature review into account. Some details on the individual methods (e.g. numbers of participants) can be found in Table 1 in the introduction.
Observations
Approach: For the observations of online meetings, identified exclusion mechanisms were integrated into the observation protocol (e.g. microaggressions and discriminatory practices such as interruptions, overlooking or misgendering, sexist/xenophobic/racist remarks, etc.). For each observation, there were two observers from the project team. Each observer took notes on the demographical characteristics of each participant and after each observation, both observers exchanged their respective notes on these characteristics in order to calibrate them.
Difficulties/learnings: We are aware that an external assignment of diversity characteristics to people is by no means reliable. However, since we also had the interviews and the survey, in which the same participants could express their insider perspective, we deemed it appropriate to concentrate on the outside perspective in the observations. This also seemed suitable because exclusion mechanisms in online meetings are partly based on external perception. In addition, it would also have been practically challenging to capture the demographics with small questionnaires, as we did not want to overly influence the observation and prevent sharing individual characteristics within the group. As observers, we found it difficult to complete this section and felt uncomfortable answering these questions from an outside perspective. However, it was helpful to discuss this with the other observer. In addition, it is important to clarify in the description of diversity characteristics in the report that they were not self-reported (e.g. perceived as women). Overall, we experience the handling of diversity characteristics in observations as challenging and want to explore this matter further.
Interviews
Approach: The interview guideline touched on similar topics as the observation (e.g. micro-aggressions), but focused more on the interviewee’s perception and experience of online meetings and on needs and ideas of improvement. For example, one question was about how comfortable the interviewee felt speaking up in online meetings and the reasons behind it. We asked quite open-ended questions and also spontaneous follow-up questions during the interviews to give space to the experiences of the interview partners. Following the interview, we distributed a brief demographic questionnaire in order to capture the diversity characteristics of the interview partners.
Difficulties/learnings: The interviews worked quite well and the brief demographic questionnaire was an appropriate tool to obtain self-reported information. Specific challenges with the individual questions of the brief demographic questionnaire are described below in the section on the online survey, as the wording of the demographic questions was the same for both. In addition, during the interviews, we encountered some language barriers with a few interview partners, where a translator would have been helpful.
Online Survey
Approach: Similar to the interview guide, we incorporated the exclusion mechanisms of the literature review into the online survey by inquiring the participants’ experience with microaggressions such as interruptions or being overlooked, having difficulties speaking up, among others. Additionally, we focused on capturing their usage behaviour in online meetings (e.g. camera use, chat use etc.). On the other hand, we included gender and other diversity dimensions that appeared to play a role in the literature review (e.g., age, education, language) in the demographics section in order to analyse the results accordingly. In developing the specific questions for the quantitative online survey, we aimed to formulate questions and response categories that were easily comprehensible and sensitive to various life circumstances and affected groups.
Difficulties/learnings: We now want to critically reflect on the development process of the questionnaire and the challenges we had in this context:
First, we discussed the manner in which we intended to inquire about gender: Since we planned a rather small sample size for the survey and this option was recommended in the guidelines of Scheuerman et al. (2020), we chose an open text field. The question was not a compulsory, so participants could decide whether they felt comfortable answering it. In this open text field, survey participants could describe their gender (“Geschlecht”) in their own words. The majority of the participants identified themselves as female or male, few used the terms woman or man. Some participants identified as non-binary (some in English, others in German) and others as inter*, intergendered, genderqueer-/fluid or inter/male. One participant proactively gave feedback that they strongly support the option of open text fields[2].
In order to capture gender transitions, we also asked: „Entspricht Ihr aktuelles Geschlecht jenem, das Ihnen bei Ihrer Geburt zugewiesen wurde?“ [Is your current gender the same as the one ascribed to you at birth?]. We requested this information, because we wanted to include the perspectives of inter* and trans* individuals on online meetings[3]. This question was also not compulsory, so participants could decide whether they felt comfortable answering it. One participant from the inter* community gave feedback that asking for gender in this manner can be confusing, as it is unclear whether we ask for the legal gender or the gender identity. Therefore, as Pöge et al. (2022) point out, it should be phrased in a more understandable and precise way. Another participant in the online survey, a moderator from one of the recruited teams whose members have a diverse cultural background and first languages other than German, also told us that the team found this question confusing. This shows that addressing the needs of several groups can be contradictory and therefore challenging. So a potential approach could be to ask explicitly and precisely about individuals’ identification as inter* or trans*, but also to include info boxes that explain these terms in a comprehensible way for people who are not yet familiar with them.
In order to capture whether participants might have marginalisation experiences in online meetings for being BIPOC and/or having a migration background, we included the question: Werden Sie von manchen Personen z.B. aufgrund Ihrer Sprache, Hautfarbe, Herkunft oder Herkunft Ihrer Eltern nicht als Österreicher*in gesehen? [Are you not perceived as an Austrian by some individuals, for example because of your language, skin colour, origin or the origin of your parents?] In contrast to other questions, we did not receive any feedback on this question, but are considering an alternative wording because it was challenging in the report to describe this group as it covers many life realities at once. For the future, it might be easier to be more precise and ask about the groups individually. However, this would unduly lengthen the questionnaire.
To inquire about physical limitations that could impact participation in online meetings, we posed the question, “Haben Sie eine Seh- oder Hörbeeinträchtigung?” [Do you have a visual or hearing impairment?]. In our case, it would have been better to ask about disability instead of impairment, as individuals who can see well with glasses also responded affirmatively to this question, although they have visual difficulties in online meetings. However, the primary intention was to use the question to identify people whose vision cannot be fully compensated for by glasses. This resulted in a decrease in the significance of the collected data on this item.
Regarding educational background, we asked for the highest level of education with a closed question. The answer categories were highly aligned with the Austrian educational system, which makes it difficult for individuals with education from other countries to classify themselves. In addition, it was unclear for the participants if they could include their foreign degree even if it was not officially recognised in the Austrian system. In the future, it might be better to use an international classification system of education levels.
As a general learning, we can conclude that being as precise as possible makes it easier for the participants to answer questions, as well easier for us to use the results. In addition, if one aims for a diverse sample, the survey instruments also have to reflect these different life realities (e.g., language). However, this also leads to longer questionnaires with info boxes and more detailed questions.
4. How to consider diversity dimensions in selecting a sample for the needs assessment?
Simultaneously with the development of the data collection instruments, we started to recruit teams for the needs assessment. In compiling the sample of eight teams, we faced the challenge of representing the following diversity dimensions:
We initiated a search among our extended networks to identify teams that meet specific diversity requirements and have a maximum membership size of 15 for the purpose of observation. In order to identify teams, we requested the following information from teams via email:
- What is your team working on, and what is the purpose of your meetings?
- How often do you meet?
- How many members does your team or group have? What is the approximate composition of your team in terms of gender and age?
- What software do you use for your online meetings?
The selection of the teams to be observed followed a selective sampling strategy (Ganz and Hausotter 2020), for which the diversity dimensions to be covered were defined. The search for the teams was modified on the basis of the information provided by the teams already acquired in order to meet the dimensions that were still missing.
The teams that we were able to recruit for our study covered a wide variety of individual dimensions:
We were able to cover nearly all diversity dimensions to at least some extent, as table 3 shows, but none of the designated teams had an individual that identified gender-divers. As it was a big concern and part of our research interest to think gender not only in binary terms, we decided to reach out for a team of trans*, inter* and non-binary people within our research project. A search on the web and in our personal networks resulted in 16 possible groups/associations/networks/sub-organisations of advocacy LGBTQIA groups/parties for FLINTAs, queer individuals, trans* and inter* individuals. All individuals were contacted via e-mail and, to a lesser extent, through telephone communication, with the purpose of extending an invitation to participate in our project. In the end, we did not succeed in finding a group that was willing to be observed in online meetings. In hindsight, it might be argued that the decision to propose the observation of one of their meetings to the target group was inadvisable. Due to the stigma and prejudice faced by marginalised groups, it can be difficult to obtain consent from members of these groups to participate in research projects (Else-Quest und Hyde 2016) as this requires a relationship of trust between the group and the research team.
As the online survey of the needs assessment later revealed, trans- inter and non-binary people in our sample activate the camera in meetings less often than other participants (Reidl et. al 2022). This suggests that they would also not like to be observed by researchers in a meeting. The potential cause of this outcome could also be attributed to the intricate nature of the study design, which may have appeared labour-intensive and hence acted as a disincentive. In addition, we suspect that LGBTQIA groups and associations are receiving an increasing number of requests from the research community. This can be attributed to the rising prevalence of investigating gender diversity in research endeavours, which is increasingly regarded as a scientific standard. Therefore, these entities are unable to fulfil these requests due to time constraints. In addition, the timing of our survey in spring 2022 was probably also inconvenient, as many groups were returning to face-to-face meetings at that time after a long period of online meetings caused by the COVID-19 pandemic.
After we had realised that a team of trans*, inter* and/or non-binary individuals was unattainable, we therefore decided to alter our strategy and asked members of the researched and requested organisations, our personal networks and on social media for interviews and participation in the online survey. In most cases, we did not receive a response to our requests; but, in two instances, we successfully arranged interviews, only to have them subsequently cancelled. Finally, only one trans* woman agreed to be interviewed. Furthermore, the online survey was ultimately completed by six trans*, inter* or non-binary individuals.
In addition, to the observations and the online survey, we interviewed three to five members from each team whose meeting we observed. Again, the respondents were selected using the method of selective sampling (Ganz und Hausotter 2020).
In each case, the moderator of the respective meeting was asked to participate in an interview, as they could report both from the perspective of moderator and participant in online meetings. In addition, we interviewed between two and four participants from the observed meetings. On the one hand, we paid attention to the coverage of various diversity dimensions (gender, age etc.). On the other hand, we examined the team’s structure and the role of its members. We wanted to ask both seemingly more extroverted individuals, who had a large share of speaking time in the observed meetings, and seemingly more introverted, quieter individuals. In addition, three interviews were conducted with experts on moderation and (online) communication via online meetings.
In total, 63 individuals were involved in the needs assessment: 53 individuals were observed, 60 individuals participated in the online survey and 27 individuals were interviewed – so the individual samples overlap to a large extent (see table 1). Given the comprehensive data available on the participants of the online survey, we will provide a concise overview of the sample in light of this information:
Online Survey
The link to the online survey was sent to all team members, facilitators and few interest groups and associations of gender diverse, trans*, inter* or non-binary individuals. There were 60 questionnaires included in the evaluation. The sample is not at all representative of the population. Significantly more women participated in the survey, followed by 28% men and 17% trans*, inter* and non-binary individuals. Six people have not answered this question.
The participants’ ages ranged relatively evenly between 18 and 70 years. 42% of respondents were between the ages of 36 and 50. Up to 35-year-olds accounted for 30% of the sample and those over 50 years made up 28%. Men tended to be younger than women. Additionally, individuals whose native language is not the meeting language tended to be younger.
Among the participants, a particularly large number have completed higher education at a college, post-secondary institution, university of applied sciences, academy or university (68%). Just under 20% have completed an apprenticeship or similar programme or hold the high school graduation as their highest level of education. Twelve percent of the participants have a compulsory school leaving certificate or no completed school education at all.
Almost 90% of the respondents speak German as their native language, while 10% of the participants have another mother tongue. Among the participants, about 18% stated others perceive them as non-Austrians based on factors such as appearance or language.
Due to limited resources and a small sample size, we were unable to conduct an intersectional analysis of the data. Therefore, we decided to analyse the quantitative and qualitative data step by step according to diversity dimensions. On this basis, we were able to identify differences based on age, gender or origin, but cannot make any assertions regarding the overlap of diversity dimensions. This additive approach is a deficiency of our project because social identities and inequality are interdependent and not mutually exclusive for groups such as Black lesbians (Bowleg 2008). An intersectional analysis of all types of data would be preferable.
5. How to consider diversity dimensions in selecting participants for Co-Creation?
For a common understanding of the target group’s desires and needs, we developed personas based on the results of the needs assessment. Personas are fictional user profiles that help visualize a typical user, foster empathy, and improve designers’ understanding of the people they are designing for (Miaskiewicz und Kozar 2011). We used these personas in remote co-creation workshops with potential users to develop solution ideas for different application scenarios. To find suitable and appropriate participants, a questionnaire was sent to a user panel in advance. Based on the answers, a diverse group of people was put together for the workshops.
Personas
When creating the personas for our project, we paid special attention to gender and diversity dimensions and attempted to avoid stereotyping. Following the work of Himmelsbach et al. (2019) we tried to consider the four layers of diversity (Gardenswartz und Rowe 2009) in the design of our personas. The first layer of diversity highlights the distinctive personality, which in our project represents technology affinity and extraversion. For the second and third layers, we mapped age, ethnicity/race and gender in the first dimension and education, geographic location, language, migration biographies and also parental or relationship status in the second dimension. The individual aspects in the work context are displayed in the fourth layer and were also represented in the personas and their scenarios through the use of online communication, either in the business context or in the private context, as well as through the role as moderator or participant.
We started in a coordinate system with the dimensions technology affinity and extraversion (see figure 3). Within the coordinate system, four personas formed the basis for further development. The female and male gender components of the dimensions were equally distributed. In order to avoid these stereotypes, the female persona was not assigned with non-tech proficient and introverted.
In addition to the four basic personas, a fifth non-binary persona was created with medium expressions of the two dimensions. All other previously mentioned diversity dimensions were distributed among the meanwhile five personas.
This set of five personas and their corresponding user stories were further enriched by incorporating the findings from the preceding needs assessment. This approach aligns with the recommended practice of developing personas by aggregating user research, combining insights from numerous users into a cohesive narrative (Adlin und Pruitt 2010). After a detailed review of the results of the needs assessment, that took place in advance (see chapter 4), it immediately became apparent that another persona was crucial, whose primary focus was on mobile online communication. Thus, a total of six personas were created, which are described in more detail in table 4.
Co-Creation Workshops
Fostering the involvement of a wide range of individuals, in human-computer interaction research is essential for the development of technologies that are both safe and inclusive, thereby promoting fairness and equality.
To facilitate the upcoming co-creation activities, efforts were made to form diverse groups of participants for the workshops. Consequently, a short questionnaire was sent to a user panel in advance with the purpose of soliciting their participation in complementing the questionnaire and engaging in one of the co-creation workshops. On the basis of those answers we selected the participants for the upcoming co-creation workshops. This questionnaire inquired about various dimensions of diversity, including gender, age, ethnicity, experiences of discrimination, and educational background. Additionally, participants were asked about their familiarity with technology (Wessel et al. 2019), their extraversion (Rammstedt und John 2007), and other factors relevant to online communication, including their role in the meeting, the context, the device they would be using, the environment, and their frequency of engagement.
The response rate of the questionnaire was high, but we were only able to cover the female and male gender dimensions but could not engage inter*, trans* or non-binary individuals. Thus, several individuals who had previously been asked for an interview as part of the needs assessment were subsequently contacted via email and asked if they would be interested in attending one of the subsequent co-creation workshops. We also re-contacted the various groups, associations, networks, or sub-organizations that represent the interests of FLINTAs (women, lesbians, inter*, non-binary, trans*, and gender individuals), queer individuals, LGBTQIA communities, and trans* and inter* individuals who had already been contacted for the needs assessment. The goal was to ask them to complete the questionnaire and participate in one of the co-creation workshops. Unfortunately, these efforts did not produce the anticipated outcome.
The three co-creation workshops took place online. Within two hours, suggestions for solutions and ideas for possible interaction concepts that address the hurdles described in the scenarios were developed together with users. Therefore, the created personas and user stories served as input and starting points.
A total of 24 people participated in our workshops, including 11 females and 13 males. They were all between 26 and 76 years old, with an average age of 45. Seven participants reported that they had experienced discrimination in their life. The range of affinity for technology was from relatively low (three participant) to very high (five participants). On average, the affinity for technology was slightly above the mean. The average extraversion of all participants was 3.66 on a Likert scale ranging from 1 not at all to 5 very much. While three people had only engaged in private online communication, all other participants use online communication more in a professional context, with six individuals having no moderation experience and one participant acting solely as moderator.
Difficulties and Learnings
In all workshops, we managed reasonably well to assemble a diverse group according to the previously queried dimensions age, experience of discrimination, extraversion and technology affinity, but we were unable to adequately cover the gender and education dimension.
During the workshops it became apparent that at least one participant is in a same-sex relationship. This individual was able to empathize well with various gender and diversity obstacles faced by the exhibited personas. This prompted us to question whether we had asked the correct question in our pre-questionnaire to select the participants, whether it was sufficient to ask about gender, or whether other dimensions, such as sexual orientation, would have been relevant here as well.
During a workshop, one individual was clearly discriminated against on multiple occasions due to their origin. This situation was extremely unpleasant and stressful for the individual and also for the facilitator and all other participants. It took the facilitator several attempts to stop the discriminating person from continuing to discriminate against a participant. This incident once again emphasized the importance of the project because on the one hand, the moderator does not always notice discrimination and may first have to be made aware of it by other participants, and on the other hand, the moderator may not even know how to deal with it.
Within the co-creation workshops, participants actively contributed by generating a multitude of ideas and formulating initial interaction concepts specifically tailored to overcome the obstacles presented in the personas and scenarios. Three main topics could be identified. The first topic was that users support a distribution of speaking time among all participants based solely on task and contextual factors rather, irrespective of their individual characteristics. For example, participants who present something in the meeting should get more time than those who just listen and give feedback. This objective can be supported by technology, specifically by using visual information, such as displaying the amount of time already spent speaking. Furthermore, participants should have opportunities to provide non-verbal feedback on meeting experiences, both during and after the session, including instances of inappropriate behaviour such as discriminatory language. Moreover, making diversity visible was frequently addressed and discussed. Existing solutions in online communication platforms only provide limited options, such as displaying pronouns alongside names. It is crucial to develop solutions that comprehensively address diversity in a more inclusive manner.
6. How to shed more light on the topic of “Making Diversity visible”?
As the participants found it difficult to develop concrete ideas on the topic of “making diversity visible”, the project team decided to conduct an internal workshop to further enhance our understanding and broaden the range of potential technological and moderation solutions for the further use of the project. Seven members of the FairCom project team discussed the significance of emphasizing diversity and collected ideas for technical and moderation solutions.
The results of the user survey were reviewed in preparation of the workshop, and all pertinent information was presented in condensed form during the workshop. This included the problematic areas of inappropriate jokes and other micro aggressions, misgendering, varying speaking times, and different levels of attentiveness to speaking inputs. The survey revealed that trans*, inter* and non-binary individuals were more likely than cis-woman and cis-man respondents to not want to be filmed during a meeting. In addition, clear rules and the chat were especially relevant for trans*, inter* and non-binary individuals. In the interviews, it was also mentioned that an outing can be more difficult when individuals only know each other online as trust is easier to develop in personal meetings. Consequently, the subject of visibility is complex and ambivalent. This prompted the following questions: What visibility options does the digital environment provide? Who is doing it for whom and to what extent? Who wishes to be made visible? What benefits can feedback mechanisms provide? In addition, since this topic frequently came up in the process, (how) could avatars contribute to this?
After the presentation of the results, possible solutions were collected. For this, the method of collaborative brainwriting[4] was used, in which each participant was given several minutes to independently generate ideas and record them on a whiteboard without any critical evaluation. After this phase, participants had time to silently read and add to the ideas proposed by others. The collected ideas were then grouped and discussed with the project team in plenary. One group of proposed solutions centred on providing feedback to and attracting the moderator’s attention (e.g., some sort of emergency button). Another group of solutions focused on awareness and sensitivity for needs of gender diverse individuals (e.g., different options of stating pronouns). The third group was about visualisation and options of anonymous communication in online meetings (e.g., anonymous message channels to moderator). The last group addressed sensitization of moderators and moderation techniques and methods (e.g., post-meeting feedback questionnaires and interaction cards). The ideas were refined and prioritised. These ideas then contributed to the subsequent process.
Resumé
This project was conducted in the field of applied research and carried out to the best of our abilities. When we started the project with the literature review, it became evident that we would have to consider a variety of diversity dimensions for a needs assessment in online meetings. However, due to limited resources, restrictions had to be imposed. For instance, we decided not to address the question of how to develop online meetings for people with disabilities, despite the fact that there is definitely a significant demand for this. Finally, we succeeded in covering many diversity dimensions when recruiting participants for the needs assessment, which led to relevant results for the co-creation workshops. Additional personas had to be developed to reflect these findings in the personas and user scenarios developed for the co-creation workshops. Overall, this contributes very positively to the further development of inclusive moderative and technological solutions in the further course of the project. However, we have been insufficiently successful in acquiring non-binary participants. To attract this target group, we should not have asked them for observation. Also, we would have had to think of a strategy to build trust, e.g., by involving a non-binary person in the research team (e.g., an expert). The experiences from the co-creation workshops have also shown us that under certain circumstances it can be helpful not only to focus on non-binary persons but also to involve persons with a homosexual orientation in research projects. The co-creation workshops have shown that they empathized well with gender and diversity obstacles faced by the personas. In addition, this target group is larger and perhaps easier to reach than non-binary people.
In conducting the needs assessment, we learned a lot about how to better serve a diverse group of participants. We need to use an international classification system to identify levels of education. We need to ask about disability, not impairment, to get meaningful data on people who cannot compensate for vision problems with glasses. In order to accommodate participants whose native language is not German, the utilisation of interpreters for interviews and the inclusion of information boxes in surveys would have been essential. This would have had to be calculated in the project costs. However, we also recognised that different target groups have different needs for question wording (clear and in simple language for people with different cultural background and language, more open text boxes and differentiation between legal gender and gender identity for LGBTQIA* people). Designing intersectional survey instruments that work equally for all participants is an art that requires further development. We also contemplate the feasibility of eschewing a one-size-fits-all questionnaire in favor of target group-specific surveys, allowing for nuanced adjustments in aspects like wording to better cater to diverse needs. In this case, however, the challenge is to ensure that differently formulated questions measure the same. In addition, this approach also raises the question of resources, as the data analysis is much more time-consuming. Solutions in this regard need to be developed in following research projects.
We also found that we should have addressed the individual steps of instrument development, sampling, and data analysis in more detail for claiming an intersectional approach than was possible in this development project. An intersectional analysis of the interview data, for example, would have been beyond our budgetary scope. We were therefore left with the approach of analysing the data according to individual diversity dimensions. Yet, with our exploratory approach, we succeeded to collect a wide range of suggestions for improvement and ideas for solutions that, if implemented, can benefit a broad range of individuals.
All in all, we discovered that dealing with diversity in research and development projects means a continuous learning process. Based on our experience with FairCom, we would do some things differently in subsequent projects:
In future endeavors, we aim to assemble a more diverse research team in terms of gender, origin, and other factors. This will enable us to better reach various target groups when recruiting test subjects and to incorporate a broader range of knowledge and perspectives into the project. One option here would be to calculate fees for experts to conduct more specific sensitivity checks with survey instruments. We anticipate additional resources for a qualitative intersectional analysis.
In conclusion, we expect that our insights from our applied project will contribute to advancing and further developing the treatment of diversity in research and development, as well as promoting an open and honest exchange within the research and innovation community.
[1] The technological and moderation solutions are not part of this paper as they are still in progress, which is why they are shown behind a dashed line in Figure 1.
[2] We did not ask for feedback directly, but the person chose to give feedback to the project team in an email.
[3] Currently, there is little research that includes the experiences of inter* and trans* individuals in online formats.
[4] More about collaborative brainwriting can be found here: https://www.lucidchart.com/blog/how-to-use-brainwriting-for-idea-generation or here https://conceptboard.com/blog/brainwriting-technique-free-template/
References
Adlin, Tamara; Pruitt, John (2010): The essential persona lifecycle. Your guide to building and using personas. 1st edition. Burlington, Mass.: Morgan Kaufmann.
Amarasekara, Inoka; Grant, Will J. (2019): Exploring the YouTube science communication gender gap. A sentiment analysis. Public understanding of science (Bristol, England) 28 (1), S. 68–84. DOI: 10.1177/0963662518786654.
Arellano, Evelyn (2020): Power Dynamics and Inclusion in Virtual Meetings. AspirationTech. Online verfügbar unter https://aspirationtech.org/blog/virtualmeetingpowerdynamics.
Armentor-Cota, Janet (2011): Multiple Perspectives on the Influence of Gender in Online Interactions. Sociology Compass 5 (1), S. 23–36. DOI: 10.1111/j.1751-9020.2010.00341.x.
Bowleg, Lisa (2008): When Black + Lesbian + Woman ≠ Black Lesbian Woman. The Methodological Challenges of Qualitative and Quantitative Intersectionality Research. Sex Roles 59 (5-6), S. 312–325. DOI: 10.1007/s11199-008-9400-z.
Buchebner-Ferstl, Sabine; Geserick, Christine; Kaindl, Markus (2020): Elternbildung im digitalen Zeitalter. Chancen – Risiken – Grenzen (ÖIF Forschungsbericht, 34).
Callahan, Kristin (2021): Valuing and Supporting Introverted Students in the Virtual Design Classroom. Int J Art Design Ed 40 (4), S. 714–722. DOI: 10.1111/jade.12383.
Castelli, Frank R.; Sarvary, Mark A. (2021): Why students do not turn on their video cameras during online classes and an equitable and inclusive plan to encourage them to do so. Ecology and evolution 11 (8), S. 3565–3576. DOI: 10.1002/ece3.7123.
Catalyst (2020): The Impact of COVID19 on Workplace Inclusion: Survey. Online verfügbar unter https://www.catalyst.org/research/workplace-inclusion-covid-19.
Connley, Courtney (2020): 45% of women business leaders say it’s difficult for women to speak up in virtual meetings., 03.09.2020. Online verfügbar unter https://www.cnbc.com/2020/09/03/45percent-of-women-business-leaders-say-its-difficult-for-women-to-speak-up-in-virtual-meetings.html.
Cullinan, Renee (2016): Run Meetings That Are Fair to Introverts, Women, and Remote Workers. Harvard Business Review, 29.04.2016. Online verfügbar unter https://hbr.org/2016/04/run-meetings-that-are-fair-to-introverts-women-and-remote-workers.
da Silva Figueiredo Medeiros Ribeiro, Karen (2020): A Gender Analysis of Interaction in Online Work Meeting Tools. IHC ’20, October 26–30, 2020, Diamantina, Brazil.
Else-Quest, Nicole M.; Hyde, Janet Shibley (2016): Intersectionality in Quantitative Psychological Research. Psychology of Women Quarterly 40 (3), S. 319–336. DOI: 10.1177/0361684316647953.
Frey, Regina (2020): Geschlecht und Gewalt im digitalen Raum. Eine qualitative Analyse der Erscheinungsformen, Betroffenheiten und Handlungsmöglichkeiten unter Berücksichtigung intersektionaler Aspekte (Expertise für den Dritten Gleichstellungsbericht der Bundesregierung).
Ganz, Kathrin; Hausotter, Jette (2020): Intersektionale Sozialforschung: transcript Verlag.
Gardenswartz, L.; Rowe, A. (2009): Diverse teams at work. Capitalizing on the power of diversity. Hg. v. Society for Human Resource.
Hauk, Nathalie; Hüffmeier, Joachim; Krumm, Stefan (2018): Ready to be a Silver Surfer? A Meta-analysis on the Relationship Between Chronological Age and Technology Acceptance. Computers in Human Behavior 84, S. 304–319. DOI: 10.1016/j.chb.2018.01.020.
Heath, Kathryn; Flynn, Jill (2014): Women, Find Your Voice. Harvard Business Review. Online verfügbar unter https://hbr.org/2014/06/women-find-your-voice.
Herring, Susan C.; Stoerger, Sharon (2013): Gender and (A)nonymity in Computer-Mediated Communication. In: J. Holmes, M. Meyerhoff und S. Ehrlich (Hg.): Handbook of Language and Gender.
Himmelsbach, Julia; Schwarz, Stephanie; Gerdenitsch, Cornelia; Wais-Zechmann, Beatrix; Bobeth, Jan; Tscheligi, Manfred (2019): Do We Care About Diversity in Human Computer Interaction: A Comprehensive Content Analysis on Diversity Dimensions in Research. Do We Care About Diversity in Human Computer Interaction: A Comprehensive Content Analysis on Diversity Dimensions in Research, S. 1–16.
Hui, Sun; Milin, Petar; Divjak, Dagmar (2021): Native perception of non-native speech: Speaker accent mitigates penalization for language errors in non-native speech unless the listener is conscientious (UC Merced – Proceedings of the Annual Meeting of the Cognitive Science Society). Online verfügbar unter https://escholarship.org/uc/item/54r77777.
Kawsar, Ali (2021): Zoom-ing in on White Supremacy. M/C J 24 (3). DOI: 10.5204/mcj.2786.
Laitinen, Kaisa; Valo, Maarit (2018): Meanings of communication technology in virtual team meetings. Framing technology-related interaction. International Journal of Human-Computer Studies 111, S. 12–22. DOI: 10.1016/j.ijhcs.2017.10.012.
Lowenthal, Patrick R.; Borup, Jered; West, Richard E.; Archambault, Leanna (2020): Thinking Beyond Zoom: Using Asynchronous Video to Maintain Connection and Engagement During the COVID-19 Pandemic. Journal of Technology and Teacher Education 28 (2), S. 383–391.
Luk, Artesimia (2021): Zoom 2.0: Designing a Video Conferencing Platform for Students with Social Anxiety at Wellesley College. Master’s Thesis. Online verfügbar unter https://repository.wellesley.edu/object/ir1600.
Mendelberg, Tali; Karpowitz, Christopher F.; Oliphant, J. Baxter (2014): Gender Inequality in Deliberation. Unpacking the Black Box of Interaction. Persp on Pol 12 (1), S. 18–44. DOI: 10.1017/S1537592713003691.
Meyer, Zlati (2020): Do you show your face in Zoom meetings? Your gender may play a role. FastCompany, 23.06.2020. Online verfügbar unter https://www.fastcompany.com/90514033/do-you-show-your-face-in-zoom-meetings-your-gender-may-play-a-role.
Miaskiewicz, Tomasz; Kozar, Kenneth A. (2011): Personas and user-centered design. How can personas benefit product design processes? Design Studies 32 (5), S. 417–430. DOI: 10.1016/j.destud.2011.03.003.
Mori, Akira S. (2020): Next-generation meetings must be diverse and inclusive. Nat. Clim. Chang. 10 (6), S. 481. DOI: 10.1038/s41558-020-0795-z.
Nadim, Marjan; Fladmoe, Audun (2021): Silencing Women? Gender and Online Harassment. Social Science Computer Review 39 (2), S. 245–258. DOI: 10.1177/0894439319865518.
NeXR (2020): Digitale Meeting-Kultur. Online verfügbar unter https://nexr-seminar.com/wp-content/uploads/2020/09/Studienergebnisse_NeXRSeminar.pdf.
ÖIAT (2014): Maßnahmen für Senior/innen in der digitalen Welt. Online verfügbar unter https://www.digitaleseniorinnen.at/fileadmin/redakteure/Downloads/studie_massnahmen_fuer_senorinnen_in_der_digitalen_welt.pdf.
Pöge, Kathleen; Rommel, Alexander; Starker, Anne; Prütz, Franziska; Tolksdorf, Katharina; Öztürk, Ilter et al. (2022): Geschlechtliche Diversität in der Studie GEDA 2019/2020-EHIS – Ziele, Vorgehen und Erfahrungen. Journal of Health Monitoring 7 (2), 52-71.
Rammstedt, Beatrice; John, Oliver P. (2007): Measuring personality in one minute or less: A 10-item short version of the Big Five Invento-ry in English and German. Journal of Research in Personality 41 (1), S. 203–212.
Reidl, Sybille; Streicher, Jürgen; Hock, Marlene; Hausner, Beatrix; Waibel, Gina; Gürtl, Franziska (2020): DIGITALE UNGLEICHHEIT – Wie sie entsteht, was sie bewirkt … und was dagegen hilft. Online verfügbar unter https://www.femtech.at/sites/default/files/pictures/Reidl%2C%20Streicher%2C%20Hock%2C%20Hausner%2C%20Waibel%2C%20G%C3%BCrtl%20%282020%29%20Studie_Digitale_Ungleichheit_barrierefrei_final.pdf.
Reidl, Sybille; Beranek, Sarah; Greithanner, Julia; Sauer, Angelika; Schiffbänker, Helene: D2.2 Ergebnisbericht zu Nutzer*innenerhebungen. Wien 2022
Rini, Rini; Noorman, Safrina S.; Nafisah, Nia (2021): Zoom-Storytelling. An Activity to Reduce Students’ Speaking Anxiety. JPP 21 (2), S. 112–121. DOI: 10.17509/jpp.v21i2.36653.
Rubin, Jennifer D.; Blackwell, Lindsay; Conley, Terri D. (2020): Fragile Masculinity: Men, Gender, and Online Harassment (CHI ’20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems April 2020 Pages 1–14). Online verfügbar unter https://doi.org/10.1145/3313831.3376645.
Scheuerman, Morgan Klaus; Jiang, Jialun Aaron; Spiel, Katta; Brubaker, Jed R. (2021): Revisiting Gendered Web Forms: An Evaluation of Gender Inputs with (Non-)Binary People. CHI’21.
Scheuerman, Morgan Klaus; Spiel, Katta; Haimson, Oliver L.; Hamidi, Foad; Branham, Stacy M. (2020): HCI Guidelines for Gender Equity and Inclusivity. Unter Mitarbeit von Os Keyes, Kate Ringland, Brianna Dym, Leo Stewart, Danielle Lottridge, Jes Feuston et al. Hg. v. Alex Ahmed, Os Keyes und Danielle Lottridge. Online verfügbar unter https://www.morgan-klaus.com/gender-guidelines.html#About.
Sohn, Emily (2018): The future of the scientific conference. Nature 564 (7736), S80-S82. DOI: 10.1038/d41586-018-07779-y.
SPECTRA Marktforschung (2018): Spectra-Digi-Monitor Österreich (Teil 1).
Tannen, Deborah (1995): The Power of Talk: Who Gets Heard and Why. Harvard Business Review. Online verfügbar unter https://hbr.org/1995/09/the-power-of-talk-who-gets-heard-and-why.
Thomas, Rachel; Cooper, Marianne; Konar, Ellen; Bohrer, Ali; Mohsenin, Ava; Yee, Lareina et al. (2019): Women in the Workplace 2019. Hg. v. McKinsey & Company.
Veletsianos, George; Kimmons, Royce; Larsen, Ross; Dousay, Tonia A.; Lowenthal, Patrick R. (2018): Public comment sentiment on educational videos. Understanding the effects of presenter gender, video format, threading, and moderation on YouTube TED talk comments. PloS one 13 (6), e0197331. DOI: 10.1371/journal.pone.0197331.
Wessel, D.; Attig, C.; Franke, T. (2019): ATI-S – An Ultra-Short Scale for Assessing Affinity for Technology Interaction in User Studies. Hamburg: Gesellschaft für Informatik e.V. Mensch und Computer 2019 – Tagungsband.