When the data analysed is not specific or intersectional so it doesn’t really tell you much without generalisations: For example, “10% of our staff are Black” but you don’t know if they are all the same gender, age or anything else about them! Plus you don’t know whether they all work in one Department or are the lowest paid staff.
When there is a lack of trust and psychological safety in the organizational culture so people are dishonest, or have omitted information during data collection: For example, when an engagement survey reads positively but people didn't add many comments and the quantitative data doesn't correspond with the high staff turnover and sickness-absence rates.
When it fails to centre those who are marginalized by focusing on the majority of responses being positive, rather than seeing the negative responses as an issue to explore: For example, “89% of our colleagues feel their voices are listened to... so we are doing great!” or "75% of our staff feel proud of our DEI work". Meanwhile, the 11% & 25% don’t feel the same and could be more likely to be part of marginalised communities…
When the data collection exercise is seen as an outcome in itself and uses up all of the DEI resources for the year: For example, software or consultants used for the data collection cost $$$ and your DEI budget is inadequate, and by the time new budget is allocated or the DEI team has capacity to take action, the data is actually out of date!
When someone's lived experience is not seen as enough to act upon, and instead, multiple surveys and focus groups are 'needed' to get examples from 'enough' people to prove that the organization is impacted by systemic issues: For example, it's been a year since a Black colleague shared multiple experiences of racism in an exit interview and the company has spent the past 12 months trying to collect data that tells them it was a one-off problem so they don't need to do anything about it.
Data collection is done in a way that is triggering for marginalized folks: For example, listening groups are conducted by untrained senior leaders who want to be educated on issues by those with lived experiences, but the senior leaders re-trigger the participants by asking unsafe questions and there is no after care for those who have taken part.
When there is insufficient or no communication about how marginalised people's feedback has led to changes, or worse - nothing is actually done with the information: For example, the data collection takes months and culminates in a final report that isn't shared beyond senior leadership level and there is no buy-in to take forward the recommendations.