2.18.1 Confidence in Bullying / Harassment Data

We realise the bullying and harassment results might be surprising, especially to the quarter of the profession who has never experienced or witnessed such behavior and to whom these experiences might be completely alien.

We are highly confident in the data for the following reasons:

• the Profiling the Professional sample (as a percentage of the entire ‘population’) is 100,000x larger than what is used in current opinion polling;
• this survey was a general survey and would not have generated a ‘selection bias’;
• three independent studies, using different methods, found similar results;
• the results are similar to other sectors;
• the cut off for p-values was 50x greater than the common conventions.

Sample Size

Profiling the Profession (PP) sample came from 1,200 respondents, 1,000 of which are active professionals, representing 1/7^th of all professional archaeologists (7000). To put this into perspective, modern political polling might sample 1,000 respondents from a population of 66 million, which is a sample of 0.000002 vs 0.2 for PP (100,000x larger sample). To further check for potential bias. PP collected data from employers on some of their staff characteristics, such as location, gender and working patterns. This was a sample of 2,500 full-time equivalent archaeologists, which is ~40% of the estimated FTE archaeological posts (6,300). Both the survey of individuals and organisations produced almost exactly the same demographic results. Slightly higher number of women responded to the individual survey (48% vs 47% listed by employers), which is where there data originate, and they are more likely to report harassment – but this could only inflate the numbers by roughly a quarter of one percent.

This indicates that, even if we increased our sample size by 150%, we would only be marginally (<1%) changing the composition.

While not a random sample, the matching demographic data indicates that this is a representative sample and possibly the equivalent of a random sample. If that were the case, then the PP individuals sample carries a 4% error rate with 99% confidence, 3% at 95% confidence level. This means the actual rates might differ from the sampled results but by negligible amounts.

Selection Bias

It has been suggested that these results could be the outcomes of selection bias – that those who have been harassed/bullied would have wanted to respond to this survey, and those that have not, would not have responded to a bullying / harassment survey might have been because they would have thought they would have nothing to contribute.

However, Profiling the Profession was a general survey of the sector, not focusing solely on harassment, which is only a small portion of the data gathered. Those responding would have done so to contribute to a general survey of archaeology, not to provide harassment data. Harassment and bullying was was not mentioned in the intro to the survey:

‘This is gathering information about the people who are, who have been or who will be professional archaeologists. It asks everyone the same three sets of questions – about ascribed, personal characteristics demographic information your health and wellbeing’.

This greatly reduces the chances that there might have been a response bias in regards to harassment/bullying. This also presupposes that any selection bias would be to increase the results, which there is no evidence to support. Furthermore, Prospect’s survey (see below) was a bullying and harassment survey in which half of the participants did not report incidents. So the strength of the selection bias appears to be minimal to non-existent in regards to selecting for archaeologists that have been harassed/bullied.

Matching results

There have now been three recent published reports on harassment and bullying in UK professional archaeology:

• Profiling the Profession (~1200 respondents, 2020)
• Prospect Archaeologists Branch Research Report 1: Archaeologists’ responses to Prospect’s Workplace Behaviours Survey (~300 respondents, 2018)
• Reporting Bullying and Sexual Harassment: a workplace survey by BAJR Respect (~280 respondents, 2020)

Three surveys, conducted by three different groups and using different collection methodologies, all produced similar results. As discussed, the overall rates and responses for the Prospect and Profiling the Profession are within 10% of each other. The BAJR and Profiling the Profession responses are within ~5% of each other when it comes to confidence in reporting future incidents. The Prospect report also found a similar correlation between age and incidents as found in Profiling the Profession.

It is a vanishingly small chance that three surveys, using different methods finding similar results would all be wrong.

Comparable to Other Sectors

As outlined in the results, archaeology falls between NHS England staff and Civil Servants in terms of incidents in the last 12 months and in reporting of incidents at the time. Both those surveys have hundreds of thousands of responses, were general surveys, and having run for several years with the same results, which makes them excellent and reliable data-sources. These results show that the findings of this survey are not outliers in terms of experiences for millions of employees in the UK.

These results are not outliers in archaeology or in the UK.

Standards Used

The analysis of the Profiling the Profession data use Chi-squares to determine correlations between harassment and other variables; we used a cut-off point of p-values of .001. The common convention is for p-values of .05 so our analysis used a standard that is 50x greater than that. Only very strong relationships have been presented.

Conclusion

All of these factors indicate that these results are not unexpected and not biased. Of course, with all surveys there will be some variation in results due to sampling but this is likely to be in the order of a few percentage points – rather than representing a complete inversion of the true picture.