Robustness of field studies evaluating biodiversity responses to invasive species management in New Zealand

Benefits of invasive species management for terrestrial biodiversity are widely expected and promoted in New Zealand. Evidence for this is presented in policy and scientific reviews of the literature, but the robustness and repeatability of the underpinning evidence-base remains poorly understood. We evaluated the design of field-based studies assessing biodiversity responses to invasive species management in 155 peer-reviewed articles published in 46 journals from 2010–2019. Each study was assessed against nine principles of experimental design, covering robustness of sampling and avoidance of bias. These principles are important in New Zealand to detect treatment effects from environmental variability driven by underlying gradients such as soil fertility, climate, and disturbance. Across all publications, about half defined the sampling universe (52%) or were unreplicated (54%), whereas most (74%) did not representatively collect data across the sampling universe. Management treatments were specified, with or without only influencing the target species, in 68% of publications. Relatively few studies quantified invasive species (15%) and biodiversity responses (27%) representatively within replicates. Initial conditions and accounting for the effects of experimental implementation were not used in 57% and 84% of publications respectively. No publications avoided observer/analyst bias using blinding methods, despite this being widely adopted in other scientific fields. We used ordinal logistic regression to understand how these principles varied among categories of biodiversity responses and for major groups of invasive species. Our findings suggest that greater attention to experimental design principles is desirable: supported by researchers, funding agencies, reviewers, and journal editors. Greater resources are not necessarily a solution to these design issues. Undertaking fewer studies that are individually more expensive because they better adhere to experimental design principles, is one alternative. The challenges of meeting experimental design principles suggests a significant role for other approaches such as systematic monitoring and natural experiments, although many of the design principles we discuss still apply. Our intent in this article is to improve the robustness of future field studies for at least some principles. Robust designs have enduring value, reduce uncertainty, and increase our understanding of when, where and how often the impacts of invasive species on biodiversity are reversible.