We were keen to ensure that it wasn't just us getting frustrated by the peer review process or, even more worryingly, trying to pass off poor quality research that could have been designed better! We therefore used a survey of ecologists to gain an idea of the extent to which other researchers encountered the problem and faced issues when trying to publish.
The online survey was disseminated through our professional network and advertised on the Ecolog‐l mailing list and Twitter. There was much interest in the topic, and 103 responses were collected which revealed the following key findings:

Fifty‐eight percent of respondents had faced a research question where they felt pseudoreplication was an unavoidable issue (Table 2).

Categories of pseudoreplication problem identified in the questionnaire and the frequency with which respondents described them

Landscape‐scale treatments/monitoring (including manipulations of forest stand structure)10
Nested designs with insufficient replication at site level9
Wildlife behavior/physiology (including repeated measures on a small number of individuals)9
Confounded site/stand and treatment (including multisite vegetation chronosequences)8
Demography and disease and – what is the appropriate analysis level site, plot, or individual?8
Exclosures at a single site (including grazing and irrigation studies)6
Aquatic ecology + hydrology ‐ unreplicated ponds/lakes/watersheds5
Fire behavior and effects (including studies of individual wildfires)5
Single‐site case studies or phenomena limited to one location5
Spatial autocorrelation3
Repeated measures of vegetation change (including studies on a single relevé)2
John Wiley & Sons, Ltd

Of those who'd faced the problem, 85% were aware of the concept before they started their research although most (89%) were not discouraged by it.

Nearly 70% of respondents had read Hurlbert (1984).

Two‐thirds of respondents tried to deal with the issue during their statistical analysis or by acknowledging the limits of statistical inference possible given their design. A third dealt with the issue during hypothesis formation by clearly defining their population, and a fifth framed their conclusions as new hypotheses. Only four respondents admitted they just hoped no one would notice (which is honest but naughty!).

Half of the respondents admitted they'd had difficulties getting their research published, and 17% were never able to get their studies published at all. Of those who experienced publication difficulties, 41% received major corrections but 55% had their paper rejected (with less than half of those being given the option to resubmit). A quarter of the respondents had ended up in prolonged arguments with reviewers and/or editors.

When completing peer reviews, reviewers who had not encountered pseudoreplication issues in their own research, though a relatively small proportion of all reviewers (25%), appeared to be considerably more likely to reject or ask for resubmission of papers with pseudoreplication (59%) than those who'd had to deal with the issue themselves (36%).

The survey revealed that we aren't the only ecologists who are frustrated by their experiences during peer review, and it was clear a number of respondents had particularly strong views (Box 1). All but one of the respondents who provided comments expressed frustration with the way pseudoreplication was dealt with during review. The sample of respondents was probably self‐selecting (Olsen 2008), but it does indicate that there is a proportion of scientists genuinely concerned about the issue. This is also evidenced by the continuing and ongoing debate (Hargrove and Pickering 1992; Oksanen 2001; Cottenie and De Meester 2003; Feeberg and Lucas 2009; Schank and Koehnle 2009; Ramage et al. 2013). Most views expressed in the survey could be categorized as feeling that:

pseudoreplication was inevitable in many types of research due to cost, scale, and other “real‐world” environmental issues such as a wildfire, drought, or flood only occurring once;

many kinds of pseudoreplication can be dealt with statistically using appropriate nesting or random effects.

Free full text: Click here