Data provenance checks in R
Almost every diagnostic an ecologist is taught measures the model against the data. The residual plot, the dispersion check, the posterior predictive draw: all of them take the dataset as given and ask whether the model fitted to it behaves. That leaves one question unasked, and it is the question a field season is most likely to answer badly. Are these the data the design asked for? The frame said one set of plots and the crew reached another. The protocol said one method and the second half of the series used a different one. The field card said a recording grid and the recorder rounded to something friendlier. The species list changed under a name that never moved.
None of that shows up in a residual. Each of these tutorials takes one gap between the written record and the returned dataset, simulates it with a known answer underneath, and prices what it costs the estimate you would have reported. That is the method here, and it is worth naming, because it is the only way to find out whether a check is worth running: build the fault deliberately, run the ordinary diagnostics, and watch which ones stay quiet. Most of them stay quiet. The value of a provenance check is exactly the damage the silent case does.
When the returned data are not the data that were drawn
A design is a set of decisions written down before the field season. What comes back is a set of decisions made during it, usually by somebody standing in the rain, and the difference is rarely recorded anywhere.
- Non-response and site substitution follows a crew that cannot reach every plot it drew and walks to the nearest reachable one instead, and separates the part of the damage that lands on the estimate from the part that lands on the interval, which is the larger of the two and the one nobody looks at.
- Splicing a monitoring series takes a scheme that swapped its method halfway through and shows the step reading as a slope, with an era indicator recovering the real trend even where no calibration data were ever collected.
- Baseline selection and the return to the mean is the case where the design itself creates the artefact: money goes to the worst sites, the worst sites recover on their own, and every pre-treatment diagnostic agrees that nothing is wrong.
When the number carries the ruler as well as the measurement
A measurement is a claim about an organism and a claim about an instrument, and the second one gets lost on the way from the field card to the spreadsheet.
- Rounded and coarsened measurements works with a known recording grid, adds up the variance the grid contributes, and shows a normality test rejecting the folding rule rather than the biology before an interval-censored likelihood puts both back.
- Digit preference and heaping in field data covers the rounding that is not symmetric, where the mean itself moves and keeps moving no matter how many fish go over the measuring board, and where two arms of a study that heap differently manufacture a contrast out of nothing.
When the labels move under the data
Species lists are records of taxonomy, not of organisms. A long series records names, and names have their own history.
- Taxonomic revision and species trends puts a split and a lump partway through a quarter-century of quadrat data and prices the richness trend they invent in a community that never changed, together with the two repairs available and what each one costs.
- The price of harmonising a species list is the other half: pushing two recording eras onto one backbone looks like tidying, and it hides opposite trends, drops richness, and loses records in a trait join that reports no error.
The checks themselves
Two closing tutorials collect the diagnostics rather than the failures, one for data that had a design and one for data that never did.
- Checking your data against the design runs five checks over a handover folder: the returned sample against the frame, the visit dates against the protocol, the measurements against the recording grid, the group sizes against the allocation, and the labels against the code list. Each one is stated with the case it cannot see, and the honest limit is stated too: where the design was never written down, the check does not weaken, it stops existing.
- Checking an unstructured-data analysis is the version for records nobody designed, where the provenance has to be reconstructed from the records themselves: effort in space and in time needing separate corrections, reporting bias between species, the first-record trap, and validation against a structured survey.
What these tutorials assume
Base R and one plotting package throughout. Every fault is simulated, so the true answer is known and the cost of the fault is a measurement rather than an opinion. You need no special package: the point of writing the checks out is that a provenance check is a comparison between a dataset and a document, and no package can read the document for you. Reading a GLM output and a confidence interval is enough background. Where a tutorial reaches for a heavier method, it says why the light one failed first.
Where this connects
Provenance sits between data cleaning and modelling, and it borrows from both sides.
- R for ecologists is the layer underneath: reading, cleaning, reshaping and joining field data, which is where several of these faults are quietly introduced rather than inherited.
- Reproducible workflows in R applies the same argument to the code rather than the dataset, and its closing tutorials are the ones that ask whether a script reports the numbers it actually computed.
- How long a calibration overlap do you need? turns the splicing problem around and asks it before the change instead of after, as a question about how much dual running to pay for.
- Testing a monitoring series every year is a provenance fault in the analysis history rather than the dataset: the trend test rerun each autumn, and the false alarm rate that follows.
- Reporting rates and effort drift ranks four kinds of drift in a casual recording scheme by the damage each does to a trend, which is the unstructured counterpart of a protocol change.
- Effect sizes from incomplete reports moves the same problem into synthesis, where the record is a published paper and the missing piece is the standard deviation it never printed.
- Measurement error and regression dilution is what happens once the instrument error is accepted rather than removed, and how far a slope moves because of it.
- Missing data: MCAR, MAR and MNAR asks why the gaps are where they are, which is a provenance question with a well developed statistical answer attached.