Non-random Temporal Patterns of Citizen Science Biodiversity Recording¶
Category: Biodiversity · Size: 3.0 MB · Format: ZIP License: CC-BY-4.0 · Zenodo record · Data sheet on the CSDH
Data and code to analyse when citizens record biodiversity, identifying non-random temporal patterns in sampling effort and their associated drivers.
The data is mounted read-only at /srv/data/temporal-patterns-biodiversity/.
Save anything you produce in your personal folder (~/).
What's in the archive¶
The dataset ships as a single ZIP holding one data table plus the R scripts the authors used. We list the contents and read the short descriptor without extracting — /srv/data is read-only.
import zipfile
from pathlib import Path
DATA = Path('/srv/data/temporal-patterns-biodiversity')
z = zipfile.ZipFile(DATA / 'data&code.zip')
for info in z.infolist():
if not info.is_dir():
print(f"{info.file_size/1e6:7.2f} MB {info.filename}")
print('\n--- descriptor.txt ---')
print(z.read('data&code/descriptor.txt').decode('utf-8', 'replace').strip())
12.45 MB data&code/All_TAXA_10_replicates_with_holidays.csv 0.00 MB data&code/descriptor.txt 0.00 MB data&code/scp1.R 0.00 MB data&code/scp2.R 0.00 MB data&code/scp3.R 0.01 MB data&code/scp4.R --- descriptor.txt --- Companion data and code for paper: When do citizen scientists record biodiversity? Non-random temporal patterns of recording effort and associated factors. Authors: Inês.T. Rosário1*; Patrícia. Tiago1; Sergio Chozas1; César Capinha2,3* 1 cE3c – Centre for Ecology, Evolution and Environmental Changes & CHANGE - Global Change and Sustainability Institute, Faculdade de Ciências, Universidade de Lisboa, Campo Grande, 1749-016 Lisboa, Portugal 2 Centre of Geographical Studies, Institute of Geography and Spatial Planning, University of Lisbon, Rua Branca Edmée Marques, 1600-276 Lisbon, Portugal 3 Associate Laboratory TERRA, Lisbon Portugal
The clever part: a case–control design¶
The single CSV is All_TAXA_10_replicates_with_holidays.csv. It does not just list recordings — it is built to test when people record. Each real observation of a tree (Dependent = 1) is paired with 10 pseudo-absences (Dependent = 0): the same location, but a random date drawn from the same year.
So the Dependent = 0 rows describe the calendar and weather that were simply available, while Dependent = 1 rows describe the days people actually chose. If recording were random in time, the two groups would look identical. Any systematic difference — more weekends, more holidays, nicer weather — is the sampling bias we are hunting.
We load the whole table (~78k rows, ~30 MB — comfortably within budget) and read only the columns we need.
import pandas as pd
import numpy as np
usecols = ['Species', 'Country', 'year', 'month', 'Month', 'Weekday',
'Dependent', 'Holiday', 'Temperature', 'Precipitation', 'Wind', 'Temperature_K']
df = pd.read_csv(z.open('data&code/All_TAXA_10_replicates_with_holidays.csv'),
usecols=usecols, low_memory=False)
# A handful of rows have Temperature_K == 0 (missing weather); mask those weather values.
bad = df['Temperature_K'] == 0
df.loc[bad, ['Temperature', 'Precipitation', 'Wind']] = np.nan
df = df.drop(columns='Temperature_K')
print('rows:', len(df))
print('real records (Dependent=1):', int((df.Dependent == 1).sum()))
print('pseudo-absences (Dependent=0):', int((df.Dependent == 0).sum()))
print('years:', sorted(df.year.unique()))
print('countries:', df.Country.value_counts().to_dict())
print('species:', df.Species.nunique(), '->', list(df.Species.unique()))
df.head()
rows: 77561
real records (Dependent=1): 7051
pseudo-absences (Dependent=0): 70510
years: [np.int64(2017), np.int64(2018), np.int64(2019), np.int64(2022), np.int64(2023)]
countries: {'Spain': 48631, 'Portugal': 28930}
species: 7 -> ['Pinus pinea', 'Quercus suber', 'Quercus rotundifolia', 'Pinus pinaster', 'Pinus halepensis', 'Pinus sylvestris', 'Pinus nigra']
| Species | month | year | Dependent | Weekday | Month | Temperature | Precipitation | Wind | Country | Holiday | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | Pinus pinea | 1 | 2017 | 0 | Sunday | Jan | 11.080652 | 0.02 | 3.166559 | Portugal | 1 |
| 1 | Pinus pinea | 1 | 2017 | 0 | Sunday | Jan | 11.175470 | 0.02 | 3.077367 | Portugal | 1 |
| 2 | Quercus suber | 1 | 2017 | 0 | Monday | Jan | 7.776483 | 9.36 | 6.392357 | Portugal | 0 |
| 3 | Quercus rotundifolia | 1 | 2017 | 0 | Monday | Jan | 10.986963 | 0.08 | 4.979019 | Portugal | 0 |
| 4 | Quercus rotundifolia | 1 | 2017 | 0 | Monday | Jan | 7.712793 | 4.40 | 4.119776 | Portugal | 0 |
A reusable measure: the selection ratio¶
For any category (a weekday, a month, a holiday flag) we compare its share among real records to its share among the available background:
$$\text{selection ratio} = \frac{P(\text{category}\mid \text{recorded})}{P(\text{category}\mid \text{available})}$$
A ratio of 1 means "recorded exactly as often as chance would predict". Above 1 = people prefer that condition; below 1 = they avoid it.
def selection_ratio(frame, col):
p_rec = frame.loc[frame.Dependent == 1, col].value_counts(normalize=True)
p_avail = frame.loc[frame.Dependent == 0, col].value_counts(normalize=True)
out = pd.DataFrame({'recorded': p_rec, 'available': p_avail})
out['ratio'] = out['recorded'] / out['available']
return out
selection_ratio(df, 'Weekday').round(3)
| recorded | available | ratio | |
|---|---|---|---|
| Weekday | |||
| Friday | 0.140 | 0.143 | 0.977 |
| Monday | 0.112 | 0.144 | 0.782 |
| Saturday | 0.198 | 0.142 | 1.396 |
| Sunday | 0.204 | 0.143 | 1.432 |
| Thursday | 0.123 | 0.144 | 0.857 |
| Tuesday | 0.117 | 0.143 | 0.822 |
| Wednesday | 0.105 | 0.142 | 0.739 |
1. Day of the week¶
We order the weekdays Monday→Sunday and plot the selection ratio. The dashed line at 1.0 is the "no preference" baseline.
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme(style='whitegrid')
order = ['Monday','Tuesday','Wednesday','Thursday','Friday','Saturday','Sunday']
wd = selection_ratio(df, 'Weekday').reindex(order)
fig, ax = plt.subplots(figsize=(9, 4.5))
colors = ['#4c72b0' if d < 5 else '#dd8452' for d in range(7)]
ax.bar(wd.index, wd['ratio'], color=colors)
ax.axhline(1.0, ls='--', color='grey', lw=1)
ax.set_ylabel('Selection ratio\n(recorded / available)')
ax.set_title('Citizens record biodiversity far more on weekends')
for x, v in zip(range(7), wd['ratio']):
ax.text(x, v + 0.02, f'{v:.2f}', ha='center', fontsize=9)
plt.xticks(rotation=30)
plt.tight_layout(); plt.show()
print('Weekend vs weekday mean ratio:',
round(wd.loc[["Saturday","Sunday"],"ratio"].mean(), 2), 'vs',
round(wd.loc[order[:5], "ratio"].mean(), 2))
Weekend vs weekday mean ratio: 1.41 vs 0.84
Saturday and Sunday sit well above 1, while mid-week days fall to ~0.75. People clearly log wildlife in their leisure time — the very first non-random pattern the paper reports.
2. Season (month of year)¶
Do certain months attract more recording than their availability would predict?
month_order = ['Jan','Feb','Mar','Apr','May','Jun','Jul','Aug','Sep','Oct','Nov','Dec']
mo = selection_ratio(df, 'Month').reindex(month_order)
fig, ax = plt.subplots(figsize=(9, 4.5))
sns.barplot(x=mo.index, y=mo['ratio'], color='#55a868', ax=ax)
ax.axhline(1.0, ls='--', color='grey', lw=1)
ax.set_ylabel('Selection ratio')
ax.set_xlabel('')
ax.set_title('Spring peaks, deep-summer and winter dips in recording effort')
plt.tight_layout(); plt.show()
Spring months are over-represented (nature is active and the weather is pleasant), while the hottest and coldest months are visited less than chance — another temporal bias that shapes what the data can tell us about phenology.
3. Public holidays and weather¶
Two more candidate drivers. Holidays: are free days over-used like weekends? Weather: do people wait for warmer, drier conditions? For weather we compare the full distribution recorded vs available, not just the mean.
fig, axes = plt.subplots(1, 3, figsize=(14, 4.5))
# --- Holidays: selection ratio ---
hol = selection_ratio(df, 'Holiday').reindex([0, 1])
axes[0].bar(['Ordinary day', 'Public holiday'], hol['ratio'], color=['#8c8c8c', '#c44e52'])
axes[0].axhline(1.0, ls='--', color='grey', lw=1)
axes[0].set_ylabel('Selection ratio')
axes[0].set_title('Holidays')
for x, v in enumerate(hol['ratio']):
axes[0].text(x, v + 0.02, f'{v:.2f}', ha='center')
# --- Temperature distribution ---
lab = {0: 'available', 1: 'recorded'}
for d in (0, 1):
sns.kdeplot(df.loc[df.Dependent == d, 'Temperature'].dropna(),
ax=axes[1], label=lab[d], fill=True, alpha=0.3)
axes[1].set_title('Temperature (°C)')
axes[1].set_xlabel('Temperature (°C)'); axes[1].legend()
# --- Precipitation: share of dry days ---
dry = (df.assign(dry=df.Precipitation < 1)
.groupby('Dependent')['dry'].mean().reindex([0, 1]))
axes[2].bar(['available', 'recorded'], dry.values, color=['#8c8c8c', '#4c72b0'])
axes[2].set_ylim(0, 1)
axes[2].set_title('Fraction of dry days (<1 mm)')
for x, v in enumerate(dry.values):
axes[2].text(x, v + 0.01, f'{v:.0%}', ha='center')
plt.tight_layout(); plt.show()
print('Holiday recording rate: {:.1%} of records vs {:.1%} of background'.format(
df.loc[df.Dependent==1,'Holiday'].mean(), df.loc[df.Dependent==0,'Holiday'].mean()))
Holiday recording rate: 5.6% of records vs 3.4% of background
Reading the panels. Public holidays behave like extra weekends (ratio well above 1). The temperature curves overlap almost entirely — once season is accounted for, day-to-day temperature is a weak driver here. Dry days are recorded slightly more than they occur, hinting at a mild fair-weather preference.
Takeaway — the sampling bias¶
Recording effort is not random in time: it clusters on weekends, public holidays and spring. Any study that treats these citizen records as an even sample through time will over-weight leisure days and the spring season, and could mistake observer behaviour for biological signal (e.g. an apparent spring "peak" in a species that is really just a peak in human activity). Correcting for these patterns — as the pseudo-absence design here does — is essential before drawing ecological conclusions.
Your turn¶
This is just the starting point. Some ideas:
- Check the dataset challenge on its CSDH data sheet.
- Work on a copy: right-click the file → Duplicate (or Save Notebook As…). Your changes only live in your Hub space — they're never pushed to GitHub.
- Edited this notebook and want the original back? Use the Restore cell
below (or the
restore.ipynbnotebook). - Questions and results: on the platform forum.
Attribution: data from Non-random Temporal Patterns of Citizen Science Biodiversity Recording, license CC-BY-4.0. Notebook from the Citizen Science Data Hub (CSDH) — Fundación Ibercivis.
# ⚠️ RESTORE: this DISCARDS YOUR CHANGES to this notebook and resets it to the original.
# 1. Uncomment the line below (remove the #) 2. Run this cell
# 3. Then: menu File → Reload Notebook from Disk
# !git -C ~/citizen-science-data fetch -q origin && git -C ~/citizen-science-data checkout origin/main -- temporal-patterns-biodiversity.ipynb && echo "Restored. Now: File → Reload Notebook from Disk"