Tropical Rainforest Birds: Acoustic + eBird Distribution Models

Category: Ornithology · Size: 8.3 GB · Format: CSV, R, ZIP License: CC-BY-4.0 · Zenodo record · Data sheet on the CSDH

Automated acoustic monitoring data and eBird observations combined for distribution models of suboscine birds in southwestern Amazonia.

The data is mounted read-only at /srv/data/tropical-rainforest-birds/. Save anything you produce in your personal folder (~/).

⚠️ Large dataset (8.3 GB). Your Hub session has 4 GB RAM — do not load the whole file into memory or the kernel will crash. Work like the pros: read only the columns you need, process the file in chunks, or query it straight from disk with DuckDB (no full load). Copy-paste patterns are in "Working with data larger than memory" near the end of this notebook.

What's in this dataset

This is the tabular output of a study that mapped five suboscine bird species (antbirds, antthrushes and relatives) in the rainforest around the Los Amigos station (CICRA), Madre de Dios, southwestern Peru.

Two very different survey methods were combined:

• Audio – automated acoustic recorders that listen continuously and are scanned for each species' song.
• eBird – checklists submitted by human birdwatchers.

For every species the authors built a zero-filled detection history: one row per survey with species_observed = True/False, so absences are recorded, not just presences. Those files live inside data.zip and are the clean product we analyse here — no need to re-run the R occupancy models.

We read the tables straight from inside the ZIP (never extracting the whole 1.5 GB archive) to respect the memory budget.

import zipfile
import pandas as pd
from pathlib import Path

DATA = Path('/srv/data/tropical-rainforest-birds')

# The per-species zero-filled checklists are entries inside data.zip.
# We skip the "__MACOSX" resource-fork entries that macOS adds to ZIPs.
zf = zipfile.ZipFile(DATA / 'data.zip')
species_files = {
    Path(n).stem.split('-')[-1]: n
    for n in zf.namelist()
    if 'checklists-zf' in n and n.endswith('.csv') and '__MACOSX' not in n
}
species_files

{'rucant2': 'data/checklists-zf_CICRA_relOct-2024-rucant2.csv',
 'whtant1': 'data/checklists-zf_CICRA_relOct-2024-whtant1.csv',
 'thlant2': 'data/checklists-zf_CICRA_relOct-2024-thlant2.csv',
 'blfant1': 'data/checklists-zf_CICRA_relOct-2024-blfant1.csv',
 'goeant1': 'data/checklists-zf_CICRA_relOct-2024-goeant1.csv'}

The dictionary keys are eBird species codes. We give each a short readable label just for the plots, then load one species to see the columns.

LABELS = {
    'blfant1': 'Black-faced Antbird',
    'goeant1': "Goeldi's Antbird",
    'whtant1': 'White-throated Antbird',
    'rucant2': 'Rufous-capped Antthrush',
    'thlant2': 'Thrush-like Antpitta',
}

def load_species(code, cols=None):
    """Read one species' zero-filled checklist straight from the ZIP."""
    with zf.open(species_files[code]) as f:
        return pd.read_csv(f, usecols=cols)

demo = load_species('blfant1')
print('rows:', len(demo), '| columns:', list(demo.columns))
demo[['checklist_id', 'source', 'protocol_type', 'species_observed',
      'observation_date', 'latitude', 'longitude', 'effort_hours']].head()

rows: 30510 | columns: ['checklist_id', 'observer_id', 'observation_count', 'species_observed', 'state_code', 'locality_id', 'latitude', 'longitude', 'protocol_type', 'all_species_reported', 'observation_date', 'year', 'day_of_year', 'hours_of_day', 'effort_hours', 'effort_distance_km', 'effort_speed_kmph', 'number_observers', 'source']

	checklist_id	source	protocol_type	species_observed	observation_date	latitude	longitude	effort_hours
0	G7641541	eBird	Stationary	False	2022-01-02	-12.861573	-69.489448	1.233333
1	G7641542	eBird	Stationary	False	2022-01-01	-12.861573	-69.489448	0.383333
2	G7641547	eBird	Traveling	False	2021-12-30	-12.861573	-69.489448	4.083333
3	G7678829	eBird	Stationary	False	2022-01-03	-12.861573	-69.489448	0.750000
4	G7206243	eBird	Stationary	False	2021-09-16	-12.337929	-70.722304	0.050000

Each species has the same ~30,500 surveys, split between the two sources. species_observed is our response variable; source tells us which method produced each survey. Let's answer the core question first.

1. How often is each species detected, by data source?

If the two methods agreed perfectly, the detection rate (share of surveys where the species was found) would be the same for Audio and eBird. We compute it for every species and compare side by side.

rows = []
for code in LABELS:
    d = load_species(code, cols=['source', 'species_observed'])
    for src, g in d.groupby('source'):
        rows.append({
            'species': LABELS[code],
            'source': src,
            'n_surveys': len(g),
            'n_detections': int(g['species_observed'].sum()),
            'detection_rate': g['species_observed'].mean(),
        })
rates = pd.DataFrame(rows)
rates.pivot(index='species', columns='source',
            values='detection_rate').round(4)

source	Audio	eBird
species
Black-faced Antbird	0.0291	0.1732
Goeldi's Antbird	0.0057	0.0568
Rufous-capped Antthrush	0.0050	0.0163
Thrush-like Antpitta	0.0120	0.0152
White-throated Antbird	0.0112	0.0156

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme(style='whitegrid', context='talk')

fig, ax = plt.subplots(figsize=(11, 6))
order = rates.groupby('species')['detection_rate'].max().sort_values().index
sns.barplot(data=rates, y='species', x='detection_rate', hue='source',
            order=order, palette={'Audio': '#2a9d8f', 'eBird': '#e76f51'}, ax=ax)
ax.set_xlabel('Detection rate  (fraction of surveys with the species)')
ax.set_ylabel('')
ax.set_title('Audio vs eBird detection rate by species')
ax.legend(title='Source', loc='lower right')
plt.tight_layout()
plt.show()

The methods disagree, and not in one consistent direction. For Rufous-capped Antthrush and Thrush-like Antpitta the acoustic recorders pick the species up more often — these are vocal, cryptic understory birds that human observers easily miss. For Black-faced Antbird and Goeldi's Antbird it is the reverse: birdwatchers report them at several times the audio rate. This is exactly the kind of method-dependence the study was built to quantify.

2. Where does each source detect the species?

A difference in overall rate could hide a spatial disagreement: maybe the two methods sample different parts of the landscape. We map one focal species, Black-faced Antbird, showing all surveyed sites in grey and the detections coloured by source.

focal = load_species('blfant1',
    cols=['source', 'species_observed', 'latitude', 'longitude'])

fig, axes = plt.subplots(1, 2, figsize=(13, 6), sharex=True, sharey=True)
for ax, src in zip(axes, ['Audio', 'eBird']):
    sub = focal[focal['source'] == src]
    ax.scatter(sub['longitude'], sub['latitude'], s=6, color='0.8',
               label='surveyed (no detection)')
    det = sub[sub['species_observed']]
    ax.scatter(det['longitude'], det['latitude'], s=28,
               color='#c1121f', edgecolor='k', linewidth=0.3,
               label='detected')
    ax.set_title(f'{src}  ({len(det)} detections / {len(sub)} surveys)')
    ax.set_xlabel('Longitude')
    ax.set_aspect('equal', adjustable='box')
axes[0].set_ylabel('Latitude')
axes[0].legend(loc='upper left', fontsize=10, framealpha=0.9)
fig.suptitle('Black-faced Antbird: where each method detects it', y=1.02)
plt.tight_layout()
plt.show()

The audio recorders are clustered at a handful of fixed monitoring sites, while eBird surveys spread out along trails and rivers wherever birders go. So the two sources not only differ in rate — they cover different geography, which matters a lot when you combine them in a distribution model.

3. Do detections line up with the environment?

The dataset ships an environmental table keyed on checklist_id. We join it to our focal species and ask whether detections favour particular conditions — here elevation and HAND (Height Above Nearest Drainage, a proxy for how far a site sits above the nearest stream).

env = pd.read_csv(
    DATA / 'environmental-variables_checklists.csv',
    usecols=['checklist_id', 'elevation_mean_300m', 'HAND_mean_300m'])

focal_env = load_species('blfant1',
    cols=['checklist_id', 'species_observed']).merge(env, on='checklist_id')
focal_env['detected'] = focal_env['species_observed'].map({True: 'detected',
                                                           False: 'not detected'})

fig, axes = plt.subplots(1, 2, figsize=(12, 5.5))
for ax, col, name in zip(
        axes,
        ['elevation_mean_300m', 'HAND_mean_300m'],
        ['Elevation (m)', 'HAND – height above nearest drainage (m)']):
    sns.boxplot(data=focal_env, x='detected', y=col, hue='detected',
                order=['not detected', 'detected'], legend=False,
                palette={'not detected': '0.8', 'detected': '#c1121f'}, ax=ax)
    ax.set_xlabel('')
    ax.set_ylabel(name)
fig.suptitle('Black-faced Antbird detections vs local environment', y=1.02)
plt.tight_layout()
plt.show()

print(focal_env.groupby('detected')[['elevation_mean_300m',
                                      'HAND_mean_300m']].median().round(1))

              elevation_mean_300m  HAND_mean_300m
detected                                         
detected                    242.4            17.1
not detected                260.8            27.4

Detections sit at noticeably lower HAND values — i.e. closer to streams and floodplain — than the surveys where the bird was absent. That is an ecologically sensible signal (this antbird favours low, wetter forest) and it is the kind of habitat relationship the study's occupancy models formalise. From here you could swap in the other land-cover columns (pland_c04_terra_firme_300m, canopy_mean_300m, …) or another species and see how the story changes.

Working with data larger than memory

Your session has a 4 GB RAM limit, but you can analyse files of 10 GB or more without loading them whole:

• Read only the columns you need: pd.read_csv(f, usecols=[...]) / pd.read_parquet(f, columns=[...]).
• Process in chunks and keep only the result:

  total = 0
  for chunk in pd.read_csv(file, chunksize=1_000_000):
      total += len(chunk)
  

• Query with SQL without loading anything — DuckDB (already installed) reads CSV and Parquet straight from disk and only brings the result into memory:

  import duckdb
  duckdb.sql("SELECT column, count(*) FROM '/srv/data/.../file.parquet' GROUP BY column").df()
  

Your turn

This is just the starting point. Some ideas:

• Check the dataset challenge on its CSDH data sheet.
• Work on a copy: right-click the file → Duplicate (or Save Notebook As…). Your changes only live in your Hub space — they're never pushed to GitHub.
• Edited this notebook and want the original back? Use the Restore cell below (or the restore.ipynb notebook).
• Questions and results: on the platform forum.

Attribution: data from Tropical Rainforest Birds: Acoustic + eBird Distribution Models, license CC-BY-4.0. Notebook from the Citizen Science Data Hub (CSDH) — Fundación Ibercivis.

# ⚠️ RESTORE: this DISCARDS YOUR CHANGES to this notebook and resets it to the original.
# 1. Uncomment the line below (remove the #)   2. Run this cell
# 3. Then: menu File → Reload Notebook from Disk

# !git -C ~/citizen-science-data fetch -q origin && git -C ~/citizen-science-data checkout origin/main -- tropical-rainforest-birds.ipynb && echo "Restored. Now: File → Reload Notebook from Disk"