SONYC Urban Sound Tagging (SONYC-UST)¶
Category: Urban Acoustics · Size: 1.9 GB · Format: CSV, CSV.gz, MD, YAML License: CC-BY-4.0 · Zenodo record · Data sheet on the CSDH
Multi-class dataset of 2,794 ten-second audio recordings from the SONYC acoustic sensor network in New York, annotated by Zooniverse volunteers for 23 urban noise classes.
The data is mounted read-only at /srv/data/sonyc-urban-sound/.
Save anything you produce in your personal folder (~/).
⚠️ Large dataset (1.9 GB). Your Hub session has 4 GB RAM — do not load the whole file into memory or the kernel will crash. Work like the pros: read only the columns you need, process the file in chunks, or query it straight from disk with DuckDB (no full load). Copy-paste patterns are in "Working with data larger than memory" near the end of this notebook.
What's in the dataset¶
The mounted folder holds one lightweight annotation table plus the raw audio and
the label taxonomy. All the analysis lives in the small CSV, so we never need to
touch the 1.9 GB audio.tar.gz. Let's list what's there.
from pathlib import Path
DATA = Path('/srv/data/sonyc-urban-sound')
for f in sorted(DATA.rglob('*')):
if f.is_file():
print(f"{f.relative_to(DATA)} ({f.stat().st_size/1e6:,.1f} MB)")
README.md (0.0 MB) annotations.csv (2.6 MB) audio.tar.gz (1,892.5 MB) dcase-ust-taxonomy.yaml (0.0 MB)
The annotation table¶
annotations.csv is the heart of the dataset. Each row is one person's tagging
of one 10-second clip — so the same clip appears several times:
annotator_id > 0→ a Zooniverse citizen-science volunteerannotator_id < 0→ a SONYC team member (validation set only)annotator_id == 0→ the expert ground truth agreed by the SONYC team
For each of the 8 coarse noise classes there is a *_presence column: 1 = the
annotator heard it, 0 = they didn't, -1 = not labelled. There are also 23
fine-grained classes and *_proximity columns, but we'll focus on the 8 coarse
presence columns. We only read the columns we need.
import pandas as pd
# The 8 coarse-class presence columns (id_name_presence, no dash in the id part)
coarse_cols = ['1_engine_presence', '2_machinery-impact_presence',
'3_non-machinery-impact_presence', '4_powered-saw_presence',
'5_alert-signal_presence', '6_music_presence',
'7_human-voice_presence', '8_dog_presence']
labels = [c.split('_')[1] for c in coarse_cols] # short names for plots
meta_cols = ['split', 'sensor_id', 'audio_filename', 'annotator_id']
ann = pd.read_csv(DATA / 'annotations.csv',
usecols=meta_cols + coarse_cols, low_memory=False)
# Some presence columns were read as mixed types -> force numeric, treat -1 as 0
for c in coarse_cols:
ann[c] = pd.to_numeric(ann[c], errors='coerce').fillna(0).clip(lower=0).astype(int)
print(f"{len(ann):,} annotations · {ann.audio_filename.nunique():,} clips "
f"· {ann.sensor_id.nunique()} sensors")
print("annotators:",
f"volunteers={int((ann.annotator_id > 0).sum())},",
f"team={int((ann.annotator_id < 0).sum())},",
f"ground-truth={int((ann.annotator_id == 0).sum())}")
ann.head()
10,349 annotations · 2,794 clips · 43 sensors annotators: volunteers=8382, team=1524, ground-truth=443
| split | sensor_id | audio_filename | annotator_id | 1_engine_presence | 2_machinery-impact_presence | 3_non-machinery-impact_presence | 4_powered-saw_presence | 5_alert-signal_presence | 6_music_presence | 7_human-voice_presence | 8_dog_presence | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | validate | 0 | 00_000066.wav | 95 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 |
| 1 | validate | 0 | 00_000066.wav | 108 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 2 | validate | 0 | 00_000066.wav | 127 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 3 | validate | 0 | 00_000118.wav | 45 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
| 4 | validate | 0 | 00_000118.wav | 58 | 1 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
From many opinions to one soundscape¶
Every clip was tagged by at least three volunteers, who don't always agree. To describe the city's soundscape we collapse the crowd into a single verdict per clip using a majority vote: a class is "present" in a clip if half or more of the volunteers who heard it tagged it. This turns the long annotation table into one clean clip × class presence matrix.
volunteers = ann[ann.annotator_id > 0]
# mean presence per clip = fraction of volunteers who tagged the class
vote = volunteers.groupby('audio_filename')[coarse_cols].mean()
present = (vote >= 0.5).astype(int) # majority-vote clip x class matrix
present.columns = labels
print(f"Clip × class matrix: {present.shape[0]:,} clips × {present.shape[1]} classes")
print(f"Clips with at least one noise class present: "
f"{(present.sum(axis=1) > 0).mean()*100:.0f}%")
present.head()
Clip × class matrix: 2,794 clips × 8 classes Clips with at least one noise class present: 70%
| engine | machinery-impact | non-machinery-impact | powered-saw | alert-signal | music | human-voice | dog | |
|---|---|---|---|---|---|---|---|---|
| audio_filename | ||||||||
| 00_000066.wav | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 00_000118.wav | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 |
| 00_000275.wav | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 00_000277.wav | 0 | 0 | 0 | 0 | 1 | 0 | 0 | 0 |
| 00_000357.wav | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
1. How common is each type of noise?¶
The first question: which sounds dominate New York's streets? We simply count, for each coarse class, the share of clips where the crowd agreed it was present.
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme(style='whitegrid')
freq = (present.mean() * 100).sort_values()
fig, ax = plt.subplots(figsize=(8, 4.5))
ax.barh(freq.index, freq.values, color=sns.color_palette('crest', len(freq)))
for y, v in enumerate(freq.values):
ax.text(v + 0.3, y, f"{v:.1f}%", va='center', fontsize=9)
ax.set_xlabel('share of clips where the class is present (%)')
ax.set_title('How common is each urban-noise class in SONYC clips')
plt.tight_layout()
plt.show()
Engines and human voices are by far the most common ingredients of the NYC soundscape, each in roughly a quarter of clips, followed by alert signals (horns, sirens, alarms) and machinery impact (construction). Music and dogs are comparatively rare.
2. Which noises occur together?¶
Urban sounds rarely happen in isolation. We measure how often two classes share the same clip with the Jaccard index — clips where both are present divided by clips where either is present. A heatmap makes the pairings pop out.
import numpy as np
P = present.values.astype(bool)
K = P.shape[1]
jac = np.zeros((K, K))
for i in range(K):
for j in range(K):
inter = (P[:, i] & P[:, j]).sum()
union = (P[:, i] | P[:, j]).sum()
jac[i, j] = inter / union if union else 0.0
mask = np.triu(np.ones_like(jac, dtype=bool), k=1) # show lower triangle
fig, ax = plt.subplots(figsize=(7.5, 6))
sns.heatmap(jac, mask=mask, xticklabels=labels, yticklabels=labels,
cmap='rocket_r', annot=True, fmt='.2f', square=True,
cbar_kws={'label': 'Jaccard overlap'}, ax=ax)
ax.set_title('Co-occurrence of noise classes in the same clip')
plt.tight_layout()
plt.show()
The strongest overlap is engine ↔ human-voice — the everyday street mix of traffic and people. Alert signals and machinery impact also travel with engines, consistent with busy-street and construction scenes. The diagonal is 1 by definition and is masked away.
3. Do volunteers and experts agree?¶
A core citizen-science question: can the crowd be trusted? The validate split
has expert ground truth (annotator_id == 0) for the same clips the volunteers
tagged. We compare, per class, the share of clips flagged positive by the expert
versus by the volunteer majority vote.
val = ann[ann.split == 'validate']
# expert ground truth: one row per clip
expert = (val[val.annotator_id == 0]
.set_index('audio_filename')[coarse_cols])
expert.columns = labels
# volunteer majority vote on the same clips
vcrowd = (val[val.annotator_id > 0]
.groupby('audio_filename')[coarse_cols].mean() >= 0.5).astype(int)
vcrowd.columns = labels
clips = expert.index.intersection(vcrowd.index)
expert, vcrowd = expert.loc[clips], vcrowd.loc[clips]
compare = pd.DataFrame({
'expert': expert.mean() * 100,
'volunteers': vcrowd.mean() * 100,
}).loc[labels]
fig, ax = plt.subplots(figsize=(9, 4.5))
compare.plot.bar(ax=ax, color=['#d1495b', '#30638e'], width=0.8)
ax.set_ylabel('share of validate clips positive (%)')
ax.set_title(f'Crowd vs expert on {len(clips)} validated clips')
ax.set_xticklabels(labels, rotation=35, ha='right')
ax.legend(title=None)
plt.tight_layout()
plt.show()
# recall: of expert-positive clips, how many did the crowd also catch?
recall = {lab: ((expert[lab] == 1) & (vcrowd[lab] == 1)).sum() / max(int(expert[lab].sum()), 1)
for lab in labels}
print("Crowd recall vs expert (fraction of expert-positive clips also caught):")
for lab in labels:
print(f" {lab:22s} {recall[lab]*100:5.0f}%")
Crowd recall vs expert (fraction of expert-positive clips also caught): engine 43% machinery-impact 19% non-machinery-impact 3% powered-saw 39% alert-signal 80% music 80% human-voice 76% dog 83%
Volunteers and experts agree closely for loud, unambiguous sounds (music, dogs, alert signals), but the crowd systematically under-reports subtler or ambiguous classes — especially engines, human voice and construction — tagging fewer clips than the experts. That conservative bias is exactly the kind of insight a citizen-science project needs before using volunteer tags as training labels.
Engines and human voices are by far the most common ingredients of the NYC soundscape, each in roughly a quarter of clips, followed by alert signals (horns, sirens, alarms) and machinery impact (construction). Music and dogs are comparatively rare.
2. Which noises occur together?¶
Urban sounds rarely happen in isolation. We measure how often two classes share the same clip with the Jaccard index — clips where both are present divided by clips where either is present. A heatmap makes the pairings pop out.
import numpy as np
P = present.values.astype(bool)
K = P.shape[1]
jac = np.zeros((K, K))
for i in range(K):
for j in range(K):
inter = (P[:, i] & P[:, j]).sum()
union = (P[:, i] | P[:, j]).sum()
jac[i, j] = inter / union if union else 0.0
mask = np.triu(np.ones_like(jac, dtype=bool), k=1) # show lower triangle
fig, ax = plt.subplots(figsize=(7.5, 6))
sns.heatmap(jac, mask=mask, xticklabels=labels, yticklabels=labels,
cmap='rocket_r', annot=True, fmt='.2f', square=True,
cbar_kws={'label': 'Jaccard overlap'}, ax=ax)
ax.set_title('Co-occurrence of noise classes in the same clip')
plt.tight_layout()
plt.show()