MedSens: Mediterranean Marine Species Abundance (Reef Check Italia)

Category: Marine Biodiversity · Size: 2.3 MB · Format: ZIP License: CC-BY-4.0 · Zenodo record · Data sheet on the CSDH

Abundance of 25 Mediterranean marine species collected by volunteer divers under the Reef Check Mediterranean protocol, used to compute the MedSens biotic index.

The data is mounted read-only at /srv/data/medsens-mediterranean/. Save anything you produce in your personal folder (~/).

What's in the dataset

The download is a single ESRI shapefile (MedSens_data.shp plus its .dbf/.shx/.prj sidecars) packed in a ZIP. Each row is one species recorded during one diver survey, so the table is in long format: 44,800 rows covering 25 indicator species observed at thousands of georeferenced dive sites across the Mediterranean.

We can read it straight from inside the ZIP with GeoPandas — no need to extract anything to disk.

import zipfile
from pathlib import Path

DATA = Path('/srv/data/medsens-mediterranean')
zpath = sorted(DATA.rglob('*.zip'))[0]

with zipfile.ZipFile(zpath) as z:
    for info in z.infolist():
        print(f"{info.file_size/1e6:6.2f} MB   {info.filename}")

  0.00 MB   MedSens_data.prj
  0.02 MB   MedSens_data.qml
  1.25 MB   MedSens_data.shp
  0.36 MB   MedSens_data.shx
 22.67 MB   MedSens_data.dbf

It is a shapefile, so we open it with GeoPandas using the zip://...!file.shp virtual path. That gives us a GeoDataFrame: an ordinary table plus a geometry column of point locations (one per observation), in geographic coordinates (EPSG:4326, i.e. lon/lat degrees).

import geopandas as gpd
import pandas as pd

gdf = gpd.read_file(f'zip://{zpath}!MedSens_data.shp')

print('rows, cols:', gdf.shape)
print('CRS       :', gdf.crs)
print('geometry  :', gdf.geom_type.unique())
gdf.drop(columns='geometry').head(4)

ERROR 1: PROJ: proj_create_from_database: Open of /opt/tljh/user/share/proj failed

rows, cols: (44800, 26)
CRS       : EPSG:4326
geometry  : ['Point']

	id	lat	lon	id_survey	year	date	time	source_cod	source	place	...	min_found	max_found	abundance	ab	NIS	Plevel	MSVtot	MSVphy	MSVchem	MSVbio
0	1155	40.85225	24.34598	2379	2012	05/05/2012 13:00:00	13	1565	Davide Poli	Ecd01	...	3.0	10.0	100	6	0	3	1.26087	1.53846	0.71429	1.33333
1	1156	45.23041	12.48941	2208	2011	17/09/2011 12:00:00	12	1630	Discovery Tegnùe 2011 (by Eva Turicchia & C.)	Mr08_c.s. Chioggia	...	20.0	22.0	100	6	0	3	1.26087	1.53846	0.71429	1.33333
2	1157	44.33120	14.63240	6456	2022	17/08/2022 15:00:00	15	2	Massimo Ponti	Uvala Kalpic Premuda	...	2.0	6.0	100	6	0	3	1.26087	1.53846	0.71429	1.33333
3	1158	44.08700	14.98350	6454	2022	16/08/2022 17:00:00	17	2	Massimo Ponti	Brbinj - Dugi Otok	...	3.0	7.0	100	6	0	3	1.26087	1.53846	0.71429	1.33333

4 rows × 25 columns

What the columns mean

The key fields for the MedSens biotic index are:

• taxon — one of 25 Mediterranean indicator species (fish, corals, sponges, algae...).
• ab — a semi-quantitative abundance class from 0 (species absent at this survey) to 6 (very abundant). The abundance column is the same thing coded as 0/1/2/5/10/50/100.
• MSVtot — the MedSens Sensitivity Value of that species: how sensitive it is to environmental stress. It is a fixed trait per species, split into physical (MSVphy), chemical (MSVchem) and biological (MSVbio) components.
• NIS — flag for Non-Indigenous (invasive) Species.
• lat / lon, year, habitat, place, source (the volunteer diver).

Let's confirm that the sensitivity values really are constant per species, and see the range of surveys.

df = gdf.drop(columns='geometry').copy()
df['year'] = pd.to_numeric(df['year'], errors='coerce')

print('Unique species          :', df['taxon'].nunique())
print('Unique surveys           :', df['id_survey'].nunique())
print('Year range               :', int(df['year'].min()), '-', int(df['year'].max()))
print('Records where species present (ab>0):', int((df['ab'] > 0).sum()))

# Sanity check: one sensitivity value per species?
per_taxon_variation = df.groupby('taxon')[['MSVtot','MSVphy','MSVchem','MSVbio']].nunique().max()
print('\nMax distinct MSV values within any species (should all be 1):')
print(per_taxon_variation.to_string())

Unique species          : 25
Unique surveys           : 5738
Year range               : 1981 - 2023
Records where species present (ab>0): 22611

Max distinct MSV values within any species (should all be 1):
MSVtot     1
MSVphy     1
MSVchem    1
MSVbio     1

1. Which indicator species are the most sensitive?

Because each species carries a single MedSens Sensitivity Value, ranking the 25 taxa by MSVtot directly answers the first half of the scientific question. Higher = more sensitive to environmental stress, so its presence signals good ecological quality; low-sensitivity, stress-tolerant species (including invasive algae) sit at the bottom.

import matplotlib.pyplot as plt
import seaborn as sns
sns.set_theme(style='whitegrid')

sens = (df.groupby('taxon')[['MSVtot','MSVphy','MSVchem','MSVbio','NIS']]
          .first()
          .sort_values('MSVtot'))

fig, ax = plt.subplots(figsize=(9, 8))
colors = ['#c0392b' if nis else '#2c7fb8' for nis in sens['NIS']]
ax.barh(sens.index, sens['MSVtot'], color=colors)
ax.set_xlabel('MedSens Sensitivity Value (MSVtot)')
ax.set_title('Sensitivity of the 25 Mediterranean indicator species\n(red = Non-Indigenous / invasive species)')
for y, v in enumerate(sens['MSVtot']):
    ax.text(v + 0.02, y, f'{v:.2f}', va='center', fontsize=8)
ax.set_xlim(0, sens['MSVtot'].max() * 1.12)
plt.tight_layout()
plt.show()

print('Most sensitive :', sens.index[-1], f"({sens['MSVtot'].iloc[-1]:.2f})")
print('Least sensitive:', sens.index[0],  f"({sens['MSVtot'].iloc[0]:.2f})")

Most sensitive : Paramuricea clavata (2.57)
Least sensitive: Caulerpa cylindracea (0.58)

The two soft-coral gorgonians Paramuricea clavata and Eunicella cavolini, together with red coral (Corallium rubrum), top the list — these are exactly the slow-growing, structurally important species that reef monitoring worries about. At the bottom sit the invasive Caulerpa algae (flagged in red), whose spread is itself a symptom of degradation.

We can also see what kind of stress each species is most sensitive to by comparing the physical, chemical and biological components for the ten most sensitive taxa.

top10 = sens.tail(10)[['MSVphy','MSVchem','MSVbio']]
ax = top10.plot(kind='barh', figsize=(9, 6),
                color=['#41b6c4','#fecc5c','#78c679'])
ax.set_xlabel('Sensitivity value')
ax.set_title('Stress-type sensitivity components (top 10 most sensitive species)')
ax.legend(['Physical','Chemical','Biological'], title='Stress type')
plt.tight_layout()
plt.show()

2. A MedSens quality index per survey

The MedSens index summarises the ecological quality of a site from the sensitive species living there. For each survey we compute an abundance-weighted mean sensitivity over the species actually observed (ab > 0):

$$\text{MedSens}_{survey} = \frac{\sum_i \text{MSVtot}_i \cdot ab_i}{\sum_i ab_i}$$

A high value means the site is dominated by abundant, sensitive species (healthy); a low value means only stress-tolerant species remain (degraded).

present = df[df['ab'] > 0].copy()

def medsens_index(g):
    w = g['ab']
    return (g['MSVtot'] * w).sum() / w.sum()

survey = present.groupby('id_survey').apply(medsens_index).rename('medsens')

# attach one location + year + habitat per survey
meta = (present.groupby('id_survey')
        .agg(lat=('lat','first'), lon=('lon','first'),
             year=('year','first'), habitat=('habitat','first'),
             n_species=('taxon','nunique')))
survey = meta.join(survey)

print(survey['medsens'].describe().round(3).to_string())
survey.head()

count    5528.000
mean        1.617
std         0.347
min         0.583
25%         1.342
50%         1.618
75%         1.825
max         2.565

/tmp/ipykernel_43512/2231762788.py:7: FutureWarning: DataFrameGroupBy.apply operated on the grouping columns. This behavior is deprecated, and in a future version of pandas the grouping columns will be excluded from the operation. Either pass `include_groups=False` to exclude the groupings or explicitly select the grouping columns after groupby to silence this warning.
  survey = present.groupby('id_survey').apply(medsens_index).rename('medsens')

	lat	lon	year	habitat	n_species	medsens
id_survey
1	40.56976	14.33953	2006	Rocky cliff	6	2.245250
2	40.57460	14.34605	2006	Rocky cliff	6	1.984587
3	40.56955	14.32545	2006	Rocky cliff	4	2.133652
4	40.56976	14.33953	2006	Rocky cliff	6	2.115564
5	40.58828	14.37273	2006	Rocky cliff	3	1.763410

Where is ecological quality lowest?

Now we map every survey across the Mediterranean, colouring each point by its MedSens index. Because the coordinates are already in lon/lat we can plot them directly on equal-aspect axes (fully offline — no map tiles needed). Cool/dark points are low-quality sites dominated by tolerant species; bright points are high-quality reefs rich in sensitive species.

fig, ax = plt.subplots(figsize=(12, 6))
sc = ax.scatter(survey['lon'], survey['lat'], c=survey['medsens'],
                cmap='RdYlBu', s=14, alpha=0.75, edgecolor='none',
                vmin=survey['medsens'].quantile(0.02),
                vmax=survey['medsens'].quantile(0.98))
ax.set_aspect('equal')
ax.set_xlabel('Longitude'); ax.set_ylabel('Latitude')
ax.set_title('Survey-level MedSens index across the Mediterranean\n(blue = low ecological quality, red = high)')
cb = fig.colorbar(sc, ax=ax, shrink=0.8)
cb.set_label('MedSens index (abundance-weighted sensitivity)')
plt.tight_layout()
plt.show()

The surveys trace the Italian and Adriatic coasts where Reef Check Mediterranean divers are most active. To see the pattern more clearly through the overplotting, we bin the sites onto a coarse spatial grid and show the average index per cell, plus a ranking of the lowest-quality areas.

import numpy as np

# ~0.5 degree grid
survey['glon'] = (survey['lon'] // 0.5) * 0.5
survey['glat'] = (survey['lat'] // 0.5) * 0.5
grid = (survey.groupby(['glon','glat'])
        .agg(medsens=('medsens','mean'), n=('medsens','size'))
        .reset_index())
grid = grid[grid['n'] >= 5]   # only cells with enough surveys

fig, ax = plt.subplots(figsize=(12, 6))
sc = ax.scatter(grid['glon']+0.25, grid['glat']+0.25, c=grid['medsens'],
                cmap='RdYlBu', s=grid['n']*3, alpha=0.85,
                edgecolor='k', linewidth=0.3)
ax.set_aspect('equal')
ax.set_xlabel('Longitude'); ax.set_ylabel('Latitude')
ax.set_title('Mean MedSens index on a 0.5deg grid (marker size = number of surveys)')
fig.colorbar(sc, ax=ax, shrink=0.8, label='Mean MedSens index')
plt.tight_layout()
plt.show()

print('Lowest-quality grid cells (mean MedSens, >=5 surveys):')
print(grid.sort_values('medsens').head(8).round(2).to_string(index=False))

Lowest-quality grid cells (mean MedSens, >=5 surveys):
 glon  glat  medsens   n
 14.0  44.0     0.84   5
 19.0  40.0     1.15   6
 14.5  42.0     1.20  69
 14.0  36.0     1.25  26
 12.5  44.0     1.32 802
 13.5  45.0     1.32 183
 12.5  43.5     1.33   5
 18.0  39.5     1.35   9

3. Does quality differ by habitat?

Finally, a quick look at how the index varies across the main seabed habitats gives ecological context to the map — deep offshore and rocky-cliff reefs typically host the most sensitive gorgonian communities, while sedimentary or port habitats score lower.

top_hab = present['habitat'].value_counts().head(8).index
sub = survey[survey['habitat'].isin(top_hab)]
order = sub.groupby('habitat')['medsens'].median().sort_values().index

fig, ax = plt.subplots(figsize=(10, 6))
sns.boxplot(data=sub, y='habitat', x='medsens', order=order,
            palette='viridis', ax=ax)
ax.set_xlabel('Survey MedSens index'); ax.set_ylabel('')
ax.set_title('Ecological quality by habitat type')
plt.tight_layout()
plt.show()

/tmp/ipykernel_43512/2721824484.py:6: FutureWarning: 

Passing `palette` without assigning `hue` is deprecated and will be removed in v0.14.0. Assign the `y` variable to `hue` and set `legend=False` for the same effect.

  sns.boxplot(data=sub, y='habitat', x='medsens', order=order,

Takeaways

• The most stress-sensitive indicator species are the gorgonian soft corals (Paramuricea clavata, Eunicella cavolini) and red coral; the least sensitive are the invasive Caulerpa algae.
• A simple abundance-weighted MedSens index per survey lets us map ecological quality; it varies markedly between sites and habitats.
• The Your turn section below has ideas to take this further.

Your turn

This is just the starting point. Some ideas:

• Check the dataset challenge on its CSDH data sheet.
• Work on a copy: right-click the file → Duplicate (or Save Notebook As…). Your changes only live in your Hub space — they're never pushed to GitHub.
• Edited this notebook and want the original back? Use the Restore cell below (or the restore.ipynb notebook).
• Questions and results: on the platform forum.

Attribution: data from MedSens: Mediterranean Marine Species Abundance (Reef Check Italia), license CC-BY-4.0. Notebook from the Citizen Science Data Hub (CSDH) — Fundación Ibercivis.

# ⚠️ RESTORE: this DISCARDS YOUR CHANGES to this notebook and resets it to the original.
# 1. Uncomment the line below (remove the #)   2. Run this cell
# 3. Then: menu File → Reload Notebook from Disk

# !git -C ~/citizen-science-data fetch -q origin && git -C ~/citizen-science-data checkout origin/main -- medsens-mediterranean.ipynb && echo "Restored. Now: File → Reload Notebook from Disk"