About Watkins Marine Mammal Sound Database (Remasterd Delux Edition)

About the Watkins Marine Mammal Sound Database

See the about page at Woods Hole Oceanographic Institution (WHOI) for a detailed description of the database.

About this Remaster

Goal

This project aims to make the 15,000 annotated digital sound clips more accessible for research.

Initial Situation

The current (2018) WHOI website is very limited in its features to explore sound clips and recordings. Most of the detailed metadata, that describe all sound clips, can not be used as search filters and are hardly explained.

As the WHOI about page states:

Due to the outdated and nonstandard format of the metadata, letters and codes may appear in some of the tables. These codes are described in the publication of the original DOS-based database.

Cleaning up that metadata is therefor one of the main tasks of this project.

All sound clips, recordings and metadata can be directly downloaded or scraped from the WHOI website.

Project Outline

Implementing the goal means:

• Understanding what questions and requirements researchers have
• What the database contains to offer answers
• How to implement the requirements
• How to translate questions into user interface features

Example Questions

(TODO)

Requirements

• A researcher needs to be confident, that that data is accurate. The transformation should not introduce errors, and if any occur they should be documented.
• A researcher needs to be able to reproduce the data transformation
• A researcher needs to reference a specific state of the (transformed) data

Tasks

These goals and requirments translate into the following tasks.

Aggregate the data

• Download all metadata and sound clips from the WHOI website

Bring the data to modern formats

• Extract, normalize and transform all information from the database
• Document the details of the database contents
• Develop the data transformation process as open source code
• Document all steps required to reproduce the resulting data
• Make all transformed data available as versioned download
• Code and data should follow the same version schema, to clarify what code produced what data

Build a user interface for research

• Provide a search interface to explore all recordings
• Provide search filters for various metadata properties (animal, observation, location, signal, sound, etc)
• Provide a World Map to explore all records by location

Enrich the data

• Analyze all sound clips and expose gained statistical values as search filters
• Use the location description to get more precise location coordinates

Source Code

The source code GIT repository is currently hosted on Codeberg and mirrored on RocketGIT.

It contains:

• Shell-Scripts to download, transform and index the data
• Scripts to analyze the audio clips
• Configuration for Apache Web-Server and ElasticSearch index.
• Website with Search UI and World Map