Polymath: Revolutionizing Music Production with AI

Polymath is an innovative AI tool that transforms any music library into a music production sample-library. It offers a seamless and efficient workflow for music producers, DJs, and ML audio developers.

Key Features:

• Automatically separates songs into stems (beats, bass, etc.).
• Quantizes songs to the same tempo and beat-grid.
• Analyzes musical structure, key, and other important infos.
• Converts audio to MIDI.

Use Cases:

• Easily combine elements from different songs to create unique compositions.
• Quickly create a polished, hour-long mash-up DJ set.
• Simplify the process of creating a large music dataset for ML developers.

How It Works:

• Music Source Separation is performed with the Demucs neural network.
• Music Structure Segmentation/Labeling is performed with the sf_segmenter neural network.
• Music Pitch Tracking and Key Detection are performed with Crepe neural network.
• Music to MIDI transcription is performed with Basic Pitch neural network.
• Music Quantization and Alignment are performed with pyrubberband.
• Music Info retrieval and processing is performed with librosa.

Requirements and Installation:

• Requires ffmpeg and python version >=3.7 and <=3.10.
• To install, follow the steps: git clone , cd polymath, pip install -r requirements.txt. If you encounter an issue with basic-pitch, run pip install git+.
• Most libraries used come with native GPU support through cuda. Follow the steps on to setup tensorflow for use with cuda.

Docker Setup:

• If Docker is installed, use the provided Dockerfile to build a polymath docker image. Create the necessary directories for input and output file exchange.

Running Polymath:

• Add songs to the Polymath Library, including from YouTube or local audio files.
• Quantize songs in the library to a specific tempo or the tempo of the song.
• Search for similar songs in the library and optionally quantize them.
• Convert processed audio files and stems to MIDI.

Audio Features Extracted:

• Stems: bass, drum, guitar, other, piano, vocals.
• Features: tempo, duration, timbre, pitch, intensity, volume, loudness, beats, segments boundaries, segments labels, frequency.

Polymath is released under the MIT license and is a powerful tool for those working in the music and AI fields.