Mini toolsMuScriptor
By Mirelo AI
MuScriptor is a decoder-only transformer that performs automatic multi-instrument music transcription. Given a mel-spectrogram of a 5-second audio chunk, it autoregressively generates a sequence of discrete tokens encoding pitch, timing, and instrument type, similar to the MIDI standard, producing a separate piano-roll track per detected instrument. It was pre-trained on 1.45 million synthetic MIDI-derived recordings (DSynth), fine-tuned on 170,000 real recordings totaling over 11,000 hours (DReal), and post-trained with reinforcement learning on a curated high-quality subset (DRL). It supports optional instrument conditioning to customize transcription output for specific instrumentations. Model weights, inference code, and the accompanying research paper are publicly released for researchers.
New
Audio
Overview
An open-source multi-instrument audio-to-MIDI transcription model that converts full song mixes into separate MIDI tracks for each instrument (voice, drums, bass, keys, and others), representing music transcription as a token-based language modeling task.
About Mirelo AI
Mirelo AI builds foundation models that generate synchronized sound effects and music for videos. Users upload footage and the AI analyzes scenes, actions, and motion to produce custom audio that aligns perfectly to visual cues in seconds, accessible via Mirelo Studio and a developer API.
Industry:
Artificial Intelligence
Company Size:
5
Location:
Tubingen, Baden-Wurttemberg, DE
Website:
mirelo.ai
Last updated: July 10, 2026
