AI Music Composer: Generating Original Music using Deep Learning

Objective: The objective of this project is to create an AI Music Composer that can generate original music compositions. The AI will be trained using deep learning techniques, specifically recurrent neural networks (RNNs) and long short-term memory (LSTM) networks. The project will involve collecting a dataset of MIDI files, preprocessing the data, training the AI model, and developing a user-friendly interface to showcase the generated music. By completing this project, you will gain a strong understanding of deep learning for music generation, data preprocessing techniques, and user interface design.

Learning Outcomes:

  1. Understand the fundamentals of deep learning for music generation.
  2. Gain hands-on experience in preprocessing MIDI data for model training.
  3. Learn how to implement and train a recurrent neural network (RNN) with LSTM layers.
  4. Develop skills in evaluating and fine-tuning a music generation model.
  5. Acquire knowledge in building a user interface to showcase the generated music.

Steps and Tasks:

  1. Collecting MIDI Dataset:

    • Download a diverse collection of MIDI files to serve as the training dataset for your AI Music Composer. You can find MIDI files on websites like MIDI World and FreeMIDI.
    • Save the MIDI files in a dedicated folder on your local machine for further processing.
  2. Preprocessing MIDI Data:

    • Install the mido library using pip to handle MIDI file processing.
    • Load the MIDI files from the dataset folder and extract the note sequences.
    • Perform data cleaning and transformation, including normalizing the note values and removing unnecessary metadata.
    • Split the processed data into input sequences (previous notes) and target sequences (next note to predict).
    • Convert the processed data into a format suitable for training the AI model (e.g., numpy arrays).
    • Split the data into training and validation sets.
  3. Building the AI Model:

    • Import the necessary deep learning libraries, such as TensorFlow or Keras.
    • Set up the architecture for your AI Music Composer using a recurrent neural network (RNN) with LSTM layers.
    • Define the model’s input shape and the number of LSTM units or layers.
    • Add a dense layer for output, and consider using techniques like dropout to improve the model’s generalization.
    • Compile the model, specifying the appropriate loss function and optimizer for a generative task.
    • Print a summary of the model’s architecture to get an overview of its structure.
  4. Training the AI Model:

    • Fit the model to the preprocessed MIDI data, using a batch size and number of epochs that balance computational resources and training performance.
    • Monitor the model’s training progress, including the loss function value, to ensure it is converging.
    • Visualize the training process using libraries like matplotlib to gain insights into the model’s performance.
    • Experiment with different hyperparameters and model architectures to improve the quality of the generated music.
  5. Generating Music with the Trained Model:

    • Develop a function that can generate music using your trained AI model.
    • Initialize the generation process with a random starting sequence of notes.
    • Use the trained model to predict the next note in the sequence.
    • Incorporate randomness or temperature parameter to add variation to the generated music.
    • Generate a longer sequence of notes to create a complete musical composition.
    • Convert the generated note sequence back into a MIDI format for playback and further analysis.
  6. Creating a User Interface:

    • Install the necessary libraries, such as Flask or Streamlit, for building a user interface.
    • Design a user-friendly interface that allows users to interact with your AI Music Composer.
    • Implement a feature that lets users initiate the music generation process.
    • Enable the interface to play back the generated music for users to listen to.
    • Add any additional features or visualizations that enhance the user experience.


  • You can evaluate the success of your AI Music Composer based on the quality and originality of the generated music. Solicit feedback from users to gauge their perception of the generated compositions.
  • Additionally, you can assess the model’s performance during training by monitoring the loss function value. A decreasing loss function indicates that the model is learning and generalizing from the data.

Resources and Learning Materials:

  1. DeepBach:

    • DeepBach is a deep learning project that focuses on generating chorales in the style of Johann Sebastian Bach. The project’s website provides a detailed explanation of the underlying techniques and offers a live demo for users to experience the generated music.
    • Website: DeepBach
  2. Magenta:

    • Magenta is a research project by Google that explores the role of machine learning in the creative process, including music generation. The project offers a wide range of tools, models, and datasets for musicians and developers interested in AI music composition.
    • Website: Magenta
  3. Music Generation with Magenta:

    • This book, written by the Magenta team, provides a comprehensive guide to using Magenta’s tools and models for music generation. It covers various aspects, including data preprocessing, model training, and performance evaluation.
    • Book: Music Generation with Magenta
  4. Deep Learning for Music:

    • This online course from the National University of Singapore explores the application of deep learning techniques in music analysis and generation. It covers a wide range of topics, including recurrent neural networks, music language modeling, and generative adversarial networks (GANs).
    • Course: Deep Learning for Music
  5. How to Generate Music using a LSTM Neural Network in Python:

    • This tutorial by Sigurður Skúli provides a step-by-step explanation of building an LSTM-based music generator in Python. It offers a beginner-friendly approach to the topic and includes code snippets for each stage of the process.
    • Tutorial: How to Generate Music using a LSTM Neural Network in Python

Need a little extra help?

Here are some code snippets to help you get started on your AI Music Composer project:

1. Installing the Required Libraries:

!pip install mido
!pip install tensorflow

2. Preprocessing MIDI Data:

import os
import numpy as np
from mido import MidiFile

# Define the path to your MIDI dataset folder
dataset_folder = 'path/to/your/dataset/folder'

# List to store the processed note sequences
note_sequences = []

# Iterate through each MIDI file in the dataset folder
for file in os.listdir(dataset_folder):
    # Load the MIDI file
    midi = MidiFile(os.path.join(dataset_folder, file))
    # Extract note sequences from the MIDI file
    notes = []
    for msg in midi:
        if msg.type == 'note_on':
    # Perform data cleaning and transformation as needed
    # ...
    # Append the processed note sequences to the main list

# Convert the note sequences into a format suitable for training
# ...

# Split the data into input sequences and target sequences
# ...

# Convert the processed data into numpy arrays
input_data = np.array(input_sequences)
target_data = np.array(target_sequences)

# Split the data into training and validation sets
split_index = int(0.8 * len(input_data))
train_input, val_input = input_data[:split_index], input_data[split_index:]
train_target, val_target = target_data[:split_index], target_data[split_index:]

3. Building the LSTM Model:

import tensorflow as tf
from tensorflow.keras.layers import LSTM, Dense
from tensorflow.keras import Sequential

# Define the model architecture
model = Sequential([
    LSTM(256, input_shape=(sequence_length, 1), return_sequences=True),
    Dense(num_unique_notes, activation='softmax')

# Compile the model
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')

# Print the model summary

4. Training the Model:

# Define the number of epochs and batch size
num_epochs = 50
batch_size = 64

# Fit the model to the training data, train_target, 
          validation_data=(val_input, val_target),

5. Generating Music with the Trained Model:

# Generate music using the trained model
def generate_music(model, starting_sequence, num_steps=100):
    generated_sequence = starting_sequence.copy()
    for _ in range(num_steps):
        # Reshape the input sequence to match the model's input shape
        input_sequence = np.reshape(generated_sequence, (1, len(generated_sequence), 1))
        # Use the trained model to predict the next note
        predicted_note = np.argmax(model.predict(input_sequence))
        # Add the predicted note to the generated sequence
        # Remove the first note to keep the sequence length consistent
        generated_sequence = generated_sequence[1:]
    return generated_sequence

# Define a random starting sequence
starting_sequence = np.random.randint(low=0, high=num_unique_notes, size=sequence_length)

# Generate music using the trained model
generated_music = generate_music(model, starting_sequence, num_steps=100)

# Convert the generated music back to MIDI format for playback
# ...

These snippets should provide you with a solid starting point for your AI Music Composer project. Remember to experiment with different model architectures, hyperparameters, and data preprocessing techniques to achieve the best results. Enjoy the journey of creating your own AI music composer!

@joy.b has been assigned as the mentor. View code along.