Trainspotting in Code

If you're anything like me, at some point you've been listening to a DJ set and suddenly overwhelmed with the thought, "what exactly is this track?!" Maybe you've even held your phone up to your speaker and tried to Shazam it. As a DJ, this happens to me constantly. Listening to other DJ's set is almost exclusively how I discover new music. And I got tired of that cumbersome workflow. So I decided to say goodbye to holding my phone up to things to Shazam and decided to solve this with code.

Identifying Tracks in a DJ Set with Ruby

The basic process is simple.

Download the set.
Split the set into chunks.
Submit each chunk to ACRCloud.
Lookup the results.

The lookup service we'll use is ACRCloud, which provides audio fingerprinting and music identification. It's a similar technology as Shazam, but they provide an API.

Download the Set

First, some setup and structure for our SetIdentifier class.

require 'fileutils'
require 'audioinfo'

class SetIdentifier
  attr_reader :sample_rate, :sample_length, :set_url

  def initialize(set_url, sample_rate = 90, sample_length = 10)
    @sample_rate = sample_rate
    @sample_length = sample_length
    @set_url

    create_tmp_set_dir
  end


  private

  def create_tmp_set_dir
    FileUtils.mkdir_p(tmp_dir)
  end

  def tmp_dir
    @tmp_dir = set_url.split("/").last.gsub(".", "_")
  end
end

This class takes in a set_url, generally a Soundcloud or Youtube link, and will create a temporary directory to store the assets related to our set.

With that we can build the first step, downloading the set.

def download(setlist_id)
 download_command = "yt-dlp -o '#{tmp_dir}/%(title)s.%(ext)s' --extract-audio --audio-format mp3 --audio-quality 0 '#{set_url}'"

 # Pipe the command to execute it.
 result = `#{download_command}`


 @set_filepath = Dir.glob("#{tmp_dir}/*.mp3").first
 @set_name = File.basename(file_path, ".mp3").split("/").last
end

We're using the awesome library yt-dlp, which is probably a violation of YouTube and Soundcloud's TOS but this is for education purposes only, of course, to download an mp3 version of the set. Ruby's backtick operator allows us to execute a command in the shell and return the result. We're using Dir.glob to find the downloaded file and set the @file_path and @set_name instance variables. With that, we're ready to continue to the next step.

Splitting the Set into Chunks

In theory, you want to take as many samples as you can from the larger set but in practice, you don't want to make 200 requests to identify a 2 hour set. So I decided to take 10 second samples every 90 seconds. I might miss a track here and there but it seemed like a good balance and I can always take more samples. Chunking is basically a two step process:

Identify the length of the track so you can figure out where the chunks start and stop, we'll use audioinfo for that.
Chunking the larger mp3 into the individual samples and we'll go back to piping to our command line using ffmpeg.

def chunk_audio
 AudioInfo.open(@set_filepath) do |info|
   total_chunks = info.length / sample_rate
 end

 chunk_complete_mutex = Mutex.new

 threads = []
 (0...total_chunks).each_slice((total_chunks / 4).ceil).to_a.each do |chunk_slice|
   threads << Thread.new(chunk_slice) do |chunks|
     chunks.each do |i|
       start_time = i * sample_rate
       `ffmpeg -i "#{@set_filepath}" -ss #{start_time} -t #{sample_length} -acodec copy "#{tmp_dir}/chunk#{i}.mp3" > /dev/null 2>&1`
     end
   end
 end

 threads.each(&:join)
end

We're using Mutex to make sure we don't write to the same file at the same time and we're using Thread to run the chunking process in parallel. Without theading it would look like:

def chunk_audio
  chunk_length_in_seconds = "#{sample_length}"
  total_length_in_seconds = AudioInfo.open(file_path).length
  chunk_count = (total_length_in_seconds / sample_rate).ceil

  chunk_count.times do |i|
    chunk_start = i * sample_rate
    output_chunk_path = "#{tmp_dir}/chunk#{i}.mp3"

    `ffmpeg -i "#{@set_filepath}" -ss #{chunk_start} -t #{sample_length} -acodec copy "#{output_chunk_path}"`
  end
end

Simpler. But you get the idea.

Submitting the Chunks to ACRCloud

Let's look at the code I actually used to do this and take it apart.

 def submit_chunks
   @scan_results = []
   @scan_results_mutex = Mutex.new
   chunks = Dir.glob("#{tmp_dir}/chunk*.mp3")

   # Divide the chunks into equal-sized groups
   chunk_groups = chunks.each_slice((chunks.size / 4.to_f).ceil).to_a

   threads = []
   chunk_groups.each do |chunk_group|
     threads << Thread.new(chunk_group) do |group|
       group.each do |chunk|
         process_chunk(chunk)
       end
     end
   end

   threads.each(&:join)
end

And then our process_chunk and submit_chunk methods:

def process_chunk(chunk)
 result = submit_chunk(chunk)
 json = JSON.parse(result)
 File.write(chunk.gsub(".mp3", ".json"), JSON.pretty_generate(json["data"]))
 @scan_results_mutex.synchronize do
   @scan_results << json["data"]
 end
end

def submit_chunk(chunk_file)
  filescan_curl = <<-SH
    curl -s -k --location --request POST 'https://api-v2.acrcloud.com/api/fs-containers/#{CONTAINER_ID}/files' \
      --header 'Accept: application/json' \
      --header 'Authorization: Bearer #{BEARER_TOKEN}' \
      --form 'file=@"#{chunk_file}"' \
      --form 'data_type="audio"' \
      --compressed
  SH
  @curls_logger.info(filescan_curl)
  result = `#{filescan_curl}`
end

Okay there are a few things going on here.

The submit_chunks method does the following:

It initializes two instance variables: @scan_results, an empty array, and @scan_results_mutex, a Mutex object (Mutex is again used for synchronizing access to some shared resource, which in this case, is the @scan_results array).

It then divides the chunks into equal-sized groups (as nearly as possible) using each_slice. It calculates the size of the groups using the total number of chunks divided by 4 and rounding up (.ceil).

It then creates a new thread for each group of chunks using Thread.new. Inside each thread, it processes each chunk in the group by calling the process_chunk method.

Finally, it waits for all threads to finish processing with threads.each(&:join) before moving on. This ensures all chunks are processed before the method exits.

The `process_chunk method`` is pretty straightforward:

It submits a chunk file to some remote service with the submit_chunk method and parses the response as JSON.

It then writes the JSON data to a file, using the same name as the original chunk but with a .json extension.

Inside a @scan_results_mutex.synchronize block, it appends the JSON data to the @scan_results array. The synchronize block ensures thread-safe access to the @scan_results array.

Lastly, the submit_chunk method:

Constructs a cURL command as a string (a POST request) and executes it returning the results.

The reason I'm writing the data to a file is for resuming purposes should the process fail.

With all the chunks submitted and our instance having the @scan_results array populated with the results, we're ready to move on to the final step, actually looking up the results. We submitted requests to identify the sample but ACRCloud is asynchronous and takes a second, so given the data in @scan_results, by the time all the chunks are submitted, we can look up the results using the lookup ID in that data.

def lookup_results
 @full_results = []
 @full_results_mutex = Mutex.new

 result_groups = scan_results.each_slice((scan_results.size / THREAD_SIZE).ceil).to_a

 threads = []
 result_groups.each do |result_group|
   threads << Thread.new(result_group) do |group|
     group.each do |scan_result|
       result = lookup_result(scan_result["id"])

       parsed_result = parse_result(result.merge(chunk: scan_result["name"].match(/chunk(\d+)/)&.captures&.first.to_i))

       @full_results_mutex.synchronize do
         @full_results << parsed_result
       end
     end
   end
 end

 # Wait for all threads to finish
 threads.each(&:join)
 @full_results = @full_results.select { |r| r["acrid"] }.sort_by { |r| r["order"] }

  File.write("#{tmp_dir}/#{results_filename}", JSON.pretty_generate(@full_results.sort_by { |r| r["order"] }))
end

So this method is again threaded and it is iterating through each scan result and submitting the lookup request through the lookup_result method, shown below.

def lookup_result(id)
 result_curl = <<-SH
   curl -k -s 'https://eu-api-v2.acrcloud.com/api/fs-containers/#{CONTAINER_ID}/files/#{id}' \
     -H 'Accept: application/json, text/plain, */*' \
     -H 'Authorization: Bearer #{BEARER_TOKEN}' \
     --compressed
 SH

 result = `#{result_curl}`
 match_data = JSON.parse(result)
end

Another cURL request to the API submitting the ID of the lookup we got from the submit chunks step.

With the JSON we can look at parsing it.

def parse_result(result)
 if track_data(result)
   chunk = result[:chunk]
   track_data = track_data(result)
   release_date = track_data["release_date"]
   score = track_data["score"]
   title = track_data["title"]
   label = track_data["label"]
   acrid = track_data["acrid"]
   album = track_data["album"]
   artists = track_data["artists"].map { |a| a["name"] }
   external_ids = track_data["external_ids"]

   if track_data["external_metadata"].present? && track_data["external_metadata"]["spotify"]
     spotify_track_id = track_data["external_metadata"]["spotify"]["track"]["id"]
     artists = track_data["external_metadata"]["spotify"]["artists"].map { |a| a["name"] }
   end

 end
 track = {chunk:, release_date:, score:, title:, label:, acrid:, album:, artists:, external_ids:, order: chunk}
 track[:spotify_track_id] = spotify_track_id if spotify_track_id
 track[:metadata] = track_data
 track.stringify_keys!
end

def track_data(result)
 result && result.is_a?(Hash) && result["data"] && result["data"][0] &&
   result["data"][0]["results"] &&
   result["data"][0]["results"]["music"][0] &&
   result["data"][0]["results"]["music"][0]["result"]
end

That returns the data I cared about, but ACRCloud actually provides so much more. I'm just not using it. I'm also using the Spotify API to get the track data from the Spotify track ID. I'm not going to show that code here but it's pretty straightforward.

With that, we're done. We have a list of tracks identified in the set. The last part of lookup_results is writing the ID'd tracks to a JSON file.

The very last step is putting it all together.

def run
  download
  chunk_audio
  submit_chunks
  lookup_results
end

The full class would look something like this:

require 'fileutils'
require 'audioinfo'

class SetIdentifier
  attr_reader :sample_rate, :sample_length, :set_url

  def initialize(set_url, sample_rate = 90, sample_length = 10)
    @sample_rate = sample_rate
    @sample_length = sample_length
    @set_url

    create_tmp_set_dir
  end

def run
  download
  chunk_audio
  submit_chunks
  lookup_results
end

def download(setlist_id)
 download_command = "yt-dlp -o '#{tmp_dir}/%(title)s.%(ext)s' --extract-audio --audio-format mp3 --audio-quality 0 '#{set_url}'"

 # Pipe the command to execute it.
 result = `#{download_command}`


 @set_filepath = Dir.glob("#{tmp_dir}/*.mp3").first
 @set_name = File.basename(file_path, ".mp3").split("/").last
end

def chunk_audio
 AudioInfo.open(@set_filepath) do |info|
   total_chunks = info.length / sample_rate
 end

 chunk_complete_mutex = Mutex.new

 threads = []
 (0...total_chunks).each_slice((total_chunks / 4).ceil).to_a.each do |chunk_slice|
   threads << Thread.new(chunk_slice) do |chunks|
     chunks.each do |i|
       start_time = i * sample_rate
       `ffmpeg -i "#{@set_filepath}" -ss #{start_time} -t #{sample_length} -acodec copy "#{tmp_dir}/chunk#{i}.mp3" > /dev/null 2>&1`
     end
   end
 end

 threads.each(&:join)
end

 def submit_chunks
   @scan_results = []
   @scan_results_mutex = Mutex.new
   chunks = Dir.glob("#{tmp_dir}/chunk*.mp3")

   # Divide the chunks into equal-sized groups
   chunk_groups = chunks.each_slice((chunks.size / 4.to_f).ceil).to_a

   threads = []
   chunk_groups.each do |chunk_group|
     threads << Thread.new(chunk_group) do |group|
       group.each do |chunk|
         process_chunk(chunk)
       end
     end
   end

   threads.each(&:join)
end

def process_chunk(chunk)
 result = submit_chunk(chunk)
 json = JSON.parse(result)
 File.write(chunk.gsub(".mp3", ".json"), JSON.pretty_generate(json["data"]))
 @scan_results_mutex.synchronize do
   @scan_results << json["data"]
 end
end

def submit_chunk(chunk_file)
  filescan_curl = <<-SH
    curl -s -k --location --request POST 'https://api-v2.acrcloud.com/api/fs-containers/#{CONTAINER_ID}/files' \
      --header 'Accept: application/json' \
      --header 'Authorization: Bearer #{BEARER_TOKEN}' \
      --form 'file=@"#{chunk_file}"' \
      --form 'data_type="audio"' \
      --compressed
  SH
  @curls_logger.info(filescan_curl)
  result = `#{filescan_curl}`
end

def lookup_results
 @full_results = []
 @full_results_mutex = Mutex.new

 result_groups = scan_results.each_slice((scan_results.size / THREAD_SIZE).ceil).to_a

 threads = []
 result_groups.each do |result_group|
   threads << Thread.new(result_group) do |group|
     group.each do |scan_result|
       result = lookup_result(scan_result["id"])

       parsed_result = parse_result(result.merge(chunk: scan_result["name"].match(/chunk(\d+)/)&.captures&.first.to_i))

       @full_results_mutex.synchronize do
         @full_results << parsed_result
       end
     end
   end
 end

 # Wait for all threads to finish
 threads.each(&:join)
 @full_results = @full_results.select { |r| r["acrid"] }.sort_by { |r| r["order"] }

  File.write("#{tmp_dir}/#{results_filename}", JSON.pretty_generate(@full_results.sort_by { |r| r["order"] }))
end

def parse_result(result)
 if track_data(result)
   chunk = result[:chunk]
   track_data = track_data(result)
   release_date = track_data["release_date"]
   score = track_data["score"]
   title = track_data["title"]
   label = track_data["label"]
   acrid = track_data["acrid"]
   album = track_data["album"]
   artists = track_data["artists"].map { |a| a["name"] }
   external_ids = track_data["external_ids"]

   if track_data["external_metadata"].present? && track_data["external_metadata"]["spotify"]
     spotify_track_id = track_data["external_metadata"]["spotify"]["track"]["id"]
     artists = track_data["external_metadata"]["spotify"]["artists"].map { |a| a["name"] }
   end

 end
 track = {chunk:, release_date:, score:, title:, label:, acrid:, album:, artists:, external_ids:, order: chunk}
 track[:spotify_track_id] = spotify_track_id if spotify_track_id
 track[:metadata] = track_data
 track.stringify_keys!
end

def track_data(result)
 result && result.is_a?(Hash) && result["data"] && result["data"][0] &&
   result["data"][0]["results"] &&
   result["data"][0]["results"]["music"][0] &&
   result["data"][0]["results"]["music"][0]["result"]
end


  private

  def create_tmp_set_dir
    FileUtils.mkdir_p(tmp_dir)
  end

  def tmp_dir
    @tmp_dir = set_url.split("/").last.gsub(".", "_")
  end
end

Identifying Tracks in a DJ Set with Ruby

Comments

More from this blog

Design Previews for Ruby on Rails

Turbo View Transitions in Rails

Turbo Sortable Paginated Tables

Turbo Frame Search Filters

Turbo Frame Slide Over

Trainspotting in Code

Identifying Tracks in a DJ Set with Ruby

Download the Set

Splitting the Set into Chunks

Submitting the Chunks to ACRCloud

Command Palette

Comments

More from this blog

Trainspotting in Code

Identifying Tracks in a DJ Set with Ruby

Download the Set

Splitting the Set into Chunks

Submitting the Chunks to ACRCloud