Tuesday, July 7, 2009

Procedure and ruby script to remove duplicated tracks from iPod in Last.fm client before scrobbling

Since I bought an iPod Touch, it became my main Last.fm scrobbler device. However, I am facing a recurrent problem, every track that I listen in my iPod is scrobbled between 2 and 20 times with the same timestamp! The picture bellow shows the result:

I am using the Last.fm official client with the iTunes plug-in. I have searched a solution in several sites (e.g, here), lots of people are facing the same problem, but I did not find any solution yet.

To make the problem even worse, lately the remove button in the Last.fm tracks page don’t remove a single occurrence of a track anymore, instead it removes all occurrences in a given timestamp. So, once a track is scrobbled multiple times, I can not remove the duplicates anymore, because once I remove one, it removes all the entries.

So, while I don’t find a good solution, I created a manual procedure that I am using to solve the problem:

  1. Turn off the Internet connection;
  2. Ask the last.fm client to scrobble the tracks;
  3. Since it cannot scrobble the tracks because the lack of connection, the tracks (with the multiple occurrences) are kept in the local history (C:\users\windows_user\Local Settings\Application Data\Last.fm\Client\lastfm_user_submissions.xml);
  4. Close the Last.fm client;
  5. Run the ruby script bellow that I have written that excludes the duplicates tracks from the local history xml file;
    • ruby scrobbler_fix.rb "C:\users\windows_user\Local Settings\Application Data\Last.fm\Client\lastfm_user_submissions.xml"
  6. Connect to the internet;
  7. Open the last.fm client and scrobble the history;
  8. Check the tracks page to see if everything was ok.

As usual, I kept the source code in my utility scripts repository at GitHub, but the code is listed bellow as well. I hope I find a better solution soon, because this problem even with this procedure sucks!

# Script to remove duplicate items from last.fm submission xml file
# author: Douglas Fernando da Silva

require "rexml/document"
include REXML

history = nil
File.open(ARGV[0]) do |file|
history = Document.new(file)
tracks_to_remove = []

last_timestamp = nil
history.elements.each("submissions/item") do |element|
ts = element.elements["timestamp"].text
if ts == last_timestamp
puts element.elements["track"].text
last_timestamp = ts

for i in 0..tracks_to_remove.length-1 do
element_to_rm = tracks_to_remove[i]
element = history.elements["submissions/item"].parent.delete(element_to_rm)


formatter = REXML::Formatters::Default.new
File.open(ARGV[0], "w") do |result|
formatter.write(history, result)