Procedure and ruby script to remove duplicated tracks from iPod in Last.fm client before scrobbling

Since I bought an iPod Touch, it became my main Last.fm scrobbler device. However, I am facing a recurrent problem, every track that I listen in my iPod is scrobbled between 2 and 20 times with the same timestamp! The picture bellow shows the result:

I am using the Last.fm official client with the iTunes plug-in. I have searched a solution in several sites (e.g, here), lots of people are facing the same problem, but I did not find any solution yet.

To make the problem even worse, lately the remove button in the Last.fm tracks page don’t remove a single occurrence of a track anymore, instead it removes all occurrences in a given timestamp. So, once a track is scrobbled multiple times, I can not remove the duplicates anymore, because once I remove one, it removes all the entries.

So, while I don’t find a good solution, I created a manual procedure that I am using to solve the problem:

1. Turn off the Internet connection;
2. Ask the last.fm client to scrobble the tracks;
3. Since it cannot scrobble the tracks because the lack of connection, the tracks (with the multiple occurrences) are kept in the local history (C:\users\windows_user\Local Settings\Application Data\Last.fm\Client\lastfm_user_submissions.xml);
4. Close the Last.fm client;
5. Run the ruby script bellow that I have written that excludes the duplicates tracks from the local history xml file;
• ruby scrobbler_fix.rb "C:\users\windows_user\Local Settings\Application Data\Last.fm\Client\lastfm_user_submissions.xml"
6. Connect to the internet;
7. Open the last.fm client and scrobble the history;
8. Check the tracks page to see if everything was ok.

As usual, I kept the source code in my utility scripts repository at GitHub, but the code is listed bellow as well. I hope I find a better solution soon, because this problem even with this procedure sucks!

# Script to remove duplicate items from last.fm submission xml file
# author: Douglas Fernando da Silva 

require "rexml/document"
include REXML

history = nil
File.open(ARGV[0]) do |file|
 history = Document.new(file)
 tracks_to_remove = []

 last_timestamp = nil
 history.elements.each("submissions/item") do |element|
   ts = element.elements["timestamp"].text
   if ts == last_timestamp
     puts element.elements["track"].text
     tracks_to_remove.push(element);
   else
     last_timestamp = ts
   end
 end

 for i in 0..tracks_to_remove.length-1 do
   element_to_rm = tracks_to_remove[i]
   element = history.elements["submissions/item"].parent.delete(element_to_rm)
 end

end

formatter = REXML::Formatters::Default.new
File.open(ARGV[0], "w") do |result|
 formatter.write(history, result)
end

Douglas

Ruby script for setting lyrics in iTunes Library tracks

One of the cool features of the iPod Touch and iPhone is that is possible to see the lyrics of a track while it’s playing. The picture bellow shows that.

 

But the problem is that you have to insert the lyrics manually using the iTunes, for each track you have to use the “Get Info” option and then set the track, just like the pictures bellow.

 

That’s a problem, because if you have hundreds of tracks, it will take lots of clicks and time. In order to solve that problem, I created a Ruby script that tries to set lyrics for all the tracks in my iTunes Library.

The script interacts with the iTunes COM components to get all the tracks in my Library, after that, it uses the LyricWiki web service to get the lyrics for each track and set to it.

The script works only on Windows and it’s available in my github repository:

git://github.com/dougfernando/itunes-helper.git

http://github.com/dougfernando/itunes-helper

Now, every time I get a new album, all I have to do is to run the script again,  like the following images :

The script code is also bellow:

require 'win32ole'
require 'soap/wsdlDriver' 
require 'iconv'

class ITunesHelper
  def initialize(lyrics_service)
    @itunes = WIN32OLE.new('iTunes.Application')
    @tracks = @itunes.LibraryPlaylist.Tracks
    @lyrics_service = lyrics_service
  end

  def show_all_tracks
    @tracks.each do |track|
      print track.Name() + "\n"
    end
  end

  def set_all_lyrics
    puts "Number of tracks to be analyzed: #{@tracks.count}"
    index = 1
    @tracks.each do |track|
      if self.is_lyrics_empty(track.lyrics)
        lyrics = @lyrics_service.get_lyrics(track.Artist(), track.Name())
        
        if self.is_lyrics_empty(lyrics)
          self.print_track(index, @tracks.count, track, "lyrics WAS NOT found")
        else
          track.lyrics = @lyrics_service.get_lyrics(track.Artist(), track.Name())
          self.print_track(index, @tracks.count, track, "lyrics WAS found and set")
        end
      else
        self.print_track(index, @tracks.count, track, "already had lyrics")
      end
      index = index + 1
    end
  end

  def is_lyrics_empty(lyrics)
    lyrics.empty? or lyrics == "Not found"
  end

  def print_track(index, total, track, result) 
    puts "Track #{index}/#{total}: #{track.Artist()} - #{track.Name()}: #{result}"
  end
end

class LyricsServiceProxy
  def initialize
    @driver = SOAP::WSDLDriverFactory.new("http://lyricwiki.org/server.php?wsdl").create_rpc_driver
    puts "Lyrics web service initialized"
  end

  def get_lyrics(artist, track_name)
    Iconv.iconv("LATIN1", "UTF-8", @driver.getSong(artist, track_name).lyrics).to_s
  end
end

itunesH = ITunesHelper.new(LyricsServiceProxy.new)
itunesH.set_all_lyrics

Douglas

Powershell script for converting Word documents to PDF format

In my job, we only send documents to clients in pdf formats. Sometimes we have to send several documents at once and it takes a good time to open each one and save it as a pdf, it’s not a very pleasant task.

In order to minimize this problem, I’ve created a simple powershell script that opens a Word document and saves it as pdf without any user interaction. With such script, I can convert a lot of files at once, just like this:

ls . *.doc* –Recurse | %{ ~\scripts\doc2ps1.ps1 $_.fullname }

With this command, I list recursively each doc/docx file in a directory and save each one as PDF.

By default, Office 2007 apps are not able to save files as PDF, so you have to install the add-in from Microsoft first to use the script. The add-in is available here.

Bellow is the script, it’s also available in my PowerShell scripts repository at GitHub (http://github.com/dougfernando/utility-scripts/tree/master) as doc2pdf.ps1.

param (
 [string]$source = $(Throw "You have to specify a source path."))

$extensionSize = 3
if ($source.EndsWith("docx")) {
$extensionSize = 4
}

$destiny = $source.Substring(0, $source.Length - $extensionSize) + "pdf"
$saveaspath = [ref] $destiny
$formatPDF = [ref] 17

$word = new-object -ComObject "word.application"
$doc = $word.documents.open($source)
$doc.SaveAs($saveaspath, $formatPDF)
$doc.Close()

echo "Converted file: $source"

ps winword | kill

An important note, the script closes every Word instance in the end, so save any work previous using it.

ASPY Player Beta 4 released - S60 Player with Last.fm scrobbling built-in. Open signed online now possible!

I've just released the beta 4 version of the ASPY Player. Currently I have only few hours (~3h) a week to work on it, so that's the reason for few enhancements in this release, however I think some of them are very important.

The changes in this release are:

• Possible to sign the sis file through open signed online;
• Blinking "now playing" screen bug fixed;
• Better handled id3v2 only files (id3v2 not supported yet);
• Track number info being used in the app;
• Better tracks ordering handling;
• Better handling of non-US ASCII characters in directories names;
• Application code design improved;

As usual, all kinds of feedbacks are welcome. They have been essential to improve the player quality, thank you guys!

ASPY Player Beta 3 released - S60 Player with Last.fm scrobbling built-in

After coming back to work from vacations, it has been hard to find time to work on the player. Currently I'm leading 5 projects in my work! However I got some time (most of it while traveling) and released the beta 3 of my player. Bellow is a list of the major changes:

• Non-US ASCII bug fixed, now files path and id3 content can have é, è, ã, etc..
• Now it's possible to view the current music history, ie, tracks not sent to last.fm yet
• After selecting an artist, now it's possible to select all tracks of that artist as before or drill-down by artist album, just like the native player
• AudioScrobbler protocol hard error handled correctly
• Application icon added
• Correct version number set (major/minor)
• Now it's possible to play tracks in random mode
• Several bugs fixed

As usual, the source are here and here there's a step-by-step tutorial on how to run it without signing using the python shell (thanks Brixton).

The python snake image I found using Google Images so I don't know who created it. Any information about it, please, tell me so that I can ask the creator the usage authorization.

I also have to find a more beautiful image for the "Now playing screen", preferentially a dark one to help with the energy consumption. I have also to make the now playing screen stop blinking too. I'll try to stabilize the current feature set (fix bugs and refine the design) then start adding more features.

Ahh, BTW, because of my efforts to build this player, the guys from Build Last.fm gave me 12 months of subscription!! Great incentive! Thank you guys again!

I'd like to thank the guys who are using the player and sending me feedbacks.

Douglas

Some new pics from this version:

Second release of my Last.fm S60 Client - aspyplayer

Today I released the second beta version of my last.fm client for S60 devices (although only tested in my N95-1). I rewrote the whole UI, now it's a little less ugly and it kinda looks like the native player. Now it's also possible to play/stop/forward/back/volume up/volume down using the keyboard in the "Now playing" screen like the native player.

The major bug in the moment is that it cannot handle well files with path or title that contain special characters, such as 'é', 'è', 'ç', 'ã'. I'm Brazilian, so it's a problem for me because in portuguese, words with that kind of characters are so common. However, for me it's strange how the pyS60 handles unicode, I didn't get it yet. It pissed me off and I gave up for the moment, so the bug is there. I have to correct some spelling errors too and now, I don't know why, the sis only works on C:

For those who can't sign the sis files and want to try the player, here can be found a good tutorial explaining how to sign sis files. Besides that way, it's possible to run the application using the sources. To do so, get the aspylayer.py and now_playing_bg.jpg files from the repository and copy them to c:\python or e:\python. After that, run it using the Python Shell like in the image bellow.

Thank you guys for the feedbacks (I'm waiting for more :)). I think soon a more stable/usable version will be released...

Some screeenshots for this release:

My Last.fm S60 Client - aspyplayer

In the last few days I've been working on a S60 music player with Last.fm scrobbling capabilities because most of time I listen to musics in my N95. I'd the idea to build this application months ago, but only now, in my last week of vacation I had time and today I released the first beta version (ugly UI and barely working) at: http://code.google.com/p/aspyplayer/

Some screenshots for this first beta release are available at: http://code.google.com/p/aspyplayer/wiki/screenshots

I'm writing the application in python (pyS60), which for me, currently is the best development platform for Symbian devices. However, it has a lot of limitations compared to C++ on S60 devices.

The application architecture kinda follows DDD, with a basic MVP for the presentation layer. The last time I did some development in python was about 5 years ago, so the code is not that good yet. The sources can be seen at: http://code.google.com/p/aspyplayer/source/browse. Code reviews are welcome.

Now I'll work on the UI, which currently is very ugly and I have to refactor some code to decouple the file system libraries from the model putting it in the infrastructure layer. Besides that, I have to create more unit tests.

Any kind of feedback is welcome.

BTW, yesterday I found an amazing last.fm client for iPhone/iPod Touch, it's called MobileScrobbler. It's too bad that the S60 interfaces sucks when compared to iPhone interface which is fantastic.