Adventures With AirTunes
The main addition to Signal 1.1 was the ability to control AirTunes speakers, something I was very happy to finally be able to offer as it was by far the most requested feature over the life of the application. Of course, with this implemented one of the new most requested features is “show which speakers are actually active!”. This is something I’d very much like to see added myself, and the reason it’s not supported stems from the same limitation of the iTunes programming interface that caused AirTunes control to be delayed for so long.
When the AirPort Express first came out, I was incredibly excited. Here was a way to put together a whole-house audio system that integrated seamlessly with iTunes, all that was missing was a slick way to control it via Wi-Fi. Of course, I soon discovered that there was in fact no way to control AirTunes through the iTunes API. Ok, no problem, I’ll just file a feature request (rdar://problem/3821346 for any Apple folks). I’m sure they’ll add it soon.
That was nearly four years ago. It’s still not there.
Meanwhile, the requests for AirTunes support kept pouring in, sometimes on a daily basis. Finally I had to relent. So with no support from Apple, and no clever back-door into iTunes available, how can we add support for this feature? Really, there’s only one way: Simulate the user’s input in the iTunes interface. In other words, send mouse clicks exactly at the positions where you would make them to select a speaker through iTunes. This is known as UI scripting and should make every developer who hears those words cringe.
At first glance UI scripting doesn’t seem like such a bad thing. Simulate a click on the speaker selection drop-down, how hard can it be? There’s just one problem. That drop-down? It moves. Try clicking around between different playlists and source types and you’ll see how its position changes. What’s more, in order to send those mouse clicks iTunes has to be the foreground window and there are all kinds of reasons why that might not be the case. Here are just a few of the things that Signal has to account for in order to make those simulated clicks work reliably:
- The user’s language
- The type of playlist selected
- The iTunes window being hidden
- The iTunes window being minimized
- The Mini Player view being selected
- The screen saver being active
And of course, my personal favorite:
- The user’s language not being English, with the radio tuner playlist selected, with iTunes in the Mini Player view, minimized, with the screen saver active, plus some random process that just happens to act up right then and slow everything down.
You have no idea how much cursing was involved in the creation of this feature.
The worst part is that despite all the efforts to make this work as smoothly as possible, it’s still based on a broken model. If Apple decides to update the iTunes interface and move the speaker selection drop-down to a new location it will break. If there is some odd mouse or application behavior that Signal doesn’t account for it will break. And of course, although Signal can send the mouse clicks it can’t get any information back about whether those clicks worked or what speakers are actually turned on.
Sure, Signal could start taking little screenshots of iTunes and try to figure this out from the image. “Well, that kind of looks like ‘Computer’ in Simplified Chinese, I guess we can check the box”. But then I would go insane and then there would be no further product development.
Applications will always have bugs, but a good application tries to be as reliable as possible so that it “just works”. When the limitations of an API prevent your application from doing that, it’s incredibly frustrating. All of which is to say to Apple: Please, please, for the love of shiny plastic, put AirTunes control in the iTunes API.
April 28th, 2008 at 3:57 pm
You have done a great job. I rarely have issues using my airtunes over signal. Love signal and couldn’t be happier with the program. My only suggestion would be to put it in the status bar and get it out of my dock and maybe stop if from popping up the dialog box on logon. But neither of these affect signals usability at all. Thanks so much!
April 28th, 2008 at 4:19 pm
Thanks Ryan, I appreciate it! For your feature request, the plan is to move Signal to a system preference pane so that it’s completely hidden unless called upon. This way it will always be available (unless turned off) without adding anything to the UI.
April 28th, 2008 at 4:25 pm
I enjoy reading what goes on behind the scenes to make the magic happen. It’s too bad apple doesn’t make it easier, but at least you were able to pull it off.
April 28th, 2008 at 4:30 pm
Matt,
This post along with some recent news/rumor got me thinking. You may have seen certain tech or Apple sites reporting that certain code was found in the latest iphone 2.0 beta firmware release that suggests Apple is working on some sort of “remote” or “control” app. While some are speculating that it will allow iTunes content to be streamed directly to the iPhone/Touch, I think it’s much more likely that it would be an app similar to Signal.
I wonder if Apple is intentionally withholding AirTunes control from the API. There is no way they are so oblivious to not realize how important AirTunes control would be. I’m guessing that this is one way for them to give their eventual “official” app a leg up over other 3rd party apps. I wouldn’t be surprised if after Apple releases their official app we finally see AirTunes control in the API.
Maybe I’m being too cynical. But it just seems like such an obvious function. Just like A2DP seems like such a no-brainer for the iphone but I’m guessing it will not be added until Apple releases its own stereo bluetooth headset.
Anyway, Signal is still amazing app and the AirTunes function works great, even as a work around. Reading what you had to go through to make it work makes me appreciate it even more. Keep up the good work. We all appreciate it.
April 28th, 2008 at 8:39 pm
Brad, Kevin: Thanks, glad you enjoyed the read! I’ll try to add some more “behind the scenes” posts as interesting things crop up during development.
As for the iControl rumor, it’s really impossible to speculate. I’d like to think that Apple hasn’t been holding back on adding useful remote control functions to the API just so that they can release their own competing product, but anything is possible. We’ll just have to see what happens.
April 29th, 2008 at 7:54 am
Hi Matt,
I must /sign Kevin’s post. I am really looking forward to see what Apple got in mind for such a control app – and I am sure you will do it better when the new firmware is released for 3rd party developers to sell their software as apps.
Btw – For some reason I feel religious about my apple stuff, and so I never hack it. I hear rumours that software can ONLY be obtained through iTunes app store when all this becomes available.
If this is the case – will current customers be able to download it for free anyway, or will Signal be released as 2.0 pulling anohter fee from customers?
April 29th, 2008 at 8:40 am
Stoffer: Absolutely, I don’t think there should be any need to hack a device to get an application working. This is why a native version of Signal has waited for the official SDK – so it can work right for everyone and anyone can use it.
Apple has announced that iPhone/iPod apps will be distributed through iTunes and the App Store exclusively, so they have complete control. The plan is to release the native version of Signal as a free upgrade to all existing users, but this does depend on what rules Apple sets for the App Store which have yet to be announced.
May 23rd, 2008 at 3:14 am
would it perhaps be easier to patch into RogueAmoeba’s product Airfoil
( http://rogueamoeba.com/airfoil/mac/ ) than trying to manipulate AirTunes via iTunes?
May 23rd, 2008 at 7:45 am
Airfoil support is something that’s being considered as an additional feature, but I think AirTunes control is a critical part of Signal and shouldn’t require purchasing another app in order to provide it. That’s why everything described in the post was gone through in order to get this in, and although it’s not perfect it does work pretty well. Thanks for the suggestion though!
May 24th, 2008 at 6:17 am
What an awesome web app. I’m pretty paranoid (and obviously uninformed) about running 3rd party apps on my iPhone. I was falsely under the impression that one had to unlock their iPhone to get cool functionality like this. Not the case!
A co-worker (who has in fact hacked an iPhone and installed all kinds of apps) showed me how easy Signal works…I bought w/o a trial version based on his demo. The install took less than 2 minutes.
Support for multiple speakers works great, and is why I’m all over this. Our home audio is zoned and all controlled by any one of 4 Macs in the house. All AirTunes, of course.
Thanks for a great solution.
May 24th, 2008 at 5:40 pm
Thanks Paul, glad to hear you like the app!
June 3rd, 2008 at 6:29 am
Matt, I love Signal. I have been looking for something like this ever since I adopted Itunes and Airtunes about 12 months ago.
I trialled it for one day, and just had to buy it immediately! It is the final piece in the puzzle. Now I can control everything, from anywhere.
I use a smartphone, and XP pc. Very non Apple background, yet it works brilliantly.
If I can put in my feature request, can you add party shuffle support? (Or is it there and I am missing something?) I never listen to music in the same order twice if I can help it.
June 3rd, 2008 at 9:49 pm
Thanks Andrew! It sounds like you’re looking for the ability to queue songs in the Party Shuffle playlist rather than the Signal Queue, is that right?
Although queuing into Party Shuffle is not available today, you can certainly start Party Shuffle playing through the media library interface and control it just like a standard playlist, moving through tracks with next / previous and jumping ahead to a particular track using the playlist view.
June 15th, 2008 at 1:30 am
I’d like to add my thanks and congratulations for a fantastic app. My multi-room setup plus Airtunes and Signal provides the perfect whole house audio solution for me. I do however like to dream so here is my wish list for the future;
Airfoil style functionality so we can access non iTunes content;
Stream video back to the iPhone from the Mac;
Stream audio directly to Airtunes from the iPhone;
Hope you are able to add some of these to the to do list.
thanks again. Wayne
June 16th, 2008 at 10:07 pm
Wayne: Thanks for the kind words, I’m very glad to hear that you’re enjoying the app!
Airfoil integration has been requested a few times and is on the list for consideration in a future release. Streaming to the iPhone is something that’s being looked at in the native app version, although there hasn’t seemed to be a whole lot of interest in this. Streaming directly from the iPhone would be cool, but is unfortunately not possible today. In addition, I think having the Wi-Fi radio transmitting constantly would drain your battery life in nothing flat, so even if this becomes possible in the future it may not be practical.
June 27th, 2008 at 2:35 pm
Matt, just seen this article – http://www.iclarified.com/entry/index.php?enid=1425 – Looks like some competition is coming …
June 27th, 2008 at 4:19 pm
Wayne: Michael made a similar observation in the group and I posted a quick response.