Step 2 is the hard part.

Matt Ginzton writes here.

Off-the-cuff Reaction to OS X Mountain Lion

| Comments

After reading about Apple’s new Mountain Lion, including coverage from Daring Fireball and Ars Technica, my immediate reaction is:

  • Address Book is being renamed Contacts? Great. I can’t tell you how many times I’ve typed Command-Space, C-O-N, trying to launch Address Book via Spotlight.
  • So long Growl? I already said “so long” to Growl months ago, since I don’t like the way old notifications pile up. Notification Center, at least in its iOS incarnation, is much less intrusive.
  • Messages (the app, with support for, er, messages via iMessage) from the desktop makes perfect sense.

But top to bottom, from today’s announcements, this feels much more like a collection of iOS apps appearing in OS X, not a revamp of the OS itself. Where are the system-level changes? OK, Notification Center is system-wide, and there’s more iCloud support including document save/load directly to iCloud, and there’s Gatekeeper (the ability to ban installation of unsigned apps). AirPlay mirroring will be useful, but it’s too bad the current AppleTV limits the output to 720p. Still, most of the changes (as documented today) are at the surface.

On the other hand, recent OS X releases (especially Snow Leopard, but also to a large degree Lion) were about laying new system infrastructure, so maybe this is perfectly appropriate. Also, compared to Windows, OS X distinguishes itself more and more by its useful collection of built-in apps, so improving and extending that collection is fair game. And finally, the name signifies it’s a new lion, not a whole new cat.

From which I conclude the meta-news here is that with a plan to update the OS annually and the ability to roll out these upgrades via the Mac App Store for $30, we should get used to regular incremental updates, not revolutions every blue moon. Which is a pretty sane way to develop quality software, really.

Space Limiting Your Time Machine Network Backups

| Comments

As part of the aforementioned office-quieting project, I wanted spinning disks out of the office, so I garbage collected 2 1TB drives from external enclosures that had served for Time Machine, and moved them into a NAS enclosure in the basement.

That solved the noise problem and gave me a bunch more network-attached storage, but turned off Time Machine; the next step was to re-enable TIme Machine but back up to the network.

A few years ago setting up Time Machine to back up over the network, to anything but the Time Capsule mini-NAS that Apple designed for it, took some minor rocket science (and in my experience caused no shortage of kernel panics on the client machines); now it’s more stable and easy to set up, especially if the network file services are provided by Apple’s own AFP server. I have some free space on another RAID array attached to a Mac Mini also in the basement, perfect for this sort of thing, and so all I had to do was mount that drive from the client machine, then go to the Time Machine prefpane and select it for backup. Time Machine creates itself a disk image and goes to town.

The one problem with this is that it creates a disk image with the same size as the underlying physical volume. It’s a sparse image, so it doesn’t immediately fill the whole volume, but it will grow to do so over time. That’s not good, since I want multiple Time Machine backups to be able to share that volume, and they’re not the only thing that lives there.

Googling for solutions to this, I found an article on how to pre-create the sparseimage with whatever size you want. I tried that, but when I enabled Time Machine, it ignored the hostname_macaddress.sparseimage directory and just created a new hostname.sparseimage directory next to it. (Which, IMHO, is a good thing, since keying the backup name from the MAC address is not going to work well with machines with multiple network interfaces, for example a laptop which is sometimes using Ethernet and sometimes using Wi-Fi.) Maybe that’s a holdover from a previous OS version; who knows. So then I tried precreating the image file as just hostname.sparseimage, but then when I enabled Time Machine to the same volume, it noticed the existing one, decided not to use it, and created “hostname 1.sparseimage” instead.

Then I stumbled on a simpler recipe:

  1. Enable Time Machine the normal way (mount a network volume, open System Preferences, go to Time Machine preferences, click Select Disk, and choose the network volume).
  2. Let Time Machine do its thing. It will create a sparse image there with the same size as the underlying volume, and do the initial backup, and (eventually) unmount the sparse image.
  3. Later, when Time Machine is not running, use hdiutil to resize the sparseimage to something smaller. I used “hdiutil resize -size 750g hostname.sparseimage” (down from its original 3TB size).

Boom. Seems to work fine. After having backed up a little over 400GB, Time Machine now displays the backup status with “Available: 385 GB of 3 TB”, so after backing up another 385GB, it’ll start pruning the backup set, instead of filling the volume and getting confused.

Observations on Mobile Platform Speed

| Comments

Over the past 5 years, I’ve used smartphones from Apple and Palm (powered by iOS and webOS), and played around with Android devices over the same timeframe though I’ve never owned one for daily use.

Still, I find the different design choices at the system level to be very different, and the results pretty much what you’d expect for each one:

  • webOS as Palm originally shipped it: apps are written in Javascript and run in an interpreter; the platform doesn’t swap out virtual memory. I never saw apps crash; it was pretty common to get out-of-memory errors (in the guise of a message that says you’re trying to do too many things at once and need to close some apps). Apps were pretty slow.
  • webOS as reinterpreted by the hacker/modder community: apps are still Javascript running in an interpreter, but now there’s a swapfile pretending there’s more memory. (I think Palm liked this patch enough they eventually folded it back into their official kernels, actually. And if swapping caused a hit to speed, it was compensated for by the fact that the hacker kernels also overclocked the CPU.) The result with this was a platform on which apps never crashed, and also never run out of memory, but they could be glacially slow. The poster child here was the Google Maps app, which often took so long just to launch — over a minute, no kidding — that it would apparently become confused about why it had launched in the first place, and just draw a white screen. (The bit about confusion is just my narrative, but it often would draw nothing but a white screen, and that seemed to correlate highly with the times it took forever to launch.) You can certainly make the argument that an app that takes a minute to launch to a broken state is no more helpful than an app that crashed; there’s no panacea here.
  • iOS: apps are written in Objective C and compiled to native code; there’s no swapfile I’m aware of. There are two results here: apps run fast, and they crash often. They crash because they segv from bad code, they crash because malloc returns null and they segv instead of checking for that, and they crash because the platform kills them for using too much memory.
  • Android: apps are written in Java and run in a JIT-compiled VM (which has gotten faster over time, notably in the 2.3 release); I don’t know if there’s a swapfile, but I suspect stock Android releases don’t have one and mods like CyanogenMod allow one. As I said above, I don’t have deep experience with Android devices, but my general perception is that on contemporary roughly equivalent hardware, it feels significantly faster than webOS and significantly slower than iOS. (I’m talking about user perception of speed here, which has to do both with how responsive the UI is and how fast real work gets done. iOS and webOS both pay more attention than Android to keeping the UI responsive, to the best of my knowledge.)

So. This evidence is relatively unsurprising and easy to explain — interpreted code that can have as much memory as it wants never crashes but takes forever to do anything useful; native code under tight memory management is much faster and much more crash prone — what’s interesting to me is that all 3 of these approaches were considered viable in the marketplace. And I don’t even know that the dog slowness of webOS is the reason it failed in the marketplace (and if HP had poured the right resources into it and actually shipped what they said they would in 2011, a better optimized webOS 3 running on 2011-class hardware might not have even felt slow). But even discounting webOS, the other two approaches are both provably market-viable.

Nothing Is as Easy as It Should Be: SSD Upgrades

| Comments

Santa, bless the guy, heard me getting annoyed at the sounds of hard drives spinning up and down in the night, and decided to help me out by bringing me SSD’s for my and Vanessa’s computers.

So, the week after Christmas, I replaced the spinning rust in my (2006) Mac Pro and her (2009) iMac with Crucial M4 solid state drives.

This was really easy in the Mac Pro; this machine is the easiest to work on I’ve ever owned or used, and that’s probably why a machine from 2006 is still ticking along so happily in 2012 — I’ve upgraded the RAM twice, the hard drives three times for bigger/better hard drives and twice more to supplement and now replace them with SSDs — I don’t really feel bottlenecked by 5-year- old CPUs.

Drive upgrades are not as easy in the iMac. First of all, there are no screws on the outside of the case; all the guts are behind the glass of the screen, which is held on by magnets, and you need to remove it with a suction cup. Seriously. (If you already know this, you probably don’t find it surprising, but when I first learned it, I was really surprised.) It’s actually not as hard as it sounds, but I did have to buy a suction cup. Then you have to carefully disconnect a bunch of miniature ribbon cables. I followed the walkthrough here because it has nice pictures, except I couldn’t actually follow it because the 2009 iMac I was working on, unlike the more recent models, doesn’t have a 2nd SATA port. Which means I had to replace the hard drive, instead of adding the SSD alongside it. That was actually the goal, since I didn’t want to hear drives spinning up and down, but it means I had to contend with the temperature sensor issue.

There seem to be 5 ways people deal with this:

  • ignore it and deal with loud fans
  • use aftermarket software to control the fan speed, ignoring drive temperature
  • buy the cable from the previous generation iMac, which had the temperature sensor outside the hard drive
  • short out the temperature sensor circuit, which is apparently temperature-sensor code for “it’s cool”
  • (only on newer iMacs with 2 SATA ports) leave the OEM factory hard drive attached, and add the SSD as a 2nd drive

I opted for shorting the pins, as I don’t want loud fans, don’t really trust a software solution, don’t want to pay and wait for the extra part which seems to be hard to find now anyway, and don’t have the newer model with the 2nd SATA port.

This just leaves the question of how to short the pins. Jumpers from old PC hard drives would probably do the trick, but I don’t have any lying around. I could cut the cable and just short the wires, but I wanted to leave the option to reverse this procedure if things didn’t work out.

I ended up making a jumper out of a short piece of one conductor from solid- core Category 6 network wire. Strip that, bend it in a tight U, jam it inside the connector at the end of the sensor cable, and it fits nicely. Seems to be working fine.

Nothing Is as Easy as It Should Be: NAS Upgrades

| Comments

After the SSD upgrade project, I now have a stack of unused 1TB hard drives. I also have an old NAS unit (ReadyNAS NV) which is currently hosting 4 320GB hard drives, is almost full, and while old is still compatible with the 1TB drives. Sounds like a good match — time to upgrade.

Reading about ReadyNAS’ vaunted X-RAID technology, a proprietary extension to RAID that allows resizing, this sounds easy — one by one, replace each drive with a larger one, and wait for the RAID to rebuild onto the new drive. When the drives are different sizes, the array will be limited to 4x the size of the smallest, but after upgrading all 4 to 1TB, I simply reboot and will have access to the full new size. And since the ReadyNAS supports hot swap and has the drives mounted in nice hot swappable sleds, I don’t even have to reboot except for that final resize (and I’m not really clear on why I have to then, but whatever).

Sounds easy… until I actually try swapping out the drives. The drives are in sleds, see, and the sleds have this latch, and the latch has this release button… and nothing happens when I press the release button. I try reading the manual, which talks about how easy it is to add drives but doesn’t go into any details on what to do if you have trouble removing one. I try reading “The definitive guide to the ReadyNAS NV+” (a slightly newer but very similar model) for tips; it talks about how disk replacement eases the stress of disk failure; my mind jumps to bad puns about stressing the ease while easing the stress, and thinking I’m glad I’m not dealing with the stress of a failed disk that I can’t remove.

Eventually I find a forum thread on the problem, with the solution. It’s 5.5 years old, and still kicking. Much like my ReadyNAS NV, I suppose.

Nest Learning Thermostat Review

| Comments

I bought two Nest Learning Thermostats for my house, which has two furnaces. Here’s how they’re working out for me.

Ordering

I almost bought them on the day they were announced back in November, but dithered, and the initial batch sold out; I got on the waiting list and waited. In early January they sent me an email allowing me to place an order (for up to 5 units); I ordered two; they arrived Friday and I’ve spent the weekend playing (er, working) with them.

One thing that happened between when I didn’t order them the first day and when my turn to order came around was Marco Arment’s post on Nest incompatibility with his existing wiring. Having read that, I semi-carefully checked my own wiring against Nest’s online compatibility wizard before ordering (which I might not have bothered doing if I hadn’t read Marco’s post, given that my furnaces and wiring are only 3 years old, so I assumed they’re modern). When I say “semi-carefully”, I mean I looked at my wires and verified them against Nest’s tool (which says I’m fine), and I considered Marco’s problem (2 wires, red and white, are enough for a basic thermostat to control heating, but not to power or recharge the thermostat without activating said heating) and made sure I had a 3rd wire and figured I wouldn’t have that problem.

In fact, when my thermostats arrived and I installed them, I also uttered a triumphant but premature tweet about having modern wiring. If only. If you’re reading carefully — more carefully than I had read Marco’s post — you’ll note I said I had 3 wires, but not which 3 wires. And, as it turns out, I had it wrong. No, I didn’t have a C (common) wire; my third wire was a G (fan) wire.

Unboxing and setup

The packaging is nice; the actual thermostat unit is nice; the base which you attach to the wall has a built-in level, and pressure-fit connectors for the wires instead of screw terminals. It’s obvious the device was carefully designed and built without taking shortcuts — which had better be true, given the price.

It was fairly easy to take my old thermostat off the wall, note how it was using the wires, reconnect the same wires (Rh, W and G, again) in the same capacity to the Nest base, attach the Nest base to the wall, and then press the Nest device onto the base. It didn’t take long to do this twice, in fact — once upstairs and once downstairs.

The next step is to connect the thermostat my Wi-Fi network — entering Wi-Fi passwords on small devices without keyboards is usually no fun, but entering text on the Nest’s rotating dial is surprisingly satisfying. Kind of like using a combination lock from high school again, but less confusing. As soon as it connected to the network, the Nest found itself a software update, invoking the download-and-reboot cycle that’s the bane of many modern smart devices. In fact, it seemed to reboot twice. The downstairs device survived this fine and next wanted me to join it to a cloud management account; the upstairs device asked me what time it was, which is a bad sign for something that’s supposed to be connected to the internet. It turns out it had forgotten how to connect to the network, so I had to enter my Wi-Fi password again.

After this setup phase — which took a little longer than I’d expected due to all that rebooting — the thermostats were active, ready to use manually, and, ostensibly, learning my habits so I don’t need to tell them what to do.

Use

These thermostats are easy on the eyes and easy to use; you just turn the dial to change the temperature, or push in on the dial to bring up the onscreen UI with easy-to-understand options for setting home/away, turning the whole thing off, or invoking manual scheduling settings I haven’t bothered playing with.

If you create a Nest account and install the smartphone app or use their website, you can monitor and change the temperature from anywhere, which is pretty darn space age if you ask me.

One minor glitch is that the just-approach-to-wake-screen feature seems to only sometimes work; the Nest is supposed to recognize when you approach and greet you by turning on the screen, but sometimes it doesn’t turn on until I actually touch it, sometimes it only turns in when I stick my hand within inches, and sometimes it turns on when I just walk by. This would be a minor annoyance, but it also makes me wonder how well the auto-away feature will work, since that’s also based on a proximity sensor (maybe the same one, maybe a different one, I don’t know).

Two simple things my old, not-worth-blogging-about thermostat did that the Nest doesn’t: show you the time on the main screen (my wife and I are used to using the thermostat as a clock on our way out the door, as it’s conveniently located for that) and remind you to change the furnace filters. Oh well, not a dealbreaker, and maybe these features will appear in a future software update.

One design deficiency that wouldn’t be worth complaining about with any other thermostat, but is noteworthy with the Nest now that they’re trying to apply good design to elevate thermostats to the level of art object: off-angle viewing of the display is terrible. It looks great from straight on, but thermostats are usually mounted at chest level, not at eye level, and used from fairly close by, so in normal use I’m looking down at the display at a fairly steep angle. Part of the problem is the LCD display itself, and part of the problem is the convex lens sitting over it; it’s really not putting its best face forward in actual use.

Problems

The second day after I installed the two Nests, I heard a clicking sound coming from the heating ducts. Upon inspection, the downstairs furnace was working fine, but the upstairs furnace was making the clicking noises, which were resonating through the ducts. I fiddled with the thermostat; it wanted the furnace on but it wouldn’t come on. I turned the thermostat off; the clicking stopped. I turned the thermostat back on, and it turned on the furnace for about a minute, then the furnace turned off and went back to clicking. I fiddled with the thermostat for a while longer, and was able to provoke various permutations of this behavior, but not a working upstairs furnace.

I called Nest support, and got a really helpful and knowledgeable guy who suggested I swap the upstairs and downstairs units, to help rule out a problem with the thermostat itself. After the swap, I still had problems upstairs and not downstairs — apparently the problem is with the furnace, and not the specific Nest unit. His hypothesis was that my furnace was so sensitive to voltage that it would work fine if the thermostat just bridges the Rh and W wires, but if the thermostat pulls any power at all from those wires, the furnace relays get confused and start toggling instead of staying on. Just for fun, I measured the wiring at both thermostats with a multimeter and couldn’t detect much difference between upstairs and downstairs (in both cases, the multimeter saw 28 V across Rh and W with the thermostat detached and the furnace not running, and 0.485 A across Rh and W with the furnace running), but the Nest’s built-in technical details monitoring report claimed voltage dropped from 29 V upstairs when not running the furnace to 7.77 V when running the furnace (and compared to 9.19 V downstairs). Is that 1.4 V difference enough to explain why one would work and the other wouldn’t? I don’t know.

When I called back to Nest support with these results, the tech support guy said this sounds like a power stealing problem and the probable solution is to run a C wire. I asked why two nearly identical (same model, different size) furnaces that are only 3 years old would be different in this regard, and he didn’t really know.

Note this isn’t the same problem Marco wrote about, though I’d likely have that problem over time too; his problem happens if the furnace doesn’t run very often and the Nest needs to charge itself without running the furnace; in my case the fully charged Nest couldn’t even run the furnace when it should be on.

Gripes

Beyond the fact that the thermostat can’t run one of my furnaces, it’s annoying that the Nest compatibility tool doesn’t even warn of this problem.

A few other things that came up with I was on the support call: first, when I was first describing the problem and giving information on how I’d connected the Nest et cetera, I asked if they were already collecting diagnostic information in the cloud since the thermostat is smart and internet-connected, and was told no. I find that surprising and disappointing — a missed opportunity. (There’s maybe a privacy concern, and I don’t want them selling data on how often I run my furnace or how often I’m home vs away, but I certainly wouldn’t mind them collecting diagnostics to improve their product.)

Also while chatting with the support tech, I asked if there was a way to get the Nest to display a clock on the main screen, and to remind me when to change the furnace filter, and was told no on both accounts; then I asked if I could file a feature request for those and was told that not only do they not take feature requests from customers, but that the support techs are prohibited from talking to the product designers, both for legal/intellectual property reasons. I joked that this must be because Nest was founded by people from Apple — famous for not being persnickety about taking ideas from people who might then claim ownership of said ideas; what can you say about a company with an official “unsolicited idea submission policy”? — and he said yeah, if you read our privacy policy it has a lot in common with Apple’s. But come on. It’s not like there’s any intellectual property in the idea of a thermostat displaying the time of day. And more than that: a policy that says a company can’t listen to its customers is a customer-hostile policy.

Solution

Basically, to get the Nest working with my upstairs furnace, I need to give it a C wire. I did consider running new thermostat control wiring, but when I looked at where the wires run, it’s not worth the trouble — there’s no way to rerun the wires without opening walls.

But. What’s this G wire for, anyway? Following the Transtronics wiki Marco linked to, and confirmed by observation, the thermostat doesn’t need to tell the fan when to run while heating — I have gas heating and the furnace itself controls the fan. Now, the thermostat can also tell the fan to run without the heat, and it turns out that’s all the G wire is good for. After checking with my wife, I confirmed we almost never use that feature (and it was relatively easy to access before, with a switch on our old thermostats, whereas with the Nest it requires digging through the settings menu a bit, so we’re even less likely to do it).

Thus, the solution: repurpose the G wire as a C wire. We don’t really need direct control over the fan, and if I do, I can run new wiring for that, to locations that aren’t as hard to get to as the thermostat locations. (Note that this works for me because we have gas heat; electric heaters don’t control the fan themselves, so removing the G wire would not be a good idea.) So, I had to turn off the furnace and reconnect the green wire on that end too, moving it from the G terminal to the C terminal. This quick-and-dirty solution satisfies my hacker instincts, and after a couple days of use, the Nest is much happier: the Nest’s voltage monitoring reports 36 V at 100 mA, instead of 8 V at 20 mA without the C wire, and the furnace runs when it should.

Summary

I like it; I’ll keep mine.

But based on my experience, I can’t recommend the Nest thermostat unless you can give it a C wire. And I’m disappointed that all the information Nest will give you before buying doesn’t point out even a whiff of such problems.

Update: Nest support tweeted me back to say they’ve updated the compatibility tool with a note about the C wire, but in my opinion it’s still buried where it’s too easy to miss.

Nest “C” Wire Update

| Comments

My Nest thermostat review complained about up-front lack of obviousness about “C” wire compatibility issues, and Nest support replied with a tweet saying

We’ve updated the compatibility tool so there’s a note about C wire, but you’re right – it could be clearer.

I took a trip back through the compatibility wizard, and (unless there are more changes coming) it’s there but it’s still not obvious.

First, I tell it what I’ve got:

Then, it tells me what to expect:

I’m compatible, no reservations, it says. Unless I click through the “Learn more” link, which from the context sounds like it’s going to help me decide whether I can install it without help, not whether it’s compatible with my wiring. That link goes to a “Does Nest need a common wire?” article, which has some useful information, including recognition of the possible problem, a suggestion to do what I did (turn another wire into a C wire if possible), and a link to even more details.

That’s the right information, but I wouldn’t expect to find it hidden behind that “Learn more” link, and I still think anyone who clicks through the compatibility wizard without a C wire should see it. Maybe these problems are rarer than my small sample size indicates, but on the other hand, this “power stealing” problem doesn’t seem like an easy one.

C’mon, Nest — I like the product, and now that I’ve got mine working, I expect to remain a happy customer. But for an issue this subtle and important, I think you’d do well to err on the side of caution.

Blog Comment Semi-spam

| Comments

Seems like spammers are taking to spamming weblog comments wherever they are allowed with innocuous-looking content-free posts just to link back to themselves. (Is this somewhere between spam and bacn, since it’s not quite spam? I was unsure at first, but now I think it’s just spam.)

Typically these posts will be written in a fairly reasonable facsimile of English, and not overtly spammy — a bunch of compliments I’m almost egotistical enough to believe, except there’s no relation to the post they’re commenting on, and somewhere at the end, a link to some site they’re pimping.

These range from the fairly obvious link

to a link trying to masquerade as a signature

to links only in the author’s name

and then there’s the variant using Tumblr’s “like” feature, as in

or

and for a final really blatant example, there’s this guy:

These last ones might actually require more than 2 seconds of thought to recognize as spam, if they didn’t appear in batches of 8 on the same post.

Needless to say, I just delete all these, except for the Tumblr “like” spam which I don’t know how to target directly. (The next step would be turning off both comments and Tumblr “notes” in the theme, which would make these problems go away entirely.)

The ironic thing is these people are probably doing this for Google link juice, but the way comments work via Disqus, it’s all Javascript and not in the real HTML as the page loads anyway, so I doubt indexers follow it. Plus I just delete it anyway. Spammers, if you’re reading this, you’re just wasting your time and mine.

Configuring iOS for Google’s Cloud Data Services

| Comments

If you’re a Gmail user, you’ve already got cloud email, a contact list or address book, and a calendar (actually a set of calendars) hosted by Google. And if you have an iPhone, you want to connect the built-in email, contacts and calendar applications to those Google data sources.

It’s possible, but it’s not as easy as it should be.

“Not easy”, you’re thinking? Go to Settings, “Mail, Contacts, Calendars”, “Add Account”, and there’s Gmail — enter your credentials, the account is set up, what could be easier than that?

Well, that isn’t hard, but it also doesn’t work very well. If you use the “Gmail” account type, it doesn’t sync contacts at all. Email is there, via IMAP, but Gmail via IMAP has some notable deficiencies: push email doesn’t work (so the iPhone doesn’t see new mail or other changes right away), the delete behavior is weird, and there’s this extra “All Mail” folder which might sync an extra copy of all your messages. Your calendars are there — well, by default, only the main calendar, and not any others you may have created, or calendars others have shared with you, or delegated/published calendars like the one TripIt can automatically publish your travel plans to. (You can fix this with an extra step, after which multiple calendars work normally.)

There’s another way: add your Google account using the “Microsoft Exchange” account type. (If this seems confusing, note that what this really means is to access Google’s data services using the Exchange ActiveSync, or EAS, protocol, which Microsoft has licensed to both Apple and Google among others. Even more confusing, Google brands this method of access to its services not as Exchange or EAS, but as “Google Sync”. Whatever.)

If you do this, things get quite a bit better: contacts sync to the phone; email gains push (and in my opinion, the folder list and delete button work better). This is actually the method Google recommends for use with iOS. Again, only the main calendar syncs to the phone, but again, there’s a way to enable the other calendars, which is even more of a pain than the CalDAV method.

(Aside: I think the ability to get multiple calendars via Google Sync and EAS was recently added, because when I first set up my iPhone I couldn’t find this method, and resorted to adding my Gmail account twice: once using the “Exchange” account type for email and contacts with calendars disabled, and once using the “Gmail” account type for calendars with email disabled. This no longer seems necessary, but it may be a useful tip for anyone who wants Google contacts synced to their phone but for whatever reason don’t like the EAS/Google Sync behavior for email or calendars.)

You may also run into some problems with contact sync, documented on the “Google Sync known issues with iOS” page, under “Limited Contact Information”: “Phone number synchronization is limited to 2 Home numbers, 1 Home Fax, 1 Mobile, 1 Pager, 3 Work (one will be labeled ‘Company Main’) and one Work Fax number.” Another problem not described here is that phone numbers not exactly matching the expected labels (which are case sensitive) will not sync: I had dozens of friends who in in my Google contacts list had their mobile phone number labeled as “Mobile” instead of “mobile”, and these people showed up on my iPhone with no phone number at all.

The end result is that it’s possible to make iOS and Google accounts play nicely together, but it requires some up front knowledge, some extra steps in account step on the iOS device, and some extra massaging of, potentially, every entry in your address book to make sure the expected phone numbers sync.

Is it possible to do better than this? Sure. The late webOS synced all my Google data (email, calendars and contacts) easily with no extra setup beyond choosing the Gmail account type and adding my credentials. Presumably Android is capable of similarly smooth integration. Apple may not be interested in smoothing the process in iOS because they’re wary of Google’s dominance and want to push their own cloud data services, but given the number of people who use both Gmail and iOS, I think they do their users a disservice.

Using iTunes Match as iTunes Library Sync

| Comments

Apple recently added two new features to iTunes: “iTunes in the Cloud”, which adds all your iTunes music purchases to a cloud library which can be streamed or downloaded as often as you want to any of your devices (iPhone, iPod, iPad, computer with iTunes), and “iTunes Match”, which adds all the music you didn’t buy from iTunes to that same cloud library, enabling all the same “iTunes in the Cloud” features.

Apple is billing this primarily as a way to get your music from your iTunes library onto your phone or iPad without having to plan ahead and pre-sync the music onto the device.

However, in addition to using iCloud to sync music between devices, iTunes Match also has another benefit: it syncs the library metadata between multiple iTunes libraries that enable iTunes Match using the same account.

This subtle addition is actually something important that I’ve wanted from iTunes as long as I’ve been using it: iTunes has a bunch of organization features including smart playlists which work best if it knows what you listen to and how much you like it. It tracks how many times you play each song and allows you to rate songs, and it automatically syncs this between one iTunes library and any connected iPods, but until now each iTunes library has been an island: if you have two computers, say a desktop and a laptop, each has its own library, with its own play counts and ratings and playlists, and its own idea of your preferences, and any iPods or other Apple devices you have were able to sync songs and metadata from a single one of these libraries.

Now, with iTunes Match enabled, all my computers with iTunes see the same library — all of the songs on any one of them are synced up to iCloud and then back to the other computers, and metadata including play counts, ratings and playlists are also kept in sync. This joining of the libraries also means it matters a lot less which of my computers I sync a given iPod with, since effectively they all use the same library.

There’s one minor remaining annoyance given the way I use iTunes at home: I have a file server with all my music, a Mac Mini in the living room which can play music and movies, and my main desktop computer, and each of these Macs uses the network file server for the music files instead of storing a local copy. (My laptop spends enough time disconnected from the home network that it does deserve a local copy of the music.) This means that if I add music to the library on one of the desktop computers, it’s already accessible in the expected location from any of the other computers that can access the file server, though the other computers won’t know it until I tell iTunes to add the files to the local library. The storage is shared; the library metadata is still per-computer. Using iTunes Match, the libraries are kept in sync automatically, but iTunes doesn’t notice that some of the files are magically already local — it will show such files with the cloud icon, and offer to stream them or download them even though that’s unnecessary. Streaming works fine, and downloading works, but actually downloads the file to the same folder, notices that the file already exists, and adds a numeric suffix to make the filename unique.

It would be nice if there was a way to tell iTunes to notice local copies of files that it thinks are only in iCloud. Also, I occasionally found myself having downloaded extra copies of songs, noticing this cluttering up the music folder, and deleting the extra files outside of iTunes (using Finder) — this left me in a broken state where iTunes has an entry for the song in the library, and the file exists in the filesystem and in iCloud, but iTunes thinks it’s local so won’t stream or download from iCloud and thinks it has a different name so it won’t play the local file, at which point things are pretty broken. What’s needed here is a way to stuff the file back into the cloud, making iTunes think there is no local copy so it will go get it from iCloud again the next time it’s needed — I couldn’t figure out how to do this until I read this Q&A article at Ars Technica which has the secret: select the song, hold the option key while pressing the delete key, and the normal delete confirmation dialog will grow a new checkbox “also delete this song from iCloud”. Leaving that option unchecked, you can delete the local copy of the song (or, a broken link to a missing local copy of a song) while leaving it in the cloud library, the cloud icon comes back, and iTunes will go back to getting the data from iCloud as needed again.