Step 2 is the hard part.

Matt Ginzton writes here.

Swimming Upstream

| Comments

VW’s safety/recall department just sent me these janky stickers for my 2011 Golf TDI. The accompanying notice wasn’t phrased as a recall, exactly, but given the return address, I can’t help but think they’re trying to avoid liability, and they must be having a larger than normal problem with people using the wrong fuel.

Stickers for VW Golf TDI fuel door

They specially called out loaning the car to friends or family; I haven’t had any problems with borrowers putting in the wrong fuel, but when I have loaned out the car I’ve reminded people 3 times that they shouldn’t refuel it but if they do they need to use diesel (the fact the car goes 500 miles between refueling helps a lot here).

So now my fuel filler area looks like this:

VW Golf TDI fuel door after stickers

(Note how the stickers are wrinkly and poor-fitting, and this isn’t only because I applied them poorly.)

What I don’t understand is, why don’t they just use green plastic for the fuel cap itself? Not green as in tree hugger; green as in the color of the handle on almost all diesel fuel pumps in the US. (I realize the green handle isn’t a legal requirement or a universal convention, and notably BP doesn’t follow it, but it’s still pretty widely recognized.) That would be a much more obvious visual cue to remember which fuel to use. Doesn’t this make more sense and look both cleaner and more obvious?

VW TDI fuel door with green aftermarket cap

I wondered this when I first bought the car; I wonder this every time I refuel it; I especially wonder this every time I see these stickers.

iPhone Hardware Wishlist

| Comments

It’s almost September, so time for a new iPhone.

What’s coming this fall from the software side (iOS 6) is mostly public already.

What’s coming this fall from the hardware side is just rumor, but seems to revolve around a larger 16:9 screen, a unibody enclosure, and a smaller replacement for the dock connector.

Here, with no connection to reality, the above rumors, or any inside information, is what I wish Apple would add on the hardware side:

  • Wireless charging (a la Palm Touchstone, or, like, every electric toothbrush ever). Since iOS 5 added over-the-air upgrades and wireless cloud backup and iTunes sync, pretty much the only reason I need to plug my iPhone into a cable is to charge it. Even better than a smaller “foolproof” dock connector is no connector.
  • USB 3. Probably a no-brainer, but now that the iPhone shoots HD video, it accumulates some pretty darn big files. After charging, the only other reason I plug my phone into a cable is to transfer photos and videos; sure, this could be done over Wi-Fi, but for gigabyte video files, neither USB 2 nor Wi-Fi is satisfyingly fast. USB 3 should be.
  • NFC. Google is turning Android phones into wallets; Apple is adding this new Passbook feature in iOS 6; it would work better with NFC. But even more than replacing my wallet, I’d like my phone to replace my keychain, and NFC would help with that too.
  • Water resistance. How many iPhones (or other phones, music players, cameras, or other devices that could be replaced by iPhones) have you seen destroyed by water? Me, a lot. I’m not a hardware engineer, but there aren’t all that many openings on an iPhone (especially if they’re about to redesign the biggest one anyway); the battery is sealed inside; I wonder how hard it would be to make iPhones survive at least casual contact with water. (Apple’s gone to the trouble of adding a special moisture sensor to detect liquid damage and rule it out of warranty coverage; wouldn’t it be nice if neither they nor their customers had to worry about it?)

Mac App Store DRM

| Comments

Apparently apps from the Mac App Store are wrapped in DRM (not surprising) and this DRM is somehow tied to the motherboard hardware, not the OS install or the disk drive hardware (more surprising).

I wouldn’t have realized this until I “upgraded” my Mac Pro by buying a newer one, and plopping the SSDs from the old one into the new one. This let me avoid installing or configuring the OS or apps; everything ran just fine but faster; everything, that is, except apps from the Mac App Store, which all (individually) prompted for my MAS credentials with a dialog about how “this app was purchased on another computer”.

(I expect this scenario and dialog are more common when copying the app bundles from one computer to another; that’s an obvious case of purchasing an app on one computer and running it on another; moving the whole drive and OS install is less obviously a new computer — hey, it’s still got the same hostname — but it triggers the same scenario.)

Retina MBP First Impressions

| Comments

I ordered the Retina MacBook Pro the second day they were available. (In that day, the order backlog went from 1 week to 3 weeks, so I just got it a week ago.)

The screen It’s super high resolution, as you’d expect. The front glass covering it is super thin, like on the “retina” iPad and iPhone models, giving the appearance that the (invisible) pixels are painted on the surface, instead of trapped inside. It looks great. But it’s still too glossy for my taste. (Apple says it’s 4 times less glare-prone than their previous glossy screens; I don’t know how to measure that; it may be less glare-prone but it’s still glare-prone.)

Apple’s using their usual pixel-doubling trick (same as on “retina”-branded iOS devices) to increase the number of physical pixels while pretending not to, to improve compatibility with existing software; as deployed on the rMBP with OS X 10.7.4, it seems to work better than the experimental “hiDPI” pixel- doubled modes on other recent Macs running OS X 10.7. While the rMBP doesn’t claim a new OS version, the Displays panel in System Preferences looks different than on other machines running 10.7.4, with the list of pixel resolutions removed and replaced with a choice between “best for Retina display” and 5 “scaled” choices ranging from “more space” to “larger text”. (The option to show the display settings in the menubar is missing, somehow, though the search feature in System Preferences still thinks it should be there.)

The physical design It’s thin. Apple achieved this thinness, controversially, by removing a bunch of components you’d expect in a high-end laptop — an optical drive, ethernet and Firewire ports — and also sacrificing upgradability by using custom components and connections to save space. I don’t care at all about the missing optical drive (in fact, in my last laptop, I removed it and replaced it with an SSD), but I do care about the lack of upgradability. Typically I’ve extended the life of my laptop by upgrading memory and replacing the hard drive with an SSD. With this machine, the RAM is soldered in, and upgrades are not gonna happen. I countered that by ordering 16GB, more than my old desktop machine had, and hopefully enough for the life of this machine. I also ordered the Thunderbolt to gigabit Ethernet adapter; the way I use laptops, Wi-Fi is usually fine but occasionally not; the performance of the external Thunderbolt dongle is just fine, as long as I have it where I need it when I need it (the perennial problem with dongles/adapters).

And finally, in what may sound like a nitpick, I’m sad that Apple removed the external battery status light. I always found that useful, especially if I haven’t used the machine in a couple days and am about to take it somewhere and don’t know if I need to take the charger with me. (I guess the MacBook Air doesn’t have this light either, so there’s precedent, but I don’t have to like it.)

The speed The CPU is fast. Most of what I (and most of us) do with computers is no longer CPU-bound, but I was coming from a dual-core Core 2 machine, and this rMBP’s Core i7 cores are palpably faster at the same clock speed, and this machine has 4 of them, and they’re hyperthreaded so Mac OS thinks it has 8 of them. The only taxing thing I’ve tried so far is telling Serato Itch to analyze an 800-song music library, which took 3 or 4 hours on my old laptop and about 15 minutes on this one. I don’t think I’ll be complaining about CPU speed for a while.

Meanwhile, while I’ve been all-SSD for a while, this is the first time I’ve had Apple install the SSD as a factory option instead of going aftermarket and choosing and installing it myself. Mostly because I had no choice, but also because it looks like Apple finally chose a top-shelf model.

The software Given the pixel-doubling trick Apple is using on all their retina-branded models to avoid text and UI elements getting too small to see/interact with normally, software needs to be updated to actually take advantage of the additional resolution. Otherwise, it’ll run as well as before but at the old resolution, and look somewhat blocky/pixelated. Apps that are mostly text and standard UI widgets using Apple’s text rendering and widget libraries turn out fine; apps with a lot of custom graphical UI elements need to provide double-resolution versions of their images but look mostly okay in the meantime; apps that do their own text rendering look pretty much horrible without an update.

I normally use Google Chrome as my web browser. Chrome (and Firefox) is one of those rare apps that does its own text rendering, which means it severely needs an update, not just to be “optimized for Retina display” but really to look good at all. This update has already happened in the “Canary” channel (12 weeks into the future of the stable version, because Chrome updates every 6 weeks, and updates flow from the Canary double-beta version to the beta version to the stable version); this means if you want text to look good, you can wait 12 weeks, use Safari, or use Chrome Canary. I’m using Canary, and for the most part it works fine, but Google Maps doesn’t work. At all. (And more into nitpick territory, the Canary version is clearly not intended for daily use; the yellow dock icon is an eyesore, and it won’t allow you to set it as the default system browser without resorting to external means.) So I’ve got Chrome Canary for most things, and normal Chrome and Safari as fallbacks for whatever doesn’t work in Canary, which so far is just Google Maps.

The summary It’s got the highest resolution screen most of us have ever seen; it’s thin and light given that screen size; it’s fast, packing very respectable compute+storage hardware; it’s basically the right silicon wrapped around an amazing screen in an amazingly tiny package. I’m theoretically annoyed by the need for an Ethernet dongle, but I expect it won’t be a real problem. I’m theoretically annoyed by the non-upgradable RAM, but 16GB should be enough for anybody. Seriously, over the years usually RAM upgrades are the last straw that force me to upgrade laptops; it’s hitting the RAM ceiling that forces me to upgrade the whole machine; time will tell if this rMBP model is a good or bad decision for me, but by ordering it with the maximum (in place of starting lower and upgrading later) it feels relatively well future-proofed.

I like it. I have a hard time imagining a better 4½ pound computer today.

Apple’s June 2012 MacBook Pro Pricing

| Comments

Apple updated the MacBook Pro lineup this week, with both evolutionary enhancements recognizable as successors to the previous lineup, and also adding a new option branded as a member of the same family (“MacBook Pro with Retina Display”) which is actually a whole new model. It’s not an improved display option for the same machine, as the name implies: it’s thinner and lighter with an entirely different case design, different set of external ports, no optical drive, and vastly different (and less upgradeable) internals.

To my thinking, it’s closer to a 15” MacBook Air than a true MacBook Pro.

That doesn’t affect whether you should buy one, but I do find it interesting.

Predictably, people are gushing about the display and complaining about the price. Apple positions it as the high-end option in the MacBook Pro lineup (13” starting at $1199, 15” starting at $1799, and 15” Retina model starting at $2199). No, that’s not cheap, but let’s consider: the lowest-speced Retina model has a 2.3 GHz quad-core processor, 8GB of RAM, and a 256GB SSD for $2199; the 15” non- Retina model with the $1799 price tag has the same processor but 4GB of RAM and a spinning hard drive.

If you equalize the storage and memory options, Apple wants $100 for the memory upgrade and $500 for the SSD upgrade. Leave aside the question of whether those are fair component prices — they’re the same components Apple is including in the base level of the Retina model. So this brings the price of the non-Retina model to $2399, or $200 above the Retina model. With equivalent specs, the Retina model is actually cheaper!

This point is exacerbated by the display options — the non-Retina model actually offers a choice of displays, and the base price includes the lower- resolution 1440x900 display. For $100 more, you can get a 1680x1050 display “with 36 per cent more pixels”. Or, for $200 less, you can get the Retina model with, um, 300% more pixels, if you’re into that sort of thing.

This difference carries over to the higher-spec model too. In chart form:

CPU2.3 GHz quad-core i72.6 GHz quad-core i7
RAM8GB 8GB
SSD256GB512GB
Pr ice with 1440x900 (non-Retina) display$2,399$3,099
Price with 1650x1050 (non-Retina) display$2,499$3,199
Price with 2880x1800 (Retina) display$2,199$2,799

(In fact, note that the spread between the Retina model and its 1440x900 sibling has grown to $300 in the 2.6GHz model, because once you cancel out the cost of the storage and RAM upgrades bundled into the faster model, Apple wants $200 for the faster CPU in the non-Retina model, and $100 for the same CPU upgrade in the Retina model.)

Now, the older (non-Retina) “MacBook Pro” models do have some advantages over the Retina model: Firewire and Ethernet ports, the option of a matte screen, an optical drive, room for larger hard drives, standard components that can be upgraded, and (combining these last 3 qualities) the ability to replace the optical drive with a second hard drive or SSD. For some people, these qualities might actually be worth more money. And true, these machines do start at a lower price point than the Retina model, if you’re willing to go with spinning storage and less RAM. But comparing apples to apples (can… not… resist… use… of… phrase), it’s not true that the Retina model is more expensive.

In fact, what I find interesting here is what the pricing says about Apple’s priorities. They’re actually trying to push people in the direction of the new model, at least at the high end. I presume it costs more to build than the non-Retina model, and I think you could build an argument it should sell for more than the non-Retina model. It would probably still sell (it’s lustworthy and has no direct competition). But it would not sell as well.

Now we’re dealing with all Apple prices here, and they do charge well above market rates for the RAM and SSD upgrades which the Retina MBP forces you to opt into. So feel free to argue that it’s overpriced, if you think so. You can also argue that all this proves is that the non-Retina MBPs are even more overpriced. But the bottom line is that for anyone already considering a high- end MacBook Pro with SSD anyway, the price on the Retina models is low — surprisingly low, even.

OpenWrt Is Losing Its Luster

| Comments

Ever since I discovered it in 2006, I’ve used, and been a huge fan of, OpenWrt for my home network router. I started running the venerable Whiterussian release on a genuine WRT54G, at a time when I wanted to tweak some configuration options that the stock firmware didn’t give me control over; I dabbled in alternate firmware like Tomato and HyperWRT but quickly found that their filesystem layout made them too hard to customize; then I discovered OpenWRT and found the design far saner.

While my original purpose in looking past the factory firmware (power boosting to increase range) didn’t work out well (it boosted the signal and the noise, with little effect on range — go figure — replacing the antenna worked far better), I soon found myself addicted to several features of a fully configurable router. Probably at least half of what I loved about OpenWRT can be attributed directly to dnsmasq, a wonderfully clever piece of software that acts as both DHCP and DNS servers, and coordinates across the two protocols, so that in general, all DHCP clients get entered in the DNS with reasonable hostnames. (This solves the same problem that Microsoft tried to solve with NetBIOS and WINS, and that Apple tried to solve with Rendezvous/Bonjour, but the dnsmasq approach doesn’t require any new protocols or software on any of the clients — DNS just works.) Beyond dnsmasq, here is a short list of things I ended up configuring my OpenWrt router to do, which I hadn’t realized I needed, but once available proved to be very useful:

  • OpenVPN server: set up an offsite file server at a friend’s house for offsite backup
  • avahi: to forward mDNS across the OpenVPN link as appropriate
  • VLAN support: in addition to my private LAN, I ran a separate semi-public network with wireless access points using a separate SSID bridged onto a separate VLAN which had access to the internet but not my LAN
  • tcpdump: whenever having network problems, the ability to sniff traffic on the router is invaluable in troubleshooting them

This worked great for years, but as time passed eventually the WRT54G got flaky and started rebooting every few days. When it came time to replace it, I wanted newer and faster hardware — compared to 2002-era hardware, I wanted more RAM, more flash storage, gigabit wired networking, and N-speed wireless networking. This was 2010, and such hardware was readily available. I picked a suitable-looking model with good hardware (Netgear WNDR3700), installed the current OpenWrt release (10.03, Backfire), and at first things were great1.

Then, what’s the problem, you may ask? Well, over time I started experiencing a raft of weird problems, to be detailed below, and slowly but surely I started associating them with the current version of OpenWRT. I tried newer versions, filing bug reports and asking for help in the forums, and even switching to different hardware, but eventually these problems piled up to the point I decided to switch away. I still have a lot of respect for the project and its volunteer developers, but it just wasn’t working out for me. My unprovable hypothesis is this: Openwrt originally started as a fork of the GPL-mandated open source drop of Linksys’ own WRT54G firmware, and thanks to Linksys’ own engineering and binary drivers for the specific hardware they used, the result wasn’t particularly clean but worked great. Over time, as OpenWrt started running on more and more hardware, and started tracking closer to stock Linux instead of vendor-provided customizations, and using open source drivers instead of opaque vendor binary drivers, it has to deal with more configuration sprawl and gets less benefit from the vendor QA and the result, while much cleaner, is also flakier. I don’t know that this is true, but in terms of pure stability, I never had a problem with Whiterussian on 2002-era hardware (Linksys WRT54G and Asus WL500G-P), whereas I saw all sorts of weirdness with Backfire on 2010-era hardware (Netgear WNDR3700 and Buffalo WZR-HP-300N).

On to the problem list:

  1. On the WNDR3700, internet connectivity would occasionally just stop. My LAN would be fine, except no contact with the outside world. I first blamed this on my cable modem and provider (hence this series of articles), and at the time I’d had such a good experience with OpenWrt that it was a long time before I thought to blame the router, but eventually I looked in that direction, and found eth1 would get into a stuck state that I could reset either by replugging the cable or using mii-tool to reset the media interface. I posted about this on the OpenWrt forum, without hearing much.

  2. Because of that problem, and because I couldn’t tell whether it was hardware or software at fault, I did the extremely scientific thing of changing two variables at once, and bought a different router (Buffalo WZR-HP-300N) and installed a newer Backfire (10.03.1) on it. I didn’t have the eth1 problem any more. But I did have a different and equally annoying problem at approximately the same frequency: dnsmasq would stop serving requests. Since it’s responsible for both DHCP and DNS, this was fairly crippling. I’m loathe to blame dnsmasq itself since I’ve never seen this behavior from it in any other install, and also because this was accompanied by weird system level behavior: when it got into this state, I was unable to kill -9 the dnsmasq process, nslookup processes on the router would also become unkillable, and the router would fail to soft-reboot — I had to hard power cycle it. I posted about this on the OpenWrt forum as well, to deafening silence.

  3. I’d see a low but present rate of DNS lookup request failures — a valid request would return NXDOMAIN, and repeated immediately would succeed. This caused a small amount of application-layer collateral damage and general flakiness. Of course, I don’t know whether to blame Backfire or dnsmasq or something else. But I saw this problem during the 2 years I used Backfire as my main router, and not before or since.

  4. The OpenVPN link would eventually stop carrying traffic. Traffic that should be routed over the VPN link would just disappear. At this point, I’d try troubleshooting first with tcpdump, then by adding verbose logging like echo-a-character-for-each-packet in OpenVPN (which requires restarting the server), then restarting the firewall a few times, and none of these changes would make a difference immediately, but after restarting both OpenVPN and the firewall a few times each (without making any configuration changes other than enabling logging), suddenly I’d start seeing the echo characters and packets would flow; then I’d disable the logging and restart OpenVPN and it would work fine for a few more weeks. Again, I have no proof that Backfire was to blame here. But as in #3, I saw this problem during the 2 years I used Backfire as my main router, and not before or since.

  5. My Xbox 360 suddenly found itself unable to sign into Xbox Live. I noticed this in January 2011 and while it’s hard to know exactly when it started or what changed, I hadn’t changed anything in the router configuration recently, so I suspect the Xbox dashboard update shortly before that made the critical change. But I blame OpenWrt, not the Xbox software, because I sniffed the traffic it was sending, and found weirdness. Specifically, the Xbox likes to use UPnP to open network ports, and I had miniupnpd enabled on the router to allow this, and in the broken state, the Xbox would make a UPnP request to forward UDP port 3074 to itself, then it would send outgoing UDP packets to something.xboxlive.com:3074, and the router would see that as eligible for the forwarding rule it just set up, and immediately reflect the Xbox’s own packet back to it. (It would also get NATted and go out on the WAN, and a few hundred milliseconds later a response would arrive from the real Xbox Live service, but by then the Xbox was already confused.) Meanwhile, the port forwarding wasn’t actually necessary because of the OpenWrt NAT implementation. So if I stopped miniupnpd, the Xbox would be able to sign into Live, but other things I run that want UPnP or NAT-PMP would break. I was able to have it both ways by blacklisting the Xbox from talking to miniupnpd, but this didn’t inspire confidence.

Once I mentally assembled this list into one place, I realized there was enough general weirdness here that I was no longer happy with OpenWRT, and I wanted something that would be completely stable and dependable. That made it tempting to buy a router with the right capabilities as advertised features out of the box, and stick with the stock firmware, but I didn’t want to give up on the flexibility I’d become accustomed to (especially, dnsmasq, ability to run tcpdump, and ability to install extra services). What I actually did about this is a story for another time. Here, I’ll just say that after replacing the OpenWrt router 3 months ago, I haven’t seen any of the above problems recur.

Note 1: Actually things took a little hacking before they were great, speaking of Backfire on WNDR3700 — I initially started setting this up before the final Backfire release, so I had to compile from source, which was fine because then and even now, 5GHz radio support doesn’t work in the precompiled releases. Too bad, since dual radio support was my reason for choosing the WNDR3700 in the first place.

Custom-built Linux Router, (No) Thanks to Realtek

| Comments

In my last post, I spelled out my requirements for a home router: dependable and not requiring babysitting or monthly rebooting, but flexible enough to let me run and control dnsmasq, tcpdump, and VLANs.

When I realized I was seeing so much weirdness at once from my OpenWrt router as to be circumstantial evidence against OpenWrt itself, I mentioned this to my officemate and he said “why don’t you stop screwing around and install full-blown Linux?” Sure, I thought, but that brings up two problems: it sounded like a huge time suck, and where am I going to find appropriate hardware to use as a router?

With help from friends I eventually solved the second one, not without paying a heavy tax in terms of the first (and not in the way I expected): this is that story.

On the time suck question, it seemed like I would have to learn a lot of new things to set up and, possibly, maintain a lot of tasks that I was accustomed to having OpenWrt do for me. I already knew how to install and administer Linux for standard desktop or server use, but I’d never myself configured any advanced networking topologies and my few interactions with iptables had been painful, so configuring NAT and firewalling and routing and dealing with multiple network interfaces was daunting (and this box is by definition going to be exposed to the Internet, so I’d better get the firewall right). I poked around and found shorewall, which exists basically to configure the parts that I didn’t already know how to do, and the more I read about it, the more it seemed a good match for what I was trying to do.

On the hardware question, I wanted something small and quiet and low-power, which would fit in my server rack and stay on all the time without running up my power bill or generating so much heat that it either fails or needs a leaf blower of a fan. (That basically describes most consumer routers, for which, generally, the closest thing you can find to a standard Linux distribution supporting them is OpenWrt. Ahem.)

I also wanted it to have multiple network interfaces, as a router should. (This may or may not be relevant to the hardware decision, though; read on.) A router needs a minimum of two interfaces by definition: one for each network it routes between, so at the minimum, one for the LAN and one for the WAN. The scenario I had in mind was more complicated, with two separate LANs (one for my family and one for guests who just want to get their tablet on the internet), and leaving room for the possibility of multiple internet providers, so I’d need at least 3 interfaces, with the option to expand to 4 or more in the future. Now, these don’t all have to be physical interfaces built into the router. If the router has USB ports, you can add more that way; also if you have other physical infrastructure supporting VLANs, you can multiplex several networks over one physical port. (Again as a comparison to OpenWrt: standard consumer routers that OpenWrt runs on tend to have 5 ports and 2 network interfaces; one network interface is connected directly to one port labeled WAN, and the other interface is connected through a switch chip to the other 4 ports, which by default are bridged onto a single VLAN but which can be configured for 4 separate VLANs if that floats your boat.)

After getting some advice from friends and discussing it ad nauseum, I ended up buying a fit-pc2i, notable because it’s a standard x86 PC (so I can choose really any standard Linux distribution or even Windows to run on it), in a tiny passively cooled case, drawing 6W, and with 2 physical network interfaces. (I didn’t like the idea of depending on a bunch of USB network adapters, and I wasn’t sure I could rely only on VLAN support to get extra ports, so I wanted a 2nd port for insurance. Now that I’ve used it, I think a single reliable physical network interface + VLAN support would work out fine.) Those 2 network ports are not enough for my scenario, so I also bought a Cisco SG200-08 switch, which I use solely to add ports, turning 1 into 8.

Having made these decisions, I bought the fit-pc2i and SG200, installed Linux (Ubuntu 11.10 Server) and dnsmasq and shorewall, configured VLANs between the router and switch so that various switch ports acted like they were connected to additional eth1.x interfaces in the router, and started testing things. It worked fine until I tried a speed test (from a client connected through the new router which was connected behind my old router); the speed test promptly hung from the client’s point of view, and I couldn’t access the new router over the network at all. I power cycled the new router, tried again, same result. I poked around log files, tried to enable the Linux NMI watchdog, and generally looked for clues without finding anything until I visited the fit-pc forums and read “solution for freezes when scp/ftp/nfs with most Linux dist”. This pointed the blame squarely at the Realtek network interfaces, and suggested an alternate driver as a solution. Once I started investigating fixes for this, I got really pessimistic at first: a Google query for “r8169 freeze” shows a dismaying number of hits, many in distribution-specific bug reports going back years and years. I’d been under the assumption that networking is Linux’s lifeblood and that wired networking has long been a solved problem — wireless network hardware flaky under Linux, sure, any network hardware flaky under Windows, sure, but wired network hardware flaky under Linux? That was a rude surprise.

Long story shorter, the in-tree driver (open source and provided with Linux kernels) for this class of Realtek hardware is named r8169. It actually supports a family of Realtek chips named RTL8111/RTL8168, of which there are apparently many variants with important programming differences even inside the same PCI ID, so using lspci won’t necessarily tell you enough about which one you have. Realtek also has their own driver, also ostensibly open source but not included in the standard Linux kernel, called r8168. For years now, you can find blog posts saying “I had such and such a problem with r8169 and I switched to r8168 and it worked better.” So naturally, I tried r8168, and found it didn’t work at all. Upon further investigation, it has completely broken VLAN support (at least on my hardware, in the 8.027 driver that was current at the time, in the phase of the moon that obtained at the time): on a non-virtual interface it worked fine (and without freezing the kernel); frames that should have an 802.1Q tag added or removed had it done incorrectly, and would either (outgoing) get ignored by the switch, or (incoming) get ignored by the kernel. After spending hours running 3 instances of tcpdump (on the fitpc on the raw interface, on the fitpc on the virtual interface, and on a separate machine plugged into a switch port on the SG200 mapped to the same VLAN), I could characterize the problem: outgoing frames were transmitted with no tag, and get dropped by the switch. Incoming frames with a tag actually had it stripped and were dispatched properly. I found out about “ethtool -K” to control hardware acceleration of VLAN tagging (does this really benefit from hardware acceleration? More than it loses from the possibility of someone screwing it up?), disabled VLAN tag hardware acceleration in both directions, and found the opposite problem. Just by luck, at this point I re-enabled hardware acceleration for VLAN tagging only on the RX path, and things started working. But only on certain ports.

As a recap of what I found to be broken with r8169 and 802.1Q: as the driver loads by default, it improperly tags packets on the TX path. If I use “ethtool -K txvlan off”, TX works but RX packets are ignored. If I use “ethtool -K txvlan off rxvlan off” followed by “ethtool -K txvlan off rxvlan on”, TX and RX both work, but flakily — some ports and protocols work, some don’t, and I don’t know why but I’ve spent too much time staring at packet traces and I don’t care any more. The driver is broken out of the box, can be made to almost work by enabling and disabling VLAN tag acceleration in the right order through an order-dependent set of transitions reminiscent of port knocking, but still doesn’t entirely work, and I’m not going to trust it.

Then, back in r8169-land, I found an Ubuntu bug report, Network problem with the r8169 driver and RTL8111/8168B, in response to which people said the 3.1 kernel driver seems to work better than the 3.0 kernel driver, and Leann Ogasawara produced a 3.0 kernel with the 3.1 r8169 driver grafted in for people to try. So I tried it, and: lo and behold, while my repro scenario would still provoke a nasty warning and stack trace in system.log, there was no freeze.

At this point, I reported my findings to both the r8168 maintainers (Realtek) and the r8169 drivers (Linux netdev mailing list and Francois Romieu). Realtek didn’t respond at all. Francois did reply, saying he’d been fixing a bunch of problems in this area recently, and the 3.2 driver should work even better (this was last December, in the final throes of the 3.2 kernel release). I grabbed a 3.2RC7 kernel, installed it under the Ubuntu 11.10 install I was using, and it worked fine. No warnings, no backtraces, no freezes.

I haven’t touched the configuration since; after another week or so of testing I installed the fitpc + Ubuntu 11.10 + the 3.2RC7 kernel as our main router, and we haven’t had any problems with it. Hopefully, the Ubuntu 12.04 release (which already uses a 3.2 kernel) will install and run fine, and I won’t have to worry about this for another another 5 years since 12.04 is an LTS release.

Lessons learned here:

  • VLANs are cool, and I don’t really need more than one physical interface on the router the way I’m using it. I recommend the VLAN + separate switch as port splitter technique. But you do want a gigabit network interface if you’re going to do that.
  • Realtek was the bane of my existence for a few weeks in December. It looks like I just had bad timing, and if I’d done the same setup in April using Ubuntu 12.04 with a Linux 3.2 kernel I wouldn’t have had to learn any of this r8168/r8169 business. But given the history, I wouldn’t recommend their products. I went so far as to reconsider the whole fitpc choice. But in this form factor, it seems all the alternatives (including other fitpc products) use Realtec NICs.
  • shorewall makes configuration of NAT routing, firewalling, and traffic shaping much easier, in my opinion, than raw iptables and tc.
  • Aside from dealing with the Realtek issue, this was less of a time suck than I was expecting.
  • Including dealing with the Realtek issue, this was more of a time suck than I was expecting.
  • I’m happy with the result, though. Treating the fitpc-2i and SG200 as one unit, I have something that’s about the size and power consumption of the OpenWrt router, except now it’s got a 1 GHz x86 CPU, 1GB of RAM, 32GB of flash storage, 9 individually addressable network ports, and is still entirely solid state. Those hardware specs only matter inasmuchas they give me plenty of breathing room for future expansion (I don’t think my actual usage was taxing the much-lower-speced OpenWrt router), but the real bonus is it’s stable: OpenVPN, dnsmasq and miniupnpd are all behaving as they ought to.

Siri, Keeping the “Artificial” in “Artificial Intelligence”

| Comments

A lot has already been written on Siri, most of it pretty polarized — either of the “it’s a gimmick nobody uses”, or the “wait, but I use it!” variety. I don’t mean to throw fuel on the fire, but I keep trying to use Siri and haven’t found it to work well enough to be worth the effort; here I’ll present some examples which are both edifying and amusing.

(There’s one glaring exception to “I haven’t found it to work well enough”: for straight dictation, it’s uncannily accurate. If it starts and stops listening at the right time and doesn’t encounter a network error — that is, when it works at all — it almost always transcribes exactly what I said. It’s strange to me that Apple brands the dictation feature as part of Siri, because to my way of thinking they’re entirely separate layers — dictation or transcription for turning spoken sounds into text, and then Siri for assigning a meaning to that text and acting on the meaning — but as far as I can tell, Apple considers dictation to be a Siri function, so I have to give her credit: she’s great at understanding the words I say.)

In the following real-usage examples Siri, while great at understanding what I say (in all these cases the transcribed query is exactly what I meant to say) is not so great at understanding what I mean.

Flight status:

IMG_0326

“I can’t help you with flights.” Actually she did understand what I meant, but doesn’t know how to help. That’s too bad; that seems like a really natural function for a personal assistant, and one that lends itself well to machine structured representation and constrained search. I expected Siri would knock this one out of the park.

Appointment reminder:

IMG_0327

“3 weeks from today” gets interpreted as “1 week from today”. (This screenshot was taken on 2/9, as you can see from the blue marking, and 2/16 where Siri put the appointment is decidedly not 3 weeks later.) Again, Siri is often credited (and advertised) as being good at setting up appointments; I expected her to knock this one out of the park.

Another appointment reminder:

IMG_0364

“5 weeks from today” gets interpreted as “tomorrow”. Otherwise, so close.

Knowledge question:

IMG_0438

This one is more of a stretch, and I wouldn’t have been surprised had Siri fallen back on generic web search, but instead she tries to interpret it and get the answer from Wolfram Alpha, so points for that. But — and the reason this rates inclusion in this post — the interpretation she (and/or Wolfram) chooses is straight up bizarre. (I really had to scratch my head to deconstruct this. It decided that “king” refers to a magazine which apparently has a circulation of 223,000; it decided that “bed” is an abbreviation for “banana equivalent dose”; I have no idea why or how it thinks those concepts are related. If you actually go to wolfram alpha and search for “king size bed”, it actually understands the topic is “bed sizes”, though it has no information. If you search for “king bed”, it does take “bed” to mean the radiation unit, does not take “king” to mean the magazine, and clarifies “assuming king is a unit” while offering to “use king bed as a bed size topic instead”. So I think Siri deserves the credit for taking this from crazy to awesome.)

And finally, accidental bonus humor:

IMG_0436

OK, this wasn’t meant for Siri’s ears at all (it’s not even a question); in fact I thought I was dictating a text message in the Messages app. (Having input focus in the wrong spot is a strange and niggling complaint for a touch platform like iOS, but this is what happens.) Siri’s interpretation of why I’d be telling her about a rat eating my compost is, however, both noteworthy and disturbing, especially for the establishments it recommends (neither of which deserve this treatment, I can vouch for).

On a final note, I’ll reiterate that these are not contrived examples; excluding the compost-eating rat which was a mistake, this list represents the last 4 times I tried to use Siri with real intent, and at least 3 of these are clearly in her domain. With results like these, I don’t know that I’ll keep trying.

Headphones That Let Sound in but Not Out

| Comments

When our son Dominic was born, I knew that my wife and I (and he) would start having diverging sleep schedules for a while, and I suddenly discovered a need for something I’d never considered before: headphones that let ambient sound in but not out (and also umpteen pieces of more traditional baby paraphernalia, natch). The reason being: if they’re resting or sleeping and I’m not tired but want to stay in the same room, can I work or entertain myself without bothering them, and also without completely isolating myself from them? If I’m listening to music and Vanessa talks or Dominic squawks, I want to be able to hear them.

This is the opposite of what people usually want when using headphones: usually, you want the best fidelity possible for the sound playing in the headphones, without outside noise or distractions. Allowing ambient noise to intrude on the music is not normally the goal. So, it’s not a problem with many ready solutions.

I’m familiar with the basic headphone designs: in-ear, and open- and closed- back on-ear. Open-back headphones tend to be comfortable and provide a good soundstage, but don’t isolate your ears from the environment (in either direction), so they’re not suitable for noisy environments (you’ll hear the noise mixed with your music) and they’re not suitable for quiet environments with other people nearby (they’ll hear a tinny, audible, annoying version of what you’re listening to). Closed-back and in-ear headphones tend to have a physical seal between your ears and the outside world, with the headphone drivers on the inside of that seal, so they isolate you and your music from the environment — good if you don’t want to annoy nearby people, or if you don’t want to hear ambient noise, but not so good if you need to react to what’s going on around you.

In addition to these different physical and acoustic designs for allowing or blocking sound transfer, there are also active noise-canceling headphones, which are of the closed or in-ear variety (to physically block most sound transfer), but further have microphones and some active electronics to pick up ambient noise, negate it, and play its negative mixed into your audio stream, to really minimize ambient noise. These are popular for use on airplanes or other loud environments.

What I’m looking for here can’t be an open design because those let too much sound out, and it can’t be a passive closed design because those let too little sound in. Instead, what I want is basically a variant of the active noise-canceling design with a reverse switch, so that it uses the same circuitry but instead of negating ambient noise and mixing it with the audio stream, actually amplifies ambient noise (to a degree proportional to the volume of the audio stream) and mixes it with the audio stream. Trouble is, I don’t know of, and couldn’t find, any noise-canceling headphones with a reverse or “uncancel” feature. (When I attempted looking for this, I did find a few people looking for the same thing, mostly for use while running/biking near traffic, and most of the time they were told to use open headphones, since outgoing noise transfer isn’t a concern in that case.)

While I couldn’t find any noise-uncanceling headphones specifically designed or marketed as such, I did find two families of products that do essentially the same thing. The first is video gaming headsets with voice chat support; in addition to headphones they have a microphone for chat, and while that’s there for the purpose of sending what you say to other people, they also mix your own microphone input back into your ears so you can hear yourself talking. The second I wouldn’t have come up with on my own, but was suggested by a gun-nut friend when I posted a question about this on Facebook: electronic ear muffs designed for use around intermittent loud noise sources like heavy machinery and, er, guns, which attempt to block dangerously loud noises but otherwise keep you immersed in your environment. By design, they’re more concerned with incoming sound than outgoing, but the best way of protecting your ears is to seal them away from external sound and then electronically add back only what they think you want to hear, and that sealed design minimizes outgoing sound leakage too.

Having learned this, I tried both of these approaches. I bought Astro’s Mixamp 5.8, which has quite the bag of tricks: it connects to any headphones and makes them wireless; it also uses signal processing tricks to try to make a Dolby Digital signal sound like it’s coming from a multi-speaker surround sound system even though the headphones only have 2 drivers; it also mixes the microphone input back into the headphone output, and even has a mixer dial to fade between “chat” and “game” audio. But you don’t have to use this with a game system; you can also connect the transmitter to a music player, in which case “chat” will just be your own voice, and “game” will be the music.

I also borrowed an Impact Sport Electronic Earmuff from another gun-nut friend. It’s battery powered and by default self contained; it feels like wearing construction earmuffs until you turn it on, at which point you hear a faint hiss as it channels ambient noise from external microphones into speakers on the inside of the sound seal. But it also has an “aux in” jack which lets you feed in music or other audio from any sound source with an analog output.

How’d they work? The Mixamp, not so well for this purpose — it has many other nice properties and does a good job of what it was actually designed for, but I couldn’t get the mic on any headset I tried to pick up enough ambient noise to un-isolate me from the room. Now, the chat headsets I tried have directional mics designed to to pick up the wearer and nothing else, so they’re completely not designed for the purpose I was using them for, and I can’t really blame them for doing a poor job. I also tried a couple of separate omnidirectional mics, but couldn’t get them to pick up any sound at all when used with the Mixamp. I’m no microphone expert, and don’t know what kind of microphone you’re supposed to use with this, but I’m guessing the output level of the ones I tried was too low. Anyway, I think the theory is sound, and if I had the right mic (omnidirectional, sensitive enough, and with the right output level), the Mixamp would do what I want, but I couldn’t get it to work in practice.

On the other hand, the electronic earmuffs? Uncanny. When you first put them on, you can hear almost nothing (like passive ear protection, they’re designed to present a strong acoustic seal). Then you turn them on — and auditorily, they disappear, and you can hear what’s going on around you. Then you plug in an external audio source, and suddenly you’re marching to the beat of a drummer only you can hear, while still hearing what’s going on around you. Exactly the effect I was looking for, and, I repeat, the effect is uncanny. I had no idea things like this existed. Now, this specific set of earmuffs is not exactly perfect for my needs — they’re stiff and not comfortable to wear for long periods, and they’re certainly not audiophile sound quality — graded as headphones they’re not great, but as something that lets me hear ambient noise while mixing in a private audio track no one else can hear, they totally win. And these are just one example of a product category; there are many others available from other companies, some of which are probably more comfortable and/or better sounding. (Some of these are designed for all-day wear for police and soldiers, so they’d better be comfortable.) All at a price, of course. But they do exist.

Moral of the story: these electronic earmuffs are pretty cool. And it’s good to have gun-nut friends.

Diesel

| Comments

When Vanessa and I visited France in the fall of 2010, we spent a week driving around in a rental car. I didn’t think too much about the car when renting it — one broken thing about the rental car industry is it’s really hard to know what kind of car you’re going to get until you pick it up — so I just reserved something cheap, and when we picked it up, it turned out to be a Citroën C4 Picasso with a diesel engine. Nondescript enough, at first, but after a week on the road with it, I noticed impressively high fuel mileage — somewhere above 50 mpg (approximate since I was doing the calculations in my head and converting from liters and kilometers).

This 50+ mpg result was while driving around on the good highways where the speed limit is 130 kph (80 mph). That got me thinking. The only mainstream cars in the US that gets gas mileage like that are gas-electric hybrids (e.g. Prius), and even those not at 80 mph; meanwhile, compared to the Prius the C4 has more usable space, and is more fun to drive thanks to its manual transmission. A car that beats the Prius on performance, packaging and mileage — why hasn’t this caught on in the US?

Fast forward most of a year to when Vanessa and I found out she was pregnant, and I decided that’s enough reason to sell my 2-seater convertible and get something more practical, i.e. with back seats and a hard top. And thanks to that Citroën C4, I decided it should be a manually-shifted diesel.

Once I started shopping, I realized I had a pretty short list to choose from. The US auto manufacturers think diesel is for trucks; there are no US-made diesel passenger cars sold here. The Japanese manufacturers make diesel passenger cars for sale elsewhere but don’t import them to the US. That leaves European companies: Audi, BMW, Mercedes and Volkswagen all sell diesel cars here. But only Volkswagen offers even a single stick-shift model (shame on you, Audi and BMW — I get pretty annoyed surfing audi.de or bmw.de and looking at the drivetrain combinations available in Europe and not here, but that’s a story for another time).

What I would have chosen if it were available here, or if we lived in Europe: either an Audi A3 or BMW 3-series with all-wheel-drive, stick shift and turbodiesel engine. What’s available here in the US combining a stick shift with a diesel engine: Volkswagen Golf, Jetta and Passat.

So I’m now driving a 2011 VW Golf TDI. But before I made that decision, I realized I needed to understand whether and why diesel vehicles are really more efficient. Actually, I had 3 questions: – What is diesel fuel, actually, and does refining it from crude oil come at the expense of usable gasoline? – Why do diesel engines get better fuel economy than gasoline engines? – Why does diesel generally cost more at the pump, compared to unleaded gasoline?

which combined into a suspicion that perhaps diesel is effectively condensed gasoline, and that it costs more and drives you farther because it’s just using up more of the original crude oil energy.

Off to Wikipedia: I needed to read the articles on Diesel fuel, Diesel engines and petroleum and crude oil. It turns out my half-baked hypothesis was well-intentioned but completely wrong. You can read the article yourself, but to summarize and answer my questions: crude oil is a mix of hydrocarbons of various chain lengths, ranging from heavy and less volatile (diesel) to light and more volatile (gasoline, kerosene, etc). Refining oil is the process of separating out these different hydrocarbons; when you refine a given amount of crude oil you get some fixed quantity of diesel, gasoline, and various other products. It’s not the case that there’s a tradeoff between producing diesel vs. the other fuel products; in fact when you produce one (in a modern refinery) you produce them all, and you want to use and sell them all for best efficiency. (Contrast this with the situation in the 1800s, when the only petroleum product we knew how to use was kerosene, so they’d separate that out and throw the rest away.)

This also means there’s no fixed price relationship between diesel and gasoline; the prices are set by market forces and supply and demand; that explains why diesel (in California in 2012) generally costs more than even premium unleaded, but in other places and times it’s been cheaper than regular unleaded.

Finally, the efficiency advantage: it turns out diesel is denser than gasoline, both in terms of mass per volume and energy per volume (this falls out from the fact that they’re roughly equivalent in energy per mass). Diesel’s energy content is about 15% higher than gasoline, by volume. This translates directly to a 15% miles-per-gallon for diesel engines. Additionally, the Diesel cycle is inherently about 20% more efficient than the gasoline powered Otto cycle, apparently due to higher compression ratio.

What this means is that a diesel engine will typically get 30-40% better fuel mileage than a comparable gasoline engine, about half of this advantage due to the energy density of the fuel and half due to the higher compression ratio.

So if your goal is to use less of a scarce natural resource and generate less polluting emissions, diesels do have a real advantage over gasoline engines. A pure diesel powertrain is cheaper and lighter than a hybrid gas-electric powertrain like that of the Prius, and compares favorably for highway mileage, though not so favorably for stop-and-go driving. A hybrid diesel-electric should be substantially better than gas-electric for both city and highway driving, I would imagine (diesel-electric hybrids exist in transit buses, but at this point, there are no such passenger vehicles).

As for my Golf TDI: I get about 42 mpg door to door on my standard commute (37 miles, most of which is 75 mph highway driving). I can also typically go 500 miles on a tank of fuel, which is a real convenience advantage over most cars. I wouldn’t call it a sports car, but it is zippy and fun to drive. Compared to other high-mpg options available in the US, it strikes a pretty good balance between performance and fuel economy — I’d like to see more diesel options available here including diesel-electric hybrids, though this may be a passing phase as pure electric cars become more widely available. (Note that the Citröen C4 I started this post with was significantly more efficient, probably due to a smaller engine. I already bemoaned the lack of powertrain options that the Europeans deign to import here, and chose Volkswagen because they alone import manually shifted diesels, but they’re not immune to the curation effect either; in Europe they sell a smaller diesel engine that gets 50+ mpg, but the US gets only the 2 liter version for which mpg figures in the low 40s are typical.)