Ergonomic keyboards: Kinesis vs. Microsoft

I used to be pretty hardcore as my keyboard choices go. I have a stack of vintage IBM and Lexmark Model M keyboards, and can grade them like a wine connoisseur ("these '96 models just doesn't have the bounce I expect from even a good vintage '93 or '94", even though they're all far superior to the brand new Unicomp models still on the market). But like many computer users, I sometimes suffer from pain in my hands and arms from excessive typing. A while ago I decided I had to give up the Model M, due to both the excessive typing force it encourages and my increasing discomfort with the standard keyboard layout. A succession of keyboards aimed at ergonomic use have followed. I've been though enough configurations of those now to feel comfortable passing on some shopping recommendations, gathered as I considered what I wanted for a second keyboard after settling on a primary one.

Obligatory warning note: please be very careful here! If you're in enough pain from typing to want or need an alternate keyboard, you could have a problem much more serious than just a keyboard change will fix. My own experiments to improve what was diagnosed as classic "tennis elbow" were carefully supported by monitoring during doctor and physical therapy visits, to make sure there wasn't really a larger issue and that I wasn't making things worse in the process. You don't want to end up like the poor author of Your wrists hurt, you must be a programmer, who was left unable to type altogether after poorly executed care here.

Shopping recommendations

There's a lot to chew on below. Here's the condensed version, which answers the most common questions I and others seem to have about the more mainstream Kinesis products:

• The Kinesis Freestyle with VIP kit is my recommended middle of the road pick. It has some parts you'll want even if you decide you want the somewhat more difficult to deal with (and much more expensive) Ascent "multi-tent" kit. This gets you most of the benefits possible here, using the most common ergonomic keyboard positioning, while still offering some flexiblity and upgrade paths. The combination is $129 as I write this at the retailer I bought mine from and would recommend, The Human Solution.
• If you think a more vertical setup might be a requirement for you one day, you can do add that later on the Freestyle. But you really should see how you do with just the VIP setup first, because the Ascent alternative is both really expensive and has its own potential issues. The most I'd recommend you consider spending right from the start is the extra $36 to get the the version of the keyboard with the wider 20" separation, because that you're going to want if you want to go vertical one day (but really only in that situation, so don't consider that important either)
• If you're OK without so much flexibility or really need PS/2 support, consider the Kinesis Maxim keyboard. It's about the same price, is easier to move around due to its more integrated design, and it supports the position you're likely to settle on with the Freestyle anyway.
• Want to experiment with a more ergonomic keyboard design to see if it helps you, but without committing so much money at first? Microsoft's Natural Ergonomic Keyboard 4000 is a decent cheap (under $50 if you shop around a bit) option here, albeit without a good feel for presses of individual keys.

Now we head backward toward how I came to these conclusions.

Keyboard ergonomic issues

In order to put all this into some context, first we need to talk about what's wrong with the traditional keyboard. Keys that require less force to activate are always good, but I have those on my laptop and it's not really comfortable to type on it, so that's clearly not enough. The main thing I've come to appreciate is that our forearms aren't designed to be horizontal for long periods of time. Try this experiment: let your arms fall loose to your sides. Note the position your hands are in relative to your arms? Your thumb is forward, your hands lined up straight with your forearms. Now hold your arms in front of you. Your arms feel more comfortable with your thumbs are pointing toward the ceiling, right? That's a natural position your body is comfortable with.

Now note how your hands and arms move when you use a standard keyboard. There's a 90 degree twist from your natural position. Your hands might be bent backwards some too; that's not good either, as it tenses the muscles in your forearms (which then pull on things all the way up to the elbow). And unless your arms are much more flexible than mine, you probably are more comfortable with them further apart than when you're using a standard keyboard, where you have to bend your elbow further inwards than is really ideal in order to get both hands on the home row. Kinesis has a good description of the various problem areas here in their awkward postures document.

On a particularly painful day where it hurt just to rotate my hands into the typing position, I noted that all it took was returning to thumbs-upward and separating them to make that feel better. The first question this raised for me was whether I could type like that.

Kinesis Ascent

You sure can; the Kinesis Freestyle with Ascent accessory (also called the "Multi-tent" kit) lets you split the keyboard in half and push it straight up if you want. The product catalog on their site isn't all that great, I found the shopping experience at reseller The Human Solution easier to navigate and order from (cheaper than direct from Kinesis, too).

The Ascent is a really expensive upgrade for the keyboard. I found it valuable for mapping out what my options were, because you can setup just about any angle/position combination with it. Ultimately I wouldn't recommend it for most people though. For me at least, the really sharp angles (near the 90 degree vertical position shown in the product picture) were hard to type on for a couple of reasons. While I touch type, I learned just how much I look at the keys for punctuation when in this position, because it's hard to do.

The biggest problem with this accessory is that while well constructed, it's still just sheet metal. There was too much flex in the design for me to really be happy with it, particularly in its vertical position. I had to tense more muscles than I expected to type that way, because I had to be so careful not to press too hard toward the center. It feels like you might collapse it inward, even though that's actually hard to do, because it does give a little in that direction. And angles short of vertical didn't turn out to be very useful to me either. Once I made the angle greater than around 20 degrees, I couldn't get a comfortable position until I reached >70, at which point I was back to being worried about collapse toward the middle again. At small angles, it was much sturdier, but there was still just a bit more play than I like. What I wanted instead was to adjust into the right position, than make it really solid in that spot. That's the approach more explicitly taken by some of the other competitors here, like Goldtouch; there's a good blog entry discussing their products, and I ruled out Goldtouch because I wanted the option of being able to separate the halves (even though it turns out I don't really need that right now).

One part of the Ascent design aimed to improve stability is a metal connector plate that attaches to the two keyboard halves. I found this to be a bit sketchy in that the halves aren't locked into place as firmly as I'd like (it's just a couple of bolts). And it also has the problem that it enforces lining up the two halves to be straight. The stock Freestyle keyboard comes with a removable "pivot tether" connecting the two halves of the keyboard. That allows splaying the two halves of the keyboard apart from one another at an angle, which I found works better for me than trying to keep them straighter. You certainly can splay with the Ascent by just not using the connector plate, but now you're dealing with two completely disjoint sections. The biggest issue I found with that is repeatability: without a way to lock into the position I want to use, the two halves tended to drift toward sloppy and bad positions without me noticing. Moving the keyboard around is a nightmare too without the connector plate.

Microsoft Ergonomic Keyboards

It's hard to ask the day job for a keyboard combination that approaches $400 with all the fixins when I wasn't even sold on it myself completely. The last two years I've mainly worked at home, but sometimes visited my far off company office for a week or two. In parallel with investigating options for my home keyboard where most of my job happens at, I also wanted a cheaper solution for my desk at the office.

After two long trips typing on every keyboard setup to play with at both Fry's and Micro Center, the cheapest option that I liked at all here was the Microsoft Natural Ergonomic Keyboard 4000. These are regularly available in the sub-$50 range even at retail (I've grabbed one on sale for $40 at both Fry's and Best Buy), and Microsoft has some other models available too you might consider as well. There's a good commentary praising the keyboard in the demanding Emacs context in a review that I found informative.

This keyboard gets a couple of things right. Note that there is some separation between the two halves of the keyboard, and that it slopes vertically downward from the center toward the left and right edges. These are the two things I found most useful about the Kinesis keyboard. What's really interesting is that the angles the Microsoft keyboard fixes those at are extremely close to what I settled into using even in the much more adjustable Kinesis+Ascent combination. The Microsoft keyboard also gets one detail perfect that Kinesis doesn't handle on their Freestyle design: it slopes downward from the nicely integrated wrist wrests to the function keys. Now, it turns out I don't like this as much as I thought. The most comfortable keyboard position I've found puts the keyboard in my lap (I'm using a Gamer's Desk w/Max Mousepad). The integrated front/back slope of the Ergonomic 4000 starts out too high for that to work really for me. I think you really need to be up higher and have the keyboard start below your lap, perhaps with an office chair and a keyboard tray, for a downward sloping keyboard to be optimally placed.

Ultimately, I never got comfortable with this Microsoft keyboard. My main issue is that the big keys on the bottom, mainly Alt and Space, have awful key action on them. The space bar clunk and doesn't feel right, and I constantly missed key chords using Alt in them because I didn't press the giant but sloppy key down fully. Having to press that hard is something I can't take in a keyboard, particularly for multiple key actions. The source for the positive review I linked to above apparently isn't bothered by this because he hits alt with his palm, which I just can't get used to (and isn't compatible with my goal of not being too different from what I do on a laptop). Ultimately I'm left thinking Microsoft's Natural Ergonomic Keyboard 4000 has the right layout and position, but ultimately the cheap keys it uses are disappointing. But if you don't want to spend serious bucks on a keyboard, or want to try out more ergonomic positioning but aren't committed to it just yet, it's cheap enough that you might find it worth trying out. The keyboard is common enough that you might easily find a store who carries one you can try out too.

Exotic Options

Kinesis does make keyboards that aims for better positioning in the front to back dimension too, where sloping downward away from your body is acknowledged to work out well in many situations. Their Advantage Pro is a very nice keyboard I was able to try out via a coworker who loves his. I ultimately rejected that choice because it's just too different from the standard keyboard to be comfortable for me, and since I do have to balance my time with a healthy dose of laptop keyboarding I couldn't see myself ever really getting used to it. If that's not important to you and you don't mind some retraining time, it's an expensive but quite nice product. It does get the downward curve bits better than the Freestyle model I settled on, and I think it has the potential to be more ergonomic in the end if you can accept those trade-offs.

Even the Advantage Pro seems quite familiar compared to the really difficult to get used to DataHand, which I was also able to borrow to evaluate for a bit at one point. The DataHand I can only see making sense if you're so bad off that you really can't handle holding your hands/arms in a standard position at all, or can't press down/move around anymore and it's easier for you to only move your fingers a little bit (you "click" a set of tiny switches in each direction around your finger to type). The retraining time and difference between other keyboards is really substantial on a DataHand. That's really not a cheap option either, if you can even buy one.

Revisiting the Kinesis Freestyle

What I came to realize here is that even with all the flexibility available with the Kinesis Freestyle with Ascent accessory, in practice what worked out well is basically the same positioning the newer Microsoft ergonomic keyboards provide. Somewhere between 10 and 20 degrees of slope downward from the center is enough to get rid of the worst of the rotation issues a standard horizontal keyboard introduces, and a moderate split between the two halves is probably all you need unless you're going full-on vertical (or have an enormous chest and tiny arms like a T-rex). Kinesis knows this perfectly well too; their Maxim keyboard provide a very similar configuration to the one settled into, ready to go as an integrated unit.

There are two viable ways to convert a Kinesis Freestyle keyboard to this sort of position. I already purchased their Freestyle VIP kit, which is the cheapest way to get a set of the wrist rests to go with the Ascent kit too. After giving up on the Ascent, I used the other portion of the VIP kit, the risers providing either a 10 or 15 degree lift, and found those to be plenty stable for typing on. While the range of adjustment here is limited, I think Kinesis and Microsoft have correctly nailed that the common case is going to want something in that range anyway. A Kinesis Freestyle with the pivot tether installed for good splay, risers at 10 degrees, and the wrist wrests is quite similar to the only position that's offered on the Microsoft Ergonomic keyboard and Kinesis's own Maxim, and unless you really have exceptional needs that required moving toward full vertical I think this is the best option available. Since I find myself more comfortable with the 15 degree lift position, I decided not to get the even more stable Kinesis Freestyle Incline platform (fixed at 10 degrees).

The only thing I wish was more adjustable on the Kinesis Freestyle is the front/back slope, which is fixed at a moderate upward slope. As I mentioned before it's hard to get a downward one to work well without a keyboard tray anyway, so that's not that critical to me, and I wasn't happy with the models on the market with an integrated slope that way.

That's where I'm at with keyboards now: Freestyle with VIP kit. Works great, wish it was a bit easier to move around though. I am concerned that I'm going to knock it off my desk one day and break the whole middle tent connector in particular. Some of the other Kinesis options here (the Maxim and the Freestyle with Incline kit) look more sturdy in that respect, if that's important to you they're worth considering. I like the extra flexibility of the Freestyle, but as I've mentioned repeatedly I don't think it's really necessary for most people; just nice to have.

P.S. Yes, I've done the same level of research and tests on ergonomic mice too; will cover those next time I get some time to write on this subject.

Running cron on Ubuntu

Automating regular admin tasks with cron is a great way to handle all sorts of chores. Every day systems around the globe e-mail me cron reports showing their backups were successful and a report of how many bad guys tried to break in (by the obvious front door of ssh at least). I run an Ubuntu desktop at home, and I'd like to automate tasks on it with cron as well. Here's a quick guide to the Ubuntu-specific bits required to get cron going, presuming you're already familiar with it on other systems.

The Ubuntu wiki has a Cron How-To. When I started my night, the description in the "Enable User Level Cron" section was wrong. It suggested that the default setup didn't allow regular users to add jobs with "crontab -e" and have them execute, and that you first had to setup cron.allow for them. This may be true on some other system, but it's not the Debian or Ubuntu default. If you run into this restriction, you'll know it--running crontab will throw the error right in your face. So if that doesn't happen, but your jobs aren't running, the allow/deny section isn't your issue. I updated that section to match the description of the behavior here you'll see with "man crontab".

I also updated the how-to with the troubleshooting bits I either needed this time or have run into in the past. I'm saving the parts I added here in case somebody decides they're not worthy and deletes them:

Troubleshooting cron problems

• Edits to a user's crontab and jobs that are run on their behalf are all logged by default to /var/log/syslog and that's the first place to check if things are not running as you expect.
• When adding a new entry to a blank crontab, forgetting to add a newline at the end is a common source for the job not running. If the last line in the crontab does not end with a newline, no errors will be reported at edit or runtime, but that line will never run. See man crontab for more information. This has already been suggested as a bug.
• If a user was not allowed to execute jobs when their crontab was last edited, just adding them to the allow list won't do anything. The user needs to re-edit their crontab after being added to cron.allow before their jobs will run.
• When creating a crontab for the root user, the user name must be specified as a parameter after the date/time parameters. Accidentally including the user name that way in a user-specific crontab will result in trying to run the user's name as a command, rather than what was expected.
• Entries in cron may not run with the same environment, in particular the PATH, as you expect them to. Try using full paths to files and programs if they're not being located as you expect.
• The "%" character is used as newline delimiter in cron commands. If you need to pass that character into a script, you need to escape it as "\%".

Mail output

The behavior I expect from a UNIX system is that if I run a job in a crontab, and that job writes something to standard output, that output will be e-mailed to me. You're supposed to get that on Ubuntu, via the Exim4 Mail Transport Agent; see Setting Up Your System To Use E-Mail for details. If that's what you've got, you may get mail that cron sends just fine, to the root-ish user setup by the default installer.

But, if like me you installed something that pulled in the postfix package as a prerequisite (but didn't bother configuring it at the time), you'll discover cron output doesn't get mailed to you. Instead you'll see something like this in /var/log/syslog:

Nov  5 20:33:01 gsmith-desktop /USR/SBIN/CRON[26173]: (root) CMD (/home/gsmith/test)
Nov  5 20:33:01 gsmith-desktop postfix/sendmail[26176]: fatal: open /etc/postfix/main.cf: No such file or directory
Nov  5 20:33:01 gsmith-desktop /USR/SBIN/CRON[26166]: (root) MAIL (mailed 5 bytes of output but got status 0x004b )

This tells us that even though postfix, the mail server, is installed, because it hasn't been configured it isn't doing anything. That's kind of poor behavior just because I was too busy to fool with it at the time that prompt flashed by; what I'd expect is that the default would provide local delivery only until I wanted something better. Here's a simple guide to a minimally install and configure postfix:

sudo apt-get install postfix
sudo dpkg-reconfigure postfix

You can answer its questions like this to setup what cron needs to deliver to your system:

• Choose "local only"
• Enter your user name for "Root and postmaster mail recipient"
• Use the defaults for the rest of the questions

Now you should see that mail is being delivered in the log files when jobs execute. In order to read the mail, you might want the standard UNIX 'mail' utility, which also isn't installed by default; fix that like this:

sudo apt-get install mailutils

And that might also help random UNIX programs you install that expect 'mail' is available to send you things. Alternative text-based mail programs include alpine and mutt. GUI mail programs like thunderbird and evolution could also be used, although those are pretty heavyweight just to read the kind of local mail cron generates.

If you want your server to mail to instead deliver mail to the outside world, that's a whole nother level of complexity altogether. I suggest getting local delivery going first, to confirm that cron is working normally, before launching into that adventure. Then you can deal with the fun that is the Postfix setup documentation. Good luck with that.

PostgreSQL at the LISA conference in Baltimore

This week the Usenix LISA Conference is running in downtown Baltimore. There will be a PostgreSQL booth in the exhibition area from noon-7pm on Wednesday and from 10am-2pm on Thursday. Robert Treat is the lead elephant for this show, and is too busy with booth setup to have time to write fluff pieces like this one. I'm co-hosting, and we have some other volunteers you might see too. We're basically the least attractive booth babes ever, but if you're looking to talk about open-source databases and in the neighborhood, we're more informative than the local dolphin alternatives.

This week is also my first working for 2ndQuadrant, as the latest addition to their global PostgreSQL and replication consulting staff. If you're in the US, were interested in 2ndQuadrant's services or array of training classes, but figured that would be too hard to coordinate with their UK or Italian offices, that's something I can help out with now.