The old fastball special!

One of the fun things about reading older comics is noticing early versions of things that became popular later.  The DC compilation "The Greatest Team-Up Stories Every Told" is filled with firsts.  I picked it up for the reprint of the 1961 Flash #123, "Flash of Two Worlds".  That's where the DC multiverse was introduced, a concept that would later be expanded into "Crisis On Earth-One/Earth-Two" (also in that collection), "Crisis on Infinite Earths", and eventually "Infinite Crisis".

I found a fun surprise there when reading "Superboy Meets Robin", from Adventure Comics #253, October 1958.  At one point Superboy throws Robin at a bad guy:

Any good comic book fan can tell you this is a move called the Fastball Special, where a strong hero throws another one toward a target.  It's most associated with Colossus throwing Wolverine in the X-Men comics, going back to X-Men #100 in 1976.  Seems it was already a pretty old move by then though!

Marvel comics used this move before then too.  In Amazing Spider-Man #19, December, 1964, Spidey is fighting a 3 man team of bad guys named The Enforcers.  The large muscle-bound one, Ox, throws martial artist Fancy Dan:

This comic is available in some of the Marvel Masterworks collections, such as The Amazing Spider-Man Volume 2.

The FDA and our government

What's right, what's wrong, and what that says about the US Government

It's easy for anti-government comments from Libertarian sources to inflame a debate focusing on good work done by our current government. Watching the reaction to a profane, vulgar, and hilarious rant from Penn and Teller supporting the Ron Paul campaign provides a good example that focuses on the Food and Drug Administration. It's easy to say that's a ridiculous agency to pick on, when the FDA is obviously doing a useful job for so many people. But there's a deeper message here about what's wrong with our government. The question this video should raise is not whether the FDA is helping anyone; they are. It's why High Fructose Corn Syrup is ubiquitous in US foods, but not in other countries?

Let's start with what the FDA does right. I recently found myself scolding someone who was advocating to an ill friend a quack herbal remedy: "colloidal silver". Through a funny twist, some people only know of this "remedy" because of its association with harming Libertarian candidate Stan Jones. Watchdog organizations like Quackwatch make it easy to debunk the claims of those selling this product to make a profit from the uninformed. In a true Libertarian environment, that sort of information gathering police work, combined with the right judicial environment for individuals to sue those making phony claims, would be sufficient to prevent these products from reaching a large market.

The FDA is serving as exactly that sort of watchdog in this case. One of the reasons ingesting this toxic brew isn't more popular is because the times their labeling and advertising has gone too far, the FDA in combination with the FTC has smacked them down, hard. "Big government" in Australia has done that one better, legislating even further reaching defenses against this sort of fraud. People like me know better, but ultimately if there wasn't an FDA, we'd be compelled to reinvent something like it. It's quite reasonable to say we can't expect individual people to cope with the scale of the problem.

It seems an organization whose job it is to collect information about good and bad things you might eat or drink, and a related one for medication, is a necessary component to managing the complexity of modern life. They would need the ability to sue on behalf of public safety when both uninformed sellers and immoral criminals tried to sell the clearly bad products to uninformed people. We have one right now, and most of the time it does the right thing. But as Penn Jillette's angry rant points out, it's not done a thing to protect the public from the obvious, documented dangers of high-fructose corn syrup. Why hasn't the FDA acted to rule HFCS harmful?

The problem here is that the legislative and organizational structure that ultimately defines the FDA is set and managed by people whose concerns go beyond the public interest. They're motivated heavily by their corporate sponsors. In the HFCS case, that's a long list going from all the food manufacturers saving money using HFCS to the corn industry producing it. It's even easier to see this at work on the drug side of the office.

In 2004, FDA whistle-blower Dr. David Graham reported the agency as "virtually incapable of protecting America". One of the drugs he called out as unsafe, Accutane, was removed from the market in 2009. Was it because the FDA shut them down? No; it was because of the massive number of personal injury lawsuits against the company. If drugs this dangerous are not being stopped, if it takes private lawsuits to get them off the market, you have to ask just what the FDA is really doing. Isn't saving individuals from needing to file their own lawsuits to eliminate dangerous items from the market the whole reason they exist?

Dr. Graham's comments came from a look into the investigation over Vioxx being withdrawn from the market, its own sad story of lax FDA work. There's more information about that available from the Union of Concerned Scientists at and further comments from him in another interview. There Graham describes how the FDA "views industry as its client, and the client is someone whose interest you represent". This is not a problem isolated to the FDA.

The easiest such case to follow is also one of the biggest, around the banking industry. Here the Securities and Exchange Commission serves a similar role to the FDA, or at least it would if it weren't actively manipulated by the people it's supposed to police. That the corporate influence of what's called the "revolving door" of the SEC prevents it from acting to protect the public is also well documented. And the stories told by SEC whistle-blowers sound awfully similar to the ones heard about the FDA.

What our current government is demonstrating in many areas is that if you allow elected officials and their offices to accept large amounts of money from companies, they will then create favorable conditions for those companies to make more money--and therefore continue financing the official or office. On the elected official side, since it's almost impossible for anyone who isn't playing that game to pay for enough advertising to win an election, this is an almost unbreakable vicious circle. It exists across every part of our government right now. This situation where corporations are seen as the "client" of a government office--rather than the voters--is far from unique. It's ubiquitous.

And it's obvious that problem goes all the way to the top of the US government. Does anyone really think the Bush and Obama administrations were capable of confronting the large financial firms that caused so many problems with the US economy, when money from them was a necessary component for both presidents to be elected? Every single one of the discussions around the TARP bailouts should have included these two tables so it was clear the money being distributed was advocated by presidents on the payroll of those receiving it: Bush contributors, Obama contributors. These companies aren't favoring the Republican or Democratic sides; to be safe they purchase as much influence as they feel necessary from every candidate who will accept it. A vote for either is supporting the same kleptocracy. If you'd like to see what a candidate sponsored by the voters looks like for comparison, see Ron Paul contributors.

The real problem isn't that there is a FDA and it's sponsored by pooled government funding. The problem is that its agenda is ultimately influenced by how much money everyone from "big pharma" to the corn industry passes through the hands of our legislators and our government offices. The extreme positions Libertarian candidates and their advocates sometimes use is reflecting the fact that every single one of our government organizations, as they exist right now, are pushed by forces that do not have the good of the voters as their primary motivator. People can be conned into doing things if you have enough resources to convince them. There's this thing called advertising, turns out it works really well. And right now we're a nation that has been conned into voting for a massive tower of corporate influenced corruption, top to bottom, every agency, as our government. We're now letting corporations act like people in every way, with campaign contributions and lobbying as a proxy for voting (which, again, turns out to work really well). Actual people are letting themselves get outvoted every time. And by definition a corporation has no morals.

The idea of little or no government may not really be viable across our large and diverse population. Advocating too strongly in that direction enables a debate based on the merit of the existing organizations. The fly in the Libertarian ointment is that the percentage of the self-interested and outright criminals in our world is so high, with these obvious examples going from herbal medicine quacks to banking industry leaders, that it's hard to imagine any way to stop them all without building a big organization. It seems the easiest way to make the cost of policing them, and they will pop up in every single place possible, is to build something that looks an awful lot like government.

But the government we have right now has already been actively reorganized by literally amoral corporations to benefit their needs. Don't let the fact that some parts of our existing government are functional despite that fool you. There are good meaning people at all levels too, and organizations like the FDA are helpful more often than harmful. But for every FDA, there's an SEC--a regulatory body so warped by corporate influence that they fail to protect the voters from the very things they were chartered to do.

The real reason to single out the FDA as a source of trouble is to highlight that even there, an organization which for the most part does enormous good for our population, there is massive corporate-driven fraud driving its agenda. Does our entire government need to be wrecked to completely eliminate that taint, to be restructured as something that really is driven by the voters this time? That's not a question you should just shrug off as ridiculous. And arguing over the details of which parts of our government currently do something useful despite that is missing the point.

Copyright 2012, Gregory Smith gsmith@gregsmith.com

Use of this text is allowed under the terms of the Creative Commons Attribution 3.0 license.

Rhythmbox playlist editing with the magic of command-line diff

As a near full-time Linux desktop user, I haven't found anything I like better for music than GNOME's Rhythmbox. The UI is a little funky sometimes, but all I really want from a music player is the ability to find all the songs in my library and make playlists.

One of the problems I run into sometimes relates to my fanatical music ripping. I extract all the audio from my CDs in both FLAC, for high quality, and MP3 formats. I only put the FLAC version into the main music library, the MP3 copies are strictly for copying over to my portable player (Sansa Fuse, which also works fine with Linux with no special software). Every now and then I make the mistake of adding the directory that contains the MP3 files to my music library, and then I'm screwed. There's now two copies of every song, and weeding them out is a giant mess.

When I did this again recently, decided to just wipe my whole library out and start over. I added most of the same songs back in again. One ugly surprise though: all of my playlists were deleted! Now that I know what not to do here, I wanted to share that info.

One of the things I like about Rhythmbox is that all its metadata is stored in simple XML files, so I've recovered from errors like this before. Depending on what version you're running, below your home directory should be .local/share/rhythmbox/playlists.xml or its older variation, .gnome2/rhythmbox/playlists.xml

Since I'm paranoid, I made a backup of this file and the music library before I touched anything, so I had the original playlist file with all the songs for reference. Apparently what happens here is that when you exit Rhythmbox, it removes any file in a playlist that isn't in the library anymore. So the procedure I had to go through went like this:

1. Restore the original big playlist file
   
2. Add the directories I think it was missing to the library, then exit
   
3. Compare the original playlist file with the new one, to see what files are missing.
   
4. Repeat until no files are missing.
   

In command line form, that looked like this:

cp $HOME/backup/playlists.xml $HOME/.local/share/rhythmbox/playlists.xml 
rhythmbox
kdiff3 /home/gsmith/personal/music/playlists.xml /home/gsmith/.local/share/rhythmbox/playlists.xml &

After a few rounds of that, I got to where the difference between the original and new playlists was down to only three files. The common thing about these files is that they had punctunation characters in them: comma and ampersand, AKA ",&". For reasons I haven't fully figured out, when I added those files back to the library, the format it saved those names in was escaped slightly differently. So they were in the music library, but didn't match the playlist perfectly, and thus deleted at every exit.

To fix this, I manually copied those files from the library back into the playlist again. Then I exited the program and figured out what order they used to be in like this:

diff -c $HOME/backup/playlists.xml $HOME/local/share/rhythmbox/playlists.xml

This form of context diff makes it straightforward to see what lines the song originally appeared in. I tweaked the file using a regular editor (vi worked fine) until the differences were all adjacent lines, so that the new file names were directly replacing the original ones, comfirming the edits with that same diff again. Save that, and finally my original playlists are back in the order I liked them in.

Now that I realize how easy it is to lose playlist entries, I've now added playlists.xml to the list of files I keep under version control. One last twist here to be aware of. Normally, the way I do that is put the file into my personal git directory, then symbolically link the original location to it. The version of Rhythmbox I have here does not respect this at all. When exiting and saving, it silently overwrote the configuration file with

This "wipe out everything I don't like when exiting" behavior from Rhythmbox is rather immature, given it's a program that could be running with a music library mounted over intermittent network storage. And not following symlinks is just absurd. But, at least with plain text, readable XML files, I can use standard UNIX tools to fix all those limitations. It's still far less stupid to recover from than what happens, say, when you screw up your iTunes library after running into the same sort of limitations, like bad behavior with intermittent network mounts. Binary configuration files suck.

Toner ripoffs on the Brother HL-4040CDN

I bought a Brother HL-4040CDN color laser printer about two months ago. Knowing I had a whole book worth of proofreading to do, I knew a lot of printing was coming. And the way my publisher marks things up, I need to see some color on each sheet to confirm formatting is right; some words are highlighted in red or blue. We're talking a few words on each page, typically.

After exactly 2310 color pages printed, the printer declared that all three toner cartridges (CMK) were empty. This was of course nonsense; there's no way I burned through that much color toner, or that all three went on the same page. This model has three rounds of defense mechanisms against using cartridges longer than they want you to:

1) Page counters per toner cartridge. This what I ran into. To fix, open the bay where the toner is stored. Hold down the Cancel button and hit Reprint. This will bring up a maintenance menu. Scroll through all 8 entries there, selecting them with the right arrow to reset the counts. Close the front again, wait a bit for it to recalibrate, and you're past this problem. As I hadn't used even a tiny fraction of the real toner yet, this is all it took for me.

2) Optical sensors for the amount of toner left. Once toner gets low, these will trigger. You can likely still print for some time before it actually runs out though. To defeat, pull out each toner cartridge. On each side, there will be a clear, round plastic window that you can see the colored toner through. Cover these with something opaque, like black electrical or duct tape. Then run through the page count reset procedure. You should be able to print again. If you notice a color starting to print less accurately, you might get some more life out of it with the usual "shake the cartridge" trick. But you don't want to print too many pages with your printer in that state, once it really has run out.

3) Gears that advance forward as you use the toner, to set how much voltage the cartridge needs to provide to print with the remaining amount in there. You definitely need to reset these if are refilling the cartridge; see refill instructions for details. I'll have to see how many pages I've printed before I reach this point before I decide if I feel this is a good idea or not. I'm not going to refill the toner, but if I suspect there's still more in there I might try it. If I really am out of toner, I don't have a problem buying more; I just don't want to get blatantly ripped off.

There is an excellent albeit rambling forum discussion covering this issue for a number of Brother printers you may find useful for additional details here. Thanks to them and to Amazon reviewer Sang Joon Lee, who turned me onto this idea before I'd even bought the printer, while researching whether it was a good idea or not. If some playing with the service menu and electrical tape is all I need to make the unit economical, that doesn't bother me if everything else about it is OK. And that's been the case so far. This printer doesn't have great print quality like the old, dead QMS it's replacing did, but it's completely acceptable for the sort of general business printing I do in color. And it works great under Linux.

A tale of three power supplies: Antec, Seasonic, Corsair

I've bought a lot of Antec cases and power supplies over the years. The original Antec Sonata was the last one I was really satisfied with though. So long as you added your own internal fan to the hard drive area, it was really great. The Sonata II is where things started to go wrong for me and Antec. They jumped onto the stupid "put a duct in it" train, and the power supply didn't look so compelling compared to the alternatives anymore. During the few years after that, increasing reports of low quality power from the actual power supplies let me further away from Antec. Off my vendor list they went.

Back in 2008 I used a CoolerMaster Cosmos 1000 case to build a giant server, with a primary goal being that it was quiet. Got a Seasonic power supply. Worked well at first. Then the power supply fan started buzzing because the blades were hitting the guard. If I smacked it I could usually free them, so put up with that for a while. Eventually, one of the internal plastic guards fell apart altogether inside the power supply case, making it dangerous to even try and start. RMA time...or so I though. After about 4 tries via different methods (web page, e-mail), I came to the conclusion that Seasonic products don't actually have a warranty. They claim to, but you can't actually get them to honor it. Eventually I took the whole thing apart, repaired the piece inside, inserted some spacers between the fan and the guard, and better than new. Seasonic: decent product, minor flaws I would normally forgive; worthless warranty puts them on the banned vendor list though. Have since heard the same warranty saga mess from someone else, so I know it's not just me.

Recently when I wanted to put together a decent mid-tower system, I found myself back at Antec's Sonata III. I refuse to buy a case without a cover over the front drive bays, both for dust and noise reasons. That's the part that's usually most exposed to the room, and I don't want every external drive bay to be a hole for dust to go in and noise to go out. The CoolerMaster cases where the whole front is exposed are the worst in that regard; every decibel of hard drive noise comes right out of those. I made the mistake of buying a CM Stacker once that was unusably bad, making me certain every reviewer who was enthusiastic about it was a quiet case was either deaf or a shill for the company.

The Sonata III was one of the few smaller cases left with where you could block everything on the front, and that had decent drive bays too. I tinker a lot, so easy to work on in a priority for me, thus the hatred of ducts. It came with their "EarthWatts 500W" power supply. Seemed more than sufficient for what I was putting in there: motherboard, lower power processor, tiny video card, and four hard drives. Maybe it was time to give Antec a second chance.

Ever since assembly, I noticed that periodically the system rebooted itself. Very intermittent, couldn't track down the reason. After a recent move, the reboots became extremely frequent; almost every day. And then I noticed what was happening: each time I turned on the stereo in the same room as the PC, it rebooted. I know the amplifier draws a lot of power, but seriously? The PC is plugged in a giant APC UPS, and that's enough to take it offline? What the hell? At first I thought this might be related to the now well known grounding issue with the Sonata III. The newer model I have definitely has the second generation design where that problem is supposed to be fixed (I bought the case anyway presuming that was a standard early adopter issue already resolved), but some report it's still an issue. My reboots always happen when I'm away, though, not when touching the front.

Maybe the APC UPS was broken...plugged in the USB cable, loaded the Linux monitoring software. When the stereo was powered on, output line voltage dropped from its normally overfull 122V to 117V. While available amperage at that voltage probably was dropping, too, a power supply has to be seriously deficient in capacitance to not survive a brownout that gentle.

So what to replace it with? Well, Corsair seems to have grabbed a lot of positive mindshare in a short period for their power supplies the last few years, particularly for a company that was just jumping into that market. Found out why: they're rebranding supplies from two other manufacturers. From what I've been able to gather, the Corsair VX550, TX750, TX850, and HX1000 are made by Channel Well Technology, AKA CWT. You can see confirmation of that in a torn apart TX750. But most of the rest of their models are made by...my old friend Seasonic! The spcr review of the Corsair TX650W confirms it's one of those models.

The TX650W is good enough to have made Silent PC's recommended list too. And I know Corsair support is excellent if I have a problem that requires warranty service or the like, from dealing with them in the past buying memory. Best of all: I could impulse buy one at my local Best Buy, where that model is stocked in some stores. It was even on sale that week to be cheaper than Newegg. Out the door I went. After a quick swap, the rebuilt system with Corsair TX650W has never rebooted unless I asked it to. Power on the stereo, leave it running a while, it just stays on like it's supposed to.

So: new guidelines. If you want the otherwise nice Seasonic designs, figure out what Corsair model that uses one of their designs is closest to your requirements and buy it. This is now my preferred power supply choice for all systems I build. Avoid the Corsair models with the Channel Well supplies instead for now, they're really unknown quantities as far as I'm concerned. And never, ever buy the Seasonic designs directly instead. As for Antec: between the grounding issue still lingering around, and now discovering this EarthWatts power supply is completely worthless, I've realized that this Sonata III is just as badly engineered as the problematic Sonata II. The problems just weren't as obvious. Antec, you are right back on the banned vendor list--again.

Testing PostgreSQL 8.5-alpha3 with peg

PostgreSQL 8.5-alpha3 was announced last week. The biggest single feature introduced in it is Hot Standby, which allows you to run queries against a server that's being used as a Warm Standby replica. Since you can make any number of such replicas from a single master database, this introduces a whole new way to scale up PostgreSQL server farms in situations where you can live with queries that won't necessarily have the very latest data in them, due to replication lag. For example, it's a great way to run large batch reports like daily business summaries against the standby, rather than beating the master server with that load. Just wait a little bit after the end of the day for the transactions to copy over, then kick the query off against the hot standby system.

Actually getting two servers up and running so you can test this feature can be a drag though, since the alpha releases aren't necessarily going to be packaged up for you to install easily. In order to test a new version of PostgreSQL built from source, there are a fair number of steps involved: checkout the source, build, create a database cluster, start the server, and then you can finally run a client. Each one of these has its own bits you likely need to customize, from needing a directory tree to keep all these pieces (source, binaries, database) organized to source configuration time options.

After a few years of building increasingly complicated scripts to handle pieces of this job, recently I assembled them Voltron-style into one giant script that takes care of everything: peg, short for "PostgreSQL environment generator". The basic idea is that you give peg a directory to store everything in, point it toward the source you want to use, and it takes care of all the dirty work.

The best way to show how peg can help speed up development and testing is to see how easy it makes testing the latest PostgreSQL 8.5 alpha3:

$ git clone git://github.com/gregs1104/peg.git
$ sudo ln -s /home/gsmith/peg/peg /usr/local/bin/peg
$ mkdir -p pgwork/repo/tgz
$ pushd pgwork/repo/tgz/
$ wget http://wwwmaster.postgresql.org/redir/198/h/source/8.5alpha3/postgresql-8.5alpha3.tar.gz
$ popd 
$ peg init alpha3
Using discovered PGWORK=/home/gsmith/pgwork
Using discovered PGVCS=tgz
Using tgz repo source /home/gsmith/pgwork/repo/tgz/postgresql-8.5alpha3.tar.gz
Extracting repo to /home/gsmith/pgwork/src/alpha3
tar: Read 6656 bytes from -
Extracted tgz root directory is named postgresql-8.5alpha3
No checkout step needed for PGVCS=tgz
$ . peg build
$ psql
psql (8.5alpha3)
Type "help" for help.

gsmith=# \q

That's 9 lines of commands to go from blank development system to working psql client, and half of that was installing peg. The above downloads and installs peg (in /usr/local/bin, you may want to use a private binary location instead), grabs the alpha3 source (from the US, not necessarily best for you), and build a server using the standard options for testing--including debugging and assertion checks. If you want to do performance tests instead, you can do that with peg by setting the PGDEBUG variable to something else. That's covered in the documentation.

Warning: if you try to do this on a RedHat/CentOS system, you will probably discover it won't work. PostgreSQL 8.5 requires a newer version of the Flex tool in order to build from an early source code build now than is available in RHEL5 and its derivatives. I've got some notes on Upgrading Flex from source RPM to compile PostgreSQL from CVS you can read. There were official RPM packages of alpha2 released that bypass this problem, you may be able to get an alpha3 set from there in the near future too.

My goal was to make learning how to use peg pay for itself in time savings the first time you use it for testing a PostgreSQL development snapshot. The script still has some rough edges, but I've been using it regularly for over a month now with minimal functional issues. The documentation is almost complete, even showing examples of how to use peg to create two PostgreSQL installs on the same host--which allows testing hot standby even if you've got only one system.

peg is hosted on github, I've heavily noted the things that need improvement with TODO tags in the source and documentation, and patches are welcome. Hope it helps you out, and that you'll be joining the 8.5-alpha3 testers.

Ergonomic keyboards: Kinesis vs. Microsoft

I used to be pretty hardcore as my keyboard choices go. I have a stack of vintage IBM and Lexmark Model M keyboards, and can grade them like a wine connoisseur ("these '96 models just doesn't have the bounce I expect from even a good vintage '93 or '94", even though they're all far superior to the brand new Unicomp models still on the market). But like many computer users, I sometimes suffer from pain in my hands and arms from excessive typing. A while ago I decided I had to give up the Model M, due to both the excessive typing force it encourages and my increasing discomfort with the standard keyboard layout. A succession of keyboards aimed at ergonomic use have followed. I've been though enough configurations of those now to feel comfortable passing on some shopping recommendations, gathered as I considered what I wanted for a second keyboard after settling on a primary one.

Obligatory warning note: please be very careful here! If you're in enough pain from typing to want or need an alternate keyboard, you could have a problem much more serious than just a keyboard change will fix. My own experiments to improve what was diagnosed as classic "tennis elbow" were carefully supported by monitoring during doctor and physical therapy visits, to make sure there wasn't really a larger issue and that I wasn't making things worse in the process. You don't want to end up like the poor author of Your wrists hurt, you must be a programmer, who was left unable to type altogether after poorly executed care here.

Shopping recommendations

There's a lot to chew on below. Here's the condensed version, which answers the most common questions I and others seem to have about the more mainstream Kinesis products:

• The Kinesis Freestyle with VIP kit is my recommended middle of the road pick. It has some parts you'll want even if you decide you want the somewhat more difficult to deal with (and much more expensive) Ascent "multi-tent" kit. This gets you most of the benefits possible here, using the most common ergonomic keyboard positioning, while still offering some flexiblity and upgrade paths. The combination is $129 as I write this at the retailer I bought mine from and would recommend, The Human Solution.
• If you think a more vertical setup might be a requirement for you one day, you can do add that later on the Freestyle. But you really should see how you do with just the VIP setup first, because the Ascent alternative is both really expensive and has its own potential issues. The most I'd recommend you consider spending right from the start is the extra $36 to get the the version of the keyboard with the wider 20" separation, because that you're going to want if you want to go vertical one day (but really only in that situation, so don't consider that important either)
• If you're OK without so much flexibility or really need PS/2 support, consider the Kinesis Maxim keyboard. It's about the same price, is easier to move around due to its more integrated design, and it supports the position you're likely to settle on with the Freestyle anyway.
• Want to experiment with a more ergonomic keyboard design to see if it helps you, but without committing so much money at first? Microsoft's Natural Ergonomic Keyboard 4000 is a decent cheap (under $50 if you shop around a bit) option here, albeit without a good feel for presses of individual keys.

Now we head backward toward how I came to these conclusions.

Keyboard ergonomic issues

In order to put all this into some context, first we need to talk about what's wrong with the traditional keyboard. Keys that require less force to activate are always good, but I have those on my laptop and it's not really comfortable to type on it, so that's clearly not enough. The main thing I've come to appreciate is that our forearms aren't designed to be horizontal for long periods of time. Try this experiment: let your arms fall loose to your sides. Note the position your hands are in relative to your arms? Your thumb is forward, your hands lined up straight with your forearms. Now hold your arms in front of you. Your arms feel more comfortable with your thumbs are pointing toward the ceiling, right? That's a natural position your body is comfortable with.

Now note how your hands and arms move when you use a standard keyboard. There's a 90 degree twist from your natural position. Your hands might be bent backwards some too; that's not good either, as it tenses the muscles in your forearms (which then pull on things all the way up to the elbow). And unless your arms are much more flexible than mine, you probably are more comfortable with them further apart than when you're using a standard keyboard, where you have to bend your elbow further inwards than is really ideal in order to get both hands on the home row. Kinesis has a good description of the various problem areas here in their awkward postures document.

On a particularly painful day where it hurt just to rotate my hands into the typing position, I noted that all it took was returning to thumbs-upward and separating them to make that feel better. The first question this raised for me was whether I could type like that.

Kinesis Ascent

You sure can; the Kinesis Freestyle with Ascent accessory (also called the "Multi-tent" kit) lets you split the keyboard in half and push it straight up if you want. The product catalog on their site isn't all that great, I found the shopping experience at reseller The Human Solution easier to navigate and order from (cheaper than direct from Kinesis, too).

The Ascent is a really expensive upgrade for the keyboard. I found it valuable for mapping out what my options were, because you can setup just about any angle/position combination with it. Ultimately I wouldn't recommend it for most people though. For me at least, the really sharp angles (near the 90 degree vertical position shown in the product picture) were hard to type on for a couple of reasons. While I touch type, I learned just how much I look at the keys for punctuation when in this position, because it's hard to do.

The biggest problem with this accessory is that while well constructed, it's still just sheet metal. There was too much flex in the design for me to really be happy with it, particularly in its vertical position. I had to tense more muscles than I expected to type that way, because I had to be so careful not to press too hard toward the center. It feels like you might collapse it inward, even though that's actually hard to do, because it does give a little in that direction. And angles short of vertical didn't turn out to be very useful to me either. Once I made the angle greater than around 20 degrees, I couldn't get a comfortable position until I reached >70, at which point I was back to being worried about collapse toward the middle again. At small angles, it was much sturdier, but there was still just a bit more play than I like. What I wanted instead was to adjust into the right position, than make it really solid in that spot. That's the approach more explicitly taken by some of the other competitors here, like Goldtouch; there's a good blog entry discussing their products, and I ruled out Goldtouch because I wanted the option of being able to separate the halves (even though it turns out I don't really need that right now).

One part of the Ascent design aimed to improve stability is a metal connector plate that attaches to the two keyboard halves. I found this to be a bit sketchy in that the halves aren't locked into place as firmly as I'd like (it's just a couple of bolts). And it also has the problem that it enforces lining up the two halves to be straight. The stock Freestyle keyboard comes with a removable "pivot tether" connecting the two halves of the keyboard. That allows splaying the two halves of the keyboard apart from one another at an angle, which I found works better for me than trying to keep them straighter. You certainly can splay with the Ascent by just not using the connector plate, but now you're dealing with two completely disjoint sections. The biggest issue I found with that is repeatability: without a way to lock into the position I want to use, the two halves tended to drift toward sloppy and bad positions without me noticing. Moving the keyboard around is a nightmare too without the connector plate.

Microsoft Ergonomic Keyboards

It's hard to ask the day job for a keyboard combination that approaches $400 with all the fixins when I wasn't even sold on it myself completely. The last two years I've mainly worked at home, but sometimes visited my far off company office for a week or two. In parallel with investigating options for my home keyboard where most of my job happens at, I also wanted a cheaper solution for my desk at the office.

After two long trips typing on every keyboard setup to play with at both Fry's and Micro Center, the cheapest option that I liked at all here was the Microsoft Natural Ergonomic Keyboard 4000. These are regularly available in the sub-$50 range even at retail (I've grabbed one on sale for $40 at both Fry's and Best Buy), and Microsoft has some other models available too you might consider as well. There's a good commentary praising the keyboard in the demanding Emacs context in a review that I found informative.

This keyboard gets a couple of things right. Note that there is some separation between the two halves of the keyboard, and that it slopes vertically downward from the center toward the left and right edges. These are the two things I found most useful about the Kinesis keyboard. What's really interesting is that the angles the Microsoft keyboard fixes those at are extremely close to what I settled into using even in the much more adjustable Kinesis+Ascent combination. The Microsoft keyboard also gets one detail perfect that Kinesis doesn't handle on their Freestyle design: it slopes downward from the nicely integrated wrist wrests to the function keys. Now, it turns out I don't like this as much as I thought. The most comfortable keyboard position I've found puts the keyboard in my lap (I'm using a Gamer's Desk w/Max Mousepad). The integrated front/back slope of the Ergonomic 4000 starts out too high for that to work really for me. I think you really need to be up higher and have the keyboard start below your lap, perhaps with an office chair and a keyboard tray, for a downward sloping keyboard to be optimally placed.

Ultimately, I never got comfortable with this Microsoft keyboard. My main issue is that the big keys on the bottom, mainly Alt and Space, have awful key action on them. The space bar clunk and doesn't feel right, and I constantly missed key chords using Alt in them because I didn't press the giant but sloppy key down fully. Having to press that hard is something I can't take in a keyboard, particularly for multiple key actions. The source for the positive review I linked to above apparently isn't bothered by this because he hits alt with his palm, which I just can't get used to (and isn't compatible with my goal of not being too different from what I do on a laptop). Ultimately I'm left thinking Microsoft's Natural Ergonomic Keyboard 4000 has the right layout and position, but ultimately the cheap keys it uses are disappointing. But if you don't want to spend serious bucks on a keyboard, or want to try out more ergonomic positioning but aren't committed to it just yet, it's cheap enough that you might find it worth trying out. The keyboard is common enough that you might easily find a store who carries one you can try out too.

Exotic Options

Kinesis does make keyboards that aims for better positioning in the front to back dimension too, where sloping downward away from your body is acknowledged to work out well in many situations. Their Advantage Pro is a very nice keyboard I was able to try out via a coworker who loves his. I ultimately rejected that choice because it's just too different from the standard keyboard to be comfortable for me, and since I do have to balance my time with a healthy dose of laptop keyboarding I couldn't see myself ever really getting used to it. If that's not important to you and you don't mind some retraining time, it's an expensive but quite nice product. It does get the downward curve bits better than the Freestyle model I settled on, and I think it has the potential to be more ergonomic in the end if you can accept those trade-offs.

Even the Advantage Pro seems quite familiar compared to the really difficult to get used to DataHand, which I was also able to borrow to evaluate for a bit at one point. The DataHand I can only see making sense if you're so bad off that you really can't handle holding your hands/arms in a standard position at all, or can't press down/move around anymore and it's easier for you to only move your fingers a little bit (you "click" a set of tiny switches in each direction around your finger to type). The retraining time and difference between other keyboards is really substantial on a DataHand. That's really not a cheap option either, if you can even buy one.

Revisiting the Kinesis Freestyle

What I came to realize here is that even with all the flexibility available with the Kinesis Freestyle with Ascent accessory, in practice what worked out well is basically the same positioning the newer Microsoft ergonomic keyboards provide. Somewhere between 10 and 20 degrees of slope downward from the center is enough to get rid of the worst of the rotation issues a standard horizontal keyboard introduces, and a moderate split between the two halves is probably all you need unless you're going full-on vertical (or have an enormous chest and tiny arms like a T-rex). Kinesis knows this perfectly well too; their Maxim keyboard provide a very similar configuration to the one settled into, ready to go as an integrated unit.

There are two viable ways to convert a Kinesis Freestyle keyboard to this sort of position. I already purchased their Freestyle VIP kit, which is the cheapest way to get a set of the wrist rests to go with the Ascent kit too. After giving up on the Ascent, I used the other portion of the VIP kit, the risers providing either a 10 or 15 degree lift, and found those to be plenty stable for typing on. While the range of adjustment here is limited, I think Kinesis and Microsoft have correctly nailed that the common case is going to want something in that range anyway. A Kinesis Freestyle with the pivot tether installed for good splay, risers at 10 degrees, and the wrist wrests is quite similar to the only position that's offered on the Microsoft Ergonomic keyboard and Kinesis's own Maxim, and unless you really have exceptional needs that required moving toward full vertical I think this is the best option available. Since I find myself more comfortable with the 15 degree lift position, I decided not to get the even more stable Kinesis Freestyle Incline platform (fixed at 10 degrees).

The only thing I wish was more adjustable on the Kinesis Freestyle is the front/back slope, which is fixed at a moderate upward slope. As I mentioned before it's hard to get a downward one to work well without a keyboard tray anyway, so that's not that critical to me, and I wasn't happy with the models on the market with an integrated slope that way.

That's where I'm at with keyboards now: Freestyle with VIP kit. Works great, wish it was a bit easier to move around though. I am concerned that I'm going to knock it off my desk one day and break the whole middle tent connector in particular. Some of the other Kinesis options here (the Maxim and the Freestyle with Incline kit) look more sturdy in that respect, if that's important to you they're worth considering. I like the extra flexibility of the Freestyle, but as I've mentioned repeatedly I don't think it's really necessary for most people; just nice to have.

P.S. Yes, I've done the same level of research and tests on ergonomic mice too; will cover those next time I get some time to write on this subject.