Emacs: P1: What Color Is My Painbow?

Last week I had a classic "Monkey Boy" moment. I decided to adjust the colors in my text editor. 10 days later I'm finishing a 3 post blog on it.

I worry myself some days.

This first post is going to be mostly theory work. Post 2 and 3 are more hands on.

Laying the Ground Work.

I use an editor called Emacs for most of my programming. It's an old editor, but its one of the most powerful editors out there. It also lets you edit files in display (windows) mode and from the shell (command.com for you Windows folks).

Now days most of the editing is done in display mode. No real surprise there. However, there are times when working from the shell makes more sense.

I routinely log into distant machines across slow connections. I could pop up a virtual session and wait for the window in the virtual session and then wait for the editor in the window in the virtual session and then wait for the file in the editor in the window in the virtual session, or I can use text mode, where are complete screen refresh is around 2000 bytes.

I hate waiting. Its a no brainer.

The down side of terminal mode is that you can only use characters to draw and you have a limited number of colors. Both of these could be overcome with modern technology, but it ain't going to happen so we have to get used to it.

Why do you have limited colors? Well, the underling technology differences between a terminal from 20 years ago and a modern graphic display is pretty significant.

RGB

Colors are made by mixing various amounts of Red, Green and Blue (RGB) together. If you crank up the RGB, you get bright colors, dial it down and you get dark. Wikipedia has a nice write up on color depth so I won't go into it here. The only thing you need to know is that by adjusting the RGB values you can change colors.

On modern display you have absolute control of every dot on the screen. Each one has it's own RGB setting which is independent of it's neighbor.

Old school color terminals were more like "paint by numbers" projects. You were given a pallet of colors (usually 8) that were hard wired into slots. If you set the color to pallet slot 0 and then printed, you got black text. If you printed in color 4 you might get blue. Unfortunately for us, these are the terminals that most terminal emulators emulate.

We have two problems when we want use Emacs in both terminal and display mode: First is that Emacs's support of terminal colors is functional, but not much more. The second is that the friendly Emacs customizers don't like it when you're a switch hitter. In fact they gets down right medieval on your monkey butt. Well, this ain't monkey butt, this is monkey boy butt. Accept no substitutions.

Terminal Colors

As I said before, most terminal emulators model the old style, 8 color pallets. There are ways for a program to ask the emulator for the number of colors available, but there isn't any way to get the actual RGB of each color.

What does Emacs do? It guesses! If you don't tell it otherwise Emacs assumes that you have an 8 color pallet with the following colors:

  Slot  Name   Red Green Blue
  ---- ------- --- ----- ----
     0 black     0     0    0
     1 red     255     0    0
     2 green     0   255    0
     3 yellow  255   255    0
     4 blue      0     0  255
     5 magenta 255     0  255
     6 cyan      0   255  255
     7 white   255   255  255

The numbers after the colors are how much Red Green and Blue that each color is supposed to have. 255 is the largest number you can express in 8 bits (1 byte) of data. There are places internally where Emacs uses 16 bit (2 byte) RGB values which go from 0 to 65535. I got bit by this more than a few times so I'll try to point them out, or gloss over them when I can.

Emacs cares about the RGB values because you (the user) set colors by name not slot values. If you set the color of something to "CadetBlue1" 152/245/255, then run in terminal mode, Emacs needs to figure out which of the eight colors CadetBlue1 is closest to. It uses the RGB values to figure it out.

Oh, by the way, the name "CadetBlue1" comes from a variable called "color-name-rgb-alist". To see it's contents, fire up Emacs in display mode and type "M-x list-colors-display". You'll see the colors and their names.

Let's do some hands on. From a terminal, type "emacs -nw". It should start an Emacs session in the terminal. In Emacs type "M-x list-colors-display". You'll get a listing of the 8 colors that Emacs knows about. Note: On some systems you get more than eight. Lucky you. The theory is still the same.

If you're like me, you notice one thing first off. These colors look nothing like their names! The Red might be brick colored. Yellow may look brown. And my white has tattle tale gray! What happened?

Easy. Emacs has no idea what colors your terminal's pallet is set to and it's guess stinks. How do we handle the miss-match?

One option is to change our terminal to Emacs's pallet. Then we can vomit and claw our eyes out. Basic colors tend to be rather harsh on the psyche.

The second option is to tell Emacs what our terminal is really packing. That's the subject of the next post.

Going Deaf at Duffs

Whenever anyone from the company has to come up to Buffalo on work, we have to feed them.

This being Buffalo, and we being cliches, we inevitably drag them off to Duffs.

The thinking goes like this: Person A comes to Buffalo. Person A must want to try our local cuisine. Our local cuisine consists of Chicken Wings and Beef on Weck. Monday is Chicken Wings, Tuesday Weck. Today is Monday. Real men like hot Chicken Wings. Duffs' wing are really hot! We go to Duffs.

Never mind the fact that there is more to Buffalo than Chicken Wings. Ignore that fact that Person A may be here for is 10th time this year. Obfuscate the fact that hot wings and good wings independent variables. Me Buffalo, Me Wings, Me Hot, Me Duffs. *Sigh*

OK. I don't have anything against Duffs. Alright, it does remind me of a low rent vomitorium and the video games run on diesel, but besides that it's a nice enough place. I'd just like to see a little more depth in our chow.

But that's not why I'm writing today's blog.

The last time we were in Duffs, a person came up to me and handed myself and a couple of other people at the table "deaf cards".

For those of you who don't know, a deaf card is a card that allegedly deaf people hand out at airports as a way to beg for money.

The card usually follows a certain formula: First the introduction "I am a deaf person.", then the pitch "10 bucks would make me feel better about being deaf.", then a blessing "May god bless you for giving me 10 dollars." and then a graphic. Usually the graphic is something like a cartoon angle or a peace sign. I got a smiley face.

The first thing that bothered me is, I'm no where near an airport. We got rules! You street beg on streets, airport beg in airports and PBS beg on the radio. It's all part of the begging ecosystem. What's next, Hare Krishna telethons?

Also, I don't know that the person's deaf. I have no problem giving help to the needy. I'm well aware that with a disconcertingly small number of bad breaks, I could be out on the street. This guy is Alpha/Omega. Either I do good by helping someone out, or I'm encouraging a rodent to make a dishonest living by pretending to be deaf.

Then I notice that everyone at my table is whispering. Whispering? Why would you whisper around a deaf person? We're surrounded by the hearing? If he's stone deaf, like he's professing, then he can't hear us. If he can hear us, then he belongs in the slammer.

And if we're afraid of people hearing us, then why aren't we whispering around the people at the tables all around us? We've been blabbing for an hour. They've heard every word!

Then I had the big epiphany. Why would I give someone $10 for being deaf? This isn't like the 1820s where deaf people starved on street corners. In modern societies there are very few jobs that aren't accessible to the deaf. We have the technology and, I'd like to think, are more enlightened about deafness. Rare is the person who thinks it's a punishment from god.

I'm not saying there still isn't ignorance, I'm just saying it isn't the albatross it once was.

Thinking about it, where I work, every jobs in the building save phone receptionist and security guard could be handled by someone stone deaf. And even that, the security guard who monitors the cameras would have no problems.

Knee jerk simpletons may pretend that I'm picking on deaf people here. They're seeing what they want to see. I freely concede that someone who has a sense, and looses it, suffers. If they have an ability, they use an ability, they loose an ability, they have to adjust their life around the loss. I just don't see where deafness would make an otherwise healthy person into a beggar.

In some cases the loss is tragic. Beethoven never heard is final symphonies. In other cases, not so much. A Sumo wrestling rarely depends on the sense of smell. Neither would require you handing out cards.

On the other hand, what about people who are born deaf? They'll never hear music, but I'll never see magnetism. Am I missing out? No idea. And maybe the born deaf experience a clarity of though and tranquility of mind that I'll never know in my noise filled head. Again, no idea. Either way it ain't worth $10.

Because I pointed out that the deaf shouldn't be pitied, everyone at work now thinks that I'm a kitten burning poltroon of the worst stripe. Far from feeling like a wag, I think that I'm being more enlightened than most.

I also noticed that none of them gave the beggar any money. Ya' hear what I'm saying?

Aw C'mon, be a Minion!

Come my minions! Gather 'round me and do my bidding!!!

Nuts! That never works.

I'm starting a minion list. It's not a list of minions, it's a list of projects for my minions.

I'm going Internet because minions are sparse on the ground around these parts. I believe it's because my readership is made up of selfish swine who won't surrender their sense of personal identity for the betterment of me. But I press on! I'm brave that way.

My minion list is a list of projects, usually computer projects, that strike me as a good idea, but I don't have time to do them. It would be nice if a minion would do them (hint, hint).

I've talked about the minion list in the abstract many times, but now I've decided to codify it. That way, when someone else commercializes one of my ideas, say "Feels on Wheels" (I'll explain later), I can sue them for plenty cash.

Wanna Taste?

My first entry for the minion list is "Voice Messaging". Voice Messaging is like Instant Messaging, augmented with sound.

The concept is simple. A person wants to send you a message. Instead of typing, they have the option to record a message. The message is sent to you, along with any typing they wish to do. On your side your VM program converts the sound file to text and displays it like a normal IM. If the speech conversion garbles it too badly, you just highlight the part you can't read and it plays that part of the sound file.

This would rid the world of emoticons. If you want to know how someone is feeling, play the sound file.

I Object!

* Speech recognition sucks.

You're right as far as you take it. However, in this case we're not producing a finished document. It doesn't have to be perfect, just readable. The recognition only has to be gud enuf 2 undrsAnd. If you're unsure what was said, play the sound file.

* What's to stop someone from saying one thing, but typing another?

The conversion is done on the receiver's end. The sender can send text along with the sound, but the final arbiter of the text is the receiver.

* What's to stop someone from screaming obscenities or pulling other "funny" jokes.

Before playing a sound file, the program would normalize the sound levels. It could even warn you if the original has loud points in it.

* Couldn't someone just use the phone?

Yea, but VM is closer to IM than a phone call. The sender can send a VM, but you can ignore it until you're ready, just like IM.

* What about people who don't stop yakking?

Put a size limit on how big you'll receive. Make it settable per person. Even if someone does get carried away, it's still better than voice mail.

Almost any sound player lets you replay selected parts of a file so you can deal with the witlings who leave 5 minute messages and wait till the end to mumble their contact information.

* What file format should we use?

This is getting a bit low level, but something open format is an obvious requirement. It turns out that there is an open format called Speex that gets the job done.

A 14 second PCM, 16 bit, mono 48000 Hz .wav file is 1.4 meg. Convert it to an .mp3 shrinks it to 123K. Speex mauls it down to 66k. All with no real loss of sound quality for what we're doing.

* What about security?

As long as an open audio format is used, and you trust your audio player, then you're all set. If you use an audio player that does odious stuff like pop up web browsers (oy!) then you're asking for trouble. Playback should be based solely on the contents of the file, not it's extension.

* What technology should the be built on?

I'd bet that most IM protocols allow file transfer. It would simply be a case of the client programs handling audio file differently.

I Submit Like the Schweinhund I Am!

You have convinced me oh minion master. I supplicate at your feet. What is the first step?

For a start, someone could write a standalone application that reads Speex files and transcribes the contents. Some college out there has to have information on phonetic translation. Lets see what the state of the art is.

After that, add playback of selected sections. Once we have that settled then the rest would be cake, and how many masters let their minions have cake?

Sound Bite Me!

One of the true pleasures of programming is finding some niggling little problem and solving it with a simple bit of code. If you can solve it in a couple of hours, even better.

When I blog I tend to speak what I'm typing. If it doesn't sound right when I say it, it probably wont sound right when you read it.

One of the problems I have when I blog is that I talk faster than I type. I talk faster than I think. I start typing, and then I get a flash of inspiration. I run the idea through my head, and then, half a virtual page later, I realize that I haven't written anything down.

Then I have to try to recreate the idea from memory, but the flash is gone. By the time I've rebuilt it, or accept that I've forgotten it, my original thought is out playing in the yard and won't come back.

It frustrates the hell out of me.

I started playing around with ideas to making blogging easier.

At first I tried to check out the state of computer speech recognition. I figured I'd just blab on in my blog, and then I'd go through and clean it up by hand.

Computer speech recognition is still slow, expensive and it still sucks. Trying to run a editing session without using a keyboard is slower than typing with 2 fingers. I also wanted to blog via Linux, so firing up a Windows product ain't getting the job done.

Next it tried to do some sort of integration of speech and text together.

I envisioned loading a sound file into a sound editor, where I could chop it up and move it around. While editing the sound, I'd join text to it. When I moved a blob of sound, text that went with it would move to. Eventually I'd piece a blog out of all my rattlings on.

I still think that this is an interesting solution, but man, it would be some work! I also I think I'd end up with a crappy sound editor linked to a crappy text editor. No dice.

I also had a minor epiphany. I'm not going to keep these sound files around forever. I'm just brain dumping to a file for a few minutes until I've finished typing my original though. After that I can replay the recording and transcribe anything I think is useful.

I already know how to record from a mic on my Linux box. Adding that to my new insight I wrote a shell script that turns on the sound recorder, dumps the contents of the microphone into a file and, when I stop recording, plays it back (that way I can tell if I forgot to turn on the mic or something).

It worked like a charm! Every time I needed to make a note, I just fired up the script, decided on a name for the sound file and away I went. It was a little awkward, but a big step forward.

I called the script "sound_bite".

After than I needed to come up with a way to play back my sound bites, so I started adding flags to play the last sound bite or the first sound bite or list the sound bites and let me pick. Then, another epiphany! They're just frigging .wav files! Maybe I could just double click on them in my file manager. Oooo. Me one smart monkey!

Actually, once I got the file manager into the game, it cleaned up a lot of code. I didn't have to tell the recorder where to put the sound files, I would always put them into the same directory and give them a time stamp for a name. If I wanted to organize them better, I'd use the file manager to rename them or move them elsewhere.

The only thing left was making it easier to use. Typing in the command every time is a minor pain. I needed a quicker way to access it. That was easy too.

I hooked up sound_bite to a shortcut which put all the .wave files into the directory "sound_bites". I made another shortcut to bring up the file manager in "sound_bites" directory. I'm sure I could come up with a few dozen little tweaks, I know enough to stop typing when I'm done.

I'm now an official audio driven blogging fool!

Here is the entire source code for sound_bite. You may have to play with it a bit because the blogger code likes to play with it.

Enjoy!


#!/bin/sh -

# Sound_bite: Written by Dale Wiles 6/2/09.

# Exit if an error occurs.
set -o errexit

if [ $# -eq 0 ]; then
  cat <<EOM
Usage: $0 directory

Move to DIRECTORY and start recording a wave file from the microphone.
The name of the wave file is yymmdd_hhmmss.wav.
EOM
else 
  sound_dir="$1"
  cd "$sound_dir" || exit 1

  # Make the output name based on the time, down to the second.
  # That way I can't overwrite existing files.
  # Alright, in theory during daylights savings time it could
  # overwrite.  I've added code for that almost impossable situation.
  while :; do
    out=`date +%y%m%d_%H%M%S`.wav
    if [ ! -e $out ]; then
      break
    fi
    echo "Waiting...."
    sleep 1
  done

  echo "Recording $sound_dir/$out"
  sox -t alsa default -v 7 $out
  echo "Playing $sound_dir/$out"
  aplay $out
fi

Spock's Two Fingered Monkey Freak

I went to see the new Star Trek movie, which is called, cleverly enough, "Star Trek". Review in a sentence: Kinda fun in a Saturday morning cartoon sort of way, but it has some of the dumbest writing in the history of the world.

If you're a fan, you've already seen it. It's the second time for one of the guys I went with. If you sorta like Star Trek then you can have fun between the winces. If you're not a Star Trek fan... Hard to say. A lot of people went to see the Transformers movie and it sucked.

I'm not going to pick the film apart piece by piece. It's not a terrible film. But on the other hand, it ain't a great film either. Besides, some people find my smartypants type reviews funny.

In To The Breach! A' Spoiling I Shall Go!

This is supposed to be a reboot of the franchise. OK. I can live with that. I'm not a big fan of movies like this. The ones that show that one scene that defined everyone's character. In real life there usually isn't one scene. People grow over time and relationships grow with them.

Oh well, this is mostly like real life actors running around like pissed off Muppet Babies, so we have to let that slide.

My first kibitz is that it didn't feel like a Start Trek movie. It has all the names and a few of the places, but the rest was generic shaky camera CGI. The original Star Wars had it's feel. The Star Trek movies, even the ones that sucked (and boy did some suck), still felt like Star Trek. This one really didn't.

It has the obligatory Kobayashi Maru (bite me, I looked up the spelling) simulator scene. In the original universe Captain Kirk is the only person to beat the simulator. W00t! The universe is in awe. He later confesses, in one of the most painfully stupid scenes in all of the Star Trek universe, the he cheated. That's right, Captain James T. Kirk is not a great star ship captain, he's a cheating weasel. Anyway, his defense? He doesn't like to loose. I hate blue balls but that doesn't give me the right to date rape.

I wonder if a more qualified recruit was screwed out of his own ship because Kirk blew a professor to get to the top. I'm sure that Diane Duane's written book about him. "Lieutenant Pango: Boned by Kirk!"

In this movie they change it up a bit. In this version, Kirk cheats, but does so in such a psychotically obvious way that everyone knows he cheats. There can be no other option. He played "Duke Nukem" in god mode. It's like he went to the final and yelling "I sure hope the second answer isn't 'Your mother and a horta!'. Oh golly gee whiz, it is! The guys in Animal House could cheat with more elan then he.

In fact, it's not even cheating. It's a psychopathic desire to fail. I wouldn't want this clown running a space fairing warship with guns in the front. I wouldn't want him driving a bus! The last words I want to hear as I wing into a yawning chasm with a bus load of other victims isn't the driver saying "I sure hope this isn't a yawning chasm below us!". It is Jimmy, but gravity is going to fix that in a couple of seconds. Splat!

Also, everything in this movie was dirty. I know that we can use CGI to make things look like crap, but that doesn't mean we have to. One of the things that I liked about the old Enterprise is they kept it clean. Even the bad guys understood hygiene. They may have had only had 3 sets, but they kept them swept and they brushed their teeth.

Here, everything was filthy, everyone needed a shave and everyone needed a spanking! Needing a shave isn't a special effect!

I know it's supposed to make it more "adult", but it doesn't. I've been (in the legal sense) an adult most of my life. I've never worked in a place as dark and dank as the Romulan super-CGI-future-miner-ship. I can see dark coal bins, but why wasn't there proper lighting on the bridge? I have florescent lights in my cubical, and it doesn't have warp drive.

And there were pipes everywhere! I mean everywhere! Remember, these ships are huge. They don't hang with gravity. The have all the room they need, but man they have a lot of pipes. It's like the all of galactic space is run by Thomas the Tank Engine. Enough with the steam punk. In the future they have wallboard!

And the people! Yeek! I've also worked with some pretty wretched people, but even in the worst of places, most people walk around with their asses blissfully board free.

How about that high tech computer talk? And when they start talking about computers, they made it sound like the Enterprise is written in AppleSoft BASIC. Instead of saying plebeian stuff like "Captain, the subroutines are interfering with our, um, SoundBlaster 16, um, thousand.", they should use hip psychological terms like "Captain, someones' boned the computer's universe of discourse and it's gestalt is full of paradoxes." It's bullshit, but it's future bullshit.

Did I mention the space chicks? Can we please have a movie with more than one female lead? And could we not make them poon targets?

In this film the nookie target was, of course, Uhura. Spock gets his freak on with her in this flick. And it isn't some Vulcan two finger smoochie. Spock does it monkey style! The thing is, it contributed nothing to the plot, it was gratuitous, and Spock looked too much like Kevin Nealon, Yea, I bet "The Kev" has to beat the chicks off with a stick.

Don't think I'm against hot SciFi bitches. Wally Wood is my spirit animal. Hell, lets do a full frontal nudity version of Star Trek where the Romulians perfect the lezbo ray and the Federation sets it's love guns on "lube". I'm there with extra popcorn.

But either do it or don't. Lets see 'um naked, or give them roles that don't involve them playing kissy face all the time.

How about weapons? I know it's kewl and all, but why would you carry a sword that folds up into a tiny sword? Wouldn't you use that space to carry a second gun? Knives have their place. I carry a geek army knife with me wherever I go. But not for battle. Any Marine worth their salt will tell you, if you carry a knife for combat, replace it with another clip of bullets.

Ze Plot!

I'm going to egress talking about the plot, such that it is.

Do you remember the first "Pirates of the Caribbean" movie? If the zombie pirates had just asked Will Turner to come with them and drop one drop of blood so they could be freed from the curse, and in exchange they would give him a bag of gold, he probably would have. No muss, no fuss, just ask nicely. The whole movie would have been about 10 minutes, including credits.

Star Trek has one of those plots.

An angry Romulan (who's really just a guy who needs a shave) comes back in time to oowie the Federation because Romulus's sun unexpectedly went nova (if you just heard a popping noise, it's the head of any physics major younger than Lord Kelvin). He blames the Federation.

Fair enough.

He's a miner, so he comes back in his mining ship (a ship for digging dirt). OK. Most mining ships are kind of like lunch boxes with engines, they just move dirt. Not Romulan mining ships. These bastards are bigger than most planets and are covered with super death weapons. Kind of like the Dodge Wrangler. Most Dodge Wranglers are encrusted in torpedoes right? That's why the get such rotten gas mileage.

Any way. Planet go boom. Ships go boom. Plot goes ARRRRRRG boom. Spock gets jungle fever. Roll credits.

Um, why couldn't the Romulan guy come back through time, take his super-CGI-future-miner-ship back to Romulus and just tell everyone "Hey, I'm from the future, look at the date on my drivers license and how much I need a shave! On stardate a hundred years from now, at around 3:15PM, the sun is going to go nova and melt our planet (pop pop pop!). Write that date down. No! Use the red felt tip marker and underline it! Now that that's settled, I happen to have this super-CGI-future-miner-ship that is chock full of 100 years in the future type technology. What am I bid?"

The whole movie would be about how this poor swine died from too many ticker tape parades while getting Romulan nookie while playing the stock market.

I'm just saying.

I Conclude!

Anyone who says this movie is more than a CGI pretty-bang is either a Trek fiend or they're going to grope you in the theater. (Not that there's anything wrong with that.) Last word? Meh. But a fun Meh.

Schleprock says: Lets Open that Id.

Through out my digital travels I tend to trip over more than my share of quirks, bugs and "obvious" ideas for improvements. I'm gifted in a Schleprock sorta way.

Unlike Schleprock (probably a pinko!) I also try to be a good netitizen and report what I find.

The problem is there are too many barriers to being a good Samaritan. The biggest one I run into is having to set up an account just to make a bug report. I have 71 separate accounts to date, of which I really use 2 or 3. I'm reticent to add more accounts just for the privilege of helping someone fix their code.

I understand the reasoning for the accounts. It's to cut down on Spam. On the surface it looks like a good solution, but in reality it's almost as bad as the problem. It cuts down on Spam, but it also cuts down on user participation. No participation is killer for open source projects.

One solution that I've seen it to allow anonymous posting, but have each posting validated before being passed through. This is a great solution. It allows anonymous posting, but puts a minor impediment in the way. If you want fast turnaround, set up an account, if you can wait, then post anonymously.

The only reasons validation isn't more popular are self important administrators, who don't make the time to deal with the human component of their project, or huge projects that legitimately generate a large number of anonymous emails. These huge, anonymous, projects are fairly rare, and Spam filtering is easier for them than for normal mail. The size of my penis is not a bug.

But how do you deal with a site, that for some obsessive reason, wants you to set up an account. If only you could create one Id and have an *OPEN* way to hand it around? Open, Id? Hmmmm. OpenId? It's got a nice ring to it.

The OpenId project exists to solve the problem of having too damn many Ids. It's really clever on how it solves it too.

You start out by going to an OpenId provider and signing up for an Id. Arg! I know, you have to sign up for another account! But this is the wish for more wishes. Once you get an OpenId account, then you can use it on any OpenId enabled site that's out there.

Let's say I'm trying to log on to newsite.com. Newsite is OpenId aware.

I type in my OpenId (http://upcracky.myopenid.com in my case). Newsite then decodes my Id and redirects me to myopenid.com. I log into myopenid.com using the password I set up there. Once I'm logged in, I'm redirected back to newsite.com and I'm in. If the password is good enough for myopenid.com it's good enough for newsite.com.

In fact, I don't believe newsite.com ever sees my password. They're probably pinkos!

How do you get an OpenId. One of the nice things about OpenId is, you may already have one. If you have a gmail account, they're an OpenId provider (It's your email address and password.) Yahoo is another. If you don't have one of those, then web search for "OpenId Provider". They're plentiful. Hell, OpenIds are cheap. You can make one for work and one for home and one for the kids and dog.

The only real problems I've run into with OpenId are:

1) They're not robust enough for super secure sites. I don't use OpenId for banking.

2) Not enough sites use it. This one is easy to fix. Whenever you have a site that is OpenId aware, use it. If you have a site that's not OpenId aware, complain to the admins. The code to implement your own OpenId log on is freely available on the net. The more sites that have it, the more users that use it. The more users that use it, the more sites that will get it.

Seriously, if you have a ton of user IDs and passwords, then it behooves you to pester all the sites you know to set up OpenId. If not, you'll end up like me, stooped over from the pain of dragging around all my BugZilla accounts.

Wowzie wowzie woo woo.

Clones Ate My Files: A Sci-Fi Geek Thriller.

OK, so I'm banging out code for the betterment of my corporate overlords. It's a conceptually simple program: You give it a command and a list of servers and it runs the command against each server.

Not impressed? Well Monkey Boy doesn't get the corporate shekels without doing major mojo. This program runs in parallel. It juggles 64 instances of the command at a time. It can ping all the severs on a netmare in less than a minute, keep track of the results and print them out in the order that they were input. I get wet just thinking about it!

All praise Monkey Boy right? We'll I did run into a problem. A sneaky problem that involved failed clones, suicidal files and a forgotten inheritance. It's good stuff. The problem is, if you don't program in Perl, you're not going to give a crap. Oh well, I've never let my complete lack of an audience slow me down before. Why start now?

Like I said before, the beast is written in Perl. Perl's not an elegant language, but if you need to leap out of the bushes, rape and strange a problem and get on with your life, then Perl's your language of choice.

The basic layout is: Start a sub-process for the first 64 severs and have each one write to it's own temp file. When one sub-process finishes, read it's results from the temp file and let the temp file disappear. Then add a new sub-process for the next server. Once all the servers are done, print out the results in order and accept smoochies from hot code groupies/naked underwear models. Simple.

The Boy of Monkey knew that he'd need lots of temporary files to hold command results. He'd also like the files to go away by themselves when he's done with them. Perl lept to his aid with File::Temp. File::Temp is kind of the anonymous underage prostitute of programming. When you say "Gimmie" it gives you access to the goodies and provides an assumed name. When you're done using it, it disappears into the aether. It all works great, until someone, or something, starts killing the temps prematurely. Then you get a mystery to solve. Foreshadowing!

I got the code working and was getting ready to document (Yes, Monkey Boy is a pro, not documenting makes you a douche bag), when I though, what happens if the sever command can't run? If the user misspells "ping" as "pong" will they get a reasonable message?

Does

Couldn't open file '/tmp/multi.1d834.pid': No such file or directory

strike you as reasonable? Me neither. Nuts!

The actual error message made sense to Monkey Boy. He spawned the beast. '/tmp/multi.1d834.pid' is one of the randomly generated temp file names. When a sub-process finished, the program asked the temp handle for it's file name. It tried to opened the file, but for some reason the file was gone! Somehow the files were being killed before Monkus Boyus could get to them. No one should know about these files. They have specially constructed names, known only to the monkey... or the monkey's clone.

The way you run a sub-process in Linux is using the fork() command. You're running along happy as a clam, as if a mucus coated bivalve is your apotheoses of happiness, and then you hit fork(). At that point your program is cloned. You have 2 running copies of the code. The only difference is that fork() will tell the parent the ID of it's child. The child is handed the ID of 0, which tells it that it's the clone.

This is kind of Star Trekie at this point ain't it? We've got parents making 64 clones (top that Octomom!) and we've got children that are one bit away from being perfect copies of their parent. It gets better. The clone's next job is to call exec() which completely obliterates it and replaces it with another program. It's this second program, the sever command, which does the real work I want done.

A clone has one job. It's job is to die and be forgotten. Programming ain't for wussies!

When all goes well, the program runs like a well oiled roach motel. The clones check in, but never checkout. They disappear on the spot and are never heard from again.

That's when all goes well. What happens when the exec() command fails?

It's simple really, the clone lives on! It also keeps it's copy of the temp files, which it believes it owns. When it dies, it takes the temp files with it. Clones can be selfish little pricks.

Eventually the grieving parent checks on the child. It notices that it's died and then tries to check the temp file for the reason. The temp file is gone baby gone, it died at the hands of junior. You can only die once in temp file land.

As for the reason the exec() failed? It was written into the temp file. You know, the temp file that's in temp file heaven? Hmm. What to do? What to do?

Suddenly Monkey Boy (you remember Monkey Boy, he's the hero of this epic) has an insight. When you delete a file it doesn't really disappear until the last program that has a hold of it lets go. It's removed from the directory, so it can't be seen, but it's still out there, in limbo, awaiting for the sweet kiss of digital death.

Who else is holding on to the file? The parent of course! The question was, could Monkey Boy get to the parent to cough up the handle and could it be used for reading?

Detective Monkey Boy began investigating. He checked the usual suspects. "perdoc File::Temp" didn't provide much. It was higher level than a kite. The Internet tubes were blocked by flame wars and almost naked pictures of some platinum blond from California. No go. Detective Monkey Boy knew what he had to do. He had to go (non-prequil) Jedi. "Use the Source Luke!" is the rallying call of the Open Source movement. But Monkey Boy's name ain't Luke.

Into the source goes the hero. Past lines of documentation. Past obscure code references. Further he goes, until he finds, what he knows in is heard must be, "use IO::Handle". Rocken!

For those that don't know, deep in the belly of the beast, a file comes down to little more than a number. When you open up a file, voodoo happens, and an entry is put in a table called the File Descriptor Table. What you deal with, either directly or through some Perl interface, is an entry in this table. If you can figure out the index number to this table, you can find your file. File::Temp was a cold fish, but what about it's ancestors? File::Temp inherits from IO::Handle. To get to the real power, you got to seduce grandma.

"Hey there Granny!"

Once I go my hands on Granny's nodes (yech!) she gave up IO::Handle. IO::Handle has the fileno() function. You got the number, you get the data.

After that it was just a hop, skip and a file dupe to get to the data so ingloriously killed off by the wayward clone. It takes more than the death of a temp file to stop a motivated Monkey Boy!

All praise Monkey Boy.