Showing posts with label Linux. Show all posts
Showing posts with label Linux. Show all posts

Monday, August 1, 2011

Hey Unix, How About a Date?

The Problem.

I needed a way to warn users that their passwords were about to expire. To give them a sporting chance, I wanted to give them 2 weeks warning.

I could get the current date with no problem. Getting the password expire date was a bit of a pain, but I could figure it out. The hard part was figuring out how close to the 2 week warning date they were.

In a nut shell, doing math with dates is a royal pain. You have abstract concepts like “July 4th” permanently glued to a large spinning rock which is whizzing around the Sun.

Also, certain concepts that we're used to, such as 1 + 1 = 2, don't always hold up when doing date math. January 31st + 2 months = March 31st. No surprise. How about January + 1 month. That would be February 28th? Or would it be March 3rd (31 days after January 31st)? Add a month to February 28th and you get March 28th. March 31st doesn't equal March
28th. Oh boy!

The Simple (Linux) Solution
.

The most common way to deal with dates is to convert them to a number, do some math, and then convert the number back to a date. Unix (and Linux) system time is based on this concept. Unix dates are implemented as the number of seconds since the January 1st, 1970. In Unix parlance 1/1/1970 is called “the epoch”.

If you have a version of Unix that uses the GNU version of the date command (almost all versions of Linux do), then date math becomes trivial. The date command can convert to and from the epoch with relative ease.

To convert “August 1st 2011” to seconds from the epoch use:

date -u -d "8/1/2011" +%s

You should get 1312156800.

To convert it back use:

date -u -d "1970-01-01 1312156800 seconds" '+%m/%d/%Y'

You should get “08/01/2011”.

The problem here is 2 fold:

One, you're limited to dates between 1970 and 2038 for 32 bit computers. If you have a 24 bit computer then you're good until somewhere around the year 292,277,026,296 so it's not really a restriction.

The second problem is that most older OS's aren't running GNU date. They have their own propriety versions of date that won't let you work with arbitrary dates.

What I needed was a date converter that would work on many versions of *old* Unix. Things like Solaris 5 and HP-UX 10. These are nasty little beasts that barely have Bourne shell. I also wasn't allowed to add more advanced scripting languages to the system so Perl and Python solutions were both out.

I poked around on the Internet tubes and found a dearth of solutions. Most of them used other languages. Some gave example code that didn't handle leap years properly. Others were built around precomputed tables.

Time to step up to the plate.

My Solution.

Below is my solution. Its date format is the number of days after January 1st, 1582. That's the beginning of the Gregorian calendar and very few of my users are that old.

Internally it's mostly AWK scripts glued together by Bourne shell. It can handle dates up to 1/1/9794 and can probably go higher. I've tested it and think it's pretty bullet proof.

#!/bin/sh -

# Convert a date to/from the number of days after 1/1/1582 using only
# basic Unix commands. By "basic" I mean commands available on an
# HP-UX 10 box.
#
# 1/1/1582 is the start of the Gregorian calendar.

# Note: The Gregorian rules for leap years is:
#
# If the year is a factor of 400
# It's a leap year.
# Else If the year is a factor of 100
# It's not a leap year.
# Else If the year is a factor of 4
# It's a leap year
# Else
# It's not a leap year.

# This mostly uses awk because awk is much faster than using raw
# Bourne shell.

# To get a date from the Unix "seconds from the epoch" time use
# int($utime / (24 * 60 * 60)) + date_as_days(1970 1 1)
#
# date_as_days(1970 1 1) = 141714 by the way.

#
# Return the number of days in the previous months.
#
# For example the second entry is the number of days in January. The
# third entry is the combined number of days in January and
# February.
#
# The only parameter is the 4 digit year.
#
days_prev_month()
{
  echo $1 | awk '{
    year = $1

    # Pick the number of days depending of whether its a leap year.
    if ((year % 4 == 0) && (year % 100 != 0 || year % 400 == 0)) {
      print "0 31 60 91 121 152 182 213 244 274 305 335 366"
    } else {
      print "0 31 59 90 120 151 181 212 243 273 304 334 365"
    }
  }'
}

#
# Convert a date in to the number of days after 1/1/1582.
#
# The parameters are YYYY MM DD.
#
date_as_days()
{
  dad_month=$2; dad_day=$3

  # Get the number of days consumed by the years and the number of
  # days remaining in the current year.
  set - `echo $1 | awk '{
    year = $1

    # The modern calendar started in 1582.
    year_days = int((year - 1581) * 365.25) - 365

    cents = int((year - 1501) / 100)
    year_days -= cents

    cents_400 = int((cents + 3) / 4)
    year_days += cents_400

    print year, year_days
  }'`
  dad_year=$1; dad_year_days=$2

  # Now add the month and day contributions.
  days_prev_month $dad_year | awk "
    BEGIN { day=$dad_day; month=$dad_month; year_days=$dad_year_days }"'
    { whole_month_days=$month
      print year_days + whole_month_days + day - 1 }'
}

#
# Take the number of days since 1/1/1582 and convert it to
# year, month, day
#
days_as_date()
{
  df_days=$1

  # This awk script computes the year contributions to the date
  # and removes the effects of those years from df_days.
  set - `
  echo $df_days | awk '{
    df_days = $1;

    # The first 400 year leap year in the Gregorian calendar is
    # 1600 so we normalize our calculations from the first block
    # of 400 years that ends on 1600. That year is 1201.
    # There are 139157 days between 1/1/1201 and 1/1/1582
    #
    # Note: We use 1201, not 1200 because we want the leap year to be the
    # *last* year of the 400, 100 or 4 year block.
    n_days = df_days + 139157

    # There is one leap year every 4 years.
    days_per_quad_year = (365 * 4) + 1
    # Years that end in 00 arent leap years.
    days_per_cent = (days_per_quad_year * 25) - 1
    # Unless its divisible evenly by 400.
    days_per_quad_cent = (days_per_cent * 4) + 1

    # Calculate the contributions of each year block.
    quad_cents = int(n_days / days_per_quad_cent)
    n_days -= quad_cents * days_per_quad_cent

    cents = int(n_days / days_per_cent)
    if (cents == 4) { cents = 3 }
    n_days -= cents * days_per_cent

    quad_years = int(n_days / days_per_quad_year)
    n_days -= quad_years * days_per_quad_year

    years = int(n_days / 365)
    if (years == 4) { years = 3 }
    n_days -= years * 365

    df_year = 1201 + (400 * quad_cents) + (100 * cents) \
      + (4 * quad_years) + years

    print n_days, df_year
  }'`
  df_n_days=$1; df_year=$2

  # Get the day and month from the given year.
  set - `days_prev_month $df_year | awk "
    BEGIN{n_days=$df_n_days}"'
    { df_month = 1
      while (n_days >= $df_month) {
        df_whole_month_days = $df_month
        df_month++
      }
      df_day = 1 + n_days - df_whole_month_days
      print (df_month - 1), df_day }'`

  df_month=$1; df_day=$2

  echo $df_year $df_month $df_day
}

# Some test code. Feed it a number and get back a date.
# Feed it a m/d/yyyy date and get back a number.
#
# Note the complete lack of error checks.
if echo "$1" | egrep '/' >/dev/null 2>&1; then
  set - `echo $1 | tr '/' ' '`
  date_as_days $3 $1 $2
else
  days_as_date $1 | awk '{ printf "%02d/%02d/%02d\n", $2, $3, $1 }'
fi

Sunday, May 23, 2010

Gators and Nautilus and DAVs, Oh My!

If you're here to get WebDAV working with Nautilus for HostGator, then just jump to the end of this article. The solution is trivial.

If you wish to bask in my purple prose, read on:

Introduction

I'm setting up a site on HostGator, which is a host provider of good repute. It's starting as place to put some of my little code snippets that don't warrant a full blown project. Who knows what it will turn in to.

So far it's been basically a positive experience, but it's not without its shortcomings. I'm trying to keep notes on dealing with HostGator so other people don't fall in to the same traps I've tripped.

One thing I ran into was getting WebDAV working with Linux. Specifically Ubunto's "Karmic Kaola" under Gnome, using the Nautilus file manager.

Of DAVs and Nautali

The first problem I ran into is that HostGator gives it's version of WebDAV the name "Web Disk".

I don't mind them trying to make web interfaces seem a bit easier to deal with, but somewhere on the page they really should tell us that "Web Disk" is really just another name for WebDAV.

Once you get that under your belt tracking down problems becomes a little easier.

The next problem I ran into is, well, I'm trying to use Nautilus.

Nautilus is the Gnome file manager. For many users it's the entire interface to the disk. It should, above all other things, be reasonably fast, accurate and reliable. All else follows from that.

Unfortunately Nautilus is notorious for running down rat holes while not getting the basics done right. It has improved a lot in the last 2 years, but it still plays amateur night a few times too often. This was one of those times.

I went to the HostGator site and navigated down to the "Web Disk" page. I selected Nautilus and followed its suggestions. They wanted me to connect to the location "https://foo.com:2078".

I did and I got "Could not display 'https://foo.com:2078'."

Crud.

I had no idea where to proceed from here. Is it the instructions, or is my account screwed up, or maybe Nautilus is broke?

Just to add to the confusion, HostGator has you create a user name when you create an account with them. No big deal, I chose "wiles". What they don't really make clear is that this user name isn't your domain user name. If you register the domain "foo.com" and want mail to come to "wiles@foo.com" then you need to set up another account called "wiles@foo.com". When I connect with Nautilus, should I use "wiles" or "wiles@foo.com"? The correct answer is neither, but that part's not HostGator's fault. More on that later.

So now I have and unknown protocol, with an untrusted browser connecting to a possibly mis-configured account using an unknown user name. You have to admit, it's kind of a challenge. Right?

I Prefer the Term "Challanged"

I'm not one to back down from a geek challenge, so into the fray I went. I ended up spending over an hour chopping around the Internet tubes trying to figure out what's going on. I got bits and pieces, but nothing I could really sink my teeth into.

I also wasn't sure of the format of the URL. For most services you can load in your user name into the URL and you don't have to type it in each time. For example, if you're using File Transfer Protocol (ftp), and the URL is ftp://foo.com, then you can include your user name with ftp://wiles@foo.com.

Does that work with "Web Disk"? What if I'm supposed to be using wiles@foo.com. Is it ftp://wiles@foo.com@foo.com or ftp://wiles%40foo.com@foo.com.

"%" is the HTML escape character and "40" is is ASCII code for "@" in hexadecimal. You knew that didn't you? I think it's a real sign of progress when connecting to a web site only requires knowledge of arcane character encoding and 2 number bases.

How about if I just create a bookmark in Nautilus, and then edit each variable separately and see if I can get anywhere? Nautilus will handle the details right? If you watch slasher moves there is always a part where some idiot says "lets split up!" The music they play at that point belongs here.

Scooby Do Takes and Ax to the Forehead

First, I'm not going to get anywhere until I figure out the real protocol being used. "Web Disk" criminy! Why not just call it "Disk Huggy Bunny" and be done with it.

I went back to the HostGator page. It listed a few other OSes and other browsers. The Mac options were mysterious. The Window options were baroque. It's not looking good. My last hope: Under Linux there was the option to use Nautilus's arch enemy "Konqueror". I selected it and it offered a download for my system. Anonymous downloads makes my toes itch, so instead I sent the file directly to a text editor to check it out.

There, large as life, is the line "URL=webdavs://foo.com:2078".

I now knew the protocol. We were moving forward.

I created the bookmark in Nautilus, set my user name, picked "Secure WebDAV (HTTPS)", and set the port.

I'm not sure about the start folder. Is it "/" or "/home/wiles" or "/public_html/wiles@foo.com"? I held my nose and left it blank.

I clicked connect and I'll be darned! I got a password prompt! Kewl. I typed in my password and crossed my fingers.

The error I got is "Could not display 'davs://wiles@foo.com:2078/'. Error: Not a WebDAV enabled share. Please select another view and try again."

This is a truly beautiful error message. First, it did *not* connect using "davs://wiles@foo.com:2078/" (more on that later). Second, the share is, in fact, a WebDav enabled share. And third, "Please select another view and try again." gives me absolutely no useful information. It was a perfect storm of rotten interface.

At this point I was pretty frustrated, and lets not kid ourselves, at this point most people have already given up. Fortunately I'm not most people. I'm a hard core uber-geek and I eat bad interfaces for breakfast! Snort!

I start creating bookmarks like they're going out of style. I try every combination I can think of. It's slow, it's tedious, it's everything a computer shouldn't be, but alas, are, and in the end, it was futile. I could not get Nautilus to connect to my HostGator site.

Nautilus: A Weasel in Drag

Hmmmm. Do I trust Nautilus? Would I lend it $10 if it asked? No. No I would not. Then why would I believe that its rotten bookmarks are doing what they say their doing? It's time to go back to first principals and start over again.

I knew, with reasonable confidence, that I'm trying to make a WebDav connection. I also think that the "s" in WebDavs" stands for "secure", as in a Secure Socket Layer (SSL) connection. When you drop the SSL requirement at HostGator, the socket number changes from 2078 to 2077. That was enough for me to make another try.

I fired up Nautilus, and in the "Location" bar I typed "webdav://wiles@foo.com:2077". I got back "Nautilus cannot handle 'webdav' locations." Wow. What a nice, simple and useful error message. Are we sure this is Nautilus talking?

"WebDAV" isn't going to work, but I've also heard it called just "dav". I tried that.

With "dav://wiles@foo.com:2077" I got the password pop up! I also got the "select another view" error message. Arg!

Blinky the Wonder Idiot

Then the dawn came. Does anyone else see the huge blinking "Dale is an idiot!" sign in the previous paragraph? It took me a while to see it.

Even though it's the standard, and Nautilus is reacting to it, do I really know that the "wiles@" part of the URL is kosher? What happens if I don't use it?

Into the breach with "dav://foo.com:2077". This time the pop up asked for user name and password. I used "wiles" again and typed in my password.

I connected like a 13 year old with an evangelist. Son of a ...

I had spent more than 4 hours trying to get this to work, and it all comes down to this one sentence. It really doesn't make HostGator or Gnome/Linux look good does it?

I wasn't where I want to be yet, so I kept pressing forward. I was going to get SSL working.

"davs://foo.com:2078". Yup. I was in. Sort of anticlimactic isn't it.

One last thing. I really wanted to be able to connect to HostGator by clicking on a shortcut on my desktop.

The solution is trivial. Right click on the desktop and select "create launcher". Give it a name, and in the "Command" field type "nautilus davs://foo.com:2078" or "nautilus dav://foo.com:2077", depending on whether you want to use SSL or not. (You do, unless it you can't get it working.)

That's it.

I'm sending this URL to HostGator so they can fix their documentation, but until they do, enjoy updating your HostGator sight via Nautilus.

Tuesday, July 7, 2009

A Window In The Ghetto.

One of the more baffling elements of Linux distributions is their constant relegation of scripting languages to the command line ghetto.

On one side you have the all the wowwie zowie windowy programs, most of which are written, for no good technical reason, in C. On the other hand you have thousands of useful Perl/Python/Java programs that rarely get used because they have command line interfaces.

If we really want to harness the power of Gnome, then we need to make it easier to write Gnome programs in the most popular languages that Gnome supports. These languages are scripting languages.

Scripting exclusively for the command line almost made sense 10 years ago. Old versions of scripting languages didn't really interface well with the Graphic User Interface (GUI) and Text User Interface (TUI) libraries. You often had to have completely separate interpreters for GUI and non GUI interfaces. Anyone remember perlx?

Now days that's bunk. Perl, Python and Java all have officially supported interfaces to GTK, and, in case you don't know, they're *MUCH* easier to use than the C interface. I'm talking an order of magnitude!

As for the TUI, how many tasks do server administrators do that would be made much easier with a simple curses interface?

Python includes a Curses interface by default (it's why I learned Python). Perl isn't too far behind with a very stable Curses package on CPAN. This could be added to any distribution in a matter of minutes. I'm not sure about Java, but if it doesn't exist it wouldn't take long to make one.

The point being, that we have to start pushing distributions to include text and graphic interface libraries with the languages they support, and give them the same status as the language. If the language is included by default, so are the GUI and TUI libraries. Don't let your hot Molly Ringwald fantasies blind you. The '80s are over. We need better interfaces.

Once the user base can depend on the libraries being there, we're going to open up Gnome to a whole to set of ideas. These new programmers greatly out number the current set of Gnome C programmers. They'll be able to fix user level problems that we hardcore programmers don't even know exist.

If you're convinced at this point, then start pestering Ubuntu. I think they're the closest to having all the pieces. Then we can go after Debian and then we'll gang up on Red Hat.

If you're not convinced, then let me over make my point with sort of a preemptive FAQ.

* We didn't do it last time, why do it now?


OK, you lead with an Ace.

First of all, modern bindings to the major scripting languages are dependable and stable. This wasn't true before.

Also, we're in a much more graphical time. Most users consider dropping to the command line a failure of the interface.

We need a bridge between the two.

* What if the bindings disappear?

The odds of the supported bindings disappearing any time soon is negligible. As for other bindings, if more people start using them, the more dependable they become. Does anyone really expect "vim" or "sendmail" to disappear any time soon?

* What's wrong with C?

Nothing, in it's place. If I were to suggest dropping all the scripting languages and doing everything in C, I'd be laughed off the Internet. Somehow we're supposed to believe that the argument is less absurd when you pop up a window.

* Distribution X has the bindings in the "Extras" section. What's wrong with that?


In the business world getting managers to allow "extras" on a system is a hard sell. Damn few middle managers get fired for failing to innovate.

Besides, if it's in it's own section then there must be something wrong with it. Right? Can I get back to you on that?

* You can get all those bindings at site X.com.

If the "Extra" argument is a hard sell, then the 3rd party site is a no-sale. They're right on this one.

A company pays for Red Hat license instead of using the free CentOs because it gives them someone to yell at if things go boom. If you depend on 3rd party software then you get sent to finger pointing hell.

I currently have a problem with my CentOS box at home. I'm using CentOS repositories and a few other reliable sites. Two sites depend on different versions of the same library. I can't upgrade until I find and remove the conflicting programs.

Run that past a manager and you'll be a Microsoft shop by the end of the day.

Besides, if the software is trustworthy and useful then it would be include with the distribution. Feel free to repeat this until your head explodes. It's what they do in corporateland.

* If you want to use the OS, learn the commands!

If you want to make me learn a bunch of esoteric commands, I'm using another OS!

Let's use the "chage" command as an example. It's a pretty simple command which I use about once a year. Every time I use it I have to look up the command flags, because I only use it once a year.

Now wrap it in a curses interface. I no longer have to use the man page. "chage-curses" pops up the user's info, lets me change it. It even uses a calendar to help me change the dates. It then munches up the interface changes into a command line call which make the actual changes.

Take it one step farther. I am a hot shot l33t Hax0r. I shouldn't be doing this peasant crap at all. I create "chage-gtk" in 30 lines of Perl, and now my helper monkey does all the chaging while I hack the cosmos.

* Surly you don't mean every scripting language?


Yep I do. For server admins the command line will live on forever. It's the easiest way to make bulk or automated changes.

Your average user isn't a server admin. We need to stop dressing them up like one and making them dance for us.

Every general purpose language that has a stable GUI and/or TUI and is already included in a distribution needs to have those bindings included. We need to show non-Linux users, and other programmers that we're serious bout getting out of the 1980s. That means that any general purpose language that can help us out needs to be embraced.

* What about languages that don't have stable GUI/TUI bindings?

Most of the popular languages have had stable bindings for years. Perl, Python and Java all have officially supported GTK/Gnome bindings.

If a language only has a stable TUI, but not GUI binding, then just include the TUI. If it has no stable bindings at all, then it remains command line only until it gets it's act together.

* Some languages have GUI/TUI bindings that are a bitch to make into packages.


Let the developer of both the bindings and the language know that that's the reason the binding isn't being included. Most will gladly fix their code or provide the distributions with packages.

* Jeez, our distribution is getting awfully big.

One: The graphic libraries shouldn't really be that big, they're just interfaces to existing libraries.

Two: This move is important! If going from a command line interface to a graphic interface is only being held back by a pack of Luddites, then Linux doesn't deserve to play with the big boys.

Three: This is also an advantage that we seem to be afraid to exploit. Does any other OS come with, by default, an easy way to create useful window based programs just using a text editor? We shouldn't be hiding this feature, we should be shoving it down peoples throats!

* Why should we include fooscript when real programmers use barscript?

You're an idiot.

This isn't about the size of your digital penis, it's about removing completely artificial barriers from the users. Gnome doesn't really serve the clever user well. This problem is trivial to fix. We need to do so in the most inclusive way.

If Perl gets the job done for you, great! If you like Python, no problem. Someone at the Guile compound must have got laid because it's perked right up in the last few months. Guess what? It has both a curses and Gnome interface. Welcome to the club Guile! (Assuming your bindings don't suck.)

* Real programmer program in C.

Actually they don't. Most casual programmers start by looking at a program that almost does what they want, and then pick at it. If the program is a script, it's much easier to play with.

For the inquisitive user, in real terms, modern computers are less powerful then they were in the days of Dos and Unix. Back then a curious user who had an itch, could scratch it with a batch file or shell script. This new command was on par with anything else on the system. If it was generally useful they could email it to anyone else and they could use the new command too. Now days a clever user has to either waste monkey cycles writing in C or they have to have other users download the script-gui library package.

Pithy Summary


This is a stupid pointless wall, and it needs to come down!

Tuesday, June 23, 2009

Emacs: P3 Separate But Equal.

In the first post, we went over some basic theory. In the second post we calibrated Emacs's concept of terminal color with the reality of our terminal program. We also created a file called "color_test.el" which is useful for showing the common "faces" used in programming.

I this part we'll explain what a "face" is and show to get terminal faces and display faces to play nice.

Faces

A "face", in Emacs parlance, is all the characteristics of a piece of text. This includes it's font, size, whether it's bold or italics and it's color.

In the olden days, faces were created by hand. They're not too bad once you get a hold of them and they're surprisingly flexible.

Emacs faces can do all sorts of snazzy things like auto-detect whether they're on a terminal, change their color if the background color changes and invert themselves if they're on a black and white screen. Here's an example taken from the Emacs Elisp manual, section "Elisp/Display/Faces/Defining Faces".

(defface region
`((((type tty) (class color))
(:background "blue" :foreground "white"))
(((type tty) (class mono))
(:inverse-video t))
(((class color) (background dark))
(:background "blue"))
(((class color) (background light))
(:background "lightblue"))
(t (:background "gray")))
"Basic face for highlighting the region."
:group 'basic-faces)

Alas, you young punks don't wanna do it by hand. You'd rather use Emacs's built in customizer. Fair enough.

The Customizer.

I'm not going into a lot of detail here as there are other web resources dedicated to customizing Emacs.

For the sake of this article we need to know:

  • "M-x list-faces-display" shows you all the faces Emacs knows about.
  • Pressing when your cursor is on the face name will let you edit it.
  • In programming mode, all the fonts we care about begin with the wildly intuitive name "font-lock-".
  • When you're customizing in terminal mode don't forget about "Weight Bold" and "Weight Light". In most terminal emulators they give you extra colors to play with.


Multiple Customs.

If you're using a new version of Emacs, you can go into the customizer, click on the "state" button and select "Show All Display Specs". Then click on "Display" and choose "Check List". This lets you select the modes that you want the changes for. If this gets the job done, then you're done. I've had trouble with edits in TTY mode stomping on my edits in display mode, so I like to keep the variables separate.

After much (and I do mean multiple days) experimenting, I've chosen a more robust solution that is easier to maintain and is a lot harder to stomp on.

Whenever you customize a face and save it, Emacs replaces the function "custom-set-faces" with a new version that has your changes in it. The change is written into your custom file. This file could be the end of your ~/.emacs file or the file named in the "custom-file" variable. I'll use the generic "custom file" because I don't care where it actually is.

The way I handle multiple customizations is to customize them via the customizer. Then load the custom file back into Emacs and rename the custom-set-faces function so it only fires when you're in either terminal or display mode, but not both.

It's very easy to do and it's mostly cut and paste.

Load your Emacs custom file. Before any "custom-set-faces" commands, add these 2 functions:

(defun my-custom-set-faces-display (&rest faces)
"Load these faces if Emacs is in windows mode."
(when window-system
(apply 'custom-set-faces faces)))
(defun my-custom-set-faces-terminal (&rest faces)
"Load these faces if Emacs is in terminal mode."
(when (not window-system)
(apply 'custom-set-faces faces)))

Now look at your current "custom-set-faces" command. Is it set up for display mode? Then rename it to "custom-set-faces-display". If it's for the terminal then rename it "custom-set-faces-terminal". Now add an empty function call for the "other" function. If you set custom-set-faces-display, then add "(custom-set-faces-terminal)". If you set custom-set-faces-terminal, then add "(custom-set-faces-display)".

Mine looks like this:

(custom-set-faces-terminal
'(font-lock-function-name-face
((t :foreground "LightlyDepressed" :weight bold)))
'(font-lock-comment-face ((t :foreground "cyan"))))

(custom-set-faces-display)

Now fire up Emacs in terminal mode (emacs -nw) and edit a face. I'll make the font-lock-comment-face "Naval" colored for this example. Then save the change. Take a look at your custom file and you should see the color change in the function "custom-set-faces". Here's my example:

(custom-set-faces-terminal
`(font-lock-function-name-face
((t :foreground "LightlyDepressed" :weight bold)))
`(font-lock-comment-face ((t :foreground "cyan"))))
(custom-set-faces-display)
(custom-set-faces
;; custom-set-faces was added by Custom -- don't edit or cut/paste it!
;; Your init file should contain only one such instance.
'(font-lock-comment-face ((t (:foreground "naval"))))
'(font-lock-function-name-face
((t :foreground "LightlyDepressed" :weight bold))))

Delete the old custom-set-faces-terminal. Rename custom-set-faces to custom-set-faces-terminal, save your work and you're done.

If you wish to edit your display faces, just fire Emacs up in display mode and run through the same process.

There's no limit to the number of face sets you can add. You can have a different face set for every day of the week if you want. Just create a "my-custom-set-faces-" for any discriminator you want and rename custom-set-faces to match it.

Hopefully my absurd 10 day journey into Emacs's faces has been rendered down into something useful for you. Let me know if you found this helpful.

Monday, June 22, 2009

Emacs: P2 Color Me LightlyDepressed.

Now that we have some color theory under our belt, let's calibrate Emacs's concept of color with the reality of the terminal's.

First we have to get the real colors being displayed. I'm using gnome-terminal which has a built in color picker. If you're using a terminal that doesn't have it's own color picker, fire up "emacs -nw", do "M-X list-colors-display", then use something like "xmag" or gimp to get the color values.

My Color List.

From a gnome-terminal, select "Edit/Current Profile" from the menu. From the "Default" screen, click on the "Colors" tab. At the bottom of the screen you should see 2 rows of 8 colors. The first row is the 8 colors that make up the terminal's pallet. Left most is entry 0, right most is 7. The second row are the colors you get when you print the first row using "bold". Gnome-terminal has a 3rd row of colors that are the first row in "dark" mode, but you can't edit them.

If Emacs was smarter about terminal colors you could tell it about all 3 rows of colors and it could use "bold" and "dark" version to increase the chance of it's making a good color choice. Alas, were stuck with our one row of 8 colors.

Click on each color in order, and write down their Red, Green and Blue (RGB) values. For example, the 4th color in my pallet is kind of brown, with yellow below it. It's RGB value is 170/85/0, so pallet entry 3 is 170/85/0.

This is my list:

0 0 0 0
1 170 0 0
2 0 170 0
3 170 85 0
4 0 0 170
5 170 0 170
6 0 170 170
7 170 170 170

Once you have all 8 values, ask Emacs (in a terminal) for help on the variable "color-name-rgb-alist" (C-hv color-name-rgb-alist). The help should list all the color names that Emacs knows and their RGB values.

Scan the list for colors that match the gnome-terminal colors. If you find a *perfect* match, put the color's name besides it's color in your list. Only use the name if it's a perfect match. 0/0/0 was the only match for me. I labeled color 0 "Black".

For the rest of the colors, give them descriptive names that are not in color-name-rgb-list. The last thing we need is 1 name for 2 colors.

Here's my final list.

Black 0 0 0 0
Brick 1 170 0 0
Greeny 2 0 170 0
Brownish 3 170 85 0
Naval 4 0 0 170
DarkishMagenta 5 170 0 170
NeonPee 6 0 170 170
LightlyDepressed 7 170 170 170

Now we have to get the colors into Emacs. It turns out that that's pretty easy.

RCS

First, make a backup of your ~/.emacs, just to be safe. As an aside, because this series isn't nearly long enough, consider using RCS to backup any config files that you hand edit. Under Emacs RCS is trivial to set up and use. It's saved my monkey boy butt more times than I care to remember.

To set up RCS for your ~/.emacs, make a directory called ~/RCS. Then load your ~/.emacs file into Emacs. Hit C-xvv. That's it. You're done. Your ~/.emacs is now write protected and checked into ~/RCS. To check out your file so you can edit it, load it into Emacs and hit C-xvv.

Back to work.

Edit your ~/.emacs, and add the following code. If your ~/.emacs has a custom-set-variables or custom-set-faces function, place this code before either. Obviously you should use your own colors and names for the my-tty-color-define-8 commands.


;; Code for handling term based Emacs.
(defun my-tty-color-define-8 (name index rgb8)
"Set the tty pallet using 8 bit rgb values."
(tty-color-define name index
(mapcar (lambda (x) (+ x (* x 256))) rgb8)))

(if (and (not window-system) (= 8 (length (tty-color-alist))))
(progn
(tty-color-clear)
(my-tty-color-define-8 "Black" 0 '(0 0 0))
(my-tty-color-define-8 "Brick" 1 '(170 0 0))
(my-tty-color-define-8 "Greeny" 2 '(0 170 0))
(my-tty-color-define-8 "Brownish" 3 '(170 85 0))
(my-tty-color-define-8 "Naval" 4 '(0 0 170))
(my-tty-color-define-8 "DarkishMagenta" 5 '(170 0 170))
(my-tty-color-define-8 "NeonPee" 6 '(0 170 170))
(my-tty-color-define-8 "LightlyDepressed" 7 '(170 170 170))))

Save your ~/.emacs file and exit. Restart with "emacs -nw" Type "M-x list-colors-display". You should see your color names listed with the colors.

This might not seem like much of an achievement, but you've actually taken a pretty big step.

To check our your results, create a file called color_test.el in Emacs (-nw). It should put you into "Emacs Lisp" mode automatically. Now type in this program:


;; Comments are in 'comment-face'.
;; defun and defvar are in keyword-face.
(defun function-name-face (&optional is-in-type-face)
"string-face `constant-face' string-face"
:builtin-face
(error "warning-face"))
(defvar variable-name-face)

;; To see the "doc-face" go into "perl-mode".
=pod
This should be in doc-face.
=cut


The program itself doesn't work. It's not even syntactically valid. All it exists for is you show all various "faces" that Emacs uses when coloring code.

To see doc-face, use "M-x perl-mode".

How do you like them colors? If you're happy happy, then you can skip post 3 of this series. If, however, you're like me and think that Red is a horrible color for comment text, then await with baited breath the last installment of the Emacs color saga.

Sunday, June 21, 2009

Emacs: P1: What Color Is My Painbow?

Last week I had a classic "Monkey Boy" moment. I decided to adjust the colors in my text editor. 10 days later I'm finishing a 3 post blog on it.

I worry myself some days.

This first post is going to be mostly theory work. Post 2 and 3 are more hands on.

Laying the Ground Work.

I use an editor called Emacs for most of my programming. It's an old editor, but its one of the most powerful editors out there. It also lets you edit files in display (windows) mode and from the shell (command.com for you Windows folks).

Now days most of the editing is done in display mode. No real surprise there. However, there are times when working from the shell makes more sense.

I routinely log into distant machines across slow connections. I could pop up a virtual session and wait for the window in the virtual session and then wait for the editor in the window in the virtual session and then wait for the file in the editor in the window in the virtual session, or I can use text mode, where are complete screen refresh is around 2000 bytes.

I hate waiting. Its a no brainer.

The down side of terminal mode is that you can only use characters to draw and you have a limited number of colors. Both of these could be overcome with modern technology, but it ain't going to happen so we have to get used to it.

Why do you have limited colors? Well, the underling technology differences between a terminal from 20 years ago and a modern graphic display is pretty significant.

RGB

Colors are made by mixing various amounts of Red, Green and Blue (RGB) together. If you crank up the RGB, you get bright colors, dial it down and you get dark. Wikipedia has a nice write up on color depth so I won't go into it here. The only thing you need to know is that by adjusting the RGB values you can change colors.

On modern display you have absolute control of every dot on the screen. Each one has it's own RGB setting which is independent of it's neighbor.

Old school color terminals were more like "paint by numbers" projects. You were given a pallet of colors (usually 8) that were hard wired into slots. If you set the color to pallet slot 0 and then printed, you got black text. If you printed in color 4 you might get blue. Unfortunately for us, these are the terminals that most terminal emulators emulate.

We have two problems when we want use Emacs in both terminal and display mode: First is that Emacs's support of terminal colors is functional, but not much more. The second is that the friendly Emacs customizers don't like it when you're a switch hitter. In fact they gets down right medieval on your monkey butt. Well, this ain't monkey butt, this is monkey boy butt. Accept no substitutions.

Terminal Colors


As I said before, most terminal emulators model the old style, 8 color pallets. There are ways for a program to ask the emulator for the number of colors available, but there isn't any way to get the actual RGB of each color.

What does Emacs do? It guesses! If you don't tell it otherwise Emacs assumes that you have an 8 color pallet with the following colors:


Slot Name Red Green Blue
---- ------- --- ----- ----
0 black 0 0 0
1 red 255 0 0
2 green 0 255 0
3 yellow 255 255 0
4 blue 0 0 255
5 magenta 255 0 255
6 cyan 0 255 255
7 white 255 255 255


The numbers after the colors are how much Red Green and Blue that each color is supposed to have. 255 is the largest number you can express in 8 bits (1 byte) of data. There are places internally where Emacs uses 16 bit (2 byte) RGB values which go from 0 to 65535. I got bit by this more than a few times so I'll try to point them out, or gloss over them when I can.

Emacs cares about the RGB values because you (the user) set colors by name not slot values. If you set the color of something to "CadetBlue1" 152/245/255, then run in terminal mode, Emacs needs to figure out which of the eight colors CadetBlue1 is closest to. It uses the RGB values to figure it out.

Oh, by the way, the name "CadetBlue1" comes from a variable called "color-name-rgb-alist". To see it's contents, fire up Emacs in display mode and type "M-x list-colors-display". You'll see the colors and their names.

Let's do some hands on. From a terminal, type "emacs -nw". It should start an Emacs session in the terminal. In Emacs type "M-x list-colors-display". You'll get a listing of the 8 colors that Emacs knows about. Note: On some systems you get more than eight. Lucky you. The theory is still the same.

If you're like me, you notice one thing first off. These colors look nothing like their names! The Red might be brick colored. Yellow may look brown. And my white has tattle tale gray! What happened?

Easy. Emacs has no idea what colors your terminal's pallet is set to and it's guess stinks. How do we handle the miss-match?

One option is to change our terminal to Emacs's pallet. Then we can vomit and claw our eyes out. Basic colors tend to be rather harsh on the psyche.

The second option is to tell Emacs what our terminal is really packing. That's the subject of the next post.

Tuesday, June 2, 2009

Sound Bite Me!

One of the true pleasures of programming is finding some niggling little problem and solving it with a simple bit of code. If you can solve it in a couple of hours, even better.

When I blog I tend to speak what I'm typing. If it doesn't sound right when I say it, it probably wont sound right when you read it.

One of the problems I have when I blog is that I talk faster than I type. I talk faster than I think. I start typing, and then I get a flash of inspiration. I run the idea through my head, and then, half a virtual page later, I realize that I haven't written anything down.

Then I have to try to recreate the idea from memory, but the flash is gone. By the time I've rebuilt it, or accept that I've forgotten it, my original thought is out playing in the yard and won't come back.

It frustrates the hell out of me.

I started playing around with ideas to making blogging easier.

At first I tried to check out the state of computer speech recognition. I figured I'd just blab on in my blog, and then I'd go through and clean it up by hand.

Computer speech recognition is still slow, expensive and it still sucks. Trying to run a editing session without using a keyboard is slower than typing with 2 fingers. I also wanted to blog via Linux, so firing up a Windows product ain't getting the job done.

Next it tried to do some sort of integration of speech and text together.

I envisioned loading a sound file into a sound editor, where I could chop it up and move it around. While editing the sound, I'd join text to it. When I moved a blob of sound, text that went with it would move to. Eventually I'd piece a blog out of all my rattlings on.

I still think that this is an interesting solution, but man, it would be some work! I also I think I'd end up with a crappy sound editor linked to a crappy text editor. No dice.

I also had a minor epiphany. I'm not going to keep these sound files around forever. I'm just brain dumping to a file for a few minutes until I've finished typing my original though. After that I can replay the recording and transcribe anything I think is useful.

I already know how to record from a mic on my Linux box. Adding that to my new insight I wrote a shell script that turns on the sound recorder, dumps the contents of the microphone into a file and, when I stop recording, plays it back (that way I can tell if I forgot to turn on the mic or something).

It worked like a charm! Every time I needed to make a note, I just fired up the script, decided on a name for the sound file and away I went. It was a little awkward, but a big step forward.

I called the script "sound_bite".

After than I needed to come up with a way to play back my sound bites, so I started adding flags to play the last sound bite or the first sound bite or list the sound bites and let me pick. Then, another epiphany! They're just frigging .wav files! Maybe I could just double click on them in my file manager. Oooo. Me one smart monkey!

Actually, once I got the file manager into the game, it cleaned up a lot of code. I didn't have to tell the recorder where to put the sound files, I would always put them into the same directory and give them a time stamp for a name. If I wanted to organize them better, I'd use the file manager to rename them or move them elsewhere.

The only thing left was making it easier to use. Typing in the command every time is a minor pain. I needed a quicker way to access it. That was easy too.

I hooked up sound_bite to a shortcut which put all the .wave files into the directory "sound_bites". I made another shortcut to bring up the file manager in "sound_bites" directory. I'm sure I could come up with a few dozen little tweaks, I know enough to stop typing when I'm done.

I'm now an official audio driven blogging fool!

Here is the entire source code for sound_bite. You may have to play with it a bit because the blogger code likes to play with it.

Enjoy!


#!/bin/sh -

# Sound_bite: Written by Dale Wiles 6/2/09.

# Exit if an error occurs.
set -o errexit

if [ $# -eq 0 ]; then
  cat <<EOM
Usage: $0 directory

Move to DIRECTORY and start recording a wave file from the microphone.
The name of the wave file is yymmdd_hhmmss.wav.
EOM
else 
  sound_dir="$1"
  cd "$sound_dir" || exit 1

  # Make the output name based on the time, down to the second.
  # That way I can't overwrite existing files.
  # Alright, in theory during daylights savings time it could
  # overwrite.  I've added code for that almost impossable situation.
  while :; do
    out=`date +%y%m%d_%H%M%S`.wav
    if [ ! -e $out ]; then
      break
    fi
    echo "Waiting...."
    sleep 1
  done

  echo "Recording $sound_dir/$out"
  sox -t alsa default -v 7 $out
  echo "Playing $sound_dir/$out"
  aplay $out
fi

Saturday, May 16, 2009

Clones Ate My Files: A Sci-Fi Geek Thriller.

OK, so I'm banging out code for the betterment of my corporate overlords. It's a conceptually simple program: You give it a command and a list of servers and it runs the command against each server.

Not impressed? Well Monkey Boy doesn't get the corporate shekels without doing major mojo. This program runs in parallel. It juggles 64 instances of the command at a time. It can ping all the severs on a netmare in less than a minute, keep track of the results and print them out in the order that they were input. I get wet just thinking about it!

All praise Monkey Boy right? We'll I did run into a problem. A sneaky problem that involved failed clones, suicidal files and a forgotten inheritance. It's good stuff. The problem is, if you don't program in Perl, you're not going to give a crap. Oh well, I've never let my complete lack of an audience slow me down before. Why start now?

Like I said before, the beast is written in Perl. Perl's not an elegant language, but if you need to leap out of the bushes, rape and strange a problem and get on with your life, then Perl's your language of choice.

The basic layout is: Start a sub-process for the first 64 severs and have each one write to it's own temp file. When one sub-process finishes, read it's results from the temp file and let the temp file disappear. Then add a new sub-process for the next server. Once all the servers are done, print out the results in order and accept smoochies from hot code groupies/naked underwear models. Simple.

The Boy of Monkey knew that he'd need lots of temporary files to hold command results. He'd also like the files to go away by themselves when he's done with them. Perl lept to his aid with File::Temp. File::Temp is kind of the anonymous underage prostitute of programming. When you say "Gimmie" it gives you access to the goodies and provides an assumed name. When you're done using it, it disappears into the aether. It all works great, until someone, or something, starts killing the temps prematurely. Then you get a mystery to solve. Foreshadowing!

I got the code working and was getting ready to document (Yes, Monkey Boy is a pro, not documenting makes you a douche bag), when I though, what happens if the sever command can't run? If the user misspells "ping" as "pong" will they get a reasonable message?

Does
Couldn't open file '/tmp/multi.1d834.pid': No such file or directory
strike you as reasonable? Me neither. Nuts!

The actual error message made sense to Monkey Boy. He spawned the beast. '/tmp/multi.1d834.pid' is one of the randomly generated temp file names. When a sub-process finished, the program asked the temp handle for it's file name. It tried to opened the file, but for some reason the file was gone! Somehow the files were being killed before Monkus Boyus could get to them. No one should know about these files. They have specially constructed names, known only to the monkey... or the monkey's clone.

The way you run a sub-process in Linux is using the fork() command. You're running along happy as a clam, as if a mucus coated bivalve is your apotheoses of happiness, and then you hit fork(). At that point your program is cloned. You have 2 running copies of the code. The only difference is that fork() will tell the parent the ID of it's child. The child is handed the ID of 0, which tells it that it's the clone.

This is kind of Star Trekie at this point ain't it? We've got parents making 64 clones (top that Octomom!) and we've got children that are one bit away from being perfect copies of their parent. It gets better. The clone's next job is to call exec() which completely obliterates it and replaces it with another program. It's this second program, the sever command, which does the real work I want done.

A clone has one job. It's job is to die and be forgotten. Programming ain't for wussies!

When all goes well, the program runs like a well oiled roach motel. The clones check in, but never checkout. They disappear on the spot and are never heard from again.

That's when all goes well. What happens when the exec() command fails?

It's simple really, the clone lives on! It also keeps it's copy of the temp files, which it believes it owns. When it dies, it takes the temp files with it. Clones can be selfish little pricks.

Eventually the grieving parent checks on the child. It notices that it's died and then tries to check the temp file for the reason. The temp file is gone baby gone, it died at the hands of junior. You can only die once in temp file land.

As for the reason the exec() failed? It was written into the temp file. You know, the temp file that's in temp file heaven? Hmm. What to do? What to do?

Suddenly Monkey Boy (you remember Monkey Boy, he's the hero of this epic) has an insight. When you delete a file it doesn't really disappear until the last program that has a hold of it lets go. It's removed from the directory, so it can't be seen, but it's still out there, in limbo, awaiting for the sweet kiss of digital death.

Who else is holding on to the file? The parent of course! The question was, could Monkey Boy get to the parent to cough up the handle and could it be used for reading?

Detective Monkey Boy began investigating. He checked the usual suspects. "perdoc File::Temp" didn't provide much. It was higher level than a kite. The Internet tubes were blocked by flame wars and almost naked pictures of some platinum blond from California. No go. Detective Monkey Boy knew what he had to do. He had to go (non-prequil) Jedi. "Use the Source Luke!" is the rallying call of the Open Source movement. But Monkey Boy's name ain't Luke.

Into the source goes the hero. Past lines of documentation. Past obscure code references. Further he goes, until he finds, what he knows in is heard must be, "use IO::Handle". Rocken!

For those that don't know, deep in the belly of the beast, a file comes down to little more than a number. When you open up a file, voodoo happens, and an entry is put in a table called the File Descriptor Table. What you deal with, either directly or through some Perl interface, is an entry in this table. If you can figure out the index number to this table, you can find your file. File::Temp was a cold fish, but what about it's ancestors? File::Temp inherits from IO::Handle. To get to the real power, you got to seduce grandma.

"Hey there Granny!"

Once I go my hands on Granny's nodes (yech!) she gave up IO::Handle. IO::Handle has the fileno() function. You got the number, you get the data.

After that it was just a hop, skip and a file dupe to get to the data so ingloriously killed off by the wayward clone. It takes more than the death of a temp file to stop a motivated Monkey Boy!

All praise Monkey Boy.

Friday, April 10, 2009

Phoning it Home.

I have a Samsung SGH-A707 cell phone. It's nothing amazing, but it has a few features that I like.

One of them is voice memo. I can hit a button and record a quick message on my phone. Later, using Samsung's crappy PC Studio, I can pull the sound files off my phone and onto my Windows box. From there it's a hop, skip and a Samba on to my beloved Linux box.

The problem I have is that the files are in a format called ".amr". Oy! God forbid they use something standard.

Fortunately it's not too hard to convert them to a format that the "sox" command can understand. From there you can go to .wav, .mp3 or .ogg.

First, get the sound file from your phone to your Linux box. I can't get the bitpim program to work on my Red Hat box so I go via Windows.

Once you have the file, I'll call it "foo.amr", on to your Linux box, use the program "amrnb-decoder" to convert it to raw format.

amrnb-decoder was already on my Red Hat box. You may have to hunt around for it.

Decode the file with:

amrnb-decoder foo.amr foo.raw

This produces a raw, signed, 2 byte word, audio file at 8000 hertz.

Convert it to an ogg file with:

sox -r 8000 -s -w foo.raw foo.ogg

You should be good to go. If you change foo.ogg to foo.wav you'll get a wave file. Creating an mp3 is left as an exorcise for the reader.

You can simplify this a little bit by naming the raw file "foo.sw". Sox sees files that end with ".sw" as being short hand for "raw, signed word" files so you don't have to include that info on the command line.

You could do the above as:

amrnb-decoder foo.amr foo.sw
sox -r 8000 foo.sw foo.ogg

If the volume of the final .ogg file is too low, use sox to increase it:

amrnb-decoder foo.amr foo.sw
sox -v 7 -r 8000 foo.sw foo.ogg

For some reason all the other instructions sets I've seen have had dumb little typos in them. Let me know if you find any here.

Saturday, March 7, 2009

Just an Artist without a Canvas.

About 3 weeks ago I posted a question on a user form at gnomesupport.org. The question was whether or not a particular widget (the Canvas) was being actively supported and, if not, how to replace it's functionality.

After 3 weeks I've had 125 looks, but no answers.

The problem is that it's not really a technical question, it's more of a "which direction is Gnome going" type question. A regular user can't answer it. It requires an opinion of someone higher up in the Gnome hierarchy. Apparently they're not reading the forums.

Gnomesupport.org is the official support forum for Gnome. We need for people higher up to scan these boards once in a while or Gnome is going to suffer from "user rot". If we don't know Gnome's direction, we waste time going in the wrong direction. If it's wasted time, then why spend it on Gnome when I can be productive somewhere else?

We seem to be falling into the "self maintaining user-base" fallacy. It's been growing pervasive in many projects in the past few years.

It goes something like this: Because users can share information, developers need only provide a platform for sharing and the users will take care of themselves.

Um, that don't work. Some questions can only be answered by the developer or someone who has a big picture view of a project. A user can't tell another user the "correct" way to do some things unless they've received the word from someone higher up.

I'm not saying that the people running Gnome should spend all day monitoring the user forums, that's petulant and wastes developer's time. However, someone with access to the top, should check the forums for questions that have been in the queue for more than a few days and don't have answers. The questions that aren't being addressed by other users should be looked over and, if relevant, kicked up to the next level.

There should also be someplace where people can have limited access to high level Gnomes (Uber-Gnomes?) to ask "big questions". Things like "I'm thinking making Fortran bindings for Gnome, does anyone else think that's awesome?" or "I don't like sound server X, let's write a replacement." or "Hey Uber-Gnomes, what do you think is most needed in Gnome?"

The place would need to have the participation of the higher ups for it to be useful. Spending a year writing a replacement for sound server X, when there is already an official project in place to replace it with Y would be a colossal waste of time and programmers. People need to know that before they waste time.

Friday, October 10, 2008

Gnome Needs a Scripting Language.

Gnome needs a scripting language that fills the same scripting role in Gnome that Bourne shell fills for the command line.

A few days back I wrote a little script that pops up a window, gets some input from the keyboard, runs a program and then exits. I banged it out in about 2 hours in Perl using the Gtk2 library. Piece o' cake.

The problem is that I really can't distribute the program. Even if I assume that all Gnome systems have Perl, it's not safe to assume that they all have the Gtk2 libraries installed.

If I want to make my little script available I'm going to have to rewrite it in C? That can't be right! Linux always has scripts. It's one of the things that separates Linux from the lower beasts!

I decided to check out the current state of Gnome scripting on Google. Um, we're exactly where we were a decade ago. 10 years ago people were bitching about the lack of an official scripting language for Gnome and we're still bitching today.

Let's stop bitching and fix this insult to Gnomehood. For now we'll call the language "gscript" to save typing. If there is already a "gscript", then accept my apologies and lets get on with life.

What Gscript Needs to Be.

Here is a list of what I think gscript needs, in approximate order of importance.

  • It has to exist. This first entry isn't as stupid as it sounds. Good Gnome interfaces have existed for many of the existing scripting languages for years, but none of these scripts/interfaces are *guaranteed* to be there. Seriously, in the past decade, how many GTK/script have you used? For it to be useful, gscript has to come with Gnome. If you're using Gnome, then gnome-terminal may be optional, gscript is required!
  • It needs to be a Gnome scripting language, not just another scripting language that comes with Gnome. This is *important*! There has to has interfaces to all major parts of Gnome, not just GTK. Things like GConf and EggTrayIcon and user menus have to be accessible to gscript. When a new version of Gnome comes out, it comes with a synced version of gscript. It's not optional.
  • It should have useful defaults. Banging out a simple script should be, well simple. About as simple as this:
#!/usr/bin/gscript

init;
guts = gtkbutton(label => "Hello World", callback => exit);
; Note: A default also menu gets created here.
start_app(body => guts);
  • It should work with both GTK and Gnome. I'm not really sure what separates GTK from Gnome, but if the script doesn't need the Gnome stuff, then it should be able to use just GTK.
  • It should do things the Gnome way. One of the big bitches about Gnome is there's always a new way to do something. No one ever explains, in simple terms, why the new way is better and how to use it properly. If the right way to read a keypress from an international keyboard is to use the foobar interface, then gscript uses the foobar interface.
  • Gscript should be simple. We're not trying to replace C or Python or COBOL. The day will never come when 90% of Gnome is written in gscript. It should exist to fill the niche that shell scripts do on the command line. Anything else is gravy.
  • Gscript shouldn't be too simple. We've all worked with languages that are so simple that they're useless. It should have all the control structures of a real language with scoped variables and real data types. At a minimum it should have native lists, hashes, numbers and strings (anything else?). It should also have modules and classes to handle medium size programming. GTK-Bash would be a lousy choice.
  • It should be embeddable. Gnome is moving to things like cellphones. Why not kill two birds with one stone? Embeddable languages are useful and this will help keep the language small and simple.
  • Screw the ASCII code. Enough already! International support should be the default, not an add on. Those strange voices across the big salty pond are other programmers. If they speak Chinese then they should be able to script in Chinese.
  • Screw C! Most scripters don't come from a C background anymore because they don't need to. Computers are a zillion times faster than they were when I was a whelp, and no one is going to put gscript code in the kernel. Gscript should be fast enough to be useful. After that ease of use and flexibility are paramount.
  • Lets not create a new language unless we have to. I believe we can leverage an existing language.
  • If we have to create a new language, lets go for it! A lot of current languages are anachronistic retreads of boring old languages. Why strive to suck?
My Opinion on Existing Options.

Let's run through some of the existing options. Despite my list above, most of what's needed already exists. It's just not included with Gnome by default.

  • Perl or Python : Both of these play very nice with Gnome/GTK. The problem I see with these two languages are that they're both serious overkill for this problem. Also choosing one over the other would come with a pre-built pissing contest. I don't want to see their GTK libraries curtailed. I personally use Perl::Gtk2 and love it. I just don't think they would be the best fit for gscript.
  • Tcl/GTk : Tcl tries too hard to be simple, which makes it painful for all but the smallest of programs. If it scaled better it would have taken over the world by now. Gscript should strive to be what Tcl/Tk failed to be.
  • Zenity : Its great for popping up windows in shell scripts, but it doesn't really let you run actual scripts. A new gscript programmer would be able to implement zenity in about a day.
  • lua-gtk, guile-gnome and ruby-Gnome2 : Of all the languages I talk about, these 3 strike me as the most plausible. They fit most, if not all of the requirements, they have and existing community and, according to http://www.gtk.org/language-bindings.html, they already have viable bindings. Anyone with experience care to chime in?
  • Squeek : Aw c'mon. The official Gnome scripting language being Smalltalk would rock! We could write Gnome scripts that expand programmer's minds and then go give the KDE folks noogies. Pull this off and we P0WN by existing. It may not be the most practical of the choices, but I'll bet it would be the most fun.
  • SomeOther : I'm open to suggestions. Give me your opinions. Any other projects already started? Just remember Gscript is not trying to replace all the other scripting languages. We just need a good choice that we can get officially distributed with Gnome.
In Summary.

Gnome not having an window aware scripting language is a decade old embarrassment. It goes against all that is Linux. The embarrassment is compounded by the fact that we have a lot of options that are ready to go, or could be ready to go in, literally, a matter of days. We just need to choose one.

Once a language is chosen, then the only big step is getting it included with the Gnome base packages. Anyone have an in with the Gnome "powers that be"? If not, then wanna write a script to storm a castle?