Hideous Bloat and Active Links.

By popular demand I've expanded my Emacs wiki from one function to an unwieldy two. The second function will scamper through a buffer and list all the active links found.

I use the same regexp that I did in the previous article so it links on CamelCase. To change the link style you just change the regexp. If you end up writing too many support functions then you may want to use that "variable" thing that all the cool kids are talking about.

The Code

(defun list-active-links (link-re &optional is-active-link-p)
  "Scan a buffer looking for links and list all the active links.

LINK-RE is a regular expression which matches the link text.
IS-ACTIVE-LINK-P is an optional function which takes the link text and
returns true if the link is active.  If not provided, `file-exists-p' is
used."
  (let ((buffer-name "*ActiveLinks*")
    (found-links ()))

    (if (not is-active-link-p)
    (setq is-active-link-p 'file-exists-p))

    ;; Gather up all the potential links and whether they're active.
    (save-excursion
      (goto-char (point-min))
      (while (re-search-forward link-re nil t)
    (let ((link-text (match-string-no-properties 0)))
      ;; If the link hasn't been checked, then save its value
      ;; and whether or not it has an existing destination.
      (if (not (assoc link-text found-links))
          (setq found-links
            (cons (cons link-text
                (funcall is-active-link-p link-text))
              found-links))))))

    ;; Now list out the active links.
    (if (get-buffer buffer-name)
    (kill-buffer buffer-name))
    (switch-to-buffer-other-window buffer-name)
    (let ((active-links
       ;; Remove any links that don't have an associated file.
       (delq nil (mapcar (lambda (x) (and (cdr x) (car x)))
                 found-links))))
      (insert "#\n# The following links have destinations.\n#\n")
      (save-excursion
    (insert (mapconcat 'identity (sort active-links 'string<) "\n")))
      (not-modified)
      (message "%d matches found." (length active-links)))))

(defun check-active-links ()
  (interactive)
  (let ((link-re "\\<[A-Z][a-z]+\\([A-Z][a-z]+\\)+\\>"))
    (list-active-links link-re)
    ;; Note: At this point we're in the "*ActiveLinks* buffer.
    (set (make-local-variable 'link-to-re) link-re)
    (local-set-key
     (kbd "C-c C-o")
     (lambda ()
       (interactive)
       (link-to link-to-re "\\<")))))

Emacs, the Wiki and the Idiot.

Sir C.A.R. (Tony) Hoare once wrote:

Inside every large problem, is a small problem—struggling to get out.

Someone else, stealing from H. L. Mencken, changed it to:

Inside every large problem, is a small problem—struggling to get out. The solution to that problem is simple, elegant and wrong.

I tend to agree with the Thief. The simple reality is that many problems don't have elegant solutions, try as we may to find them. I ran into this when I tried to come up with a simple personal wiki. Enjoy my exploits.

Dale Writes a Wiki

I think the wiki is one of the great software concepts of the last decade or so. That's not hyperbole, I really think that making it easy to link documents together, in a casual and non-invasive way, is a brilliant and under-appreciated concept.

Most wikis are web based. This makes sense if you want the world to have access to the pages. I was looking more for a personal solution.

There are personal wikis that are stand alone applications, but most of them use a special format for your data. This strikes me as, well, stupid. Why would I lock up my data in a special format when a directory of text files should easily get the job done?

I do a lot of my text editing in Emacs, so I started looking around for an wiki based in Emacs. There are a few, but most of them are either unsupported, too elaborate or too invasive. I wanted a way to link text from any kind of file, not just wiki files. I wanted to be able to link comments in source code to documentation files to cake recipes if that's what works best. I also wanted it to leave the rest of my Emacs environment alone. I didn't want to enter Wiki-Mode just to put links in a Perl program. I wanted to stay in Perl mode.

The joy of being a programmer is, if you can't find it, you can always write it. That's what I started out to do.

At first I decided to play with [[bracket style links]]. Emacs Org Mode uses them and I like much about Org Mode, so I tried to emulate its style.

The attempts where not too successful. I pulled out some of Org Mode's handler code and started playing with it. It's pretty invasive and doesn't work well with old versions of Emacs. I use an old version of Emacs at work, and I wanted something small and easy to understand.

My next idea was to write my own bracket style linking library. At it's heart it would grab the text between [[]], convert it into a file name and then open the resulting file name. 194 lines of Elisp later and I had a working minor mode. It could open files, web pages, internal links and even run commands. I was very happy with it. It was reasonably small and reasonably easy to understand and reasonably noninvasive.

Because I'm a professional, I started to document my results. The problem is, I'm a little bit insane. Sometimes, when I'm working on a project, a little idiot voice calls from the fog of experience and tells me what I'm doing is wrong. The Idiot never tells me what's right, it just picks at the back of my head until I accept, eventually, that my perfectly working code is "wrong". Arg!

The more I documented, the louder the Idiot got. I got so frustrated that I put the code aside. I'm a big believer in documentation by Idiot. You take your code and ignore it for a few weeks or even months. Then you come back and try to read it. Every time to look at a bit of code and say "What idiot wrote this?" you either re-factor or document. Maybe the Idiot could figure out what I couldn't.

The Idiot was insidious. It would go away for a week or so and then come out of the shadows to jeer hints. It asked questions that I should have asked. Do I need brackets? Do I need to open web pages and run commands from links? Are my needs the needs of others? What's more important, abstract power or agility? What is the DAO of the problem? Why won't you see it!?

This went for over a month. Part of me was trying to find the right solution. Part of me wanted me to finish what I had and get on with my life.

Then, on a Sunday night, as a long work day loomed ahead, I was laying in my bed with my beloved wife and Charlie the metal eating dog. From nowhere, it came to me. All I really want is a way to take the word under the cursor and open a file with the same name. Then it came to me. What I really want is a way to grab an arbitrary blob of text under the cursor and then open up a file based on the text. Then it came to me. What I really want is a way to grab an arbitrary blob of text under the cursor and do something with it. The Idiot had spoken!

It's a cliche, but I really wanted to leap out of bed and start hacking away. Alas, the days of 12 hour hackfests followed by 10 hour workdays are a thing of the past. I had to go to sleep or I would die at my desk. I had to go to work or I would be thrown into the street. I had to get this program written or I would crack up.

After work I lit in to my task. One of the greatest pleasures in programming is simplifying. The more I wrote, the smaller the program became. Irrelevance fell like rain. I was beginning to understand. By the end, the entire program is 15 lines long. 24 lines if you include the code to hook it up to the key of your choice.

It was simple, elegant and worked like a charm. The Thief was wrong. The Idiot stopped jeering.

Hooking Up the One Function Wiki

The simple solution is to take the text at the end of this article and append it to your .emacs.el file.

To test it, fire up Emacs, and type "This is CamelCase text.". Move your cursor to the link text "CamelCase" and type C-cC-o. It should open up the file "CamelCase".

If you want to use other link styles besides CamelCase or want to open files in different way, read the documentation. Doing things like opening files in a specific directory or opening "CamelCase.txt" are trivial to implement. You just need to do a little Elisp programming and away you go!

The Text to Append.

(defun link-to(link-re link-start-re &optional handle-link)
 "Grab the \"link\" under the cursor and open a file based on that link.

LINK-RE is a regular expression which matches the link text.
LINK-START-RE is a regular expression which matches the beginning, or text
just before the link.
HANDLE-LINK is an optional function which takes the link text and opens the
link.  If not provided, `find-file' is called with the link text.

This function is usually called via local-set-key with mode specific
regular expressions.

This example will grab a CamelCase link and open a file with the same
name.

   (global-set-key (kbd \"C-c C-o\")
     (lambda ()
       (interactive)
       (link-to
         \"<[A-Z][a-z]+\\\\([A-Z][a-z]+\\\\)+>\"
     \"<\")))  Note: The \"<>\"s above should have \"\\\" in front of them, but emacs
thinks I want to print a key map when I try to include them in the help
text.

This example will grab an alphabetic string and do a google query on it.

  (global-set-key (kbd \"C-c C-o\")
    (lambda ()
      (interactive)
      (link-to
        \"[a-z]+\" \"[^a-z]\")
        (lambda (x)
          (browse-url
            (concat \"http://www.google.com/search?q=\" x)))))"
 (let ((here (point)) link-text
   (case-fold-search nil))
   (save-excursion
     (or (re-search-backward link-start-re nil t)
     (goto-char (point-min)))
     (unless (re-search-forward link-re nil t)
   (error "No tag found to end of file"))
     (setq link-text (match-string-no-properties 0))
     (if (or (< (point) here)
         (> (- (point) (length link-text)) here))
     (error "No tag found under cursor")))
   (if handle-link
   (funcall handle-link link-text)
     (find-file link-text))))

(global-set-key
(kbd "C-c C-o")
(lambda ()
  ""
  (interactive)
  (link-to
   "\\<[A-Z][a-z]+\\([A-Z][a-z]+\\)+\\>" "\\<"
   ;; This regexp pair matches file names.
   ;;"[a-zA-Z0-9/~_][a-zA-Z0-9/._]*[a-zA-Z0-9]" "[^a-zA-Z0-9/.~_]"
   ;; This will open text files in your ~/Wiki directory.
   ;;(lambda (x) (find-file (concat "~/Wiki/" x ".txt")))
   )))

Reading Web Pages From the Android SD Card

Android 1.6 (Donut) tries to make it impossible to read web pages from the SD memory card. Fortunately there is a flaw in their security that lets a user create a specially formatted URL that allows access. This technique may work for other versions of Android but I've only tested it on 1.6.

This is an easy hack. The only skill required is the ability to drag and drop files from the computer to the phone and to connect to this web page via the Android.

Terminology

For the sake of simplicity the Android based phone is just going to be called the Android. It reads easier.

Local files are the files on the Android SD card. Any references to "the card" mean the Android SD card.

The Desktop is the computer that will be used to edit files and drag and drop the files to and from the Android.

"The browser" is the browser on the Android.

Overview

A web page is created on a computer and moved to an arbitrary, but known spot on the Android SD card. A special URL is entered in the Android browser which points to the page. The page is made into a book mark by bypassing the bookmark editor. The web page can then be edited to point to any other location on the SD card.

The Direct Way

This way is direct, but it involves a bit of typing on the Android keyboard. Most people find the longer way easier to get correct.

Put a web page the SD card. This example will use the file webpages/index.html.

Open the browser and type in the URL content://com.android.htmlfileprovider/sdcard/webpages/index.html.

If index.html shows up in the browser, go to the section "Saving as a Book Mark". If not, check the URL spelling and file location. If the problem can't be solved, use the longer way.

The Longer But Less Fussy Way

This way requires creating a web page on the desktop and the ability to drag a file from the desktop to the Android.

Open an editor ("notepad" will do) and paste the following text:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN">
<html>
<head>
<title>Found the sd card!</title>
</head>
<body>
<h1>It worked!</h1>
<ul>
<li><a href="content://com.android.htmlfileprovider/sdcard/foo.html">Self reference.</a></li>
<li><a href="webpages/index.html">Relative links work also.</a></li>
<li><a href="http://upcracky.blogspot.com">An external link.</a></li>
</ul>
</body>
</html>

Save the file as "foo.html". This is just a temporary file and the name has to be "foo.html" or the rest of the steps won't work.

Copy foo.html to the top level directory of the SD card.

Unhook the Android from the Desktop.

Connect to this blog entry via the Android browser and click on this URL. foo.html should be displayed in the browser.

Saving the Bookmark

While display the web page in the browser, press the menu key. Select "Bookmarks". Select "History". Select "Today". Find the entry for "Found the SD Card!" and click on the star symbol on the right. The bookmark will be added to the browser with no further intervention.

Do not use the "Edit bookmark" option. Just opening the book mark editor with this book mark may corrupt the link and it will stop working.

Changing the Name of the Bookmark

Since the bookmark can't be directly edited, some cleverness is required to move the SD web page and/or change the name of the bookmark.

First create a new web page and install it on the SD card in any arbitrary folder. Edit foo.html and add a link to the new page relative to this page. Bring up foo.html via the previous book mark. Click on the link to the new page. Bookmark the new file.

Once the new page is safely bookmarked, foo.html and it's bookmark can be deleted.

Limitations

The one quirk that hasn't been figured out is how to jump to the middle of a local web page. Links such as bar.html work fine. Links such as bar.html#middle give an error.

Dell OpenManage Under 64 Bit Red Hat Linux

This post was moved to here.

Beyond Help.

I'm a big fan of open source software. I do most of my work on Linux and most of my programming is done using Perl. I believe that open software has value and the more open our infrastructure is, the better it is for the world as a whole.

As programmers go, I'm pretty good at what I do. That gives me certain obligations, one of which is to give back to the open software movement when I can.

I was given an opportunity today, but I don't think I'll be giving back in this case.

As I mentioned before, I do a lot of programming in Perl. Sometimes I need to create windows and buttons to make my programs pretty. Enter GTK2. It's a really nice package. For what I do it's simple, reasonably clean and you can usually find an example out there that will give you a good head start. Therein lies the rub.

The example that I found was part of the GTK2-Perl Frequently Asked Question (FAQ) list. I copied it over and tried to run it. No go. The example was using an older version of the library and it had a typo. If only there were a programmer in the house!

Hey! I'm a programmer! And I'm in the house! It took me less that 15 minutes to get it up and running. Yay me!

Now comes the obligation part. I'm using public code, so therefore I'm obligated to send the fixes in to the GTK2-Perl folk so the next person who comes along doesn't have to figure it out. I connect to their bug tracker and try to enter a bug report, including the fix.

No dice! If you want the privilege of submitting a bug to them, you have to sign up to their web site. That's stupid but I'll bite the bullet for betterment of open sourceyness. I'm quite the patriot.

Nope. You not only have to sign up, but you have to give them an email address. Oh, and they'll publish your email address guaranteeing that you'll get spammed off the planet! It gets worse. They suggest creating a throwaway account to handle the spamming. What? Why would I set up an account just to throw it away?

In theory this is supposed to discourage spamming. Is spamming in bug reports really that big a marketing vector? Hey, my program is crashing, maybe penis enhancement will help!

I know that people want an automated system here. One that will block all spam without human intervention. These people are idiots. You have 2 choices. Get casual bug reports that will do wonders for enhancing your project while dealing with the reality of spamming, or set up barriers that will keep most spam out and will do wonders for causing inbreeding on your project.

I'll admit I don't have a lot of patience for this nonsense. I'm not jumping through hoops to help you. Besides, a better solution is trivial.

Any bug report sent via the "outside" gets put in a queue until someone who's trusted takes a look at it. Spam bug reports are rare and there are more than enough regulars on any project who will do the few seconds of grunt work for you.

You could also use a one time key. I connect to your bug tracker. You email me a tracker-putter-inner-key. You don't keep my email address, you just use it once to send me the key. I then use that key to submit the bug report. If I need to add comments to that report, then I use the number again. If I loose the number and want to add more comments then I have to sign up for an account and I become a regular user.


As for the moral obligation to open source? Here it is:

In the GTK2-Perl FAQ, question 5.3, "Show me a simple example for Drag-n-Drop" you need to make 4 changes to get it to work under Perl, v5.10.0 and GTK2 version 1:1.221-4.

1. Replace "Gtk2::Ex::Simple::List" with "Gtk2::SimpleList". It occurs in 3 places, all near the top.
2. In the function "_move_from_to", change "for my $i (0 .. $#{ @{$fromlist->{data}}) {" to "for my $i (0 .. $#{$fromlist->{data}}) {".
3. Enjoy!
   

If someone wants to send this to the GTK2-Perl bug site, feel free, but watch out for the spam.

Gators and Nautilus and DAVs, Oh My!

If you're here to get WebDAV working with Nautilus for HostGator, then just jump to the end of this article. The solution is trivial.

If you wish to bask in my purple prose, read on:

Introduction

I'm setting up a site on HostGator, which is a host provider of good repute. It's starting as place to put some of my little code snippets that don't warrant a full blown project. Who knows what it will turn in to.

So far it's been basically a positive experience, but it's not without its shortcomings. I'm trying to keep notes on dealing with HostGator so other people don't fall in to the same traps I've tripped.

One thing I ran into was getting WebDAV working with Linux. Specifically Ubunto's "Karmic Kaola" under Gnome, using the Nautilus file manager.

Of DAVs and Nautali

The first problem I ran into is that HostGator gives it's version of WebDAV the name "Web Disk".

I don't mind them trying to make web interfaces seem a bit easier to deal with, but somewhere on the page they really should tell us that "Web Disk" is really just another name for WebDAV.

Once you get that under your belt tracking down problems becomes a little easier.

The next problem I ran into is, well, I'm trying to use Nautilus.

Nautilus is the Gnome file manager. For many users it's the entire interface to the disk. It should, above all other things, be reasonably fast, accurate and reliable. All else follows from that.

Unfortunately Nautilus is notorious for running down rat holes while not getting the basics done right. It has improved a lot in the last 2 years, but it still plays amateur night a few times too often. This was one of those times.

I went to the HostGator site and navigated down to the "Web Disk" page. I selected Nautilus and followed its suggestions. They wanted me to connect to the location "https://foo.com:2078".

I did and I got "Could not display 'https://foo.com:2078'."

Crud.

I had no idea where to proceed from here. Is it the instructions, or is my account screwed up, or maybe Nautilus is broke?

Just to add to the confusion, HostGator has you create a user name when you create an account with them. No big deal, I chose "wiles". What they don't really make clear is that this user name isn't your domain user name. If you register the domain "foo.com" and want mail to come to "wiles@foo.com" then you need to set up another account called "wiles@foo.com". When I connect with Nautilus, should I use "wiles" or "wiles@foo.com"? The correct answer is neither, but that part's not HostGator's fault. More on that later.

So now I have and unknown protocol, with an untrusted browser connecting to a possibly mis-configured account using an unknown user name. You have to admit, it's kind of a challenge. Right?

I Prefer the Term "Challanged"

I'm not one to back down from a geek challenge, so into the fray I went. I ended up spending over an hour chopping around the Internet tubes trying to figure out what's going on. I got bits and pieces, but nothing I could really sink my teeth into.

I also wasn't sure of the format of the URL. For most services you can load in your user name into the URL and you don't have to type it in each time. For example, if you're using File Transfer Protocol (ftp), and the URL is ftp://foo.com, then you can include your user name with ftp://wiles@foo.com.

Does that work with "Web Disk"? What if I'm supposed to be using wiles@foo.com. Is it ftp://wiles@foo.com@foo.com or ftp://wiles%40foo.com@foo.com.

"%" is the HTML escape character and "40" is is ASCII code for "@" in hexadecimal. You knew that didn't you? I think it's a real sign of progress when connecting to a web site only requires knowledge of arcane character encoding and 2 number bases.

How about if I just create a bookmark in Nautilus, and then edit each variable separately and see if I can get anywhere? Nautilus will handle the details right? If you watch slasher moves there is always a part where some idiot says "lets split up!" The music they play at that point belongs here.

Scooby Do Takes and Ax to the Forehead

First, I'm not going to get anywhere until I figure out the real protocol being used. "Web Disk" criminy! Why not just call it "Disk Huggy Bunny" and be done with it.

I went back to the HostGator page. It listed a few other OSes and other browsers. The Mac options were mysterious. The Window options were baroque. It's not looking good. My last hope: Under Linux there was the option to use Nautilus's arch enemy "Konqueror". I selected it and it offered a download for my system. Anonymous downloads makes my toes itch, so instead I sent the file directly to a text editor to check it out.

There, large as life, is the line "URL=webdavs://foo.com:2078".

I now knew the protocol. We were moving forward.

I created the bookmark in Nautilus, set my user name, picked "Secure WebDAV (HTTPS)", and set the port.

I'm not sure about the start folder. Is it "/" or "/home/wiles" or "/public_html/wiles@foo.com"? I held my nose and left it blank.

I clicked connect and I'll be darned! I got a password prompt! Kewl. I typed in my password and crossed my fingers.

The error I got is "Could not display 'davs://wiles@foo.com:2078/'. Error: Not a WebDAV enabled share. Please select another view and try again."

This is a truly beautiful error message. First, it did *not* connect using "davs://wiles@foo.com:2078/" (more on that later). Second, the share is, in fact, a WebDav enabled share. And third, "Please select another view and try again." gives me absolutely no useful information. It was a perfect storm of rotten interface.

At this point I was pretty frustrated, and lets not kid ourselves, at this point most people have already given up. Fortunately I'm not most people. I'm a hard core uber-geek and I eat bad interfaces for breakfast! Snort!

I start creating bookmarks like they're going out of style. I try every combination I can think of. It's slow, it's tedious, it's everything a computer shouldn't be, but alas, are, and in the end, it was futile. I could not get Nautilus to connect to my HostGator site.

Nautilus: A Weasel in Drag

Hmmmm. Do I trust Nautilus? Would I lend it $10 if it asked? No. No I would not. Then why would I believe that its rotten bookmarks are doing what they say their doing? It's time to go back to first principals and start over again.

I knew, with reasonable confidence, that I'm trying to make a WebDav connection. I also think that the "s" in WebDavs" stands for "secure", as in a Secure Socket Layer (SSL) connection. When you drop the SSL requirement at HostGator, the socket number changes from 2078 to 2077. That was enough for me to make another try.

I fired up Nautilus, and in the "Location" bar I typed "webdav://wiles@foo.com:2077". I got back "Nautilus cannot handle 'webdav' locations." Wow. What a nice, simple and useful error message. Are we sure this is Nautilus talking?

"WebDAV" isn't going to work, but I've also heard it called just "dav". I tried that.

With "dav://wiles@foo.com:2077" I got the password pop up! I also got the "select another view" error message. Arg!

Blinky the Wonder Idiot

Then the dawn came. Does anyone else see the huge blinking "Dale is an idiot!" sign in the previous paragraph? It took me a while to see it.

Even though it's the standard, and Nautilus is reacting to it, do I really know that the "wiles@" part of the URL is kosher? What happens if I don't use it?

Into the breach with "dav://foo.com:2077". This time the pop up asked for user name and password. I used "wiles" again and typed in my password.

I connected like a 13 year old with an evangelist. Son of a ...

I had spent more than 4 hours trying to get this to work, and it all comes down to this one sentence. It really doesn't make HostGator or Gnome/Linux look good does it?

I wasn't where I want to be yet, so I kept pressing forward. I was going to get SSL working.

"davs://foo.com:2078". Yup. I was in. Sort of anticlimactic isn't it.

One last thing. I really wanted to be able to connect to HostGator by clicking on a shortcut on my desktop.

The solution is trivial. Right click on the desktop and select "create launcher". Give it a name, and in the "Command" field type "nautilus davs://foo.com:2078" or "nautilus dav://foo.com:2077", depending on whether you want to use SSL or not. (You do, unless it you can't get it working.)

That's it.

I'm sending this URL to HostGator so they can fix their documentation, but until they do, enjoy updating your HostGator sight via Nautilus.