December 3, 2007

How the other third lives

We're in the midst of a kitchen remodel. On Saturday, our drywall guy asked me what I did for a living. I gave him my standard short answer, "I build websites ..." That didn't get much of a reaction, so I followed that up with "for Amazon.com ..." I could tell that the name didn't register either, so I just said "I'm a computer programmer; I write software." His response, "I haven't really gotten into the whole computer thing."

Wow, he's not online, and he doesn't even own or use a computer. Here is a successful sub-contractor with a friendly, professional crew who doesn't use a computer and hadn't heard of Amazon.

In this Information Week article, found via a quick Google search, in February 2006, only 64% of US households are online. Of the 36% who aren't online, only 2% have a desire to get online. I don't really have time to find more recent data, but one can assume that things probably haven't changed too much.

I'm curious to know how much that demographic has been inadvertently marginalized by the assumption that, "everyone's online these days." This was refreshing and a good reminder that we could in fact live healthy and productive lives without computers and the internet. Unless of course you build websites for a living.

Posted by mark at 3:24 PM | Comments (0)

November 5, 2007

Sudo laugh

For my non-technical friends, who don't work with unix/linux, this won't be very funny, but I've been laughing about this for days. I don't know where this image came from, but I ran across it in this article. If you're curious you can read about sudo on Wikipedia. I can imagine some funny alternatives around the topics of beer or sex.

UPDATE: Topher pointed out that the comic's author has a website: xkcd.com. I did find this comic in the archives.

Posted by mark at 11:09 AM | Comments (4)

October 25, 2007

Gmail supports IMAP!

At some point this week, Gmail started supporting the IMAP protocol for remote email retrieval. If you own the iPhone, this is great news. Back when the only supported protocol was POP, there were only two options: Leave a copy of messages on the server (or not), and remove deleted messages from the server (or not). You didn't have access to your labels (aka tags, folders), and all of your email regardless of configured filters, showed up in one big Inbox (thankfully with the exception of the Spam label). This allowed you to read your email, but not really manage it.

IMAP is true synchronization between a client (iPhone) and a server (Gmail). On the iPhone it gives you access to your labels as IMAP folders, and your Inbox contains what's truly in your Inbox. You can effectively label an email by moving it to a folder, but it doesn't seem to be possible to move it to more than one folder so you're limited to one label per message. All labeling, deletions, and sent items are synchronized with your Gmail account in real time. If you want to access your mail from more than one location via IMAP, the synchronization features make this a lot less error prone. I noticed that if you decide to click cancel when composing a message, it asks you if you want to save it. If you do save the message it ends up in an iPhone Drafts folder instead of the Gmail Drafts folder, so it won't be synchronized. I've only used it for a day now, but it's a huge improvement over POP, and makes using my iPhone even cooler.

Update: I found a new label in my list of labels called "[Imap]/Drafts" and in that label was my draft that I had previously saved on my iPhone.

Posted by mark at 8:45 AM

October 8, 2007

Amazon Mp3 store

If you're not living under a rock, you'll know that a couple weeks ago Amazon launched an mp3 store that portends to compete with Apple in the digital music sales arena. I was initially skeptical because the Apple store is easy to use and tightly integrated with iTunes. You buy a song or TV show, and voila, it's in iTunes. The major difference between Apple and Amazon is that all songs purchased from Amazon are DRM free. Apple only sells some of their music without DRM, and I'm not sure it's possible to tell which ones are and which ones aren't -- I have no idea. For many, DRM is completely transparent. If you purchase from the Apple store (into iTunes) and only ever listen on your laptop or sync to an Apple device, then you'll never care about DRM. For me, I use a 3rd-party device that allows me to wirelessly stream music from my computer to my stereo. This works great unless you have DRM protected music. The device I use doesn't support DRM. Whenever I purchased a new, DRM protected album from iTunes, I had to first rip it to a CD and then re-import it into iTunes as mp3 files in order to remove the DRM. I then felt compelled to keep around two copies of the album, the purchased DRM version and and the non-DRM version. Needless to say, it was a pain.

As for my concern about ease of use and integration with iTunes, Amazon's service works great. You have to download and install an Amazon MP3 Downloader to help manage the download process. I'm not entirely sure why this is necessary, but I'm guessing it's to help ensure that your downloads were successfully downloaded, how many times they've been downloaded, allow you to pause and restart, etc. Since the Apple store is browsed only from within iTunes (read proprietary web browser), this sort of thing wasn't necessary. Since iTunes is such a good music player, none of us are mad that we have to use iTunes to browse their store. We weren't so happy when Microsoft did something similar with Internet Explorer, but I digress. Once the music is downloaded it's immediately available in iTunes, so integration with iTunes is seamless.

There were a few small hiccups with the store, but they weren't that big of a deal. After purchasing my first album, I tried to use the back button to browse for more albums and ended up downloading the album twice. When this happened, the first time download of the album was still processing so I had plenty of time to cancel the accidental re-download. Surprisingly, there wasn't a way to delete queued downloads in the Amazon Mp3 Downloader, so I had to let it download and then manually delete the second copy of the songs. Also, the installer didn't register the amz extension with the currently open Firefox window; perhaps that isn't possible. On restart Firefox recognizes the new extension fine and correctly launches the downloader.

From this point forward, if Amazon has the album, I'm going to purchase it there, I would suggest you do the same. If not, I'll still use iTunes. Oh yeah, just so Amazon gets the in the last word, many songs and albums are cheaper on Amazon. $0.89 cents a song versus $0.99 and $8.99 an album versus $9.99. The Crane Wife by The Decemberists was a full $2.00 cheaper.

[Update: If you enable iTunes Plus in the Apple store, you will always see the DRM free versions of songs/albums if available. The DRM free iTunes songs are more expensive individually at $1.29 a song, but it appears that you can still get the DRM free version of The Crane Wife through Apple for $9.99.]

Posted by mark at 12:30 PM | Comments (7)

April 24, 2007

Javascript closures for dummies

I was reading through a great list of 7 Javascript techniques and realized that I didn't fully understand the concept of closures. My much thumbed through Javascript Definitive Guide, which is normally quite good, wasn't much help. Thankfully I ran across Javascript closures for dummies. In 10 minutes I now fully understand something that I've run into countless times.

Posted by mark at 11:56 AM

February 21, 2007

Vim syntax files and highlighting

For about the 4th time, I've wasted an hour trying to figure out why my highlighted searches in vim would highlight the searched for phrase with white text on a yellow background; which is virtually unreadable. Since I work on about 4 different computers, I usually just scp my .vimrc to each computer and I'm good to go, however, the search highlighting problem would always reappear when I starting to use a new host.

I use vim with a black background. In the past, I've put the following command in the file .vim/after/syntax.vim:
highlight Search term=reverse ctermfg=0 ctermbg=3 guibg=Yellow

Vim will source any files in the after directory when it's done doing all of the other normal runtime sourcing. With the above command, my searches would now be highlighted with black text on a yellow background.

When troubleshooting syntax issues, I've found the following commands to be eminently helpful.
:highlight - will show you all of the defined highlight groups and with a sample of what syntax
highlighted text will look like when applied. It also gives you the definition string for each group.

:help syntax (gives you all you need to know about syntax highlighting)

:help group-name, shows you some of the most common syntax groups and if syntax highlighting is working properly, what each group looks like with your current syntax profile.

In order to make this fix a bit more portable, I took the line from .vim/after/syntax.vim and put in in my .vimrc. I'm not sure why I didn't think of putting it there in the first place.

if &t_Co > 2 || has("gui_running") syntax on set hlsearch highlight Search term=reverse ctermfg=0 ctermbg=3 guibg=Yellow endif

If you're curious about the if statement, it just means, the terminal has more than 2 colors, or is gvim. You can read more output codes like &t_Co (number of colors) with
:help terminal-options

Posted by mark at 2:49 PM | Comments (2)

February 17, 2007

MySQL hack of the day

I was doing some work on an older version of mysql (3.23.58 to be exact) today, and had to do a multi-table update, which isn't available on that version (at least I couldn't get it to work). Here's the relevant schema:

table contacts
id INT
donor_id INT (foreign key)
...

table donors
id INT
status ENUM ...
...

table temporary_donors_to_contact_ids
donor_id INT
contact_id

I was trying to backfill the donors table, and had to subsequently update the contacts table with the appropriate foreign key from the donor auto_incremented id. If I were on a current version of mysql the query would most likely look like something this: update contacts c, temporary_donors_to_contacts_ids t set contacts.donor_id = t.donor_id where c.id = t.contact_id; # untested, and sheer speculation. I do this type of thing so infrequently, there might be an elegant single query solution with this version of MySQL, but I didn't want to waste too much time since I was on such an old version.

Anyway, I was really pulling my hair out, when I thought of this terribly hacky solution that was so crazy I thought I'd post about it.

I quit mysql and ran this from the command line:

# mysql database -u mark -p -e "select 'update contacts set donor_id =', donor_id, 'where id =', contact_id, ';' from temporary_donor_to_contact_ids" > /tmp/donor_updates.sql # cat /tmp/donor_updates.sql | tail -n+2 > updates.sql # mysql database -u mark -p < updates.sql

Those of you familiar with mysql, know that when you run a query in the mysql client, it will output the results formatted in a table. When using mysql -e, it doesn't output the pipe delimiters. I was planning to strip those out using the venerable CTRL-V (Visual Block) mode in vim.

It worked so well, I just might use it again in the future -- metaprogramming sql statements ... very hacky, but pretty cool!

Update: I found in the mysql documentation that "Starting with MySQL 4.0.4, you can also perform UPDATE operations covering multiple tables." So, what I was doing wasn't possible with the version I was using.

Posted by mark at 3:39 PM

January 18, 2007

Google Blog Search

In another shameless act of Googley fervor, I want to highlight yet another Google resource: Blog Search. To be honest, I have no idea how long this has been around, but yesterday I discovered one feature that makes it really useful. The search results are RSS feeds! What this means is that I can copy the URL of a blog search, sorted by date and not relevance, paste it into Google Reader as a new subscription. I'm now tailing the blogosphere for any mention of my search.

Since I'm involved in the new Amazon UnSpun website, I've got a blog search query for any mention of Amazon and UnSpun. Very cool.

Hat tip the Google Reader blog for alerting me to this idea.

Posted by mark at 9:29 AM

Google Reader

If you have friends with blogs, then this post is for you. I use what's called a feed aggregator to collect all of the blogs that I read into a single website for easy consumption. This tool will tell me when a friend or industry maven has posted new content to their blog. I don't have to remember to visit the site. In the past I've happily used Bloglines for feed consumption, and it's still one of the best tools out there.

Based on a recent post from Jeffery Veen about viewing blog reading trends in Google Reader, I decided to check it out. Within one minute, I had imported all of my Blogline's feeds into Google Reader using the handy OPML export.

The verdict is still out on the reading trend feature that Veen mentioned, but I'm already hooked on the feed aggregation. One of the best features is that it works like email. The unread items are bold, and after I've read a blog post, it stays in the folder in an un-bolded "read" state. In Bloglines, after you've read a blog post it disappears, the only way to re-read it is to visit the website where it came from. I can't tell you how many times I've had to do that. With Google Reader, the previously read post is still there. In fact, I don't see a search feature yet, but I would bet that searching within your feeds is coming soon; it is Google we're talking about.

Anyway, if you read blogs and don't use a feed aggregator, you should. If you use something else, I think you should try Google Reader. The only Bloglines feature that's not present in it's current incarnation is the ability to display your public feeds in a javascript injected blogroll. I currently display a list of my public feeds in the sidebar of my blog. I can't replicate that feature in Google Reader, yet. They do allow for syndication of a public folder, but they syndicate the content of the blogs rather than just a list of blogs. I'm sure this will be available soon.

I feel the same sort of love for Google Mail (aka gmail), so you should use that as well. If I had to pick one feature of gmail to tout, it would be that it groups emails by subject, so an email conversation that's 20 emails deep will appear to be 1 email in your inbox. You have to use it to understand. You might have to be invited to use gmail, so if you're interested, let me know and I'll invite you.

I admit that I sound like a half-crazed, drooling, kool-aid drinking, Google disciple, and for the most part, I am. Google is just good at taking ordinary mundane things we take for granted and re-delivering them with a new spin.

Posted by mark at 8:46 AM

January 12, 2007

Firebug 1.0

It's not every day that I get as excited as I just did about a browser utility upgrade. However, Firebug 1.0 is truly amazing. If you build websites, especially ones with AJAX, Javascript, advanced CSS, etc. then you need to upgrade to Firebug 1.0. NOW.

I've been using Firebug for a long time now, but the latest release is chock full of new features. You can read about the tool in a recent article in Dr. Dobb's.

If you try to upgrade from the Extensions menu, it won't work, you have to go to the source:
http://www.getfirebug.com/.

The most obvious pre-existing feature of Firebug, is the ability to view AJAX requests and subsequent responses. For that alone, the tool is useful. Now they've included javascript and page profiling, improved javascript debugger, inline CSS editing and inheritance browsing, and much more. It's really really cool.

Posted by mark at 1:43 PM | Comments (1)

November 10, 2005

Some AJAX issues and some workarounds

In working with AJAX (which I should probably start calling AJAH, since my responseText is never XML -- but HTML), I've run across two issues. I was able to get both to work, but I thought there might be something I'm missing so here's my solutions, I'd welcome any feeback.

Scenario #1:
I have a website where the UI consists of a series of tabs. The active tab is loaded normally via the request. The unselected tabs are loaded asynchronously via AJAX into hidden divs. The responseText of one of the AJAX requests contains javascript, some that runs immediately in the context of the request, and some javascript functions that need to be stored in the DOM for later onClick events.

The problem: There's an option to the AJAX.Updater method, evalScripts: true, that you can use to cause any Javascript in the resulting responseText to run, and it works great for code that runs within the page immediately, but any declared functions that work when the page is being loaded, are subsequently lost and not stored in the DOM as expected. So, when you click on the tab, and then click an element intended to fire one of those functions, it complains. I was surprised the evalScripts didn't also load the functions into the DOM, but hey, I'm sure there's a perfectly good explanation. In order to get around this problem, I used the onComplete option in the AJAX.Updater method to run a function that searches the responseText for script blocks, and manually inserts them into the DOM. It works great. When doing this, you don't need to use the evalScripts parameter for other Javascript that needs to run immediately. Here's my function:
function insertScriptsInHead(ajaxObj)  {
    var allScriptsRegexp = /<script type="text/javascript">[\s\S]*?</script>/g;
    var results = ajaxObj.responseText.match(allScriptsRegexp);
    var head = document.getElementsByTagName("head")[0];

    if (results)  {
        // loop through all of the script blocks and append them to the head
        for (var i = 0; i < results.length; i++)  {
            var result = results[i].match(/<script type="text/javascript">([\s\S]*?)</script>/);
            var node = document.createElement('script');
            node.type = 'text/javascript';
            node.text = result[1];
            head.appendChild(node);
        }
    }
}
I should point out that I'm only developing for the latest versions of IE and Firefox, so I don't have any clue how this code might work in older browsers. If anyone has any knowledge of how to get Prototype to do this for me, I would love to hear it.

Scenario #2:
I launch a popup window with some AJAX that is fired via an onClick event in the context of the popup. After the AJAX is complete, I want the popup window to close itself via window.close().

The Problem:
For once in my life, I have to say that this works great in IE, but not in Firefox. In Firefox, a few seconds after failing to close the window, the browser will crash.

I was able to workaround this issue by storing a reference to the window object for the popup in the parent. Then, when the AJAX is complete, the popup calls a function in the parent via this command setTimeout('opener.closePopup()', 1000), which can close the window with something like this, popupWindowObj.close(). I'm not sure why I have to wrapper the opener.closePopup() command in a timout, since the AJAX is complete, but it doesn't work without a slight delay. So basically, instead of the popup closing itself, the opener (parent) does.

I actually had another issue very similar to this, in that a popup window was supposed to close itself, and launch an AJAX update method in the opener. However, this also didn't work, and I got it to work via the method described in this blog:
http://dema.ruby.com.br/articles/2005/05/06/be-careful-when-mixing-ajax-and-popup-windows-on-firefox

Again, if anyone has any feedback, I'd appreciate it. If not, I think that both of my workarounds are acceptable.

Posted by mark at 4:53 PM | Comments (6)

AJAX and Prototype

I've been doing a lot of DHTML Javascript and AJAX stuff for work lately. I was using my own library of AJAX functions, and an odd assortment DOM/Element modifying scripts. These methods actually worked quite well, but they seemed a bit disjointed. I recently discovered the Prototype Javascript Framework, and I'm really impressed. I was able to refactor my AJAX code to use Prototype in about 30 minutes. Prototype also offers a bunch of really slick DOM insertion methods that allow you to insert a block of code wherever you want in the DOM with ease. Adding event listeners also seems to be pretty easy. Here are two simple but fantastic shortcuts provided by Prototype:
$("elementid") is the same as document.getElementById("elementid")
$F("elementid") is the same as document.getElementById("elementid").value

The framework page itself has little in the way of documentation. Thankfully, I did find this document which outlines the framework, and shows some examples of use. I've also looked at the Prototype source javascript and it's remarkably clear.

The framework is in production use within the Ruby On Rails framework, so while I'm not a javascript guru by any means, I'm confident that it's sound code, and isn't going anywhere. I highly recommend checking it out.

Posted by mark at 4:11 PM

October 5, 2005

AJAX and privacy

While building a registration form for a client website this morning, I had a scary thought about AJAX (Asynchronous Javascript and XML) and security. AJAX allows us to to perform HTTP requests behind the scenes, so it's entirely possible to submit each field in a form as it's completed. So, after the user enters their first name and tabs or clicks into the last name field, AJAX could silently submit the entered text. Then when the user tabs to the next field, AJAX could submit the last name, etc.

Obviously, this flies in the face of user expectations of form behavior and security. I can't even remember how many times I've started to fill out a form, only to not submit it for any number of good reasons. I've actually entered my credit card number, expiration date, etc. and then chose *not* to submit the form because I realized that the site didn't have the requisite https:// or lock icon. Unless you're monitoring the headers of your browser behind the scenes, a form on a malicious website could be submitting your entries as you go using AJAX. Then, if you decide *not* to submit the form, they at least have all of your information up to the point at which you decided to stop filling things out.

I've been using AJAX extensively at work, and it's able to make web applications seem a whole lot more responsive. However, with mainstream acceptance of AJAX (which has actually been available without the cool acronym for years), it might be good to have a browser option to disable AJAX on secure pages, and possibly even a little toolbar icon that alerts you to the presence of asynchronous requests. Perhaps something like this already exists as an extension?

Assuming good intentions, there would be tremendous value to knowing exactly when a person decided not to submit a form. For example, if you knew that 75% of people who didn't click submit stopped at the same field, you would know that there's an obvious flaw either in design, code, or language that needs to be fixed. Of course, you could use javascript and AJAX to capture this information without actually submitting the content of the form, just the users progress through the form. Regardless, I think there are some definite ethical lines that shouldn't be crossed. Your thoughts?

Posted by mark at 11:55 AM | Comments (3)

April 25, 2005

Shell tricks and tips

Recently, I started developing exclusively on a remote Linux box. Thanks to Andy I've picked up quite a few shell tricks that have been exceedingly useful. I use zsh at work, but I'm pretty sure they all work on bash as well. Recently, when talking with some friends who are Linux veterans, I was surprised to learn that some of these commands were new to them. Of course, I also learned a new trick or two from them. Here's my list, am I missing any gems?

  1. Tab completion of directory/file names/environment variables - I'm embarrased to say that when I first started to teach myself Linux, it took me about 6 months to learn about tab completion. It was even longer until I realized that tab also completed environment variables. I mention it here because I'm sure I'm not the only one. ;-)

  2. CTRL-W delete word

  3. CTRL-E go to end of line

  4. CRTL-A go to beginning of line

  5. CTRL-R recent command completion - This is one command that wasn't known by too many people. It allows you to search quickly through your command history for previously typed commands. It helps to have a large command history: export HISTSIZE=1000.

  6. CTRL-U Discard line, this works well when you think you've mistyped a password and need to start over.

  7. CTRL-K cut line from cursor

  8. CTRL-Y paste line in buffer

  9. !! (the last command) - I mostly use this when I forget to sudo a command For example:
    rm foo
    sudo !!

  10. !$ (the last argument of the last command)
    Let's say I'm tailing a log, but I want to see more than just the last few lines.
    tail -f /var/log/cron (quit tail)
    less !$

    or you just edited your .bashrc with vim and now want to source it
    vim ~/.bashrc (quit vim)
    source !$

  11. search for any files/directories recursively starting with the current directory and look for any occurance of foo in the name.
    find . -type f | grep foo
    OR
    find . -name "*foo*"

  12. xargs - Takes the output of the preceding command and appends them to the end of the following command. I think it's actually a bit more complicated than this, but the examples that follow do work.
    Here's a use of xargs with find to search within any files/directories recursively starting with the current directory and look for any occurance of foo *within* any of the files
    find . -type f | xargs grep foo

  13. Have you ever accidentally un-zipped a file into the wrong directory and wanted an easy way to undo. You can use xargs to remove the files by first listing the files in the archive, passing that list to xargs with rm (you have to be careful with this one):
    tar xzf foo.tgz
    tar tzf foo.tgz | xargs rm -rf

  14. How about wanting to get all files with foo in them, but you don't want the vim turd files. Note the escaped ~, this is done so that the shell doesn't first convert the ~ to /home/foo. Just remember that the shell will first parse the line, and *then* pass it to the binary. This type of issue still trips me up from time to time.
    find . -type f | xargs grep foo | grep -v \~

  15. pushd/popd - pushd will store the current directory in a stack. When you're ready to return to that directory at some later time, just type popd.

  16. cd - will take you to the previous directory (Thanks to Brain Pikkaart for this one).

  17. screen - (Thanks to Joel for first introducing me to screen ages ago.) I've got a linux development server at work that I connect to from my laptop. From man: "Screen is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells)." I've got screen running with 7 shells that I can toggle between. Following Andy's lead, I've setup screen with the following:

    0 - live server
    1 - backup server
    2 - vim
    3 - shell work on shared libraries
    4 - shell work on services
    5 - logs
    6 - mysql
    7 - scratch

    When I'm ready to leave, I just detach my screen session and go home. When I'm ready to start working again, I ssh into my development box, and (screen -r) will reattach my session and I'm back exactly where I left off.


Posted by mark at 3:32 PM | Comments (9)

April 12, 2005

RFID chips for Ellie and Hop Hop

Both of my kids have stuffed animals that they are quite fond of. That fondness borders on obsession that often causes tons of grief when we have to find Ellie (James' elephant), and Hop Hop (Emma's bunny) prior to putting our kids to bed at night. Every parent out there knows what I'm talking about. EVERY NIGHT I walk upstairs to start the bedtime ritual, which is hard enough to begin with, only to realize that Ellie or Hop Hop is missing. I then have to go BACK downstairs, often all the way to the basement, to find one or both of those infernal creatures.

I want to purchase a handful of RFID chips for use in my home. There are a ton of objects that might benefit from this technolgy:

I don't know much about RFID, and I've only heard about companies using it to track inventory (Cisco, Wal-Mart). I haven't heard anything about DIY home RFID kits. I'm betting that the RFID chips are fairly inexpensive, and that the technology to locate and read them is probably the financial bottleneck.

Regardless, even with this system in place, the next time I realize that Ellie is missing, I'll probably be wondering, "Now where did I put that damn RFID receiver."

Portable phones are pretty cheap. I suppose I could just duct tape a portable phone around Ellie's head. Next time she's lost, I can use the phone's buit in paging feature to find it. James could then make phone calls from bed. ;-)

As you get older you could increase the frequency of your RFID receiver, and size of the items you track.

Posted by mark at 1:46 PM | Comments (3)

April 8, 2005

Linux is cool

I just ran uptime on my main intranet server at work. 465 days without a reboot. That's remarkable. Back when I administered Windows servers, I would be lucky if I went 465 hours without a reboot.
7:54am up 465 days, 17:08, 2 users, load average: 0.11, 0.25, 0.25

Posted by mark at 10:52 AM | Comments (1)

March 10, 2005

CSS and Internet Explorer

I've recently been troubleshooting a number of problems with IE 6.0 and CSS. I thought it might be nice to document the main IE bugs that I experienced and some workaround hacks that I found online. My project is for an intranet, so thankfully, older IE bugs prior to version 6.0 don't concern me.

IE CSS Bugs

IE CSS Hacks

Other resources

Most of the solutions that I found came from Position Everything. This was a great site that has very clear examples and explanations of IE bugs. They have two great pages that list all of the above bugs and many more here, and here. I've just highlighted the bugs that I personally experienced.
hasLayout is is a proprietary Microsoft HTML property that can help to solve float problems with IE
Meyerweb is also a great site for CSS articles.
I joined the css-discuss list which is a great resource.
While I avoided the duplicate characters bug by just removing the floats, I didn't determine this course of action before finding a number of different triggers for the same bug detailed in the positioniseverything document.

As a funny side note, I just checked my blog in IE - for the first time I think, and the right nav was entirely below the content. Oops.

Posted by mark at 3:57 PM | Comments (1) | TrackBack

Popups from a bookmarklet

I use del.icio.us to store my bookmarks. There's a bookmarklet that I use to conveniently post pages to del.icio.us. However, with Firefox, the popup would always popunder, so I'd have to Alt-Tab to it every time.

Here's the original javascript for that bookmarklet from the del.icio.us post page (added line breaks).

javascript:q=location.href;p=document.title;
void(open('http://del.icio.us/markveerman?v=2&noui=yes&jump=close&
url='+encodeURIComponent(q)+'&title='+encodeURIComponent(p),
'delicious','toolbar=no,width=700,height=250'));

Here's the new javascript (added line breaks):

javascript:q=location.href;p=document.title;
w=window.open('http://del.icio.us/markveerman?v=2&noui=yes&jump=close
&url='+encodeURIComponent(q)+'&title='+encodeURIComponent(p),
'delicious','toolbar=no,width=700,height=250');
setTimeout('w.focus()',10); void 0;

This is pure conjecture, but I think the default behavior of Firefox is to put the focus back to the page you were currently on when running a bookmarklet. When I tried to just add w.focus(), it would popup the page for a millisecond, and then Firefox would put it back to the background. By adding a 10 millisecond timeout to the focus it allows Firefox to finish it's default behavior, and then I re-focus the post to del.icio.us popup.

Posted by mark at 11:46 AM | Comments (1)

Firefox bookmarks

While trying to solve another problem, I discovered another cool feature of Firefox.

If you open the properties of a bookmark you'll see a keyword field. This allows you to assign keywords to often used bookmarks. For example, I could assign the keyword 43 to my bookmark for http://43things.com/people/view/markveerman. Then, all I have to do is type 43 in the browser address bar and voila.

Where it gets cooler is that I can use variable substitution in the bookmark url. So, for the 43 things bookmark above, I could change the location of my bookmark to http://43things.com/people/view/%s. Now, when I want to access that bookmark, I just type 43 markveerman, or 43 aharbick or any other person on 43 things that I might want quick access to.

I'm not sure how much I'll *really* use this, but can think of quite a few uses where it might helpful.

Posted by mark at 11:02 AM

December 8, 2004

Character Encoding issues distilled

I've always known that Microsoft did something non-standard with regard to character sets, but I never really knew specifically what their wrongdoing was. In attempting to convert some textual data into XML for purposes of an RSS feed, I ran into trouble. While this issue has been around for a while, I thought I might post my findings as a reference for later.

What Microsoft did wrong
The HTML 4.0 SGML specification strictly sets aside the ASCII range 128-159 as unused.



CHARSET

          BASESET  "ISO Registration Number 177//CHARSET

                    ISO/IEC 10646-1:1993 UCS-4 with

                    implementation level 3//ESC 2/5 2/15 4/6"

         DESCSET 0       9       UNUSED

                 9       2       9

                 11      2       UNUSED

                 13      1       13

                 14      18      UNUSED

                 32      95      32

                 127     1       UNUSED

                 128     32      UNUSED

                 160     55136   160

                 55296   2048    UNUSED  -- SURROGATES --

                 57344   1056768 57344

Specifically the line:



128     32      UNUSED

This means that starting at 128 through the next 32 numbers (to 159) are reserved (the range in fact is reserved by Unicode and ISO 10646 as control characters). In the character encoding Windows 1252, specific characters were assigned to numbers within this range. The most frequently used of these offending characters are the now infamous smart quotes, apostrophes, em dashes, etc. (‘ ’ “ ” —).

One workaround to this problem is to specify the character set used within the document like this:
HTML:
Content-Type: text/html; charset=Windows 1252 or
<meta http-equiv="Content-Type" content="text/html;charset=Windows 1252">
XML:
<? xml version="1.0" encoding="Windows 1252" ?>

However, this method can be overridden and should you fail to specify the character set, the content will be "broken." Also, if you ever need to merge data from different sources, having two completely disparate character sets makes this fundamentally more difficult.

What should you do?
In my opinion, at the very least, any characters that fall within the the forbidden range should be escaped with the proper Unicode Numeric Character References (NCRs). NCRs and entities are ways of representing any Unicode character in XHTML/HTML using only ASCII characters. I read a great W3C i18n tutorial regarding character-sets and encodings in HTML, XML, and CSS. You should choose a character set such as UTF-8 or ISO0-8859-1 that is most-likely to represent most characters without escaping, and then escape any characters above 127 (including the invalid range) with the Unicode NCR.

Here's a php array that maps the ASCII ordinal to the Unicode NCR that might be helpful.



$mapping = array(128 => '&#8364;',

                 130 => '&#8218;',

                 131 => '&#402;',

                 132 => '&#8222;',

                 133 => '&#8230;',

                 134 => '&#8224;',

                 135 => '&#8225;',

                 136 => '&#710;',

                 137 => '&#8240;',

                 138 => '&#352;',

                 139 => '&#8249;',

                 140 => '&#338;',

                 142 => '&#381;',

                 145 => '&#8216;',

                 146 => '&#8217;',

                 147 => '&#8220;',

                 148 => '&#8221;',

                 149 => '&#8226;',

                 150 => '&#8211;',

                 151 => '&#8212;',

                 152 => '&#732;',

                 153 => '&#8482;',

                 154 => '&#353;',

                 155 => '&#8250;',

                 156 => '&#339;',

                 158 => '&#382;',

                 159 => '&#376;');

An important distinction to remember is that a character set is a definition of a particular subset of characters for a particular purpose. (ie. The characters necessary to represent western languages). The character encoding refers to the mapping of these characters to bytes on a computer. It's important to remember that for a particular charset, there could be multiple encodings. (ie. UTF-8, UTF-16, or UTF-32). While they will all refer to the same characters, they will have different byte-mappings.

The Microsoft Windows 1252 specification shows the offending range (Hex rows 80 and 90). More importantly, here's a document that contains mappings from Windows 1252 to the proper Unicode NCR.

This is obviously NOT an exhaustive diagnosis of the challenges and proper usage of character sets and their respective encodings, but it was enough for me to transform a Windows 1252 encoded document into an XML document that rendered correctly within multiple newsreaders, blog readers, and multiple browsers.

Other References
http://www.dwheeler.com/essays/quotes-in-html.html
http://www.ascii.cl/htmlcodes.htm
Unicode Character Database (Description of the Unicode format and content)
UnicodeData.txt
FAQ regarding relationship between Unicode and ISO 10646
ASCII table for reference

Posted by mark at 7:20 PM | Comments (3)

October 11, 2003

It's official

About two weeks ago I switched my desktop at work to Linux. Which is somewhat stupid considering I support a number of Access solutions, and a few ASP-VBScript websites. One is a lost and found system for James Madison University. I'm planning to rewrite it using php classes.

I suppose I should mention that I really didn't dive in head-first. I'm dual booting my system, and in the two full weeks of *trying* to run Linux, I've only booted into Windows 4 or 5 times. The experience has had it's moments. I've learned first-hand the power of the symlink when I tried to make a backup of the data contained in a symlink, and only created a second symlink. I then proceeded to delete some of the data on the *copy*, and found out later that it wasn't a copy. Oops.

I'm running Evolution for my Contacts, Calendar, tasks - and when I can't read an HTML email in Pine, or need to work with attachments. I check my email for the most part with Pine through ssh on a remote Linux server. I'm trying hard to use vim for my editing, but get frustrated with the multiple file management through buffers, and the speed with which I can cut and paste code. We'll see. In the meantime, I'm using jedit - and the Project Manager plugin which is okay. I miss Homesite.

One of the major issues I'm dealing with in this transition is the fact that all of my clients use Windows and send me complicated documents in MS Word to post on the web. They are usually large documents with footnotes. I have yet to find a Save-As HTML program that works well for this issue. The *best* still seems to be Microsoft Word ... Open Office didn't do too well. Oh well, I'll keep looking through the oss community. Topher uses Codeweaver's CrossOver Office which allows you to run Office Applications, including Word and Access. I may have to get that.

Posted by mark at 12:32 PM | Comments (1)