Here are some links you might find useful in your efforts to avoid using X:
A couple people have asked my why I've stopped using X. The simplest answer is "boredom". One night I was tweaking with my desktop (because I was bored) and thought it might be nice to never again tweak it. Also removing X would give me something to do, since I'd have to find suitable replacements for the four GUI programs I was still using: mozilla, gaim, xmms, and logjam. In the course of switching I found a usable links and wrote grim, mope, and vilj, to replace all of them, respectively. Each of the three tools I wrote has all of the features that I was using before, none of the ones I wasn't using, and run on the command line.
Of course, it's been slightly less than easy. The hardest thing is websites. If you have ever gotten annoyed because a website only works with Internet Explorer and you use Netscape, you'll understand what I mean, but it's a thousand times worse with links. Links (the version I'm using, anyway), doesn't have a decent javascript implementation or an image renderer (it does if you use the framebuffer driver, but that's not possible in screen). The sites that I frequent (livejournal, slashdot, yahoo) all handle this fine, but certain websites (like Bank of America and HBO) are rather unusable. So I don't use them.
Also, linux framedrivers are perpetually broken. Quoth Linus: "Note that the fb stuff is ancient because it's basically not maintained as far as I'm concerned."