New computer!

aka "Holy crap, I've got a blog."

Circa 2001 I built a nice computer. Big white-box case, 750MHz athlon, 512MB memory, 80G hard drive. That thing was sweet! Of course, time passes, and in late 2008 it was showing its age. So, now that I'm older and lazier, I bought a suitably customized Dell XPS 630. Other than the stupid case (has this ridiculous jaunty angle, and LEDs bright enough to read by), it's awesome.

Of course, I put Linux on it. That was a pain, but worth it. Here's what was necessary:

Install Ubuntu from DVD

I was a long-time Debian user, decided to jump ship after one too many head-banging moments due to their overzealous free/nonfree policies. The Ubuntu install was a breeze and I loved the fact that the DVD was also 'live' so I could play with it and if I ever need to, recover with it.

Get Connected

I needed to do a hard reboot of cable modem before my machine could get network access. The modem or something upstream is doing single-MAC whitelisting. New computer wouldn't connect to the network until I hard rebooted it with just the new computer attached; now the old one won't connect.

I also needed these non-default packages:

• ssh (server): Can't live without remote access.


• vnc4server, xvnc4viewer: Remote GUI access, even.


• ez-ipupdate: So I can get into the box remotely, even with a dynamic IP

Get Video/DVD Playing Working

Much of the time I spend at my computer these days is watching DVDs. That was my first priority after getting the network up. The video card is an "(Asus) 512MB Radeon ATI HD 3870", branded "FireGL" which is new and shiny enough the drivers are still a little wack. You can use either the fglrx driver (xorg-driver-fglrx package) or the radeon driver (xserver-xorg-video-ati package). The RadeonHD driver doesn't support this card.

FGLRX driver

You get logs that show you this:

  (--) PCI:*(1:0:0) ATI Technologies Inc unknown chipset (0x9501) rev 0, Mem @ 0xd0000000/28, 0xefde0000/16, I/O @ 0xde00/8
  (--) fglrx(0): Chipset: "ATI Radeon HD 3870" (Chipset = 0x9501)
  (--) fglrx(0): (PciSubVendor = 0x1028, PciSubDevice = 0x2542)
  (--) fglrx(0): board vendor info: third party graphics adapter - NOT original ATI
  (II) fglrx(0): Using adapter: 1:0.0.
  (--) fglrx(0): VideoRAM: 262144 kByte, Type: DDR4
  (II) fglrx(0): PCIE card detected
  (--) fglrx(0): Using per-process page tables (PPPT) as GART.
  (WW) fglrx(0): board is an unknown third party board, chipset is supported

But then, if you do VideoOverlay you get a hard kernel lock and need the Big Red Button. I also played with this command: aticonfig --overlay-type=Xv to no great luck.

Radon driver

You get logs that show you this:

  (WW) RADEON(0): Direct rendering disabled
  (EE) RADEON(0): Acceleration initialization failed
  (--) RADEON(0): Chipset: "ATI Radeon HD3870" (ChipID = 0x9501)
  (WW) RADEON(0): R600 support is mostly incomplete and very experimental

It annoyingly lacks a functional XV extension, but it doesn't lock my machine. For such a beefy card, it sucks to be doing software rendering. Hopefully the drivers stabilize a bit and I can get this working better in the future.

Mplayer configuration

For DVDs you'll want these packages: ubuntu-restricted-extras,w32codecs,libdvdcss,medibuntu-keyring: this pulls in non-free extras, required to play DVDs and MP3s. The really annoying thing here is that without the decss package, mplayer will happily try to play the DVDs without decrypting them, giving you awesome garbage and unrelated error messages. That took a while to figure out; I thought something was wrong with the drive itself.

Since the XV extension isn't working, you need to pass -zoom on the command line, or add it to your ~/.mplayer/config. Occassionally the software zoom shows visual artifacts.

If you install any KDE packages, like filelight, mplayer will hang for several seconds every time you try and play a video. It's trying to kill kde screensavers. (See this bug. Just add 'stop-xscreensaver=0' to ~/.mplayer/config and it won't be stupid any more; however you'll need to manually kill any screensavers you have running with e.g. xscreensaver-command -exit to watch movies.

Get all the other bits working

Install non-default packages to actually be able to use lots of helpful stuff:

• vim: The one true editor.


• subversion: I keep swaths of my home dir under version control.


• filelight: To manage space (won't need to for a while, hello huge disk!


• rhythmbox, amarok: I miss XMMS. Enough so I kind of want to become a package maintainer... this is dumb.


• pwgen: for stupid web passwords.


• dselect: Real package management.


• xscreensaver: So the massive number of strangers wandering through my livingroom when I'm away can't access my data.


• gphoto2, python-extractor, inkscape, imagemagick, ffmpeg: Make pretty pictures/movies, download from camera.


• bittornado-gui: For downloading legally shared stuff.


• gnucash: Finance management.


• sgt-puzzles: Fun management.


• uvccapture,luvcview: Try out webcams. It works.


• gnumeric: Spreadsheets are handy.


• xautomation: So is forging X input. Yay cheating at flash games.


• mutt: Ocassionaly you need a real mail reader.


• festival festvox-kdlpc16k: And sometimes somebody to read your mail to you.

There were also a couple last configuration bits to be done:

• Set up new ssh keys/authorized keys file.


• sudo update-alternatives –config editor: Vim.


• Add a firewalling script.

WTF is going on with my keyboard?

This one pissed me off; the mouse I've been using for years, a shiny logitech number that automatically turns the LED into a lower-power mode when not in use, doesn't work with this new kernel. It feels the need to reset the device when it does that, which locks up the bus for a few seconds. If you happen to have a key pressed on the keyboard when that happens, you get stuck keys (and text that looks liiiiiiiike thisssssss). Very annoying; this happens every three seconds you're not using the mouse. I ended up switching to the dinky dell mouse that came with the computer, which doesn't have this behavior, but I miss my old one.

You see this message filling dmesg logs every time it happens

[12339.536428] usb 1-3: reset low speed USB device using ohci_hcd and address 3
[12342.800250] usb 1-3: reset low speed USB device using ohci_hcd and address 3
[12355.696772] usb 1-3: reset low speed USB device using ohci_hcd and address 3

Summary

A new linux box with new hardware is always wonderful and annoying. More wonderful than annoying these days, and Ubuntu is very cool. I'm a fan. It pretty much just worked at a basic level out of the box. After tweaking only two things are suboptimal: the video card's not fully supported (no super-fast rendering, but I'm not really gaming with it now, so I don't care that much) and I can't turn off the case LEDs. LibSMBios (dellLEDCtl, getSystemId) is two years out of date and doesn't support this model.

configure and sfslite

OK, I wasted the better part of an hour figuring this one out. I'm building sfslite-0.8 for use with Chord. It requires the GMP library. But configure reports the wrong error when you're building the thing. To use with chord, it recommends you use these parameters to configure:

../sfslite-0.8/configure --with-sfsmisc --with-dmalloc 

Which on my system (with GMP installed) gives:

checking for GMP library... checking for gmp.h... yes
checking for libgmp... configure: error: Could not find gmp library

Adding --with-gmp=/usr/lib changes nothing. Google finds nothing. Checking GMP manually and it seems to be fine. Finally, after looking through the log files I find the real error: GMP is fine, but dmalloc's not installed! The GMP test depends on gmalloc, but dmalloc wasn't tested earlier.

Removing the --with-dmalloc line fixes the problematic symptoms.

The root problem problem is the ordering of dependencies in configure, though. Bah.

Hacking direct-debit checks

A little story, with two halves.

First half: One of my roommates is somewhat disorganized. After holding a check of mine from June to mid-October last year before cashing it, we came to an agreement. He would have two weeks to cash checks written to him, or would eat the cost. I'd typically write "VOID AFTER (date)" and give him to the end of the month.

Second half: WAMU lets certain companies do direct debits from checking accounts, regardless of whether the customer has authorized them to do so. For example, SDG&E and Time Warner. This has always irritated me, because I don't get the checks back and I can't even see the check image online. All I get is the number, and there's no paper trail.

Putting it together: February 3rd I wrote the roommate a check for my portion of the cable and phone bill-- $22.45-- and wrote "VOID AFTER FEBRUARY 28th, 2007" in big letters up on the top. Sometime mid-March I wrote off the debt. May 21st a charge appears on my account: "TIME WARNER-SAND CHECK PYMT (number)" for $22.45.

WTF?

Well, it turns out the roommate had accidentally mixed up checks sent to the cable company-- and they cashed it! Not only was it not made out to them, but it was void three months prior! But the charge went through, because nobody at WAMU actually verified anything.

Moral of the story: if you want perform invalid transactions, forge checks, or other illegal behavior, Time Warner is happy to launder your money.

And the roommate's actually going to refund me the dough so I'm not going to go bitch and moan at the companies involved here-- although it would be interesting to try and track this all down.

Subversion vs. CVS

I recently decided to take subversion for a test drive. I know a couple folks who are using it and were pleased, and I was tired of some problems with CVS (e.g. having to hack the repository by hand for moves, stupidity with binary files, the pollution of my file namespace with "CVS" directories, ...).

Short version of the result: steep but short learning curve, immediate vast improvement in productivity over CVS.

Longer version:
The "CVS to SVN" howtos don't tell you some of the info you really need to know.

1. Subversion makes an explicit distinction between data and metadata ("properties", in svn). CVS uses .cvsignore files,
   SVN uses svn:ignore properties which you have to set up yourself. This bit me in a weird way-- I had put some JPEG files into CVS but forgotten to set -kb so they became corrupted. I noticed this during my check of files after importing into subversion, and copied correct versions over the corrupt ones in my working subversion directory-- but committing did nothing. This is because subversion helpfully set up file properties for these files that marked them non-binary; these had to be removed using

   svn propdel eol-style
   svn propdel svn:keywords

   before fixing it up with

   svn propset svn:mimetype application/octet-stream file.jpg

   Then, it works correctly.
   
   
2. Subversion doesn't have modules. In CVS I had a separate directory in my CVS root for each project, and could check out and commit independently. Importing this into subversion makes one repository, and committing to one project bumps the revision number on the entire repository. There's nothing technically wrong with this, but it's confusing-- when you 'update' a project and the revision number goes up, yet nothing has changed. To get the same behavior as CVS, you need to create separate repositories for each independent project.
   
   
3. Subversion doesn't like man pages. Everything is via [command] help. This is just annoying because you can't get all the info in one page to search through, you have to interact with the program's help system.
   
   
4. Subversion's repository can only be mucked around with through commands, or worst-case through a text file dump of the same. At some point, I guarantee you will have to hand edit a dump file. You will need to learn their syntax (which isn't so bad) but it's not like going in to CVS and doing file operations.
   
   
5. Subversion likes URLs for some commands. Even if you're running on localhost. And they're not too consistent on when to use one, and when another. This kills tab-completion on my shell. Annoying.
   

But once you get beyond that stuff you find it's pretty slick.

• Moving files is possible-- restructuring your source code tree is easy rather than darn near impossible.
  
• Branching is trivial.
  
• Your checked-out directories are 'clean' (no CVS directories).
  
• You can do lots of things disconnected due to the file caching mechanisms.
  
• Atomic commits.
  
• Vastly improved logging mechanisms, log-per-commit.
  
• Status checking.
  
• Knowledge that the tool has active developers and is only going to get better rather than die a slow painful dragged-out death.
  
• "Coolness" factor.
  

Python MySQLdb, warnings, mix-ins

I've been working on a side project that uses MySQLdb. Database work. Munging imperfect data inevitably spawns warnings and I want to supress the obviously unimportant ones.

Originally, one would call MySQLdb.connect with "cursorclass=MySQLdb.cursors.CursorNW" as an argument. (NW=No Warnings). That hasn't worked for years, though, so if you google and find that, you're SOL.

The next approach was to derive a new cursor class using mix-ins; the default cursor has a "Warning" mix-in that we simply want to exclude:

import MySQLdb.cursors
class EricsCursor(MySQLdb.cursors.CursorStoreResultMixIn,
                MySQLdb.cursors.CursorTupleRowsMixIn,
                MySQLdb.cursors.BaseCursor,
                ):
...
db = MySQLdb.Connect( user=uname, passwd=pword, db=dbase, cursorclass=EricsCursor)

Unfortunately, that documentation is also out of date. The CursorWarningMixIn is no more and in fact the EricsCursor is exactly what the default cursor is, now. So that doesn't work either... still SOL.

Finally, the actual, correct way to supress these messages is with the stock python warning module. You can either turn off all warnings (which is probably a bad idea) or filter exactly those you know are incorrect out, as follows:

import warnings
Class Blah:
    def supress_warnings():
        # I get a warning on "DROP TABLE IF EXISTS" query,
        # that the temp table (something_temp) is unknown.
        warnings.filterwarnings("ignore", "Unknown table.*_temp")
        return
    supress_warnings = staticmethod(supress_warnings)

This actually works, and I can piecewise supress any warnings I know are irrelevant. I made it a static method to keep the code in the same module as the Database class, but I want to call it exactly once early in my program initialization routines.

Dick's "The Man in the High Castle"

Having fallen into a rut of reading the same author's books, I found a couple of top-(whatever) lists for fiction, and I'm working my way through them at the library. I just finished Philip K. Dick's "The Man in the High Castle." It was OK, but I was rather unsatisfied with the ending. Two of the main characters just lose it, then the whole alternate 'real' reality thing seemed to come out of nowhere.

I completely missed that Tagomi was supposed to be visiting our reality. I thought it was just being pointed out that Japanese control wasn't total.

Did Juliana actually swallow a razorblade? If so, how was she able to just continue on with the story (eating, drinking, sleeping, etc) with no ensuing problems?

Hawthorne's family's bland acceptance that Juliana had killed a man seemed too much to accept.

All in all, an OK book. But mix in a little ignorance of German and history around WWII, that this was the first PKD book I've read, and the way it ended, I can't say I'd recommend it.

Vim folds

I've been a long-time Vim user. To the extent that I have a customized calendar syntax and keybindings written for it, I often do filesystem management operations (rename a bunch of oddly-named files to unified format), and so forth within the editor.

People accuse emacs of being the kitchen sink, but you can do pretty much the same stuff within Vim, it just starts up faster, has a different set of keybindings, and has that whole "Mode" thing going on.

Regardless, I've started using folds in Vim, and it's a really really useful feature. I've known about it for a long time, but never really saw the point. Now I do... it allows you to get rid of all the irrelevant context and focus on the meat of whatever you're doing. Makes paper writing much easier; you can fold each section and see the outline, or collapse everything except for the section you're working on and the abstract. Handy.

CSE Building "rooted"

There have been a rash of thefts from the CSE building over the last several months. Laptops, computers, even our couch disappeared. Today, somebody stole an HP PDA. Not just any PDA-- the PDA that is used to program the building's electronic locks.

In other words, (1) We can't change access controls to the building any more and (2) for all practical purposes, somebody now has complete access and control of the doors to the most `secure' parts of the building (e.g. machine/equipment rooms with millions of dollars of stuff in them).

All this because they wanted to save a few bucks by not installing a centralized system, and because one admin didn't lock her door. This is the admin whose job it was to walk around to every one of the 40+ doors and recode them every single time any access changed (e.g. a person joined department, left department, etc).

Brilliant.

Whoever stole it certainly knew how to attack the weakest part of the security system.

CVS's "Move away * it is in the way"

I used to have this problem with CVS: I could work in it, update it (cvs update) and so forth, but if I ever wanted to refresh the directories (cvs update -d) I'd get millions of "Move away [filename], it is in the way" messages.

The problem was that I had a trailing slash on the end of the directory name in the CVS/Root files. e.g. This fails:

:ext:eric@localhost:/home/eric/cvs/

But this works:

:ext:eric@localhost:/home/eric/cvs

Bah. Yet another reason I need to investigate subversion.

LaTeX compilation with 'rubber'

My main desktop died last week and I reinstalled from scratch (I had everything safe on a backup, not to worry). This gave me a chance to upgrade all my software, and things are working great now. I really like the x.org server and globally antialiased fonts, and other nice minor features.

What has made my day, however, is the 'rubber' tool. I do a lot of writing, and most of it in tex. It's always been irritating to have to manage dependencies, makefiles, etc. all by hand. Now, I just have a makefile I can dump anywhere and it works like magic!

TARGET_FORMAT=ps
#TARGET_FORMAT=pdf

PROBLEMS:=${shell ls *.tex}

all: figures ${PROBLEMS:.tex=.${TARGET_FORMAT}}
    gvwatch ${PROBLEMS:.tex=.${TARGET_FORMAT}}

%.pdf:%.tex
    rubber --pdf $<

%.ps:%.tex
    rubber --ps $<

figures:
    $(MAKE) -C figures

clean:
    $(MAKE) -C figures clean
    rm -f *.{aux,bbl,blg,dvi,log,ps,pdf} texput.log *~

I love cool little tools that make my life easier.

More Firefox issues

I run multiple Firefox profiles concurrently. Using simultaneous profiles allows me to, for example, have both my work and personal web mail accounts open to the same site simultaneously. It also allows me to obscure my browsing habits slightly.

This worked fine up until my upgrade to Firefox 1.5. Suddenly, even if you run firefox as 'firefox -ProfileManager', it won't start the profile manager. It will start up the last profile you used. This is stupid, broken behavior. But the firefox developers think it's ok, because "Profiles aren't for users, even for advanced users. They're for developers." Whatever.

To get firefox to actually do what you tell it to (run the profile manager, so you can pick which profile to start up in the new firefox window), you have to set an environment variable "MOZ_NO_REMOTE=1". Do that, and it works again. It's possible that to someone nerdier than I, this makes perfect sense. But I doubt it.

The fact that remote firefox interaction (read: AIM wants to open a webpage) interferes with profile management should not be the user's problem. One profile is already the default, open links in it and don't mess with this remote crap. If that profile's not open, start it. Voila.

Firefox CPU issues

For as long as I can remember, I've had this problem where firefox sucks up 100% of my CPU, freezes, locks, effectively dies but won't go away... and I have to kill it. I've always suspected it was Flash, but now I have some form of proof.

Installing the Firefox flash blocker extension made the problem go away. Now on sites where I want to see the flash (e.g. flickr) I can just click on any flash I want to see. On other sites, it doesn't start it up.

No more 100% CPU. No more locks. Eric is a happy man.

Also, in the debugging process, I discovered that pressing 'Escape' will stop all animated GIFs on the current page.

These two things combined mean that my browsing experience has dramatically improved in the last few days.

AC_CONFIG_HEADERS vs AM_CONFIG_HEADERS

I'm using autoconf and automake to deal with building issues in my software, CEP. The move was not by choice, but by necessity for interaction with other tools. It has been nothing but a pain.

Autoconf and automake have a config file which is built out of macros. Two of those macros are AC_CONFIG_HEADERS and AM_CONFIG_HEADERS. Now, here's the fun part. To use autoconf, you need to use the former. To use automake, you need to use the later. Either tool will barf processing the config file if the other is present (with error messages like `automake requires `AM_CONFIG_HEADER', not `AC_CONFIG_HEADER', which tell you how to fix the problem but not the root cause). This means that you'd need to edit the config file every time you built the software. Unfortunately, even that workaround eventually failed: the "configure" script installed an automatic dependency check into the makefile, which would try to do an automake and autoconf run to verify changes (read: any time CVS touched files). So the software would not build. Google found zero help for this problem.

Turns out, it was due to a version inconsistency between autoconf and automake. Debian keeps multiple simultaneous versions of these tools around because they have different behavior and certain code will only work with one version. My problem was that while I had installed the
newer version (automake1.9) it hadn't replaced version 1.4, and nothing had changed. As soon as I got rid of automake version 1.4, suddenly everything worked.

Lessons learned: (1) when google can't find a solution, it's something very strange. (2) You should blog it so people can find it in google later (3) Automake, autoconf, and debian are really annoying sometimes.

Test post

This is a test post.

I used to use livejournal as more of an online diary, but I'm looking to move to a more professional site that I'm willing to let search engines index.

Hopefully I'll be able to post some useful observations here. Probably mostly computer-related stuff that I want to publish.

We'll see, I guess.