Wednesday, December 20, 2006

Python MySQLdb, warnings, mix-ins

I've been working on a side project that uses MySQLdb. Database work. Munging imperfect data inevitably spawns warnings and I want to supress the obviously unimportant ones.

Originally, one would call MySQLdb.connect with "cursorclass=MySQLdb.cursors.CursorNW" as an argument. (NW=No Warnings). That hasn't worked for years, though, so if you google and find that, you're SOL.

The next approach was to derive a new cursor class using mix-ins; the default cursor has a "Warning" mix-in that we simply want to exclude:

import MySQLdb.cursors
class EricsCursor(MySQLdb.cursors.CursorStoreResultMixIn,
MySQLdb.cursors.CursorTupleRowsMixIn,
MySQLdb.cursors.BaseCursor,
):
...
db = MySQLdb.Connect( user=uname, passwd=pword, db=dbase, cursorclass=EricsCursor)

Unfortunately, that documentation is also out of date. The CursorWarningMixIn is no more and in fact the EricsCursor is exactly what the default cursor is, now. So that doesn't work either... still SOL.

Finally, the actual, correct way to supress these messages is with the stock python warning module. You can either turn off all warnings (which is probably a bad idea) or filter exactly those you know are incorrect out, as follows:

import warnings
Class Blah:
def supress_warnings():
# I get a warning on "DROP TABLE IF EXISTS" query,
# that the temp table (something_temp) is unknown.
warnings.filterwarnings("ignore", "Unknown table.*_temp")
return
supress_warnings = staticmethod(supress_warnings)


This actually works, and I can piecewise supress any warnings I know are irrelevant. I made it a static method to keep the code in the same module as the Database class, but I want to call it exactly once early in my program initialization routines.

Friday, December 15, 2006

Dick's "The Man in the High Castle"

Having fallen into a rut of reading the same author's books, I found a couple of top-(whatever) lists for fiction, and I'm working my way through them at the library. I just finished Philip K. Dick's "The Man in the High Castle." It was OK, but I was rather unsatisfied with the ending. Two of the main characters just lose it, then the whole alternate 'real' reality thing seemed to come out of nowhere.

I completely missed that Tagomi was supposed to be visiting our reality. I thought it was just being pointed out that Japanese control wasn't total.

Did Juliana actually swallow a razorblade? If so, how was she able to just continue on with the story (eating, drinking, sleeping, etc) with no ensuing problems?

Hawthorne's family's bland acceptance that Juliana had killed a man seemed too much to accept.

All in all, an OK book. But mix in a little ignorance of German and history around WWII, that this was the first PKD book I've read, and the way it ended, I can't say I'd recommend it.

Wednesday, August 23, 2006

Vim folds

I've been a long-time Vim user. To the extent that I have a customized calendar syntax and keybindings written for it, I often do filesystem management operations (rename a bunch of oddly-named files to unified format), and so forth within the editor.

People accuse emacs of being the kitchen sink, but you can do pretty much the same stuff within Vim, it just starts up faster, has a different set of keybindings, and has that whole "Mode" thing going on.

Regardless, I've started using folds in Vim, and it's a really really useful feature. I've known about it for a long time, but never really saw the point. Now I do... it allows you to get rid of all the irrelevant context and focus on the meat of whatever you're doing. Makes paper writing much easier; you can fold each section and see the outline, or collapse everything except for the section you're working on and the abstract. Handy.

Monday, August 21, 2006

CSE Building "rooted"

There have been a rash of thefts from the CSE building over the last several months. Laptops, computers, even our couch disappeared. Today, somebody stole an HP PDA. Not just any PDA-- the PDA that is used to program the building's electronic locks.

In other words, (1) We can't change access controls to the building any more and (2) for all practical purposes, somebody now has complete access and control of the doors to the most `secure' parts of the building (e.g. machine/equipment rooms with millions of dollars of stuff in them).

All this because they wanted to save a few bucks by not installing a centralized system, and because one admin didn't lock her door. This is the admin whose job it was to walk around to every one of the 40+ doors and recode them every single time any access changed (e.g. a person joined department, left department, etc).

Brilliant.

Whoever stole it certainly knew how to attack the weakest part of the security system.

Friday, August 18, 2006

CVS's "Move away * it is in the way"

I used to have this problem with CVS: I could work in it, update it (cvs update) and so forth, but if I ever wanted to refresh the directories (cvs update -d) I'd get millions of "Move away [filename], it is in the way" messages.

The problem was that I had a trailing slash on the end of the directory name in the CVS/Root files. e.g. This fails:

:ext:eric@localhost:/home/eric/cvs/


But this works:

:ext:eric@localhost:/home/eric/cvs


Bah. Yet another reason I need to investigate subversion.

Monday, April 03, 2006

LaTeX compilation with 'rubber'

My main desktop died last week and I reinstalled from scratch (I had everything safe on a backup, not to worry). This gave me a chance to upgrade all my software, and things are working great now. I really like the x.org server and globally antialiased fonts, and other nice minor features.

What has made my day, however, is the 'rubber' tool. I do a lot of writing, and most of it in tex. It's always been irritating to have to manage dependencies, makefiles, etc. all by hand. Now, I just have a makefile I can dump anywhere and it works like magic!


TARGET_FORMAT=ps
#TARGET_FORMAT=pdf

PROBLEMS:=${shell ls *.tex}

all: figures ${PROBLEMS:.tex=.${TARGET_FORMAT}}
gvwatch ${PROBLEMS:.tex=.${TARGET_FORMAT}}

%.pdf:%.tex
rubber --pdf $<

%.ps:%.tex
rubber --ps $<

figures:
$(MAKE) -C figures

clean:
$(MAKE) -C figures clean
rm -f *.{aux,bbl,blg,dvi,log,ps,pdf} texput.log *~


I love cool little tools that make my life easier.

Monday, February 20, 2006

More Firefox issues

I run multiple Firefox profiles concurrently. Using simultaneous profiles allows me to, for example, have both my work and personal web mail accounts open to the same site simultaneously. It also allows me to obscure my browsing habits slightly.

This worked fine up until my upgrade to Firefox 1.5. Suddenly, even if you run firefox as 'firefox -ProfileManager', it won't start the profile manager. It will start up the last profile you used. This is stupid, broken behavior. But the firefox developers think it's ok, because "Profiles aren't for users, even for advanced users. They're for developers." Whatever.

To get firefox to actually do what you tell it to (run the profile manager, so you can pick which profile to start up in the new firefox window), you have to set an environment variable "MOZ_NO_REMOTE=1". Do that, and it works again. It's possible that to someone nerdier than I, this makes perfect sense. But I doubt it.

The fact that remote firefox interaction (read: AIM wants to open a webpage) interferes with profile management should not be the user's problem. One profile is already the default, open links in it and don't mess with this remote crap. If that profile's not open, start it. Voila.

Sunday, February 19, 2006

Firefox CPU issues

For as long as I can remember, I've had this problem where firefox sucks up 100% of my CPU, freezes, locks, effectively dies but won't go away... and I have to kill it. I've always suspected it was Flash, but now I have some form of proof.

Installing the Firefox flash blocker extension made the problem go away. Now on sites where I want to see the flash (e.g. flickr) I can just click on any flash I want to see. On other sites, it doesn't start it up.

No more 100% CPU. No more locks. Eric is a happy man.

Also, in the debugging process, I discovered that pressing 'Escape' will stop all animated GIFs on the current page.

These two things combined mean that my browsing experience has dramatically improved in the last few days.

Thursday, February 09, 2006

AC_CONFIG_HEADERS vs AM_CONFIG_HEADERS

I'm using autoconf and automake to deal with building issues in my software, CEP. The move was not by choice, but by necessity for interaction with other tools. It has been nothing but a pain.

Autoconf and automake have a config file which is built out of macros. Two of those macros are AC_CONFIG_HEADERS and AM_CONFIG_HEADERS. Now, here's the fun part. To use autoconf, you need to use the former. To use automake, you need to use the later. Either tool will barf processing the config file if the other is present (with error messages like `automake requires `AM_CONFIG_HEADER', not `AC_CONFIG_HEADER', which tell you how to fix the problem but not the root cause). This means that you'd need to edit the config file every time you built the software. Unfortunately, even that workaround eventually failed: the "configure" script installed an automatic dependency check into the makefile, which would try to do an automake and autoconf run to verify changes (read: any time CVS touched files). So the software would not build. Google found zero help for this problem.

Turns out, it was due to a version inconsistency between autoconf and automake. Debian keeps multiple simultaneous versions of these tools around because they have different behavior and certain code will only work with one version. My problem was that while I had installed the
newer version (automake1.9) it hadn't replaced version 1.4, and nothing had changed. As soon as I got rid of automake version 1.4, suddenly everything worked.

Lessons learned: (1) when google can't find a solution, it's something very strange. (2) You should blog it so people can find it in google later (3) Automake, autoconf, and debian are really annoying sometimes.

Test post

This is a test post.

I used to use livejournal as more of an online diary, but I'm looking to move to a more professional site that I'm willing to let search engines index.

Hopefully I'll be able to post some useful observations here. Probably mostly computer-related stuff that I want to publish.

We'll see, I guess.