Author: Rory Winston

I am currently in the process of setting up a continuous integration server, which involves installing packages like CVS, Ant, CruiseControl, Tomcat, Apache, MoinMoin, ViewCVS, and MySQL, and basically glueing them all together. It’s a relatively long-winded process (especially on Solaris, as it won’t necessarily have a lot of the GNU prerequisites installed). However, a lot of it is boilerplate and individual packages can be installed relatively quickly.

During the course of installing the various packages and prerequisites involved, you find that from time to time you need to make manual adjustments to makefiles, switch libraries or library versions, or alter the parameters passed to the C compiler, for instance. This is always where I am grateful that I have a pretty good knowledge of C-based development on Unix (rusty as it is), as it comes in very handy when installing lots of open-source onto a Unix box. However, one problem almost had me stumped.

Whilst trying to install the CVS query functionality that comes with ViewCVS, I needed to install MySQL-Python as a prerequisite. This is ostensibly a simple two-step procedure, python setup.py build and (as root) python setup.py install. However, it failed immediately, citing incorrect compiler flags. I located the point in setup.py where it sets those parameters (which it gets from mysql_config), and set them to more sensible parameters for Solaris. This compiled, but failed on the link stage with ld: warning: file /usr/local/mysql/lib/libmysqlclient.a(libmysql.o): wrong ELF class: ELFCLASS64. I tracked this down to a 64-bit MySQL version installed on a 32-bit platform. After uninstalling and reinstalling the correct version of MySQL, I attempted another build of MySQL-Python. This time, it compiled and linked fine , produced a shared library, and was duly installed.

Next, I tried to run a simple Python test script to verify that it was OK. This failed with:

File “/usr/local/lib/python2.3/site-packages/MySQLdb/__init__.py”,
line 27, in ?
import _mysql
ImportError: ld.so.1: /usr/local/bin/python: fatal: relocation error:
file /usr/local/lib/python2.3/site-packages/_mysql.so: symbol
mysql_errno: referenced symbol not found

This was altogether tougher to diagnose. I first thought that mysql_errno might be defined differently for the reentrant versus the non-reentrant MySQL client libraries, so I replaced "-lmysqlclient_r" with "-lmysqlclient" on the linker command line. This didn’t make any difference. I also explicitly added the MySQL library path into the LD_LIBRARY_PATH variable, which also made no difference.

Eventually, I found the answer (at least for my version of Solaris) – pass the shared lib flag to the linker explicitly – for some reason the installer command line doesn’t do this correctly. I found this out by running gcc manually, initially replicating the flags used by the Python installer.

The full gcc command line then looked like this:

gcc -I /usr/local/mysql/include/ -I /usr/local/include/python2.3/ -Wl,-G -fPIC -Wall _mysql.c -o _mysql.so `mysql_config --libs_r`


When the shared lib had built, I copied it to the appropriate spot in the Python build directory, and then just ran setup.py install to produce a working MySQL-Python client lib.

An easy way to fix this is to modify setup.py. Under the line that reads:

extra_compile_args = config(“cflags”)

Add the following lines:

extra_compile_args.append(“-fPIC”)
extra_compile_args.append(“-Wl,-G”)

Whenever I get a new Windows development machine, the first thing I do is install some software to actually make it usable. First off, it’s usually Cygwin.
This allows me to work on a Windows filesystem using a proper set of tools, including the venerable Bash shell. This shell has so many tricks and shortcuts up its sleeve, it’s incredible. One of my favourites (and one I always forget how to use in between using it) is variable substitution. The Win32 CLI offers a watered-down version of this as well, but Bash allows you to do things like this:

for i in $(dir *.MD5); do mv -f $i ${i%.MD5}.md5; done

Note the variable substitution bit in the curly braces, that says “return everything except the shortest match starting from the right-hand-side of the variable $i that matches the pattern .MD5.

Spring 1.2 has got some nice support for database testing. Specifically, it has the class AbstractTransactionalSpringContextTests, which will execute its tests within an individual transaction, and then rollback at the end of each test, unless you specify otherwise by calling setComplete(). This class in turn extends the class AbstractDependencyInjectionSpringContextTests, which uses field injection to populate its data. Field injection looks up beans in the current context by name, rather than type. So for instance, our PersistentTestCase class can now look like:

The neat thing about this is that we have a whole bunch of reference data that is injected into the class and populated automatically – just by declaring the data as a protected field, calling setPopulateProtectedVariables(true), and declaring the reference data bean in the application-context.xml with the correct name. Field-based injection will take care of the rest. Very easy, and it makes things nice and neat. Another nice feature is that contexts can be cached, and not reloaded and reinitialized every time across tests, which can save a lot of time if you’re using Hibernate’s create-drop mode, and it spends a long time initializing constraints.