Upgrading Flex from source RPM to compile PostgreSQL from CVS

This week I'm working on reviewing a patch that's part of the current PostgreSQL CommitFest, the periodic points where all outstanding patches are looked at and committed if ready. The patch I'm looking at requires some performance testing, and all my servers I'd do that on are running CentOS 5, the popular RedHat Enterprise clone. There's a fun surprise waiting for anyone else who tries this: as of last week, you can't build PostgreSQL from the development CVS or GIT repositories anymore on that platform without updating the Flex package from the default one.

The problem started when I ran "make" to build everything and saw this:

make[3]: Entering directory `/home/gsmith/pgproject/pgsql.tpgbench/src/backend/bootstrap'
***
ERROR: `flex' is missing on your system. It is needed to create the
file `bootscanner.c'. You can either get flex from a GNU mirror site
or download an official distribution of PostgreSQL, which contains
pre-packaged flex output.
***

I knew I had flex on my system so this was kind of confusing. Usually there's more detail about dependency failures in the config.log file, and sure enough it had the details:

configure:6793: WARNING:
*** The installed version of Flex, /usr/bin/lex, is too old to use with PostgreSQL.
*** Flex version 2.5.31 or later is required, but this is /usr/bin/lex version 2.5.4.
configure:6807: result: no
configure:6815: WARNING:
*** Without Flex you will not be able to build PostgreSQL from CVS nor
*** change any of the scanner definition files.  You can obtain Flex from
*** a GNU mirror site.  (If you are using the official distribution of
*** PostgreSQL then you do not need to worry about this because the Flex
*** output is pre-generated.)

A quick check shows this change was made to allow some more powerful scanning capabilities in the PostgreSQL language parser, followed by inserting that warning.

Now, while it's possible to just build quickly from source and install directly over top of the existing flex in this case, I don't like to do that on production systems (or even development ones). It's not a good idea to mix the RPM packages on the system with stuff installed that way. It's better if you can create a new flex RPM package based on newer source code and upgrade to that one, then everything stays managed consistently with RPM.

You can locate SRPMs with a newer version of flex, the one released with Fedora 9, from the following locations:

• kernel.org


• stanford.edu


• usc.edu

Those were the first I found with a new enough version number that they should work. You could easily try to substitute the flex SRPM that comes with Fedora 10 or Fedora 11 instead, haven't tested that here myself yet to say how that goes; probably fine.

I used the kernel.org link, downloaded and installed like this:

$ wget ftp://mirrors.kernel.org/fedora/releases/9/Fedora/source/SRPMS/flex-2.5.35-1.fc9.src.rpm
$ sudo rpm -i flex-2.5.35-1.fc9.src.rpm 

This dumps the source into /usr/src/redhat:

$ cd /usr/src/redhat/
$ ls SOURCES SPECS
SOURCES:
flex-2.5.35.tar.bz2

SPECS:
flex.spec

You can then build like this:

$ cd /usr/src/redhat/SPECS/
$ sudo rpmbuild -bb flex.spec 

That produces the new RPM we want:

$ ls -l /usr/src/redhat/RPMS/x86_64/
-rw-r--r-- 1 root root 322619 Jul 22 17:14 flex-2.5.35-1.x86_64.rpm
-rw-r--r-- 1 root root 283675 Jul 22 17:14 flex-debuginfo-2.5.35-1.x86_64.rpm

Now install it:

$ cd /usr/src/redhat/RPMS/x86_64/
$ ls
flex-2.5.35-1.x86_64.rpm  flex-debuginfo-2.5.35-1.x86_64.rpm
$ rpm -qa flex
flex-2.5.4a-41.fc6
$ sudo rpm -Uvh flex-2.5.35-1.x86_64.rpm 
Preparing...                ########################################### [100%]
   1:flex                   ########################################### [100%]
$ rpm -qa flex
flex-2.5.35-1

And then cleanup:

$ cd /usr/src/redhat/SPECS/
$ sudo rpmbuild --clean --rmsource flex.spec
Executing(--clean): /bin/sh -e /var/tmp/rpm-tmp.99970
+ umask 022
+ cd /usr/src/redhat/BUILD
+ rm -rf flex-2.5.35
+ exit 0

After that, I had to run "configure" again to pick up the change, ran "make again", and now my build against CVS checkout of the future PostgreSQL 8.5 in progress compiles without any problems.

Note that normally, when you substitute a package like this you need to be careful you keep up with security patches to it because you're not going to get them automatically anymore. Since flex is a pretty low-level tool used only for developing software, I'm not too concerned about the security implication of my running a custom version here.