PATCH: ldconfig.c considers ld.so.conf entries before /lib,/usr/lib

[Dirk Eddelbuettel <edd at debian dot org>]

[ Please CC me on replies, I am not subscribed to this list. ]


I. Problem

ldconfig -v [from Debian's libc6_2.2.4-1 package] reads /lib and /usr/lib
before it considers the directories in /etc/ld.so.conf.  This is in conflict
with ld.so(8) as well as ldconfig(8).

Because /lib and /usr/lib are searched first, we cannot dynamically load
libraries from other directories instead of ones (with identical names) in
/lib or /usr/lib.  This is a problem for "use-if-and-only-if-available"
scheme we would like to apply to the Atlas linear algebra libraries [cf
http://www.netlib.org/atlas].  Atlas can transparently replace the (very
common) BLAS libraries, and thereby offer a speed gain of around 10 or basic
linear algebra operations [see below].

GNU Octave and GNU R can be linked against Atlas.  Rather than imposing a
hard dependency on Atlas and requiring an installation of Atlas at every R or
Octave installation, we have opted to configure/compile/link against the
libblas.so. If Atlas BLAS are installed (eg in /usr/lib/atlas/libblas.so or a
cpu-specific subdir of /usr/lib) they can be used instead.  This currently
requires tinkering with LD_LIBRARY_PATH even when /usr/lib/atlas is in
/etc/ld.so.conf -- because /usr/lib/libblas.so is found first, it hides the
one in /usr/lib/atlas we'd rather load. The patch below removes the need to
tinker with LD_LIBRARY_PATH and makes the dramatic performance gain more
easily available.


II. Patch

We move the parse_conf() call before the add_dir() call(s). This has been
tested against the Debian libc_2.2.4-1 package, and has worked fine for two
days.

--- glibc-2.2.4/elf/ldconfig.c.orig     Mon Jul 23 12:53:40 2001
+++ glibc-2.2.4/elf/ldconfig.c  Mon Aug 27 09:36:25 2001
@@ -1078,12 +1078,15 @@
 
   if (!opt_only_cline)
     {
+      /* Read ld.so.conf first
+       * camm@enhanced.com / edd@debian.org
+       */
+      parse_conf (config_file);
+
       /* Always add the standard search paths.  */
       add_dir (SLIBDIR);
       if (strcmp (SLIBDIR, LIBDIR))
        add_dir (LIBDIR);
-
-      parse_conf (config_file);
     }
 
   search_dirs ();


III. Illustration of Atlas' benfits

This shows some echo'ed code for R and Octave (doing simple matrix cross
products) along with timings in seconds. 

Evaluating Atlas libs at /usr/lib/atlas on size 1500
 
GNU R: m<-matrix(rnorm(1500*1500),ncol=1500); cat(system.time(crossprod(m))[1])
  Without Atlas: 92.66
  With Atlas   : 9.5
 
GNU Octave: X=randn(1500,1500); t=cputime(); Y=X'*X; disp(cputime-t)
  Without Atlas: 96.100
  With Atlas   : 9.8200

Evaluating Atlas libs at /usr/lib/3dnow/atlas on size 1500
 
GNU R: m<-matrix(rnorm(1500*1500),ncol=1500); cat(system.time(crossprod(m))[1])
  Without Atlas: 92.7
  With Atlas   : 7.86
 
GNU Octave: X=randn(1500,1500); t=cputime(); Y=X'*X; disp(cputime-t)
  Without Atlas: 96.060
  With Atlas   : 8.1900


Kudos to Camm Maguire for cooking up the blas/atlas scheme.

Regards, Dirk