Re: Performance problem of dlopen

[Joerg Budischewski <Joerg dot Budischewski at sun dot com>]

Hi Jakub,

>>In numbers, on my system the library takes 3300ms to load with one
>>dlopen, while using multiple dlopen takes 2100ms. I have actually
>>played
>>a little around with LD_DEBUG=symbols and found out, that in the first
>>case, lookup for symbols which are located in dependent libraries are
>>searched also in the higher libraries, which leads to significantly
>>more
>>lookups.
>>
>>So my question is if this is a desired behaviour or can this be
>>considered as a performance bug ?
>
>
> I'd prefer to see your testproglets first. ld.so has to obey ELF
> lookup rules, so it is questionable whether anything can be done here.
ok, I have a attached a small C program which just loads shared 
libraries given on command line and prints out the needed time. You can 
simply build it with

gcc -ldl -o shlload shlload.c
.

Additionally I have attached a textfile with a list of libraries for the 
643 (developer) build of openoffice (note: you may also use a production 
build 1.0.x, simply replace 643 by 641. Some libraries may have changed, 
but the effect is still measureable).

Make sure to have a . in your LD_LIBRARY_PATH. Starting

./shlload -v libwrp643li.so

Gives on my system 3150ms needed to load.

Using
xargs ./shlload -v < lst

gives about 1850ms on my system.

> What would IMHO help most for OpenOffice startup spent in the dynamic
> linker
> (in addition to aggressive profiling and speeding up often used
> functions)
> is to merge shared libraries which are
> a) linked statically to the OOo programs
> b) dlopened during start of every OOo program (or at least all the
> important
>    ones; IMHO spadmin or setup startup performance is not critical,
> while
>    soffice with the usual invocations is)
> into libOpenOffice.so and link it into soffice etc. programs
> statically.
Uhmm, I will comment on this later (maybe tomorrow).

> That way you kill lots of duplication (which means less relocations,
> less
> memory used and fewer dynamic symbols) and also prelink(8) can speed
> it
Where can I get prelink ?



>
>>Another point which I am interested in is that since we are using gcc
>>3.x for building, loading shared libraries lazy is significantly
>>slower,
>>one main reasons seems to be that unused symbols get relocated with
>>gcc3
>>which have not got relocated with gcc2. When I use nm on both
>>libraries,
>>the output (beside name mangling of course) looks similar. Does anyone
>>have an idea, what this is caused by ?
>
>
> The C++ ABI changed and unfortunately it is not exactly startup time
> friendly :(.
Is there some simple explanation which can be given in a few words. I 
e.g. can imagine, that every virtual function needs to be relocated or so ?


Joerg

#include <dlfcn.h>
#include <stdio.h>
#include <sys/time.h>

static struct timeval startTime;
static char bGlobalTimer = 0;

static unsigned long getGlobalTimer()
{
  struct timeval currentTime;
  unsigned long nSeconds;

  if ( bGlobalTimer == 0 )
  {
      gettimeofday( &startTime, NULL );
      bGlobalTimer=1;
  }
  
  gettimeofday( &currentTime, NULL );
  
  nSeconds = (unsigned long)( currentTime.tv_sec - startTime.tv_sec );

  return ( nSeconds * 1000 ) + (long) (( currentTime.tv_usec - startTime.tv_usec) / 1000 );
}

static char loadlib( const char *name, unsigned long *duration )
{
    char ret = 0;
    unsigned long start = getGlobalTimer();
/*  loading with RTLD_NOW increases the loadtime by 30%  */
/*      if(dlopen( name , RTLD_NOW) ) */
    if(dlopen( name , RTLD_LAZY) )
        ret = 1;
    *duration = getGlobalTimer() - start;
    return ret;
}

int main(int argc, char * argv[] )
{
    int i  = 1;
    char bVerbose = 0;
    unsigned long start;
    
    if( argc <= 1 || strcmp( argv[1] , "--help" ) == 0)
    {
        printf( "usage: shlload [-v] libname1 libname2 ....\n\n"
                "    loads the given shared libraries and \n"
                "    measures time needed for loading.\n\n"
                "Options:\n"
                "    -v  measures time for every given shared library\n" );
        return 1;
    }
    if( 0 == strcmp( argv[1], "-v" ) )
    {
        bVerbose = 1;
        i++;
    }


    start = getGlobalTimer();
    while( i < argc  )
    {
        unsigned long duration = 0;
        if( bVerbose )
        {
            fprintf( stderr, "loading %s" , argv[i] );
            fflush( stderr );
        }
        
        if( loadlib( argv[i], &duration ) )
        {
            if( bVerbose )
                fprintf( stderr, " succeeded [%dms]\n", duration );
        }
        else
        {
            if( bVerbose )
                fprintf( stderr, " failed\n" );
            else
                fprintf( stderr, "loading %s failed\n" , argv[i] );
        }
            
        i++;
    }
    fprintf( stderr, "Duration %d\n" , getGlobalTimer() - start );
    
}

Attachment: lst
Description: application/java-vm