[Various] libc/1813: dlopen silently failes or causes segmentation faults under memory starvation

[Andreas Jaeger <aj at suse dot de>]

We've received yesterday the appended bug report for glibc.  I tried
the test program with glibc 2.1.91 on Linux 2.4.0test4 - and it ate
all my swap space halting the machine :-(.  The problem can be
reproduced with glibc 2.1.3.

Any volunteers to fix this?

Andreas


Subject: Topics

Topics:
   libc/1813: dlopen silently failes or causes segmentation faults under memory starvation
   Re: libc/1813: dlopen silently failes or causes segmentation faults under memory starvation

---

• To: bugs at gnu dot org
• Subject: libc/1813: dlopen silently failes or causes segmentation faults under memory starvation
• From: lundberg at vr dot net
• Date: Wed, 12 Jul 2000 15:50:45 -0400

>Number:         1813
>Category:       libc
>Synopsis:       dlopen silently failes or causes segmentation faults under memory starvation
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    libc-gnats
>State:          open
>Class:          sw-bug
>Submitter-Id:   unknown
>Arrival-Date:   Wed Jul 12 16:00:01 EDT 2000
>Last-Modified:
>Originator:     lundberg@vr.net
>Organization:
net
>Release:        2.1.3
>Environment:
RH 6.2 CD, standard server install, with kernel update from RPMs

Linux 2.2.16-3 #14 SMP Mon Jul 3 16:31:24 EDT 2000 i686 unknown
>Description:
Under extreme memory starvation conditions, dlopen() can fail in various ways:

Sometimes it returns NULL; a subsequent call to dlerror() also returns NULL.  Workarround: when dlopen() returns NULL, if dlerror() returns NULL, assume "Not enough memory"

Sometimes it causes a segmentation fault.

Sometimes it appears to work (non-NULL return), but a subsequent call to dlclose() causes a segmentation fault.

--

This came about during stress-testing for a new version of an Inernet daemon; since reliable performance even under extreme conditions is desired, this was judged 'severe'.
>How-To-Repeat:
Use setrlimit to something small, say 10K;
malloc() all available memory;
free a few bytes and call dlopen()/dlerror()/dlclose() as shown in the examples.
vary 'a few' from 0 to 1K
>Fix:
>Audit-Trail:
>Unformatted:

---

---

• To: "Andreas Jaeger" <aj at suse dot de>
• Subject: Re: libc/1813: dlopen silently failes or causes segmentation faults under memory starvation
• From: "Gregory A Lundberg" <lundberg at vr dot net>
• Date: Thu, 13 Jul 2000 12:04:36 -0400
• Cc: <bugs at gnu dot org>
• References: <200007121950.PAA13660@delysid.gnu.org> <u8aefm4123.fsf@gromit.rhein-neckar.de>

> > Number:         1813
> > Category:       libc
> > Synopsis:       dlopen silently failes or causes segmentation
> >                 faults under memory starvation

> Can you send a complete test program, please?

ftp://ftp.vr.net/private/lundberg/gnu-bugs/

  loader is the compiled program from the target machine

  loader.c is the sample program

  output is a screen capture from the following script:

for A in 10 20 30 40 50 60 70 80 90 100 200 300 400 500 600 700
do
echo LIMIT $A
loader -s $A -l /usr/lib/libdl.so -e -u /usr/lib/libdl.so
done

---

-- 
 Andreas Jaeger
  SuSE Labs aj@suse.de
   private aj@arthur.inka.de