[ELDK] Problem with pthread_cond_wait on 8xx?

[Frank Svendsbøe]

On Wed, Sep 23, 2009 at 5:38 PM, Detlev Zundel <dzu at denx.de> wrote:
> Hi Frank,
>
>>> What perplexed me however is that I could see the same thing on my x86
>>> host.  There I even took a look with ltrace and see that without
>>> "-lpthread" there is really a continous stream of library calls, whereas
>>> with "-lpthread", it is one single blocking call...
>>>
>>
>> Thanks for sharing your observation. I can reproduce the same results
>> here on x86-64 too, which is good..
>
> Yes indeed.
>
>>>> In case I don't find any solution, I will probably need to
>>>> upgrade/downgrade the kernel, switch to using uclibc/eglibc, or upgrade
>>>> glibc to v2.9. Do you plan to upgrade the ELDK anytime soon (ie. newer
>>>> glibc for instance)?
>>>
>>> We do not currently have a project which would finance such an upgrade,
>>> I'm afraid.
>>>
>>
>> Well, ELDK is free software right? So maybe someone will volunteer for
>> that.. I wish.
>
> We wish too :)
>
>>>> FYI: I've used getconf and verified that I'm using the NPTL enabled pthreads
>>>> library on the 8xx.
>>>
>>> In your position, I'd try to reproduce my findings on the x86 host and
>>> get an explanation there first why two completely different behaviours
>>> occur.  When you have understood why this happens, maybe you can verify
>>> this for the PowerPC environment.
>>>
>>
>> I haven't studied the libc implementation on either platforms, so this may
>> take a while to figure out..
>>
>> Using ltrace, the test program compiled with -lpthreads gives:
>>
>> signal(2, 0x004008d9)                            = NULL
>> pthread_mutex_lock(0x601080, 0, 0, 0x7fffe2482b18, 0x7fffe2482bb0) = 0
>> pthread_cond_wait(0x6010a8, 0x601080, 0x7fedda059fc0, 0x7fffe2482b18,
>> 0x7fffe2482bb0  C-c C-c <unfinished ...>
>> --- SIGINT (Interrupt) ---
>> pthread_mutex_lock(0x601080, 0x7fffe24828b0, 0x7fffe2482780, -1, 0x601080) = 0
>> pthread_cond_signal(0x6010a8, 0, 0x7fedda059fc0, -1, 0x601080) = 0
>>
>> .. while without -lthread specified, I get:
>>
>> pthread_cond_wait(0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_cond_wait(0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_cond_wait(0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_cond_wait(0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_  C-c C-c0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0
>> <unfinished ...>
>> --- SIGINT (Interrupt) ---
>> pthread_cond_signal(0x6010a8, 0x7fffcd1755f0, 0x7fffcd1754c0,
>> 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_mutex_unlock(0x601080, 0x7fffcd1755f0, 0x7fffcd1754c0,
>> 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_cond_wait(0x6010a8, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>> pthread_mutex_unlock(0x601080, 0x601080, 0, 0x7fffcd175808, 0x7fffcd1758a0) = 0
>>
>> Now compare the third argument in the former and latter dumps. The
>> first dump uses a pointer
>> that seems valid, while the latter is zero.
>>
>> Dumps of same exercise done using strace:
>>
>> With -lpthreads:
>>
>> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
>> getrlimit(RLIMIT_STACK, {rlim_cur=8720000, rlim_max=RLIM_INFINITY}) = 0
>> rt_sigaction(SIGINT, {0x4008d9, [INT], SA_RESTORER|SA_RESTART,
>> 0x7fa7db11d040}, {SIG_DFL}, 8) = 0
>> futex(0x6010ac, FUTEX_WAIT_PRIVATE, 1, NULL  C-c C-c <unfinished ...>
>>
>> Without -lpthread:
>>
>> mprotect(0x7fa628fea000, 4096, PROT_READ) = 0
>> munmap(0x7fa628fd1000, 89354)           = 0
>> rt_sigaction(SIGINT, {0x400879, [INT], SA_RESTORER|SA_RESTART,
>> 0x7fa628a8c040}, {SIG_DFL}, 8) = 0
>>   C-c C-c--- SIGINT (Interrupt) @ 0 (0) ---
>>
>> The former shows that a futex is called. Why not on the latter?
>
> Thinking more about this, it may simply be that in absence of the new
> libpthreads library, glibc provides some stubs which are not very
> efficient.
>

Maybe so..

> The futex call definitely points to the new NPTL implementation -
> without futexes, the implementation would be too expensive.
>
> I can differentiate these two cases by using ldd on the binary by the
> way.  If libpthread.so shows up, we have a "good" binary.  Maybe you can
> revalidate that on the PowerPC hardware you did indeed use -lpthread...
>

The problem is... ppc_8xx-ldd on the test app using -lpthread shows that
libpthread.so.0 is being used by the "good" PowerPC binary, but the "good"
binary does evil things. Which is why I suspect something's wrong in the
MPC8xx implementation of glibc.

I'm using posix message queues on the same target, and kernel threads
, without any problems, so I don't suspect anything is wrong in the kernel.

Is there any change you or anyone you know could try to do the same
exercise on an MPC8xx target?

Regards,
Frank

> Cheers
>  Detlev
>
> --
> error compiling committee.c: too many arguments to function
> --
> DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
>