Before I spend awhile trying to implement a modular system using dlopen(), does anyone know roughly what kind of performance overhead dlopen() creates on a Mac OS X system (or in fact any BSD based UNIX)?
The kind of use I need to use it for is fairly performance critical (mainly latency) and I would hate to spend ages getting a system up and running only to find out that it is too slow to do what I want.
> Before I spend awhile trying to implement a modular system using > dlopen(), does anyone know roughly what kind of performance overhead > dlopen() creates on a Mac OS X system (or in fact any BSD based UNIX)?
> The kind of use I need to use it for is fairly performance critical > (mainly latency) and I would hate to spend ages getting a system up and > running only to find out that it is too slow to do what I want.
No specifics about MacOSX or BSD, but are you truly worried about dlopen()? You'll probably use dlopen() fewer than ten times in the course of an entire program, so it will have less total impact on your program's running time than, say, stat().
So I imagine you're worried about the use of the functions residing in the library to which dlopen() gives you access. There are two (potential) penalties I can think of: First, each call must be indirect through a function pointer variable rather than direct to an address known at link time, and second, it may be that the address-space shenanigans with which a particular system supports dynamic linking tend to punish MMU's and TLB's and the like.
To a zeroth approximation, I'd imagine neither effect would be significant except perhaps in extreme (and maybe contrived) cases. Consider that strlen() probably resides in a dynamic library, along with memset() and qsort() and atan2(). I can see no reason to worry more about mySharedLibraryFunction() than about any of these -- so I suggest you worry about none.
The extreme (and maybe contrived) cases remain as a source of concern. If you have reason to suspect that your use of a function might be particularly taxing, you could experiment: Write two versions of the same function with different names, one linked statically into your program and one resident in a shared library. Your program can then make a few gazillion calls on each and measure the elapsed time to assess the size of the penalty dynamic linkage might impose.
On 2009-07-04 02:07:39 +0100, Eric Sosman <esos...@ieee-dot-org.invalid> said:
> So I imagine you're worried about the use of the functions > residing in the library to which dlopen() gives you access. > There are two (potential) penalties I can think of: First, each > call must be indirect through a function pointer variable rather > than direct to an address known at link time, and second, it may > be that the address-space shenanigans with which a particular > system supports dynamic linking tend to punish MMU's and TLB's > and the like.
Yes, sorry I should have been more specific. I was using dlopen() as a short hand for describing the whole system rather than just the function call itself.
What I was really trying to get at was whether the dlopen / dlsym / dlclose system added any extra over head on top of what the system already uses for dynamic linking? Obviously there is going to be a small performance loss due to needing to search for the library and possibly deciding which is the correct library to load but other than that is there any difference?
Or is it the same implementation just with a developer accessable interface?
Simon Connah wrote: > On 2009-07-04 02:07:39 +0100, Eric Sosman <esos...@ieee-dot-org.invalid> > said:
>> So I imagine you're worried about the use of the functions >> residing in the library to which dlopen() gives you access. >> There are two (potential) penalties I can think of: First, each >> call must be indirect through a function pointer variable rather >> than direct to an address known at link time, and second, it may >> be that the address-space shenanigans with which a particular >> system supports dynamic linking tend to punish MMU's and TLB's >> and the like.
> Yes, sorry I should have been more specific. I was using dlopen() as a > short hand for describing the whole system rather than just the function > call itself.
> What I was really trying to get at was whether the dlopen / dlsym / > dlclose system added any extra over head on top of what the system > already uses for dynamic linking? Obviously there is going to be a small > performance loss due to needing to search for the library and possibly > deciding which is the correct library to load but other than that is > there any difference?
> Or is it the same implementation just with a developer accessable > interface?
> Hopefully that made a bit more sense.
Less sense, I fear. When you said you were worried about the performance of dlopen(), I guessed that you were actually concerned about the (possible) performance penalty of a call to an .so-resident function, possibly magnified by a thrillion such calls. But now you reiterate that you're only worried about the "framework" calls, which will (most likely) impose their costs only a small number of times. So it seems I was entirely wrong about what worried you -- and now I'm utterly without a clue about your concerns.
Could you see your way to describing your purpose in more specific terms than "implement a modular system?" And might you even go so far as to explain what kinds of "performance overhead" concern you? Or would that spoil the game for the puzzle solvers?
> Simon Connah wrote: >> On 2009-07-04 02:07:39 +0100, Eric Sosman <esos...@ieee-dot-org.invalid> said:
>>> So I imagine you're worried about the use of the functions >>> residing in the library to which dlopen() gives you access. >>> There are two (potential) penalties I can think of: First, each >>> call must be indirect through a function pointer variable rather >>> than direct to an address known at link time, and second, it may >>> be that the address-space shenanigans with which a particular >>> system supports dynamic linking tend to punish MMU's and TLB's >>> and the like.
>> Yes, sorry I should have been more specific. I was using dlopen() as a >> short hand for describing the whole system rather than just the >> function call itself.
>> What I was really trying to get at was whether the dlopen / dlsym / >> dlclose system added any extra over head on top of what the system >> already uses for dynamic linking? Obviously there is going to be a >> small performance loss due to needing to search for the library and >> possibly deciding which is the correct library to load but other than >> that is there any difference?
>> Or is it the same implementation just with a developer accessable interface?
>> Hopefully that made a bit more sense.
> Less sense, I fear. When you said you were worried about > the performance of dlopen(), I guessed that you were actually > concerned about the (possible) performance penalty of a call > to an .so-resident function, possibly magnified by a thrillion > such calls. But now you reiterate that you're only worried > about the "framework" calls, which will (most likely) impose > their costs only a small number of times. So it seems I was > entirely wrong about what worried you -- and now I'm utterly > without a clue about your concerns.
> Could you see your way to describing your purpose in more > specific terms than "implement a modular system?" And might > you even go so far as to explain what kinds of "performance > overhead" concern you? Or would that spoil the game for the > puzzle solvers?
Haha, sorry it is 5am here :). Tiredness clouds your ability to express yourself it seems.
Okay lets start from the beginning, I am writing a generic server which performs common tasks that all clients require (user authentication, database access etc etc) and am looking for ways for other programmers to implement specific features that only their clients require.
Obviously this poses a problem for me as I have no idea how often their clients are going to need to call those functions or how heavy the burden will be on the server hardware. Ultimately I just want to lay out an interface that a module developer just needs to follow and then my server can pick up said module and then talk to their clients.
Now my concern is as follows:
Given that I have no idea about what the specific implementation of each function will be (I only specifify the interface) and also that I have no idea how often the clients will need to call said function should I be worrying about the performance penalties involved with the approach I have laid out (i.e using the dlopen family of functions to load a dynamic library at runtime)?
On Jul 3, 9:06 pm, Simon Connah <simon.con...@googlemail.com> wrote:
> Obviously this poses a problem for me as I have no idea how often their > clients are going to need to call those functions or how heavy the > burden will be on the server hardware. Ultimately I just want to lay > out an interface that a module developer just needs to follow and then > my server can pick up said module and then talk to their clients.
> Now my concern is as follows:
> Given that I have no idea about what the specific implementation of > each function will be (I only specifify the interface) and also that I > have no idea how often the clients will need to call said function > should I be worrying about the performance penalties involved with the > approach I have laid out (i.e using the dlopen family of functions to > load a dynamic library at runtime)?
It's still not clear. Are you talking about the cost of calling 'dlopen' and 'dlsym' itself? Or are you talking about the cost of calling functions in a dynamic library that's been loaded by 'dlopen' and found with 'dlsym'?
On 2009-07-04 01:21:46 +0100, Simon Connah <simon.con...@googlemail.com> said:
> Hi folks,
> Before I spend awhile trying to implement a modular system using > dlopen(), does anyone know roughly what kind of performance overhead > dlopen() creates on a Mac OS X system (or in fact any BSD based UNIX)?
> The kind of use I need to use it for is fairly performance critical > (mainly latency) and I would hate to spend ages getting a system up and > running only to find out that it is too slow to do what I want.
> Thank you for any input.
For anyone interested in this question (even though I have apparently failed to express myself well) I found the answer to my questions in the Apple document "Mac OS X ABI Dynamic Loader Reference" which should be easy to find with Google.
Simon Connah <simon.con...@googlemail.com> writes:
[...]
> Okay lets start from the beginning, I am writing a generic server > which performs common tasks that all clients require (user > authentication, database access etc etc) and am looking for ways for > other programmers to implement specific features that only their > clients require.
[...]
> Given that I have no idea about what the specific implementation of > each function will be (I only specifify the interface) and also that I > have no idea how often the clients will need to call said function > should I be worrying about the performance penalties involved with the > approach I have laid out (i.e using the dlopen family of functions to > load a dynamic library at runtime)?
Probably not, because the same mechanism is used by the 'real' dynamic linker. The runtime overhead basically consists of an additional indirect function call: The calls in the code point to 'trampoline routines' whichs load a pointer to some other code (either the library function or a function of the dynamic linker which will cause a suitable library to be loaded) and jumps to this address.
|
