Discussion:
Useless `DT_NEEDED` tags after Garbage Collection
Christoph Conrads
2018-10-18 18:38:37 UTC
Permalink
My goal is to create a shared library without dead code linking only
against other needed libraries.

The situation is as follows:

I want to make my code, let us call it "Alpha", available as a shared
library. Alpha depends on "Beta". The source code for Beta is
available, I must link statically against Beta, and I cannot modify Beta
but I can compile and link as needed. Beta depends on the shared library
Gamma. Let us call this library `libgamma.so`.

Alpha has an API exposed via functions with default visibility. All of
Beta's functions have hidden visibility. To minimize the size of the
Alpha shared library (let us call it "libalpha.so"), I pass
* `-fPIC -ffunction-sections` to the compiler when compiling Alpha and
Beta,
* `--gc-sections` to the linker when linking `libalpha.so`.

This has the desired effect of removing all dead code from
`libalpha.so`. As a side-effect, linking against `libgamma.so` is not
necessary anymore but ld will still add a `DT_NEEDED` tag for
`libgamma.so`, even in the presence of `--as-needed`.
Is there a way to have ld detect and ignore unused `DT_NEEDED` tags
due to garbage collection?
This can be done manually by omitting `libgamma.so` from the command
line but this is error prone, tedious, and might not work with the next
release of Beta.


I concluded that ld must be computing `DT_NEEDED` tags before performing
garbage collection. A relocatable object file seemed to solve my problem:
* Compile an Alpha static library, pass `-fPIC -ffunction-sections` to
the compiler, ensure all API symbols have default visibility, hidden
otherwise.
* Compile a Beta static library, pass `-fPIC -ffunction-sections` to the
compiler, ensure all symbols have hidden visibility.
* Create a relocatable object from the Alpha static library and the Beta
static library with `--gc-sections --gc-keep-exported`. All dead code
is removed in this step.
* Create a shared library from the relocatable object, pass
`--as-needed` to the linker.

Unfortunately, the creation of the relocatable object does not work as I
had planned because one has to pass all of Alpha's symbols with default
visibility with the linker option `--entry`. Luckily I misspelt the
symbol names once. Lo and behold, ld simply ignored the non-existent
symbol and created the relocatable object containing all symbols with
default visibility and without dead code just as I had envisioned it.
Is there a way to create a relocatable object with `--gc-sections
--gc-keep-exported` and without `--entry` on the command line?
Assuming, a relocatable object with `--gc-sections
--gc-keep-exported` must also be built using `--entry`, can I rely on
the behavior of ld if the symbol name passed to `--entry` does not
exist? That is, ld will still create a relocatable object?
Alan Modra
2018-10-19 05:56:38 UTC
Permalink
Post by Christoph Conrads
My goal is to create a shared library without dead code linking only
against other needed libraries.
I want to make my code, let us call it "Alpha", available as a shared
library. Alpha depends on "Beta". The source code for Beta is
available, I must link statically against Beta, and I cannot modify Beta
but I can compile and link as needed. Beta depends on the shared library
Gamma. Let us call this library `libgamma.so`.
Alpha has an API exposed via functions with default visibility. All of
Beta's functions have hidden visibility. To minimize the size of the
Alpha shared library (let us call it "libalpha.so"), I pass
* `-fPIC -ffunction-sections` to the compiler when compiling Alpha and
Beta,
* `--gc-sections` to the linker when linking `libalpha.so`.
This has the desired effect of removing all dead code from
`libalpha.so`. As a side-effect, linking against `libgamma.so` is not
necessary anymore but ld will still add a `DT_NEEDED` tag for
`libgamma.so`, even in the presence of `--as-needed`.
Is there a way to have ld detect and ignore unused `DT_NEEDED` tags
due to garbage collection?
No.
Post by Christoph Conrads
This can be done manually by omitting `libgamma.so` from the command
line but this is error prone, tedious, and might not work with the next
release of Beta.
I concluded that ld must be computing `DT_NEEDED` tags before performing
garbage collection.
Correct. It's done when loading symbols for the library in question.
Post by Christoph Conrads
* Compile an Alpha static library, pass `-fPIC -ffunction-sections` to
the compiler, ensure all API symbols have default visibility, hidden
otherwise.
* Compile a Beta static library, pass `-fPIC -ffunction-sections` to the
compiler, ensure all symbols have hidden visibility.
* Create a relocatable object from the Alpha static library and the Beta
static library with `--gc-sections --gc-keep-exported`. All dead code
is removed in this step.
* Create a shared library from the relocatable object, pass
`--as-needed` to the linker.
Unfortunately, the creation of the relocatable object does not work as I
had planned because one has to pass all of Alpha's symbols with default
visibility with the linker option `--entry`.
Really? And multiple --entry options made a difference?

Did you mean --undefined (or -u) rather than --entry? Even so, I'm
surprised that you would need to use the -u trick on a linker
supporting --gc-keep-exported. What binutils version and target are
you using?
Post by Christoph Conrads
Luckily I misspelt the
symbol names once. Lo and behold, ld simply ignored the non-existent
symbol and created the relocatable object containing all symbols with
default visibility and without dead code just as I had envisioned it.
Is there a way to create a relocatable object with `--gc-sections
--gc-keep-exported` and without `--entry` on the command line?
Assuming, a relocatable object with `--gc-sections
--gc-keep-exported` must also be built using `--entry`, can I rely on
the behavior of ld if the symbol name passed to `--entry` does not
exist? That is, ld will still create a relocatable object?
--
Alan Modra
Australia Development Lab, IBM
Christoph Conrads
2018-10-19 10:22:26 UTC
Permalink
Hi,
Post by Alan Modra
Post by Christoph Conrads
* Compile an Alpha static library, pass `-fPIC -ffunction-sections` to
the compiler, ensure all API symbols have default visibility, hidden
otherwise.
* Compile a Beta static library, pass `-fPIC -ffunction-sections` to the
compiler, ensure all symbols have hidden visibility.
* Create a relocatable object from the Alpha static library and the Beta
static library with `--gc-sections --gc-keep-exported`. All dead code
is removed in this step.
* Create a shared library from the relocatable object, pass
`--as-needed` to the linker.
Unfortunately, the creation of the relocatable object does not work as I
had planned because one has to pass all of Alpha's symbols with default
visibility with the linker option `--entry`.
I made a simple demo before posting. It is available here:
https://gitlab.com/christoph-conrads/shared-libraries-and-gnu-ld-garbage-collection
Post by Alan Modra
Really? And multiple --entry options made a difference?
No, a single `--entry` or `--undefined` option sufficed.
Post by Alan Modra
Did you mean --undefined (or -u) rather than --entry?
After re-reading the man page, I see that I should have used `--undefined` here.
Post by Alan Modra
Even so, I'm
surprised that you would need to use the -u trick on a linker
supporting --gc-keep-exported. What binutils version and target are
you using?
$ gcc -dumpmachine
x86_64-pc-linux-gnu
$ ld --version
GNU ld (Gentoo 2.30 p5) 2.30.0
ld \
--print-gc-sections \
--gc-sections --gc-keep-exported \
--relocatable alpha.o libbeta.a -o reloc.o
ld: Removing unused section '.text.beta_fn_unused' in file 'libbeta.a(beta.o)'
ld: gc-sections requires either an entry or an undefined symbol
Alan Modra
2018-10-19 12:58:40 UTC
Permalink
[snip]
Post by Alan Modra
ld: gc-sections requires either an entry or an undefined symbol
OK. It used to be that -r --gc-sections needed a root symbol to start
marking kept sections, but that isn't really necessary with
--gc-keep-exported since all defined default visibility or protected
visibility symbols are roots.

I'll commit a patch to fix that, but you'll need to continue
specifying at least one --entry or --undefined option with older
linkers. Any --entry will work and most likely continue to do so.
One good option is "--entry=0".
--
Alan Modra
Australia Development Lab, IBM
Christoph Conrads
2018-10-19 15:21:33 UTC
Permalink
Hi Alan,
Post by Alan Modra
[snip]
Post by Alan Modra
ld: gc-sections requires either an entry or an undefined symbol
OK. It used to be that -r --gc-sections needed a root symbol to start
marking kept sections, but that isn't really necessary with
--gc-keep-exported since all defined default visibility or protected
visibility symbols are roots.
I'll commit a patch to fix that, but you'll need to continue
specifying at least one --entry or --undefined option with older
linkers. Any --entry will work and most likely continue to do so.
One good option is "--entry=0".
Thank you for the quick help.

Christoph Conrads

Loading...