Commit 2e9bdac
[bp/1.37]Fix hyperscan build when using non-hermetic LLVM toolchain (#43479)
Commit Message:
When using non-hermetic LLVM toolchain the path to the nm and objcopy
tools provided via environment variables to the hyperscan build script
are not correct.
When using non-hermetic toolchain, make C++ toolchain make variables NM
and OBJCOPY will have absolute paths to the host tools, so prefixing
them actually turns correct paths to incorrect onces.
So one change that this PR does is to check if we are using a
non-hermetic toolchain and if so, just pass in $(NM) and $(OBJCOPY)
without modifications.
The other part of this PR modifies the hyperscan slightly to make
build_wrapper.sh fail when one of the commands in the script fails.
The reason for this change is that currently, if this script cannot find
nm and objcopy tools (i.e., when you don't have them installed - which
could happen if you use non-hermetic compiler or, before this PR,
because we added a prefix to the host tool paths that wasn't needed) it
does not fail, but still does not produce quite correct result.
What you will have in the end is a failure during envoy-contrib linking
due to linker failing to find definitions for a bunch of the symbols.
Let me try to explain what is going on there...
Hyperscan, specifically when we use fat runtime option, does something
quite clever and quite terrible - it builds the same library source code
several times enabling different optimizations and links all those
together into the final hyperscan library.
It will build a version of the library assuming that AVX512 instruction
set is available, another version of the library that assumes that
AVC512VBMI instruction set is available and so on. Thus we will have
multiple versions of the same library built with different
optimizations.
Then it takes each of these versions and using nm and objcopy modifies
the names of exported symbols adding a prefix to them to avoid name
collisions later when it will link them together (remember all of them
are built from the same source).
Finally, it links all those versions together into one library and
provides a dispatcher function - this dispatcher function during runtime
detects what instruction sets are actually available on the machine and
calls an appropriate implementation for that instruction set.
The
[build_wrapper.sh](https://github.com/intel/hyperscan/blob/master/cmake/build_wrapper.sh)
script is what takes the compiled object files and renames symbols in it
to avoid name collision.
The build_wrapper.sh is written in such a way that when nm or objcopy
are not available - it does not fail. You can do a simple experiment for
yourself and run the following script:
```
blahblah > /tmp/test.file
if test -s /tmp/test.file
then
blahblah-again
fi
```
Even though neither `blahblah` nor `blahblah-again` command exists (and
you will see an error about it in the output) the overall script status
code when you run it will be 0 (e.g. `echo $?` will return 0).
A similar thing happens in build_wrapper.sh when it cannot find the
objcopy or nm tool - the script does not fail, but it does not produce
the object file we expect it to produce.
When later CMake links `libhs.a` out of the available object files it
links everything that exists and produces `libhs.a`, which miss a bunch
of symbol definitions, but otherwise is still a valid static library.
Because `libhs.a` build didn't fail properly, we proceed to eventually
linking envoy-contrib (that's where hyperscan is used) and at that point
linker discovers that we don't have all the symbol definitions
available.
Why just not use hermetic toolchain?
I'd be happy to, but available published LLVM toolchains are quite
limited and only support a few OS (note when it comes to hyperscan, it's
specifically limited to x86 architecture, so architecture is not the
issue here).
So if you're buildin on Linux other that RedHat or Ubuntu - you're
basically out of luck at the moment unfortunately.
Many companies and communities publish custom built LLVM toolchains, but
when they do, it's still typically done in format of a package for
whatever is the package manager of the platform (e.g., deb, rpm, etc)
and toolchains_llvm that we use to download hermetic LLVM toolchain does
not support those.
Additional Description:
NOTE: I have a PR for the hyperscan library to change their
build_wrapper.sh to be a bit more robust
(intel/hyperscan#455), but judging by the
history of the repository, it appears that they do not accept external
PRs (or at least have not accepted them for a while), so I'm not
hopleful.
NOTE: There are few other problems with using non-hermetic toolchains as
well, but I want to discuss other issues separately from hyperscan.
Hyperscan issue is a bit more straighforward and other issues with
non-hermetic toolchains may require a bit more discussion and
considering various alternatives.
Risk Level: Low
Testing: Manually that envoy-contrib builds successfully and that if nm
and objcopy aren't found, the build will fail early; +ci
Docs Changes: n/a
Release Notes: n/a
Platform Specific Features: n/a
---------
Signed-off-by: Mikhail Krinkin <mkrinkin@microsoft.com>1 parent 3efa15b commit 2e9bdac
2 files changed
Lines changed: 14 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
80 | 80 | | |
81 | 81 | | |
82 | 82 | | |
83 | | - | |
| 83 | + | |
84 | 84 | | |
85 | 85 | | |
86 | 86 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
34 | 34 | | |
35 | 35 | | |
36 | 36 | | |
37 | | - | |
38 | | - | |
39 | | - | |
40 | | - | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
41 | 50 | | |
42 | 51 | | |
43 | 52 | | |
| |||
0 commit comments