Hello,
I’m trying to port the build scripts to another Linux platform - i.e. Alpine Linux, which uses the musl-libc instead of glibc.
It seems like your building system is not very platform agnostic - only supporting Ubuntu and CentOS. I’ve seen you are building or downloading third party libs, including the gcc suite - and not using the distribution’s package repository. (At least any traditional configure + make + make install building system would). Which leads to the extend of demanding, that the compiler would be under the path /compiler-wrappers/cc or being called by CLion - thus forcing me to rewrite the entire build system i.e. CMakeList and your custom cmake functions.
Is there any reason you go this way with thirdparty libs, especially with gcc?
Why choose those compiler wrappers in a fixed path / shouldn’t it be enough to demand a specific version number of a compiler?
Is there any other way to build this from sources? Am I missing something?
Is there any way you can assist?
If you dislike the build system yourself, are you open for the community to provide another system?
Thanks a lot,
Hello Paul,
Thank you for your interest in porting YugabyteDB to Alpine Linux. While we do download GCC and Clang compilers in some cases, there are also ways to use natively installed compilers.
I am not sure what you mean by rewriting the entire build system. We should not need a big rewrite of any kind, and I think that making YugabyteDB build on Alpine is perfectly feasible.
The first step is to take GitHub - yugabyte/yugabyte-db-thirdparty: IMPORTANT: please carefully edit the squashed commit's message when merging! You can use the PR description as the starting point. DO NOT use the default squashed commit message. ================ YugabyteDB database third-party dependencies. Creating tar.gz archives for CentOS/Ubuntu/macOS, x86_64/aarch64. and try to build it on Alpine. I’ve tried to do that and it looks like the build is progressing nicely after installing a few required packages.
I’ve created the issue Support building YugabyteDB on Alpine Linux · Issue #8432 · yugabyte/yugabyte-db · GitHub to track the progress.
Also, usually, when adding a new Linux platform for YugabyteDB build, we automate Docker image creation for that platform in this repo: GitHub - yugabyte/build-infra: Infrastructure for building YugabyteDB
I like the lightweight approach of Alpine and the fact that it uses musl natively.
We are certainly open to suggestions for improving our build system, but I don’t think the answer to that is to use the standard “configure” approach which picks up whichever versions of libraries that are present on the host OS. We use well-defined versions of certain dependencies that are encoded in GitHub - yugabyte/yugabyte-db-thirdparty: IMPORTANT: please carefully edit the squashed commit's message when merging! You can use the PR description as the starting point. DO NOT use the default squashed commit message. ================ YugabyteDB database third-party dependencies. Creating tar.gz archives for CentOS/Ubuntu/macOS, x86_64/aarch64. to avoid having to support a build matrix with every version of every possible library. Also, in our test suite, we run AddressSanitizer and ThreadSanitizer extensively, so we have to build most third-party dependencies with the respective instrumentation, hence another reason that we need a third-party dependencies build system. Finally, in C/C++ there does not seem to be a standard dependency manager, although Conan is gaining popularity, so long term maybe we could use something like Conan for building the third-party dependencies in the specific required way. Another option would be to use Rust’s Cargo manager and wrap every dependency into a Rust crate. But currently I think we should just extend the current approach to whichever Linux distributions there is demand for, such as Alpine.
Regards,
Mikhail
Hello Mikhail,
thanks for your reply.
The first step is to take GitHub - yugabyte/yugabyte-db-thirdparty: YugabyteDB third-party
dependencies. Creating tar.gz archives for CentOS/Ubuntu/macOS, x86_64 architecture. and try to build it on Alpine. I’ve tried to do that and it looks like the build is progressing nicely after installing a few required packages.
Some Alpine packages apply patches to upstream code - due to some software not being POSIX conform. I’ll have to check whether that is the case for the build dependencies Yugabyte uses.
Also I’m interested to get a version into the Alpine Repositories. I’ll have to check if your way of handling dependencies is okay with them.
I’ll take a look into that,
Paul
@mbautin
It compiles with build-type common and special arguments for libedit with clang under Alpine Linux 3.13.
gcc version 10 is incompatible with older dependencies, especially with libunwind.
-fcommon throws a linker error (DSO) - see Fix gcc 10 build (or -fno-common) by djwatson · Pull Request #166 · libunwind/libunwind · GitHub
Back to clang:
The linker has some issues:
Unexpected exit code 127 from ldd, file /root/yugabyte-db-thirdparty/installed/common/lib/libfl.so.2.0.0
/lib/ld-musl-x86_64.so.1 (0x7fde39133000)
libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7fde39133000)
Error relocating /root/yugabyte-db-thirdparty/installed/common/lib/libfl.so.2.0.0: yylex: symbol not found
(flex-dev is installed)
Do I need to build uninstrumented, asan and tsan too?
What does uninstrumented stand for?
Hello Paul,
It is quite likely that we will need to upgrade some dependencies, e.g. libunwind. Also we have a bit of logic that selects different dependencies based on the compiler type, so that’s probably where the Clang / GCC difference is coming from.
Our dependencies are grouped into multiple groups:
“common” dependencies – these are never instrumented.
“instrumented” dependencies – these are built in multiple ways (uninstrumented – no instrumentation, as well as ASAN (AddressSanitizer) and TSAN (ThreadSanitizer) instrumentation).
I think if we just want debug/release builds then we don’t have to worry about ASAN/TSAN instrumentation. Also, in the future we should allow using natively installed packages for some of the libraries and not build them ourselves.
Regards,
Mikhail