I gave up on getting Ceph run on ARM 32 bit. It was a huge effort to fix the types, that diverge when switching from 64 to 32 bit. The development of Ceph is simply to fast to cope with. Since the developers decided to drop all tests for 32 bit builds, the needed fixes are too many for a single person to hunt after it.
TODO: Test this all on a virgin armhf system (raspberry, odroid hc1/2/XU4,…) and complete the TODOs for openssl and phantomjs (and the sass-dependency). Maybe with the new master tree, it is not needed to build it outside the ceph repo.
First install prerequisites:
sudo apt install python-pip build-essential libgmp-dev \
libmpfr-dev libmpc-dev reprepro
Install nodejs from nodejs.org
curl -sL https://deb.nodesource.com/setup_11.x | sudo -E bash - sudo apt-get install -y nodejs
sudo npm install -g npm
Then prepare a swap partition (you will need it 😉 )
dd if=/dev/zero of=/<some-hdd-path>/swapfile \
bs=1M count=8192 progress=status
Then we should install some dependencies
sudo apt install libgmp-dev libmpfr-dev libmpc-dev ruby
Now install a new GCC that supports C++17.
tar xfJ gcc-8.2.0.tar.xz
./configure # for armhf
# ./configure --disable-multilib # for x86_64/arm64
Building ceph with do_cmake, building a debian package with make-debs.sh or simply build packages using another compiler than the debian default one (6.3.0) requires you to change the default compiler e.g. to gcc-8.2.0 for the whole system:
sudo update-alternatives --install /usr/bin/cc cc /usr/local/gcc-8.2/bin/gcc-8.2 50
sudo update-alternatives --install /usr/bin/c++ c++ /usr/local/gcc-8.2/bin/g++-8.2 50
Checkout OpenSSL-1.0.2-stable (seems also necessary for armhf), PhantomJS, compile and install it:
git clone email@example.com:openssl/openssl.git
git checkout OpenSSL-1_0_2-stable
# Following seems only necessary on arm
# (or all platforms wihtout precompiled binary)
git clone firstname.lastname@example.org:ariya/phantomjs.git
sudo LD_LIBRARY_PATH=/opt/openssl_build_stable/lib/ \
Add the following to build.py (at L:244, just after PlatformOptions.extend)
phantom_openssl = os.getenv("PHANTOM_OPENSSL_PATH", "")
if phantom_openssl != "":
openssl = os.putenv("OPENSSL_LIBS", "-L" + phantom_openssl + "/lib -lssl -lcrypto")
openssl_include = "-I" + phantom_openssl + "/include"
openssl_lib = "-L" + phantom_openssl + "/lib"
print("Using OpenSSL at %s" % phantom_openssl)
Then install it to /opt
Build and compile Ceph
git clone email@example.com:the78mole/ceph.git
git checkout wip-32-bit-arm-fixes
./do_cmake_arm32.sh # for armhf
# ./do_cmake.sh # for x86_64/amd64 or arm64
# if it gets really slow due to swapping, break an do make -j1
# or use the scheduler-script from link below
Here you can find a rudimentary (but working) script that suspends compilers processes based on total compilers memory consumption. Running it through ‚watch‘-tool you can start e.g. 8 tasks and when memory limit is reached, it will suspend (kill -TSPT) the youngest tasks in sense of user space runtime.
cd .. # Back to ceph base dir
./make-debs-arm32.sh # for armhf
# ./make-debs.sh # fox x86_64/amd64 or arm64
If you encounter problems with setuptools (Exception –> TypeError: unsupported operand type(s) for -= ‚Retry‘ and ‚int‘) try to get a more recent version of python pip with the following commands and rerun make-debs-arm32.sh.
apt-get remove python-pip python3-pip
If I forgot anything to make it work, feel free to write some comment…