Installing MongoDB with RocksDB as a storage engine

I honestly don’t really understand most of the words I just wrote in that headline. Nevertheless, when our dba or our software team asks us to install something, install it we do.

This one was fairly tricky, and I’m hoping this post will help some people.

First, some words about our environment. We run Scientific Linux 6.x in our datacenter primarily, and we maintain a shared /usr/local directory (mounted via nfs) across most of the servers. This makes installing software somewhat tricky in two areas: Most new/Big Data/open source/whatever software these days is expecting the latest and greatest code, which tends to show up in Ubuntu. Unfortunately, Ubuntu is not all that well suited to some other stuff we do, and tends to not be as easy to manage in the datacenter. The other trickiness is that we want to avoid the default install location of /usr/local for a lot of software, unless it’s being installed for our HPC cluster.

Aaaaanyway. So that basically meant I had to install a number of packages for this to work, as well as compiling with a non-standard location for gcc, g++, etc. And I also had to install the software itself in a non-standard location. I also had to call a newer version of Python than is standard on SL (and you DON’T want to install over the standard version!)

I started out from this blog post. I’m sure those instructions work for a standard install on Ubuntu 14.xx or later, but they need a little expanding for RHEL/CentOS/Scientific Linux.

Set up the path and install some prereqs:
export PATH=/usr/local/gcc/bin:/usr/local/python-2.7.8/bin:$PATH
yum install libmpc --enablerepo=epel
yum install snappy-devel zlib-devel bzip2-devel

Install the rocksdb libraries:


git clone https://github.com/facebook/rocksdb.git
cd rocksdb
git checkout mongorocks

Change the INSTALL_PATH line in Makefile change to preferred install location (/opt/rocksdb):

INSTALL_PATH ?= /opt/rocksdb

And make/make install

make static_lib
sudo make install

Download scons, which you will need to build mongo. Note that the version of scons available in the standard repos is not new enough, so you get to compile this from source, too:


curl -O http://iweb.dl.sourceforge.net/project/scons/scons/2.3.4/scons-2.3.4.tar.gz

Install and add it to your path:

python setup.py install --prefix=/opt/scons
export PATH=/opt/scons/bin:$PATH

Download the latest version of Mongo that works with rocksdb:


git clone https://github.com/mongodb-partners/mongo.git
cd mongo
git checkout v3.0-mongorocks

Here’s the tricky bit. Scons seems to be rather poorly documented, or at least it’s so flexible that documenting it isn’t helpful. These are the commands I figured out, bearing in mind that our version of GCC (4.9.1) is at /usr/local/gcc and I installed the rocks libraries at /opt/rocksdb:


scons all --cxx=/usr/local/gcc/bin/g++ --cc=/usr/local/gcc/bin/gcc --cpppath=/opt/rocksdb/include --libpath=/opt/rocksdb/lib --rocksdb=1 --prefix=/opt/mongo

scons --cxx=/usr/local/gcc/bin/g++ --cc=/usr/local/gcc/bin/gcc --cpppath=/opt/rocksdb/include --libpath=/opt/rocksdb/lib --rocksdb=1 --prefix=/opt/mongo install

So I’m not certain I need all that jazz for the install line, but it seemed to work, so better safe than sorry. This took about 4 hours to compile in the little VM I was running it in, so I didn’t feel like rebuilding over and over again once I had it actually functional.

Now, in my case, I found that because I couldn’t figure out how to make scons use static libraries and actually work, my dba will need to use this command before running mongo:


export LD_LIBRARY_PATH=/usr/local/gcc/lib64

If anyone can figure out how to make the static libraries with a non-standard location, please include it in the comments. I tried –static, but that caused the compile to throw errors about not being able to find lpthread (??), which it finds just fine without the static libraries.

Advertisements

About kcarlile
Twitter: @overclockdlemon

5 Responses to Installing MongoDB with RocksDB as a storage engine

  1. hey!! super excited to see you playing with rocksdb. 🙂 Sorry the instructions weren’t completely coherent. I recommend joining the mongo-rocks group at https://groups.google.com/forum/#!forum/mongo-rocks. We have been posting debs, rpms, builds there and you can also ask the Rocks team for help with stuff like this. ^_^

  2. kcarlile says:

    No worries! We do run a rather weird environment, so I’m kind of use to banging my head through a lot of stuff like this. I will keep an eye on that group. Plus often times, I find that a sysadmin’s perspective on these newer technologies is a bit different than a developer’s, so I’m going to try to write a few more posts about fun stuff like this (e.g. spark) as well.

  3. Igor Canadi says:

    Nice! Thanks for writing this up. To avoid `export LD_LIBRARY_PATH=/usr/local/gcc/lib64` I think you should be able to set `RPATH=/usr/local/gcc/lib64` before invoking scons. See here: https://github.com/mongodb-partners/mongo/blob/v3.0-mongorocks/SConstruct#L428

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: