Installing Boost.MPI with a Custom Python and a Custom GCC

I was recently chasing a KDD deadline (I still am - so why am I writing this post? I don't know) and was using Boost.MPI's Python bindings for some computation.

At CMU, there's a cluster on which I use EPD-Free1 as my Python distribution which is located in:
~/opt/Python27/bin/python

I have a custom gcc/g++ (version 4.7.0) in /opt/gcc/

  • First, grab boost from here, decompress and cd into it.
  • Now, we need to install boost:
    • ./bootstrap.sh --prefix=~/opt/boost/ --libdir=~/opt/lib --with-libraries=signals,thread,python,mpi --with-python-root=/home/spalakod/opt/Python27/ --with-python-version=2.7
    • The above command generates a file called project-config.jam
    • This file contains some specifics about your python setup (it allows you to type in the path, version etc. in case it gets it wrong).
    • Despite MPI being specified, it got skipped. More on this later.
    • Now, do ./b2
  • Go to the stage/lib in the appropriate directory (for me it was the directory I got by decompressing the original tarball).
  • If you see an mpi.so there, you're good. I didn't so I had to do the following:
    • Create a file called user-config.jam
    • I placed one line in there: using mpi ;
    • Now, run ./b2 --user-config=user-config.jam
  • At this stage I had an mpi.so in stage/lib. Add /path/to/stage/lib to LD_LIBRARY_PATH and PYTHONPATH
  • Now, none of the tests will pass because the tests import mpi using
    import boost.mpi

    and the way it is installed, we will need to use:
    import mpi
  • I have attached an archive of the tests that import MPI the correct (or - if you insist - incorrect) way: https://github.com/shriphani/mpi_python_tests 2
  • Also, put the export statements (for LD_LIBRARY_PATH etc) in ~/.bashrc.
  1. http://www.enthought.com/products/epd_free.php
  2. Includes the MPI Python tests in the libs/mpi/test/python directory except skeletal_content.py

Leave a Reply