Package and Environment Managers
A brief explanation of the various ways to install and manage software on an HPC system. Constraints are
- HPC users can only install software in their dedicated areas
- HPC staff can install software in shared system areas
- capability for multiple versions of software is required
Basic system software is mostly installed from RPM packages by yum. If it were an Ubuntu system, that would be deb packages by apt. These install into shared system areas, most often the default command path /usr/bin. These aren't useful for user installation, except for rare RPM packages that are “relocatable” so can be installed in user areas. You can ask HPC staff to install an RPM package in system areas, but it won't always be feasible. One common issue is two different RPM versions can't be installed into the same directory because the command names will overlap.
This, like the great majority of HPC centers, uses modules software, originally environment modules and later rewritten and improved as the compatible lmod. Its main purpose is to allow simultaneous installation of different versions of software into different directories with an easy-to-use interface. See environment_modules. lmod does not in itself install software. In most cases it just sets environment variables PATH and LDLIBRARYPATH so that the system will reference the right version of software that is installed some other way. It is possible to create your own modulefiles for your own installed software. There are a couple of complications if you do so, please contact HPC-Support for more information.
We also support Anaconda, which was originally an easy way to install Python, but has developed into an environment manager and software installer for Python, R, Perl, and compiled software, as long as someone has packaged that software in an Anaconda format. It is often the simplest way to install software that has complicated dependencies. To install something with conda, we suggest that you use our modules to begin with an Anaconda point release (3.7,3.8,3.9,3.10, etc.). Our Anaconda modules are essentially the same result as running the miniconda installer, that is a basic version of Python at the same point release. Usually for new software, you would want to run the latest anaconda module, unless you have a dependency for an older python version. You can upgrade/downgrade python inside a conda point release, but we do not suggest doing so, as dependency issues are common.
These commands will put you in the latest anaconda base environment:
module load gcc/11.2.1 mkl/20.0.2 python/3.10-anaconda source /share/apps/bin/conda-3.10.sh
Here the source command replaces the editing that conda will try to do to your .bashrc. That editing is likely to cause issues if you run multiple versions of anaconda at different times, or run other programs that don't use Anaconda python at all. After running the above commands, you can create an environment:
conda create -n my-env conda activate my-env
Then for Python, you
