Conda is an open source package management system and environment management system. Conda as a package manager helps you find and install packages. If you need a package that requires a different version of Python, you do not need to switch to a different environment manager, because conda is also an environment manager. With just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.
As Pinnacle is a shared system, you don’t have the ability to add software to “system space”, thus it must go into your /home space.
Conda environments or “envs” essentially provide “a place for everything, and everything in its place!”
Rather than just dumping everything in one big pile and then trying to ascertain what you’ve installed, it makes much more sense to have groups of similar software together. One can (and should) also create a unique env for the more complex packages with lots of “moving parts” such as most assemblers.
To get started…
$ module load python/anaconda-3.11 $ source /share/apps/bin/conda-3.11.sh #latest version is 3.12, it and later do not use "source /share/apps/bin/conda-3.12.sh" as it is supplied by the module
Additionally, the first time you work with conda, you will need to execute the following three commands which set your software repositories for future searches…
conda config --add channels defaults conda config --add channels bioconda conda config --add channels conda-forge
With everything above resolved, one can now search for relevant software that might be useful for their research…we will actually use an enhanced command in the conda suite called mamba which is significantly faster…
mamba search <package name>
Once you find the package you wish to install, you would use the following example to perform the task:
mamba create -n <EnvName> <package name>
As most users will utilize their new software installs in conjunction with a SLURM job script, below is a quick example of how this process would be implemented:
#SBATCH -J <EnvName> #SBATCH --partition comp06 #SBATCH -o <EnvName>_%j.txt #SBATCH -e <EnvName>_%j.err #SBATCH --mail-type=ALL #SBATCH --mail-user=<userid>@uark.edu #SBATCH --nodes=1 #SBATCH --ntasks-per-node=32 #SBATCH --time=06:00:00 export OMP_NUM_THREADS=32 module load python/anaconda-3.11 source /share/apps/bin/conda-3.11.sh conda activate <EnvName>
The Anaconda group maintain a very helpful “Cheat Sheet” for all things conda. If you have further questions, you can find it here: Conda Cheat Sheet
So…just a few caveats/warnings for using conda: