Arkansas High Performace Computing Center [hpcwiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
==== Conda Package Manager====

===Conda===
[[http://https://docs.conda.io/en/latest/|Conda]] is an open source package management system and environment management system. Conda as a package manager helps you find and install packages. If you need a package that requires a different version of Python, you do not need to switch to a different environment manager, because conda is also an environment manager. With just a few commands, you can set up a totally separate environment to run that different version of Python, while continuing to run your usual version of Python in your normal environment.

As Pinnacle is a shared system, you don’t have the ability to add software to “system space”, thus it must go into your /home space.

Conda environments or “envs” essentially provide  “a place for everything, and everything in its place!”

Rather than just dumping everything in one big pile and then trying to ascertain what you’ve installed, it makes much more sense to have groups of similar software together. One can (and should) also create a unique env for the more complex packages with lots of “moving parts” such as most assemblers.

To get started...

<code>

$ module load python/anaconda-3.14

</code>

Additionally, the first time you work with conda, you will need to execute the following three commands which set your software repositories for future searches...

<code>
conda config --add channels defaults
conda config --add channels bioconda
conda config --add channels conda-forge
</code>

With everything above resolved, one can now search for relevant software that might be useful for their research...
(*The following two commands should be done on a compute node as login nodes no longer have reliable access to the outside world.)

<code>
conda search <package name>
</code>

Once you find the package you wish to install, you would use the following example to perform the following task:
<code>
conda create -n <EnvName> <package name>
</code>

One may also use the following command to see what environments are already installed on Pinnacle and activate if suitable:
<code>
conda env list
</code>

As most users will utilize their new software installs in conjunction with a SLURM job script, below is a quick example of how this process would be implemented:

<code>
#SBATCH -J <EnvName>
#SBATCH --partition comp06
#SBATCH -o <EnvName>_%j.txt
#SBATCH -e <EnvName>_%j.err
#SBATCH --mail-type=ALL
#SBATCH --mail-user=<userid>@uark.edu  
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=32
#SBATCH --time=06:00:00

export OMP_NUM_THREADS=32

module load python/anaconda-3.14
conda activate <EnvName>
</code>

The Anaconda group maintain a very helpful "Cheat Sheet" for all things conda. If you have further questions, you can find it here: [[https://docs.google.com/viewer?url=https%3A%2F%2Fdocs.conda.io%2Fprojects%2Fconda%2Fen%2F4.6.0%2F_downloads%2F52a95608c49671267e40c689e0bc00ca%2Fconda-cheatsheet.pdf|Conda Cheat Sheet]]


So…just a few caveats/warnings for using conda:

  * If a “conda install xxx” asks you to downgrade or revert anything…STOP! Don’t do it. In this case, create a unique environment for that specific software tool alone.
  * If it says “requires Python 2.7”….we’d also just create an isolated environment as python 2.x versions are no longer supported by the community.
  * If “conda install xxx” shows just an enormous number of dependencies (Rstudio, BUSCO, GATK, and Qiime are prime examples)…best to Google before installing. These occasionally have “gotcha’s” that require a bit of additional effort. 
  * Check developer page to see what the most current version of the software you need is…on occasion, the latest version on conda will be a couple of releases behind.
  * Though the situation improves daily, not ALL software packages are available thru conda repositories.
  * While conda does make GPU accelerated packages available, it isn't as straightforward getting those to run properly due to the necessity of activating the appropriate NVIDIA run time tools.
Arkansas High Performace Computing Center [hpcwiki]

User Tools

Site Tools

Page Tools
