Anaconda is a data science platform which includes Python and R.
Multiple versions of Anaconda are available on Bridges. In addition, environments tailored for AI applications which include Anaconda and other popular AI/Machine Learning/Big Data packages, such as TensorFlow, Theano, Keras, pandas, opencv, scikit-learn, and more are set up for you to use.
To see what versions of Anaconda are available, type
module avail anaconda
Note that anaconda2 modules use python2 and anaconda3 modules use python3.
To see what other modules are needed, what commands are available and how to get additional help type
module help anaconda-version
To use Anaconda, include a command like this in your batch script or interactive session to load the Anaconda module:
module load anaconda-version
Be sure you also load any other modules needed, as listed by the
module help anaconda command.
Environments for AI applications
We have built some environments that include software for AI, Big Data and machine learning applications.
The AI environments are built for Bridges' GPU nodes. Be sure to use one of the GPU partitions. See the Running jobs section of the Bridges User Guide for information on Bridges' partitions and how to choose one to use.
module avail AI/anaconda
to see the AI environments which you can use.
For additional help, type
module help AI/anaconda-version
Note that AI/anaconda2 environments use python2, while AI/anaconda3 environments use python3.
See what the PSC defined AI environment contains
To see the full list of software included in a given environment, first load the module and activate the environment with these commands:
module load AI/anaconda-version
source activate $AI_ENV
To see what is included in $AI_ENV, type
Customize the PSC defined AI environment
If you need software that is not in the pre-built environment, you can create a new environment by cloning the PSC defined one and then customizing it. First load the module and activate the PSC defined environment, as above, then clone it with
conda create --prefix path-to-your-directory/your-new-environment-name --clone $AI_ENV
Then you can activate the new environment and proceed with your customization. See the example below for more detail.
Example sessions are shown below for common uses of anaconda.
Use Tensorflow and python3
Use the following commands:
interact -gpu module load AI/anaconda3-5.1.0_gpu source activate $AI_ENV python import tensorflow
Use Tensorflow and python2
Use the following commands.
interact -gpu module load AI/anaconda2-5.1.0_gpu source activate $AI_ENV python import tensorflow
Customize the PSC defined environment
In this example, the user installs the h5py package in a new environment she is creating. Use the following commands. Note:
conda list command shows what packages are currently installed. Check to see if what you need is already available. The conda list command shows the version number of the installed packages.
conda create command clones $AI_ENV to create a new environment. This can take a long time, so ask for an hour of time with the
Here, the new environment is named clone-env-1, and is stored in the user's pylon5 home directory. The --prefix flag names the full path to the where the environment will be stored. You can name the environment anything you like and store it in any directory you like.
interact -gpu --egress -t 01:00:00 module load AI/anaconda3-5.1.0_gpu source activate $AI_ENV conda list conda create --prefix /pylon5/groupname/username/clone-env-1 --clone $AI_ENV conda activate /pylon5/groupname/username/clone-env-1 conda install h5py
Conda install will install the newest version of the package. If you want to install a version of the package not available in the public installations use the --revision option to the conda install command.
Use the additional packages you installed in a subsequent session
You can use any additional packages you install in the same session in which you install them. To access them in subsequent sessions, follow these steps. In this example, the user created the environment clone-env-1 and installed h5py in it in a previous session.
interact -gpu module load AI/anaconda3-5.1.0_gpu source activate /pylon5/groupname/username/clone-env-1 python import h5py