Data Collections

A community dataset space allows Bridges users from different grants to share data in a common space.  Bridges hosts both public and private datasets, providing rapid access for individuals, collaborations and communities with appropriate protections.

Data collections are stored on pylon5, Bridges' persistent file system.  The space they use counts toward the Bridges storage allocation for the grant hosting them.

If you would like to host a data collection on Bridges, let us know what you need by completing the Community Dataset Request form. If your data collection has security or compliance requirements, please contact compliance@psc.edu.

Request a Community Dataset

 

Publicly available datasets

Some data collections are available to anyone with a Bridges' account.  They include:

 

ImageNet

ImageNet is an image dataset organized according to WordNet hierarchy.  See the ImageNet website for complete information.

Available on Bridges at /pylon5/datasets/community/imagenet

 

Natural Languge Tool Kit Data

NLTK comes with many corpora, toy grammars, trained models, etc. A complete list of the available data is posted at: http://nltk.org/nltk_data/

Available on Bridges at /pylon5/datasets/community/nltk

 

MNIST

Dataset of handwritten digits used to train image processing systems.  

Available on Bridges at /pylon5/datasets/community/mnist

 

Genomics Data

Several genomics datasets are publicly available. 

BLAST
The BLAST databases can be accessed through the environment variable $BLASTDB after loading the BLAST module.
RepBase
Repbase is the most commonly used database of repetitive DNA elements. You must register with RepBase at http://www.girinst.org and send proof of registration to genomics@psc.edu in order to use the Repbase database.
UCSC
The University of California at Santa Clara reference genomes are available at /pylon5/datasets/community/genomics/UCSC.  The collection includes human, mouse and drosophila genomes.
Other genomics datasets
Other available datasets are typically used with a particular genomics package.  These include: 
Barrnap /pylon5/datasets/community/genomics/barrnap 
CheckM /pylon5/datasets/community/genomics/checkm
Dammit
Dammit uniref90 
/pylon5/datasets/community/genomics/dammit
/pylon5/datasets/community/genomics/dammit_uniref90
Homer /pylon5/datasets/community/genomics/homer
Kraken /pylon5/datasets/community/genomics/kraken
Long Ranger /pylon5/datasets/community/genomics/longranger
MetaPhlAn2 /pylon5/datasets/community/genomics/metaphlan2
Phylosift /pylon5/datasets/community/genomics/phylosift
Prokka /pylon5/datasets/community/genomics/prokka

System Status

  • Bridges is up

     

     Scheduled downtime June 26-28

    Read more

Pylon2 Discontinued June 19

The pylon2 filesystem has been discontinued. Use pylon5 to store all your files.

New on Bridges

Filesystems upgrades mean changes in usage for pylon5 and pylon2.
Read more

Omni-Path User Group

The Intel Omni-Path Architecture User Group is open to all interested users of Intel's Omni-Path technology.

More information on OPUG