Greenfield User Guide
File storage and transfer
There are several distinct file spaces available from Greenfield.
- Your Greenfield home directory is created the first time you log in to Greenfield.
- This is the default space for all your files. You have a home directory named /home/user-name, where user-name is your PSC username. The /home filesystem is shared across Greenfield, so you can access your files in /home from all Greenfield nodes.
- Your home directory is /home/user-name.
- The environment variable $HOME points to your Greenfield home directory.
- There is a quota of 5GB on your home directory.
- The /crucible filesystem is a scratch file system, shared across all Greenfield nodes, providing a fast, temporary file area for data read or written by running jobs. It is much faster than the /home system.
- The /crucible filesystem is not a permanent storage space and it is not backed up. Files that have not been accessed for 21 days are subject to deletion. Files in /crucible will also be deleted if we need to free up space to keep jobs running.
- Be sure to move your /crucible files to your local site or to the Data Supercell as soon as possible after your job completes.
- Your /crucible home directory is /crucible/group-name/user-name where group-name is the 7 character name for the PSC group associated with your grant.
- Find your group name
- If you have more than one grant, you will have a /crucible directory for each grant. BE SURE to use the appropriate directory when working under multiple grants.
- The Data Supercell
- The Data Supercell is a PSC-designed and built system for long-term storage and management of petabyte-scale data.
- Your home directory in the Data Supercell is /arc/users/user-name.
- The Data Supercell is mounted on Greenfield (that is, you can see your Data Supercell files by typing
ls /arc/users/user-namefrom Greenfield) and files can be copied easily between the Data Supercell and Greenfield.
- To transfer files between Greenfield and the Data Supercell, use the PSC high-speed data conduit data.psc.xsede.org with either scp or sftp. See the section on scp and sftp in this User Guide for details.
- See the Data Supercell page for a detailed description of the Data Supercell.
Globus Online, scp and sftp will all be available to transfer files from Blacklight to Greenfield.
Our recommended method for file transfer is Globus Online. If you cannot use Globus Online, but do have access to Globus client software, we recommend globus-url-copy. Otherwise, you can use sftp or scp.
Both Globus Online and globus-url-copy use high-speed data conduits, and are much faster than other file transfer methods.
In order to use Globus Online to transfer files, you must either create a Globus Online account or use InCommon credentials.
Create a Globus Online account
Go to https://www.globus.org/, click "Sign Up" in the upper right, and follow the instructions.
Once you have done that, you can use Globus Online to transfer files to Greenfield.
Use InCommon credentials
If you are affiliated with an InCommon institution you can use your username and password for that institution to authenticate to Globus. You must also register with PSC as an InCommon user before you can transfer files to PSC with your InCommon credentials.
To register with PSC as an InCommon user, follow these steps.
- Go to https://cilogon.org/
- Select your institution from the 'Select an Identity Provider' list
- Click on the 'Log On' button
This will take you to a login page for your institution.
- Enter your username and password for your institution
- Click on the 'Login' button
You will be redirected back to the CILogon Service web page.
- Find and copy your "Certificate Subject" string
Near the top of the CILogon Service web page you will see a field called "Certificate Subject" with a string like /DC=org/DC=cilogon/C=US/O=My Institution/CN=My Name A1234. Copy this string. You'll need it in step 9.
- Log off from the CILogin Service webpage
- Log in to https://dirs.psc.edu/cgi-bin/teragrid/userpage/list.pl with your PSC username and password
This site lists the certificate subjects (DNs) that we have in our PSC database for your PSC account.
- Add your CILogon Certficate Subject (DN) to this list
- Click on the 'Add DN' link at the top left
This will take you to the "Adding DN" page.
- Paste the certificate subject that you copied in step 6 into the DN: field
Make sure there are no extra spaces before or after the pasted string.
- Click on 'Create' to add your new CILogon DN (certificate subject) to the PSC database
You can click on the 'List DNs' link at the top left to confirm that your new DN was added.
- Click on the 'Add DN' link at the top left
Moving files with Globus Online
To start a file transfer, log in to the Globus site.
Globus transfers files between known endpoints. XSEDE users will use one set of endpoints and InCommon users will use another.
On the Globus site, choose 'Transfer Files' and you will be taken to a graphical interface where you will select endpoints, specify source and destination paths and identify the files to be copied.The endpoints and paths to use for Globus Online are given in the following tables. XSEDE users may need to authenticate with their XSEDE Portal username and password when they select an endpoint.
|Endpoints for||XSEDE users||InCommon users|
InCommon users, note the difference between the Greenfield endpoint and the Data Supercell and Blacklight endpoint: dxc-cilogon vs. dsc-cilogon.
|Greenfield home directory||/greenfield/home/user-name|
|Greenfield scratch directory||/crucible/group-name/user-name1|
|Blacklight home directory||/usr/users/#/user-name2|
|Blacklight scratch directory||/brashear/user-name|
1To find your group-name, log in to Greenfield and type
id-Gn. Remember that /crucible is a scratch file system and is only appropriate for temporary storage. Files that you wish to keep should be stored in the Data Supercell.
2Replace '#' with the correct digit. To find the correct digit to use, log in to Blacklight and type
XSEDE users can use GridFTP and the
globus-url-copy command line client to transfer files to the DSC. Once on the DSC, files can moved to /crucible or /home as needed.
Users with files on Blacklight can use
globus-url-copy to move files from Blacklight to the DSC until August 28. Files remaining on Blacklight after that will be lost.
globus-url-copy, first ensure that you have a current user proxy certificate for authentication with enough time on it to complete your transfer by typing
The output will look like:
subject : /C=US/O=National Center for Supercomputing Applications/CN=Joe User issuer : /C=US/O=National Center for Supercomputing Applications/OU=Certificate Authorities/CN=MyProxy identity : /C=US/O=National Center for Supercomputing Applications/CN=Joe User type : end entity credential strength : 2048 bits path : /tmp/x509up_u99999 timeleft : 11:58:33
If the timeleft is not sufficient, or you get an "ERROR: Couldn't find a valid proxy" message, then use
myproxy-logon (or if you have your own long term user certificate,
grid-proxy-init) to obtain a new user proxy certificate, e.g.:
myproxy-logon -l xsede-user-name -t 24
The output will look like:
Enter MyProxy pass phrase: A credential has been received for user xsede-user-name in /tmp/x509up_u99999.
where xsede-user-name is your XSEDE User Portal login name, -t 24 requests a 24-hour certificate, and the MyProxy pass phrase entered is your XSEDE User Portal password.
You can then use
globus-url-copy to transfer files to or from the DSC using these GridFTP server addresses:
|GridFTP server address|
Note: The gsiftp:// URLs are absolute paths to files. This means that you must give the complete path to a file. The paths to use are:
|Blacklight scratch directory||gsiftp://gridftp.blacklight.psc.xsede.org/brashear/user-name|
|Blacklight home directory||gsiftp://gridftp.blacklight.psc.xsede.org/~user-name|
|DSC home directory||gsiftp://gridftp.psc.xsede.org/arc/users/user-name|
where user-name is your PSC user name.
- Copy a file (mydata.out) from your Blacklight home directory to the DSC. Type (note that this command must be typed all on one line):
globus-url-copy -vb gsiftp://gridftp.blacklight.psc.xsede.org/~user-name/mydata.out gsiftp://gridftp.psc.xsede.org/arc/users/user-name/mydata.out
The output will be similar to:
Source: gsiftp://gridftp.blacklight.psc.xsede.org/user-name/ Dest: gsiftp://gridftp.psc.xsede.org/arc/users/user-name/ mydata.out -> mydata.out
The -vb option gives feedback on the transfer. You can omit it if you like.
- List the files in your blacklight $SCRATCH directory. Type:
globus-url-copy -list gsiftp://gridftp.blacklight.psc.xsede.org/brashear/user-name/
The output will look like:
gsiftp://gridftp.blacklight.psc.xsede.org/brashear/user-name/ file1.dat file2.dat file3.dat newdata/ olddata/
scp and sftp
You can transfer files between your local systems and Greenfield using the SSH file transfer clients scp andsftp. Both graphical and command line versions of these clients are available.
Using a graphical scp or sftp app
If you have a graphical scp or sftp client application on your local system, you can use it to transfer files to Greenfield. Connect to greenfield.psc.xsede.org and use your PSC username and password for authentication.
Using scp from the command line
For scripted transfers, or transfers that you want to execute directly from your command-line shell, you can use the SSH scp client.
General scp use
The general format of the scp command is:
scp source-file destination-file
where the full specification for both source-file and destination-file is
The machine-name for Greenfield is greenfield.psc.xsede.org.
Luckily most of the time you can rely on default settings to avoid typing the full specification. For example, you can omit:
- the machine-name of the local machine
- the user-name if it is the same on both source and destination machine
- the path-to-file when it is your home directory on the remote machine
- the path-to-file when it is your current directory on the local machine
- the destination filename unless you want to rename it
Transfer files from blacklight to Greenfield home directories
We recommend that you use scp to transfer files from your blacklight home directory to your Greenfield home directory.
To copy a file from your blacklight home directory to your current Greenfield directory:
- log in to Greenfield
- move to the directory where you want to store the file
scp tg-login1.blacklight.psc.teragrid.org:filename .
Transfer files between Greenfield and the Data Supercell
Use the command line scp client to copy files between Greenfield and the Data Supercell using the PSC high-speed data conduit data.psc.xsede.org.
To copy filename to your Data Supercell home directory, type
scp filename email@example.com:filename
Using sftp from the command line
Transfer files to the Data Supercell
Use the command line sftp client to copy files to and from the Data Supercell using the PSC high-speed data conduit data.psc.xsede.org.
- First authenticate to data.psc.xsede.org
You will be prompted for your PSC password. The first time you connect to data.psc.xsede.org, you will be asked to accept the server's host key.
Your default directory on data.psc.xsede.org is your Data Supercell home directory, /arc/users/user-name.
- Use the put command to copy files into the Data Supercell and the
getcommand to copy files from the Data Supercell.
put filename .
puts filename into your Data Supercell directory.
get filename .
copies filename from the Data Supercell into your current directory.
Transfer files to your Greenfield home directory
You can use the command line sftp client to transfer files to and from Greenfield interactively.
- First authenticate to greenfield.psc.xsede.org using a command like
The first time you connect to greenfield.psc.xsede.org using sftp, you may be prompted to accept the server's host key.
You will then be prompted to enter your PSC password.
- Transfer files using the put command to copy a file from your local system to Greenfield, or get to copy a file from Greenfield to your local system. Your default directory for sftp to greenfield.psc.xsede.org is your Greenfield home directory.
Other sftp commands
At the sftp> prompt, you can use other sftp commands to manage and transfer your files to/from Greenfield. Enter a question mark for a list of available sftp commands.
Examples (entered commands appear in bold):
- What directory am I in on Greenfield?
sftp> pwd Remote working directory: /home/user-name
- What directory am I in on my local system?
sftp> lpwd Local working directory: /Users/JoeUser/Documents
- Change directories on my local system to /Users/JoeUser/Documents/project1:
sftp> lcd /Users/JoeUser/Documents/project1
- Make a new directory called "newdata" under my current directory on Greenfield :
sftp> mkdir newdata
- Exit from this sftp session :