Pittsburgh Supercomputing Center 

Advancing the state-of-the-art in high-performance computing,
communications and data analytics.

Instructions for running MDCS R2012a on Blacklight

There are three things you must do before you can use MDCS on Blacklight:

  • You must install the MATLAB Parallel Computing Toolbox (PCT) on your local machine
  • You must request access to MDCS by submitting this form. PSC has 32 MATLAB licenses, but special permission is required to use them.
  • You must set your PSC password. Even if you access PSC resources through the XSEDE User Portal, MDCS requires your PSC password to be set.

After you have successfully installed PCT on your local machine, you need to define the correct environment to run on blacklight.  Defining your environment to run MCDS on blacklight consists of three basic tasks. First, you must download the utilties required to do the data communications and data movement between your local machine and blacklight. Second, you must define the data storage on your machine and on blacklight to handle the data generated by your application. Third, you must tie together your local and blacklight MATLAB environments together. The exact steps to do this depend on the version of MATLAB you have. The instructions below are for version R2012a.

Download utilities for data communication

  • Launch MATLAB on your local machine.
  • Enter the MATLAB userpath command. The output of this command will look similar to
        C:\Users\username\Documents\MATLAB
    
    where 'username' is your username on your local system.
  • Using the link download the file
        2012a zip file
    
    to your local machine. This will create a directory named
        2012a zip file
    on your local machine. Unzip the files in the directory if necessary. Then copy all the unzipped files into the directory you identified with the userpath command.

Define data storage locations

  • Login to blacklight and issue the command
     
       mkdir -p /brashear/username/MDCSDataLocation/R2012a
    
    where you substitute your username for 'username'.
  • On your local machine in the directory identified by the userpath command create a subdirectory named MDCSDataLocation. Within this subdirectory create another subdirectory named R2012a.

Tie together local and blacklight MATLAB environments

To tie your local and blacklight MATLAB environments together you must edit your MATLAB profile on your local machine.

  • On your local machine launch MATLAB. On the MATLAB menu bar click Parallel and select Manage Cluster Profile from the pull-down menu. The Cluster Profile Manager window will appear.
  • On the Cluster Profile Manager window click on the Input button. The Import profiles from file window will appear.
  • On the Import profiles from file window the file blacklight_12.settings should appear. Click on this file and then hit the Open button. This will load the profile setting definitions in this file into your MATLAB profile.
  • To make these settings your default profile hit the Set as Default button in your Cluster Profile Manager window.
  • You must edit this default profile. Click on the Edit button in your Cluster Profile Manager window. This will allow you to edit your profile settings. You must make two types of changes. First, you must change the JobStorageLocation to be the complete path to the data storage location you created on your local machine. The path will look similar to
        C:\Users\username\Documents\MATLAB\MDCSDataLocation\R2012a
    
    when you substitute for 'username' your username on your local system. The second type of change you must make is to the two entries in the Submit Functions section of your profile. These entries must have the full path to the data storage directory you created on blacklight. The path will look like
    /brashear/username/MDCSDataLocation/R2012a
    
    where for 'username' you substitute your username on blacklight. Hit the Done button after you have made these changes to make them permanent.
  • You can now issue MATLAB commands on your local machine that will launch parallel MATLAB jobs on blacklight.

Running jobs

Once you have set up your MATLAB environment on your local machine and blacklight you are ready to launch jobs MATLAB jobs on blacklight. There are several steps you must follow to launch MATLAB jobs on blacklight.

First, you must set parameters for your blacklight job. You must request an amount of walltime for your job and specify a blacklight queue. To set these parameters start MATLAB on you local machine. The MATLAB commands

    ClusterInfo.setQueueName('mdcs')
    ClusterInfo.setWallTime('00:30:00')

request 30 minutes of walltime in the mdcs queue.

Next, you must request MATLAB licenses. MATLAB jobs will not run if there are not enough free MATLAB licenses available. There are 32 licenses available on blacklight, but they are shared across all MATLAB users. Your job will not start until there are enough free licenses available. The number of licenses you request must be one more than the number of MATLAB workers you want your job to use. The command

    ClusterInfo.setUserDefinedOptions('-l licenses=MATLAB_Distrib_Comp_Engine:9')

requests 9 licenses, which is how many you would request if you want 8 MATLAB workers. The maximum number of licenses you can request is 32.

If your job asks for 30 minutes or less of walltime and if it asks for 16 or fewer licenses you can request to use the debug queue instead of the mdcs queue. Your queue request would then look like

    ClusterInfo.setQueueName('debug')

For short jobs, turnaround in the debug queue is usually better than in the mdcs queue. However, the debug queue should only be used when you are debugging your MATLAB program and not for production runs.

The MATLAB command

    ClusterInfo.state

shows the current setting of your MATLAB job parameters. The command

    ClusterInfo.clear

clears out your current settings. After using this command you need to set new settings before you submit your next MATLAB job.

Now you are ready to launch a MATLAB job. The MATLAB command

    batch('scriptname', 'matlabpool', 8)

submits a job from your local machine to blacklight. The job will run on blacklight. When it finishes you will be able to retrieve your job's output back to your local machine.

For the string 'scriptname' substitute the name of your script. The default location for your script is your userpath directory: the directory the MATLAB userpath command returns. If your script file is in a different location, you must use the full path to the file in place of the string scriptname. The batch command argument 'matlabpool' indicates that the job should run on blacklight. The argument 8 requests 8 MATLAB workers. This value should be one less than the number of MATLAB licenses you requested. The MATLAB command

    help batch

gives information about other arguments to the batch command. Which additional arguments you need, if any, varies based on the your job.

For example, the batch command

    batch('@lotsOfPauses', 1, {2,10}, 'matlabpool', 8)

will run the MATLAB script lotsOfPauses on blacklight. This script is one of the files you downloaded when you downloaded the 2012a zip file. The '@' in the script names indicates that this is a MATLAB function. Since lotsOfPauses is a function, the second argument to the batch command indicates the number of outputs of the function and the third argument is the list of input arguments to the function. This script is useful for testing if you have properly set up your link between your local machine and blacklight. It also provides useful timing data on your connection between your local machine and blacklight.

When you have submitted your job with the batch command it will be transferred to blacklight, where it will be placed in the appropriate queue. When your requested resources, including MATLAB licenses are available, your job wil run. Unlike when you run a MATLAB job on your local machine, your blacklight job is run in batch not interactive mode. You can end your MATLAB session once you issue the batch command. You can start a new session later to check on the status of your MATLAB jobs or to retrieve the outputs from your jobs if they have finished.

To monitor the status of your blacklight MATLAB jobs launch a MATLAB session. On the MATLAB menu bar click Parallel and select Manage Jobs from the pull-down menu. You will see a list of your jobs and their status. If a job finished normally you can right-click on the job to get a menu to fetch the job's output, the job's diary and other information about the job. If a job finished with an error its entry in your list of jobs will have a red circle with an x inside it next to the entry. If you right-click on the job you will see the error messages for the failed job. Once you have retrieved the output from a job, it is cleared from your job list. If your job is still waiting in the queue to run or is running its status in your job list will reflect its current state. To kill a queued or running job you right-click on it and select Cancel from the menu that appears.

If your MATLAB program reads from data files you should copy them to a directory on blacklight. You can use the storage location directory you created when you set up your MATLAB environment. Then, in your MATLAB batch command, use the AttachedFiles option and give it the value of the directory in which you put your data files.