POSIDONIA vs MANUAL WAY

In short…

One of the main pros of Posidonia it’s its user-friendly design and that you have one tool for everything related to the daily use of a HPC system. Moreover, you have all your jobs organized in a repository: with a lot of jobs locating the right directory or removing every trace of a job in the HPC system can be source of errors.

On the following a comparison in frequent cases of interactions with a HPC system between how to use Posidonia and how to do it with other tools (I have called it the manual way) .

Send a job to simulate in a HPC system

[ezcol_1half]

POSIDONIA

  1. Fill all the information in the main tab.
  2. Fill the tabs of Job Options and Profiles (only the first time Posidonia is used). You have more details for these two steps in the tutorial.
  3. Press Submit.
  4. Wait for the notification and the results will be downloaded automatically. You can see the progress (if the simulation is designed to do so) with the View Log button in the tab of History.

Note that all this information will be saved in Posidonia and the next time you use the application you only need to modify the existing data.

If you are an administrator of HPC system you can provide to the user an existing profile for the user only to fill its login data and the name of the job.

 

[/ezcol_1half]

[ezcol_1half_end]

MANUAL WAY

  1. Upload all the files you need to the HPC system. Depending on the operating system you work you can use is:
    • WinSCP or FileZilla for Windows.
    • Directly the command scp for UNIX systems.
  2. Write a script adequate to the scheduler system you use. This script can be of the kind of, for the example used in the tutorial:
    #!/bin/bash
    #$ -􀀀M aamor@tsc.uc3m.es
    #$ 􀀀-m e
    #$ 􀀀-cwd
    #$ 􀀀-pe mpi 8
    #$ -􀀀l h_vmem=3.6G
    #$ 􀀀-j y
    #$ 􀀀-S /bin/bash
    #$ 􀀀-V
    echo prueba | /opt/matlab/bin/matlab -nosplash -nojvm -nodisplay
  3. Upload this script (or write it directly in the HPC system).
  4. Access to the HPC system (usually they are Linux systems). Again, depending on the OS you may use:
    • Putty for Windows.
    • The command ssh for UNIX systems.
  5. Once in the HPC system, go to the folder where you have downloaded all your input files and your script to submit the job.
    • Change the permissions to the script you are going to execute. Normally: chmod +x script.sh
    • Submit the job to your scheduler system. For example, in SGE: qsub script.sh
  6. Now your job is submitted. To know about their status, you have to check:
    • The status on the cluster. You can do that in SGE with qstat, which will inform you about the status of your job (if it’s waiting for free space in the cluster, running, aborted…).
    • The progress of the job once it is running. If your application allows to do so, you can write a log with information that you can use to know about the progress of the job. You can do that with cat <logName>
  7. Once your job it’s finished (you can know that by manual checking or getting an email from the queue system) you have to download your results. You should repeat the step 1.
  8. Close all the ssh connections.

Note that you have to be organized in the HPC system. Normally this is a shared resource accessible via command line and this may suppose a big deal for users not expert in the use of these tools.

 

[/ezcol_1half_end]

Check the status of your job

[ezcol_1half]

POSIDONIA

  1. Open Posidonia and connect to the cluster.
  2. If you don’t receive a notification warning you that the job has ended, go to the tab of History.
  3. Check the End Time in the table. You can find Running or the time that the job ended.
  4. Press View Log. A Text Field will be shown with the log you specified in the profile of the job.

Image5

 

[/ezcol_1half]

[ezcol_1half_end]

MANUAL WAY

  1. Access to the HPC system with Putty or ssh.
  2. Do qstat and check by inspect if it’s your job in the list.
  3. Go to the directory where is your file and check the log you have created.
  4. Close the connection.

[/ezcol_1half_end]

Get the output/input of a job

[ezcol_1half]

POSIDONIA

  1. Open Posidonia and connect to the cluster.
  2. Go to the tab of History.
  3. Select the job whose inputs/outputs you want to download and press Get Inputs/Get Outputs respectively.
  4. Specify the directory where you want to get the files.
  5. Wait for the files to be downloaded. If the file is too big, you can see a downloading bar as a notification.

 

[/ezcol_1half]

[ezcol_1half_end]

MANUAL WAY

  1. Access to the HPC system with WinSCP, FileZilla or scp.
  2. Go to the directory in which it’s located your job.
  3. Get all the files you want to get and download them.

 

[/ezcol_1half_end]

Remove a job from the cluster

[ezcol_1half]

POSIDONIA

  1. Open Posidonia and connect to the cluster.
  2. Go to the tab of History.
  3. Select the job you want to delete and press Delete. You will asked if you are sure to do so.

 

[/ezcol_1half]

[ezcol_1half_end]

MANUAL WAY

  1. Check the status of the job (see above about how to do it).
  2. If the job is running, you have to remove it from the scheduler system. You can do it in SGE with qdel and the PID (Process Identifier) of your job.
  3. Delete all the files you have used to simulate the job. You can do it via ssh (Putty or command line depending on your OS) or via WinSCP or FileZilla (in Windows).

 

[/ezcol_1half_end]

Comments are closed.