HACC-mlcluster Jupyter Notebook Flow

This is a tutorial on how to connect to the interactive Docker container that you requested. In the interactive development environment you are able to both run python scripts and test code using Jupyter labs in addition to having sudo access to debug. If during the development process you run into situations where you need additional packages for the Batch mode please reach out to the system administrators.



Note: For the SSH-server to work properly the “authorized_key” file must have the permission 600. This can be a problem if the “authorized_key” file was created manually. If so, the user must “chmod 600 authorized_key” before launching the script. Please double check before launching the interactive job.

Once your SSH keys are configured, you can establish a connection to the Docker container using the provided IP addresses and ports. The connection is initiated via SSH, with a specific port forwarding setup to access Jupyter Notebook.

Execute the following command from your terminal, replacing <port>, <yourusername>, and <ipcluster> with the appropriate values provided to you:

ssh -L 8888:localhost:8888 -p <port> <yourusername>@<ipcluster>

This command establishes an SSH connection to the Docker container and forwards the container’s port 8888 (where Jupyter Notebook runs) to your local machine’s port 8888. This setup allows you to access Jupyter Notebook as if it were running locally.

After connecting to the Docker container, it’s essential to initialize the environment variables necessary for your development work. You can do this by sourcing the rocmEnv file located in your home directory: This step ensures that all necessary environment variables for ROCm (Radeon Open Compute) or any other GPU-related tools are correctly set up for your session.

source /home/rocmEnv

When you first connect to Jupyter Labs it may be slow, it can sometimes take upto 5 mins to fully stabilize so please be patient.