Welcome to the ML Hackathon 2022 Resources Page
All participants have been added the hackathon class account on UCI’s HPC3 cluster. You are not required to use this resource, but it is there for you. Any accounts created for just the hackathon will be closed approximately two weeks after the conclusion of the event.
|The cluster is not being used exclusively by the hackathon. There are others who are sharing this hub, and you should only request the resources that you will really use to accomplish your processing. The defaults are 1 cores and 4GB of memory. You can request up to 4 cores of memory. If you would like to use GPUs, you can request access.|
|Please do not change browsers (e.g., if you use Chrome to access your lab, always use Chrome). Jupyter writes session cookies and using multiple browsers can lead to very odd error conditions.|
This guide will help you get started to launch your own JupyterLab on one of HPC3’s existing computing nodes. The specific JupyterLab instance has access to a subset of software that is available on HPC3 and includes ones that we believe will be quite useful for the hackathon including:
Python 3.8.0 with SciKit Learn, Hugging Face, Pandas and SciPy
2. Log in and start a Server
2.1. Log in
When you see the following screen use your UCNetID and password for access:
2.2. Select Jupyter Server Options
When you have logged in you should see the JupyterHub launch screen as on the figure below.
Select the Hackathon 2022 image in the in the Select a Containerized Notebook Image menu.
You may have multiple accounts to charge the run, select hackathon in account to charge
The partition is set to standard
2.3. Start the Server
Once you made your selections press Start to launch your Lab server. If the cluster is not busy and has free resources, your lab should start within 1 minute. You may see a screen that looks similar to:
3. The Lab Environment
3.1. Access Python
Your JupyterLab server is a Singularity container that is similar to a Docker container. Singularity requires fewer system-level privileges to run and is generally considered more suitable on a shared cluster. The Hackathon container is a subset of the full HPC3 software stack. A standard Jupyter Notebook can be started from within your lab server. You also have access to a Linux command-line terminal, should you need it.
Here’s what your Lab interface should look like the first time you start. The highlighted area is where you can upload/download files from/to your laptop. The circled areas open empty Python or R notebooks.
The is also a Unix terminal application, you can use it for command-line access. This example shows the available software modules and the modules loaded by default.
3.2. Add Python packages with pip
You can use Python from .Console, .Notebook or Terminal simply via clicking on the App image in the Launcher window.
While the lab environment is reasonably complete, you may want additional Python modules installed. This example uses pip to install additional packages. In this case, we’ll install the Python package ttictoc. In your lab, click on the Terminal Icon, and then run the command
$ pip install --user ttictoc
Your output should look something like:
3.3. Access R
The R environment has about 200 R modules installed. R can be run from a jupyter notebook or from the terminal.
3.4. Add R modules
You may need to install additional R modules.
|you must be in a terminal to add new R modules as a user. Once you have added the module, you should be able to utilize it in an R notebook.|
Once you are in the R environment depending on your desired package you may need to use different options for installing:
Most packages can be installed with the regular install command giving it a desired package name, for example to install cicerone package:
Some packages can be installed using source in github. Here is an example of installing SCopeLoomR:
> library('devtools') > devtools::install_github("aertslab/SCopeLoomR")
3.5. Access Pytorch and Tensorflow
If you want to use pytorch or tensorflow, simply use them in a python notebook
4. Upload/Download Files
You have a few choices to upload/download files. Among these are
Use the upload/download capabilities of the built-in file browser
use scp from the Terminal widget in the lab (you must initiate scp from within your lab)
use curl to download files from the web.
5. Shutdown your Lab
Please shutdown your lab environment when you don’t need it. This releases backend computing resources for other active computational needs. Your notebooks and other files should be saved prior to shutting down your lab. Jupyter writes a "state" file for your lab, so that when you relaunch, you will be back where you were prior to shutting down.
|Your labs have a default 24 hour run time. If you don’t manually shutdown your lab, it will be automatically shutdown by the system.|
In the menu, go to File → Hub Control Panel
You will be presented with the window where you can press on Stop My Server