2. Support tickets

2.1. Introduction 

The resources at RCIC are here to help the UCI research community. There are 1000’s of accounts on our resources, representing disciplines from every corner of UCI.

We handle a lot of help requests and the underline time to understand your problem is at a premium. Please remember:

The better you construct our request the faster we can get to solving it.
Requests don’t need to be elaborate, but how you submit your ticket and what information you include can greatly speed up resolution and save time for you and for us to hone in on the root cause.
If we don’t have relevant information or we cannot reproduce your problem, we cannot fix it!

A pretty good perspective on How to report bugs effectively illustrates some of the reasons for the detail. High-end computing is a team sport!

Please follow the guidelines below when submitting help requests.

2.2. How to submit 

Send email to hpc-support@uci.edu describing your problem, this will automatically generate a ticket in our tracking system.
After you submit a ticket there is an automated email response that it was registered in our system.
This will be followed by a real response from one of our team members who is handling the ticket.

Note

We try to respond by the next business day, but we do not work 24x7.

Only emergency tickets are answered on weekends or during the holidays.

2.3. Good ticket practices 

Before submitting a ticket

Help yourself first. Try your best to see if the problem is a simple one.

Assume you did something wrong. Verify your commands and scripts and try to reproduce the error. Sometimes simple typos or wrong parameters are a cause of a problem.
Examine the contents of output/error files (if your programs have any) before you call for help. Often these files have additional information and even potential solutions.
Read our Frequently Asked Questions. Your question may have already been answered.
Read our user guides. They explain a few guidelines that are specific for our systems. When users follow our guidelines many of the problems are easily avoided, such as:
- file permissions
- quota problems
- Slurm jobs submission errors and OOM killed errors
Do a Google search with your error message. Others may have had the same problem and found a successful solution. Note, you need to understand what in your error is general (applicable to other similar installations) and what is specific to the resources we provide.

While submitting a ticket

DO:

Please try to submit using your @uci.edu email address.
For any problem always provide your UCInetID.
Have a descriptive subject line.
Be specific and provide only relevant essential factual information about a problem (see following sections for details). Simple text-based cut and paste directly into your email is the best info we can use. Usually there are just a few lines.
Be reasonable and polite. We know when something goes wrong, it’s very stressful. It’s stressful for us, too.

DO NOT:

Do not be vague. Statements like it’s slow, I can’t login, or my code doesn’t work give us nothing concrete.
Do not send screenshots unless you are working with a graphics application and the problem can not be described without a screenshot. From screenshots, we can’t cut and paste commands or other info, which slows down our resolution of your problem.
Do not attach multiple files to your ticket. This can overload mail attachment limit in ticketing system. It is much easier for us to see all files if you simply provide full path to them in your storage area.
Do not send multiple ticket requests for the same problem. Simply reply to our response to you, this will keep all email conversation on the same ticket and same email thread.

After receiving a response from us

Sometimes our response is no, this can’t be done or similar. This is a specific resolution of a specific ticket. There is always a reasonable cause for this and we explain it in our response.
When we ask for additional information in our response provide exactly what we ask.
Once your problem is resolved, acknowledge this so we can close the ticket.

2.4. Login problems 

When reporting login problems please include the following:

Where from are you trying to log in, from campus or over the VPN ?
What computer and Operating System (Windows, macOS, Linux) are you connecting from ?
What software and what version are you using to connect ?
Copy and paste into your email:
- exact commands you typed
- exact errors you saw

2.5. Software Problems 

When reporting software problems, please include the following:

Slurm job ID If this error was related to your software use in the Slurm job,
Your working directory (output of pwd command).
If you loaded software modules, what were they (output of module list command)?
Copy and paste into your email:
- exact commands you typed
- exact errors you saw
If the error and output are more than a few lines long, save in separate files and provide full paths to them.

Break very long commands into readable length with the use of the \ (back slash continuation character).

For example, this long line is difficult to read:

make_2d_plots.py -i wetdry_cr/beta_diveuclidean/beta_div_euclideancoords.txt -m wetdry_cr/mapping_files/merged_mapping_data.txt -b 'Elevation' -o wetdry_cr/2dplots/elevation

Same line with added \ is much easier to read and to understand:

make_2d_plots.py \
  -i wetdry_cr/beta_diveuclidean/beta_div_euclideancoords.txt \
  -m wetdry_cr/mapping_files/merged_mapping_data.txt \
  -b 'Elevation' \
  -o wetdry_cr/2dplots/elevation

2.6. Slurm Problems 

When reporting Slurm problems, please include the following:

Slurm job ID.
Your working directory (output of pwd command).
Copy and paste into your email:
- exact commands you typed
- exact errors you saw
Node where you see the issue (for interactive jobs it is output of hostname command).
Full path to the files that you reference (Slurm submit script, output/input/error files, your own scripts, etc).
If you are asking to be added to PI’s CPU or GPU Slurm lab account:
- account type (CPU or GPU)
- your PI UCInetID
- cc your request to your PI so the PI can confirm the access on the same ticket.
  
  We can’t grant any access without PI’s confirmation.

2.7. Storage Problems 

When reporting problems related to DFS or CRSP storage or quotas please include the following:

DFS or CRSP path and group ID you are trying to access. Many groups have multiple paths, we can’t guess from your name which one you need.
How do you access storage: while logged in on HPC3, via web browser, CRSP Desktop, etc.
Copy and paste into your email exact commands you typed and exact errors you saw.
If you are asking to be added to PI’s DFS or CRSP group:
- your PI UCInetID
- cc your request to your PI so the PI can confirm the access on the same ticket.
  
  We can’t grant any access without PI’s confirmation.

2.8. Requesting New software 

Because this is a research environment, we are often asked to add new software. RCIC builds and maintains an extensive collection of domain-specific software. We do our best to balance stability with the availability of latest and greatest.

Some software is very straightforward to build and deploy to the cluster, other software can be extremely challenging and time consuming.

Given realities of time:

We have to prioritize software that affects more than a single researcher or group.
We certainly are NOT here to install software that
- You might use.
- You just want to play with or evaluate.
- Is too new:
  a Beta version.
  
  the latest available version. Just because it is the latest is not a good reason.
- Is too old:
  Anything that is 5 or more years old is not a good candidate.
  
  Is no longer supported by developers (Python 2).

Even with those constraints, we are not shy about taking on complicated, time-consuming installs with many dependencies. Part of our value add to UCI is to handle as much of this as possible. We strive to say “yes” to software requests, but sometimes do have to say “no.”

Before asking us to install:

Check if the software is already installed on the cluster. See list modules for details.

Install it yourself. We encourage users first to build/install the applications in their user area. Please see the guides in User installed software.

Most R, Python and Perl packages can be installed on a per-user basis.

All conda packages and environments must be installed by the users themselves.

You do not need to install Miniconda, Anaconda or Mamba. We provide a few basic versions that can be accessed via modules and used to install your desired packages or environments.

Attention

Please note HPC3 is CentOS-based system. When you attempt to install yourself, and you run across instructions that say Ubuntu or apt get or similar, those are for a different Linux-based OS and won’t work on HPC3.

For security reasons the following is not allowed:

sudo or su access

Docker

However, many docker containers can be reused as Singularity containers.

Submit a Software Ticket

You might not be able to install/compile the software yourself without some additional system-installed software and that’s a good reason to ask us.

IMPORTANT RCIC may say “no” to your software request. We simply do not have the staff time to install every requested software component. We prioritize software that has wide applicability. Applications that you want to “evaluate” are seldom (if ever) installed by RCIC.

In the end, it’s a partnership to get new software added to HPC3. We need good information from you and a willingness to validate the installed software.

If you want to request new software or updated versions of software that are already installed please submit a ticket with the following information:

A brief statement about which lab/domain the software will impact and why this specific version is needed. Don’t write many labs will use it, we need factual usefulness info.

Software name and version.

URL for download/install instructions.

How have you tried to install it yourself, and what were exact commands and exact errors.

Any special configuration options and capabilities that should be enabled or disabled.

A brief statement about a “test” input and expected output so that we can do an initial validation.

2.9. Access Licensed Software 

When requesting an access to the licensed software please include the following:

your UCInetID
your PI UCInetID
cc your request to your PI so that the PI can
- confirm your access (response comes on the same ticket)
- provide a proof of the license purchase.
We can’t grant any access without PI’s confirmation.