About Python and JupyterHub server

This page describes both global and user command-line Python environments, and the Python environment available in the JupyterHub server web application. It does not address user-configured conda environments.

1 System and JupyterHub server Python versions
2 Understanding Python add-on packages
- 2.1 Local/Global package installation conflicts
3 About JupyterHub Kernels
4 The JupyterHub kernels
5 Troubleshooting JupyterHub server issues

System and JupyterHub server Python versions

All JupyterHub servers now provide Python 3.12. R is also available as a separate Jupyterhub kernel. Jupyterhub has now been split into multiple kernels, each targeting a distinct software set.

Users cannot install additional packages in JupyterHub, and any Python packages installed by the user on the command line are not available in the JupyterHub server environment.

This is because JupyterHub server packages are installed in virtual Python environments (venvs) that are only administratively accessible to admin users. Thus JupyterHub packages must be installed by us, so contact us (rctf-support@utexas.edu) if there is a package you'd like to see installed.

There are two versions of Python available on the command line ("system" Python) on BRCF compute servers. In Ubuntu 24.04, these versions and the commands to invoke them are:

2.7 - python2.7 (deprecated, with minimal packages installed)
3.12 - python3, python3.12

Note that Python 3.12 is the default version when you invoke python3.

There are corresponding versions of pip that should be used to install 3rd party packages:

for Python 2 - pip2
for Python 3.12 - pip3, pip3.12

Understanding Python add-on packages

Globally installed Python packages are available to any Python command-line environment of a compatible version. To see which Python packages are installed, along with their versions, use pip2 list or pip3.12 list. Similarly, user-installed packages can be viewed using pip2 list --user, pip3.12 list --user or pip3 list --user.

In addition to the many Python packages available in all the versions, users can their own install command-line-accessible packages using an appropriate version of pip install with the --user option. These user-installed packages are installed by default in the user's Home directory, in a directory with a name like ~/.local/lib/pythonN.N/site-packages, where N.N is the Python version.

Users cannot install additional packages in JupyterHub, and any Python packages installed by the user on the command line are not available in the JupyterHub server environment.

This is because JupyterHub server packages are installed in Python virtual environments (venvs) that are only accessible to admin users. Thus JupyterHub packages must be installed by us, so contact us (rctf-support@utexas.edu) if there is a package you'd like to see installed. (This is different from the R environment, where user-installed packages are automatically visible to the RStudio Server running a compatible R version. See R and RStudio Server R versions.)

Local/Global package installation conflicts

Globally-installed Python add-on packages may be updated during system maintenance. This can sometimes cause problems when users invoke Python tools, or load Python libraries, with many dependencies when some of the dependencies have been updated system-wide, but others have been locally installed and are not at a compatible level. The resulting error messages can be rather obscure, but typically show up after system maintenance has been performed.

To troubleshoot this possibility, move your local Python installation area out of the way. For example, for Python 3.9:

mv ~/.local/lib/python3.9 ~/.local/lib/python3.9.bak

If this produces a different error indicating that one or more locally installed packages are missing, the user can re-install them then see if the problem is resolved. Check the now-named ~/.local/lib/pythonN.N.bak/site-packages directory, where N.N is the Python version being used, to see the packages that were locally installed previously. Even if this resolves the immediate issue, the user may later find that they need to re-install other packages that were previously installed locally.

Finally, if renaming the local Python installation directory does not resolve the issue, it may be an issue with the globally installed packages, so contact us at rctf-support@utexas.edu.

About JupyterHub Kernels

What you need to know in one sentence

Instead of one big Python environment, there are now several smaller, purpose-built kernels, similar to conda environments. Pick the one that matches what you're doing (e.g. Bioinformatics, Machine Learning, etc).

What changed, and why

Our JupyterHub used to offer a single Python kernel, with every Python library crammed into one Python virtual environment. As the toolset grew — general data science, bioinformatics packages, and machine-learning frameworks (PyTorch, TensorFlow, JAX), each with its own and often conflicting version and GPU requirements — that one environment became fragile. Upgrading one package would break another, and GPU builds fought with CPU builds. Hence we split that one giant environment up into several smaller, isolated kernel environments, each tuned for a specific kind of work so its packages are known to work together.

What is a kernel, and how to choose one

Kernels are the engines that runs your notebook's code, each similar to a conda environment. In JupyterHub you pick a kernel when you open a notebook, and you can switch a notebook to a different kernel at any time. Switching kernels does not change your notebook file — only which environment it runs in.

To choose (or switch) a kernel:

New notebook: open the Launcher (blue + button, or File → New Launcher) and click the tile for the kernel you want, e.g. Python 3.12 (Bioinformatics).
Existing notebook: the current kernel is shown in the top-right of the notebook. Click it, or use the menu Kernel → Change Kernel…, and pick from the list.

If a kernel you expect isn't listed, see GPU kernels and the FAQ below.

The JupyterHub kernels

Start here: general & bioinformatics

Python 3.12 (Bioinformatics) — the right choice for most work. A broad data-science + bioinformatics toolkit, including:

Core data science: NumPy, SciPy, pandas, Matplotlib, seaborn, SymPy
Machine learning (classic): scikit-learn, scikit-image
Bioinformatics: Biopython, pysam, Scanpy, AnnData, hgvs, bioservices
Graphs/networks: NetworkX, igraph, leidenalg
Files & data: h5py, lxml, BeautifulSoup, and more

R — R notebooks (via IRkernel). Use this for R-based analysis; it's the same R installation and libraries available at the command line and in RStudio on these systems.

Machine learning on the CPU

Python 3.12 (ML/CPU) — deep-learning frameworks built for CPU: PyTorch (torch, torchvision, torchaudio), TensorFlow, and Hugging Face
Transformers, plus Scanpy/AnnData. Use this for ML/deep learning when you are not using a GPU.

Some older machines that lack the AVX CPU capability will not have TensorFlow installed.

Python 3.12 (ML-JAX/CPU) — the JAX ecosystem (JAX, jaxlib, and Keras with the JAX backend). JAX gets its own kernel because it conflicts with PyTorch/TensorFlow when installed side by side.

Some older machines that lack the AVX CPU capability will not have the ML-JAX/CPU kernel.

Machine learning on a GPU (on GPU nodes only)

GPU kernels appear only when you are working on a machine that has a GPU, and they are automatically matched to that machine's GPU brand. You do not choose CUDA vs ROCm yourself — only the kernels that fit your machine are offered. If you don't see any GPU kernels, you're on a CPU-only node (use a CPU kernel, or move to a GPU node).

On NVIDIA GPU nodes:

Python 3.12 (ML/GPU NVIDIA) — PyTorch / Transformers built for CUDA.
Python 3.12 (ML-JAX/NVIDIA) — JAX built for CUDA.

On AMD GPU nodes:

Python 3.12 (ML/GPU AMD ROCm) — PyTorch / etc. built for AMD ROCm.
Python 3.12 (ML-JAX/ROCm) — JAX built for AMD ROCm.

Note that not all GPU software is available on nodes running the AMD MI-50 GPU cards. MI-100 GPU cards are fully supported.

Quick "which kernel?" guide

I want to…	Use this kernel

I want to…	Use this kernel
General analysis, plots, pandas, most bioinformatics	Python 3.12 (Bioinformatics)
Single-cell (Scanpy/AnnData), genomics utilities	Python 3.12 (Bioinformatics)
Work in R	R
PyTorch / TensorFlow / Transformers, with no GPU	Python 3.12 (ML/CPU)
JAX / Keras (JAX backend), with no GPU	Python 3.12 (ML-JAX/CPU)
PyTorch/etc. on an NVIDIA GPU	Python 3.12 (ML/GPU NVIDIA)
JAX on an NVIDIA GPU	Python 3.12 (ML-JAX/NVIDIA)
PyTorch/etc. on an AMD GPU	Python 3.12 (ML/GPU AMD ROCm)
JAX on an AMD GPU	Python 3.12 (ML-JAX/ROCm)

Coming from the old setup

Your old "Python 3" notebooks will work in the Python 3.12 (Bioinformatics) kernel for most things. If a notebook uses PyTorch, TensorFlow, or JAX, switch it to the matching ML kernel instead.
Your old "R" notebooks will work in the R kernel (same as before).
To move a notebook: open it, then Kernel → Change Kernel… and select the new kernel. Re-run your cells.

Using JupyterHub Kernels from the command line

You can also run code in JupyterHub kernels from the command line. See using Jupyterhub kernels from the CLI.

Common questions

"import torch (or tensorflow / jax) fails in the Bioinformatics kernel."
That's expected — the deep-learning frameworks live in the ML kernels, not the bioinformatics one. Switch kernels (Kernel → Change Kernel…) to ML/CPU, ML-JAX/CPU, or a GPU kernel.

"A package I need isn't installed."
These kernels are shared and curated, so everyone gets a consistent, working set. If the package would be broadly useful, ask us to add it (so it's installed and kept working for everyone).

"Can I pip install into these kernels?"
No — the shared kernels are read-only. Create your own kernel instead (below).

"My notebook lost its variables / my server stopped."
Idle servers are automatically shut down after 1 hour of inactivity to free resources for others. Your notebook files are safe (and autosave), but the running state — variables in memory, loaded data — is lost. Just log back in and re-run your cells. Save often, and for long jobs keep the notebook active.

"I don't see the GPU kernels."
You're probably on a CPU-only node. Use the CPU kernels, or start your session on a GPU node (ask us which hosts have GPUs).

Troubleshooting JupyterHub server issues

Issues can arise involving JupyterHub server (or less commonly, command-line Python). These browser and disk quota issues are similar to those seen for R, so see Troubleshooting R/RStudio server issues. If those suggestions and the ones below fail, submit a help request to our rctf-support@utexas.edu support email.

"Try restarting it from the hub" message

One issue specific to JupyterHub is getting 503: service unavailable. with the message: try restarting it from the hub after logging in with BRCF credentials. Sometime just clicking on the Restart button will work; if not, try logging in again.

JupyterHub Terminal issues

While JupyterHub's Terminal window is a great way to access the Linux command line from your web browser, there are some limitations you should be aware of:

Some keystrokes and key combinations are not recognized the same way as on the real command line.
- This is especially true for Ctrl-, Function-, Option- Alt- key combinations.
- So, for example, editing a file with nano (or vi/vim or emacs) in the JupyterHub Terminal will not work when you try to enter an editor command.
Be very careful looking at large files in the JupyterHub Terminal, since you may not be able to stop output of long text.
- Always pipe data from large files to a pager (e.g. more).

Problems arising from these limitations are the main cause of "Zombie" JupyterHub processes, described below.

Zombie processes

If you are unable to login to JupyterHub on one compute server, first try other compute servers on your pod if available.

Being blocked from logging in to JupyterHub on a particular compute server can be due to "zombie" JupyterHub processes that are still in a "running but not running" state. This is particularly common after using JupyterHub's Terminal window, especially if an error was encountered.

To you can find and kill any of your "Zombie" JupyterHub processes like this, after logging in to a specific compute server:

# Find JupyterHub processes associated with <your_account>
ps -ef | grep jupyter

# The 2nd column in the listing, after your user name, is the process ID (PID).
# You can kill a process like this:
kill -9 <PID>