Running on a server#

Pixel Clarity is built for data that lives somewhere else: a JupyterHub in the cloud, a cluster’s login node, a lab machine over SSH. This page explains how the pieces fit together, what to check when something is slow, and the few environment variables that change its behavior.

How it works#

When you first call pc.show, the kernel starts a small Node.js process beside it, the viewer process. It reads files, URLs and S3 objects, cuts images into tiles, and works out statistics, histograms, region measurements and table queries. One runs per kernel, and it stops when the kernel does.

The viewer in your browser asks it for what it needs, and draws the tiles on your own graphics card. So the network carries only small tiles and replies (coarse tiles first, then full resolution, always lossless), and once a tile has arrived, panning, zooming and restretching never wait on the network.

Nothing is shared between kernels or users: each kernel’s viewer process, its temporary files and its caches are its own, readable only by you, and removed when the kernel stops.

The connection#

The dot under each viewer shows how the browser reaches the viewer process:

Green, direct: through jupyter-server-proxy, which most hubs have installed, or straight to the process when the browser is on the same machine. Tiles never wait for the kernel, so the view keeps working while a cell runs, and Open in a new tab shows the view in a tab of its own.
Amber, through the kernel: relayed as widget messages through the kernel. This works wherever widgets do, but the viewer waits while a cell is running.

The viewer picks direct when it can. connection="kernel" asks for the kernel route, and connection="direct" for direct only:

pc.show("image.fits", connection="kernel")

If a connection drops (a laptop going to sleep, a flaky network), the viewer reconnects by itself and asks again for anything it was waiting for.

Installing on a hub#

%pip install pixel-clarity

Restart the kernel afterwards. If your hub resets the kernel’s environment when the server restarts, install into your home directory instead, which persists:

%pip install --user pixel-clarity

Pixel Clarity uses its own copy of Node.js, from the nodejs-wheel-binaries package, so nothing needs to be installed on the server itself. To use another Node.js (version 20 or later), set PIXEL_CLARITY_NODE to its path.

Checking what it sees#

pc.diagnostics() reports what the viewer process found: the package version, which Node.js it runs on, the CPUs and memory it may use (a container’s limits included), where it writes temporary files, and the room left there. It’s the first thing to include when reporting a problem. Here is what it says on the machine that builds these pages:

pc.diagnostics()

{'pixel_clarity': '0.9.1',
 'node': '/workspace/pixel-clarity-python/.venv/lib/python3.12/site-packages/nodejs_wheel/bin/node',
 'node_version': 'v24.19.0',
 'cpus': 10,
 'memory_gb': 8.2,
 'pool_workers': 4,
 'backend_port': 37247,
 'jupyterhub_prefix': None,
 'temp_dir': '/tmp/pixel-clarity-h0d90u8o',
 'temp_free_gb': 696.0}

Memory and disk#

The viewer process sizes itself from the memory it may use. Each viewer’s cache of decoded frames is a share of a quarter of that memory, and a viewer nobody has used for five minutes (ten with a direct connection) frees what its data took. It reads the data again when you come back to it, which is quick, as the next section explains.

On a small server with many large viewers open, close the ones you’re done with (v.close()), or let them sleep sooner with PIXEL_CLARITY_IDLE_S.

Caches#

Large local images keep their overview, statistics and zoomed-out tiles in the viewer process’s scratch folder, so opening one again (or waking its viewer) doesn’t read the whole file again. At most PIXEL_CLARITY_CACHE_MB (2000 by default; 0 turns it off) and a quarter of the free disk.
Images from URLs keep a summary (a reduced copy and its statistics) after they are first shown, so another view of the same file reads only what is on screen at full resolution. At most PIXEL_CLARITY_SUMMARY_MB (2000) and a quarter of the free disk. A summary is used only after the URL has been read successfully again.
Data from the kernel (arrays, tables, astropy objects) is written to a temporary folder, once per distinct array, under PIXEL_CLARITY_TMP if set, else the system’s temporary folder.

All of it is removed when the kernel stops.

Environment variables#

Variable	Default	What it does
`PIXEL_CLARITY_NODE`	the bundled Node.js	The Node.js the viewer process runs on (20 or later)
`PIXEL_CLARITY_IDLE_S`	`300`	Seconds before an unused viewer frees its memory (twice that with a direct connection)
`PIXEL_CLARITY_CACHE_MB`	`2000`	Disk cache for large local images; `0` turns it off
`PIXEL_CLARITY_SUMMARY_MB`	`2000`	Disk kept for summaries of images read from URLs
`PIXEL_CLARITY_TMP`	the system’s	Where data from the kernel is written
`PIXEL_CLARITY_DEFAULTS`	`~/.config/pixel-clarity/defaults.json`	The file of saved defaults (Settings and defaults)
`PIXEL_CLARITY_PORT`	a free port	The viewer process’s port: for browser tabs over SSH
`PIXEL_CLARITY_POPUP_WAIT`	on	`0`: a script with browser tabs doesn’t wait for them at its end

Set them before the first viewer opens (the viewer process reads them when it starts), for example in a cell at the top of the notebook:

import os
os.environ["PIXEL_CLARITY_IDLE_S"] = "60"

import pixel_clarity as pc

Sky services#

Right-click › What is this? asks SIMBAD what is near a point, and the Aladin Lite panel shows a sky survey around the view. Both are reached from your browser, not the server, and receive only sky positions (and, for Aladin, the outlines of the image and its regions); image data never leaves the server. online=False turns both off:

pc.defaults["online"] = False