Reference

Written from the docstrings in src/udeep/ by tools/build_reference_page.py: 98 public functions and classes in 17 Python modules, and the 33 names the Julia package exports, from julia/UDeepCore/src/. The profile says what a figure file means; this page says what the code offers.

udeep

What import udeep gives. The matplotlib interface is its own module, udeep.mpl, imported as import udeep.mpl as hp.

caption, cmap, Declaration, hdr_view, HdrView, knee_at_percentile, Mapping, MAPS, max_data_step, pq_encode, read_gainmap_jpeg, read_metadata, read_png_data, read_png_pq, scale, to_xmp, tone, write_gainmap_jpeg, write_png_pq, write_png_sdr

udeep.mpl

src/udeep/mpl.py

matplotlib entry point: udeep.mpl.imshow / colorbar / savefig_hdr.

import udeep.mpl as hp
img = hp.imshow(ax, data, mode="extend", knee=240, stretch={"kind": "linear", "lo": -160, "hi": 1200},
                unit="HU", knee_rule="preset; top of the soft-tissue window", noise=("constant", 10.0))
hp.colorbar(img, ax=ax, label="HU")
hp.savefig_hdr(fig, "figure.jpg")                 # one file: the ordinary figure + a gain map
print(hp.caption(fig))                            # a caption generated from the declared mapping
hp.inline(fig)                                    # in a notebook: that file, shown in the cell's output

matplotlib only ever draws the fair SDR plot, so the figure, savefig PNG/PDF and the gain-map base are all the ordinary figure. savefig_hdr renders the figure four times: as it is; with each HDR layer scaled into range; and with the layers all black and then all white, which shows the pixels that an HDR layer alone makes. Only those pixels are lifted, and only where the scaled render is brighter. Axes, text and background stay at SDR white, and so does anything drawn over an image, such as a legend, with the image under it. The HDR render goes through 8-bit Agg, so gain above white is quantised to about headroom/255 of SDR white.

The headroom is fixed when the image is made (imshow(headroom=...)): the mapping, the colours and the declaration all follow from it, so a file cannot be saved at another.

classPrecisionWarning

The file's levels are too coarse for the data's stated uncertainty (profile 7.2.1).

functiontheme

theme(name: str='dark', ground: str | None=None, ink: str | None=None) -> dict

matplotlib settings for a figure's furniture on a light or a dark page: the background, and the text, axes and ticks drawn on it. Use it around the code that builds the figure:

with plt.rc_context(hp.theme("dark")):
    fig, ax = plt.subplots()
    ...
    hp.savefig_hdr(fig, "figure_dark.jpg")

ground and ink replace the theme's background and text colours, to match a particular page.

A theme changes nothing about the data: the image, its colours, the mapping and everything the file declares about them are the same in every theme, and so is the data's digest. Only the SDR figure differs, so each theme is its own file with its own SDR digest (profile 8.7). Text and lines are never lifted above white in either theme.

functionlayers

layers(fig) -> list

The figure's HDR layers whose images it still holds. A layer whose image was removed, or whose axes or figure was cleared, is dropped from the figure's list: its data is no longer part of what the file shows.

functionimshow

imshow(
    ax,
    data,
    mode: str,
    knee,
    stretch: dict,
    headroom: float = 2.0,
    cmap: str = 'fire',
    mask=None,
    unit: str | None = None,
    knee_rule: str | None = None,
    noise: tuple = ('unknown', None),
    proportional: bool = False,
    source: str | None = None,
    unit_code: str | None = None,
    float32: bool = False,
    **imshow_kw,
)

Like ax.imshow, but for UDeep's two modes; returns the AxesImage.

unit, knee_rule and noise are what the file will declare (profile section 8): the data's unit ("1" if it has none); why the knee is where it is, as "physical; ...", "statistical; ..." or "preset; ..."; and the noise model with its parameter, such as ("constant", 10.0). proportional=True declares a linear stretch from zero as proportional (data that is itself light, profile 6.6); its top must then be knee x headroom. source says where the data is kept (a DOI, URL or file name); the file also records a digest of data, so a reader holding the data can check it is the same. unit_code is the unit in machine-readable form (a UCUM code such as "K"), beside the text in unit. headroom is fixed here: savefig_hdr writes the file at this headroom and no other. float32=True keeps the image's two renderings in half the memory (udeep.view): for very large images. The colours then differ from the default's by about one part in 10 million, so the default is float64.

functioncolorbar

colorbar(img, ax, label: str='', size: str='4%', pad: float=0.08, n: int=256)

An HDR colour bar for an udeep image: data-unit ticks and the value rendered at SDR white.

functiondeclaration

declaration(fig, headroom: float | None=None) -> Declaration

What the figure declares about its data and mapping (profile section 8), for a gain-map JPEG.

A file carries one declaration, so every udeep image in the figure must share one mapping. headroom need not be given; if it is, it must be the headroom the images were made at.

functioncaption

caption(fig, headroom: float | None=None) -> str

A caption paragraph generated from the figure's declaration (profile 8.2). headroom is as for declaration: it need not be given, and must be the images' own if it is.

functionsavefig_hdr

savefig_hdr(
    fig,
    path,
    headroom: float | None = None,
    quality: int = 95,
    tight: bool = True,
    pad_inches: float = 0.1,
    gain_quality: int = 100,
    metadata: bool = True,
) -> None

Write the figure as a gain-map JPEG: the ordinary figure, plus gain on the HDR images only.

The file carries the profile's declared metadata (unit, mapping, knee and its rule, precision) unless metadata=False; without it the file does not conform to the profile.

The file's headroom is the one the images were made at (udeep.mpl.imshow's headroom=). headroom need not be given; a value that differs from the images' is refused, because the file would not be what it declares.

tight=True lays the figure out like savefig(bbox_inches="tight") -- expanding to take in labels outside the canvas -- and renders every pass that way, so the gain stays aligned with the base. The SDR figure this file's digest refers to is then fig.savefig(..., bbox_inches="tight", pad_inches=pad_inches), not a plain fig.savefig(...).

Before the file is written the gain map is decoded again and lowered wherever the figure would read above its brightest value (udeep.gainmap.write_gainmap_jpeg), which takes several coding passes on a coloured figure. gain_quality below 100 is unreliable with coloured maps (fire, ice), with this writer or any other: where one SDR channel is small each gain code is a large step. The passes may then not converge, and ValueError is raised and no file written rather than a file that reads over its headroom. Leave it at 100 unless the figure is grey.

classHdrFigure

A figure as its gain-map JPEG, for a notebook to show (inline): data is the file, byte for byte what savefig_hdr writes, and caption the caption generated from the declaration the file carries.

Jupyter shows it as an image in the cell's output. The file itself is what the page is given, so a browser that reads gain maps on an HDR display shows the images above white, and any other shows the ordinary figure.

save(self, path) -> None

functioninline

inline(
    fig,
    headroom: float | None = None,
    quality: int = 95,
    tight: bool = True,
    pad_inches: float = 0.1,
    caption: bool = True,
    close: bool = True,
) -> HdrFigure

The figure as savefig_hdr writes it, to show in a notebook: end a cell with hp.inline(fig).

The other arguments are savefig_hdr's. With caption the caption generated from the file's declaration is shown under it. close closes the figure afterwards, which stops the notebook from also showing it as an ordinary image at the end of the cell; pass close=False to go on drawing in it. Call .save(path) on the result to keep the file.

functionvalidate

validate(fig, path, tight: bool=True, pad_inches: float=0.1, sdr=None)

Check a file written by savefig_hdr against the figure: every conformance point of the profile, including the pixels (udeep.validate.validate checks what the file alone can show). Pass the same tight and pad_inches the file was saved with. sdr is the SDR figure file to check as the figure of record; without it the figure's own SDR rendering is used.

udeep.view

src/udeep/view.py

The two presets of the UDeep Figure Profile (v0.3, section 13).

highlight -- a high, meaningful knee; sparse extremes go above SDR white. Isolated pixels above
             the knee are held at white so noise cannot sparkle: a derived mask (udeep.masks).
extend    -- the knee at the top of the main range; a second range of real structure goes above
             white. No area filter: thin real structure (e.g. bone cortex) must lift too.

Both keep the plot below the knee identical to the SDR plot clipped at the same white (the fair baseline), never lift masked pixels (saturation, blooming, clipping), and take the knee in data units. A missing knee falls back to a percentile and is flagged as exploratory.

The view holds three arrays the size of the image: the two renderings and the light. The ordinary full-range SDR plot is not kept (it was a third rendering, and nothing used it): it is `view.mapping.colours( view.mapping.position(data))` when wanted.

The renderings are float64 unless float32=True is asked for, which keeps them in half the memory (32 bytes a pixel held for the view, against 56). The mapping is worked out in float64 either way and each colour is rounded once as it is stored, so a float32 rendering is within one part in 10 million of the float64 one, far inside an 8-bit file's step; but it is not the same number, so the default stays float64.

The mapping is done by the Rust core wherever it is found (udeep.core), on every processor core, to within about one part in 10^12 of the reference. The reference is the NumPy code, which does it where the core is not found or has been turned off (udeep.core.enable(False)).

The mapping itself is a Mapping record (udeep.mapping) and is pointwise. This module applies it to an image in three steps: derive any mask the preset calls for, map every pixel, hold the masked ones.

classHdrView

What hdr_view returns: the two renderings of an image, and what was done to make them.

sdr and hdr are height x width x 3, linear RGB (BT.709 primaries) in multiples of SDR white; y is each pixel's light before colour. mapping is the record that defines every colour here, colourbar samples it at 256 positions (the data value and the light at each), and warnings holds what a reader of the figure should be told, in words.

functionhdr_view

hdr_view(
    data,
    mode: str,
    knee: float | None,
    stretch: dict,
    headroom: float = 2.0,
    cmap: str = 'fire',
    mask=None,
    proportional: bool = False,
    float32: bool = False,
) -> HdrView

The fair SDR plot and the HDR plot of data, under one of the two presets.

mode is "highlight" or "extend". knee is the data value shown at SDR white, in data units; None takes a percentile of the finite data (the 99.5th for highlight, the 95th for extend) and says so in the view's warnings, since that is exploratory and not a meaningful threshold. stretch is the scale in its authoring form, {"kind", "lo", "hi", optional "width", "offset"}, headroom the light at the top of the range in multiples of SDR white, and cmap "grey" or a colour map's name, alone or with a version ("fire/1"): all as Mapping.from_stretch takes them, as it does proportional.

mask is true at the pixels that must never be lifted (saturated, clipped, invalid): they are held at their SDR light. float32=True keeps the renderings in half the memory.

ValueError for a mode that is not one of the two, and for a mapping that Mapping refuses.

udeep.mapping

src/udeep/mapping.py

The mapping from data to light and colour, as one record of parameters (design decision D3).

value x  ->  position t in [0, 1]  ->  light y, in multiples of SDR white  ->  linear RGB, same unit

A Mapping holds exactly what a file declares about that chain, under the same names as the udeep: metadata properties. Everything that computes with the mapping takes this record, so a figure can be rebuilt from its own metadata, and nothing can depend on a parameter the file does not record. The same record is meant to become the C struct of the native library and the uniform block of the shader.

Each method is a pure function of single values: no pixel looks at a neighbour (D1, D2). All are float64 and written against the Array API standard, so they run on any array library that follows it (D8, D22).

classMapping

The parameters of one mapping from data to light and colour, under the names a file declares them by.

A record is checked as it is made, and refused (ValueError, with the reason) if the curve could not draw it as declared: among other things the range must be finite and rising, the knee must lie inside it and not within KNEE_MIN of either end unless it is exactly at the top, a log stretch must start above zero, and a proportional mapping must be the straight line light = value / knee. Nothing is adjusted to make a record pass.

Build one from the authoring form with from_stretch; a file's declaration gives its own (udeep.metadata.Declaration.mapping).

from_stretch(
    cls,
    stretch: dict,
    knee: float,
    headroom: float,
    cmap: str = 'grey',
    proportional: bool = False,
) -> Mapping

kind

exit_slope

knee_position

position(self, x)

light_at(self, t, headroom: float | None=None)

light(self, x, headroom: float | None=None)

colours(self, y, dtype=np.float64)

table

value_at(self, t)

value(self, y)

udeep.cmap

src/udeep/cmap.py

y (multiples of SDR white) -> linear sRGB colour.

A colour map is a TABLE, and the table is its definition (design decision D9): two segments, light 0 to 1 and 1 to the reference headroom, each of NODES evenly spaced nodes holding linear RGB (BT.709 primaries, multiples of white, float32), with linear interpolation between nodes. cmap looks colours up in it.

cmap(y, name, headroom)    colours, from the table
table(name, headroom)      the Table: shipped for the registered headrooms (2, 4, 8, 16), built otherwise
search(y, name, headroom)  the generator that tables are made from (tools/make_tables.py); not for runtime use

A map's definition has a version, and a file names both ("fire/3"). Every version stays registered, so that a file made with an older one can always be read; new figures use the current one (VERSION). Each function takes version= for an older one.

How the generator builds a map, in Jzazbz: for y <= 1 the colour's lightness (Jz) is that of a grey of luminance y * top, where top is below 1, so SDR data never passes through white. (Its luminance is close to y * top but not equal to it: at y = 1, 0.85 for fire against a top of 0.8, and 0.78 for ice against 0.7.) For y > 1 luminance rises from top to headroom while chroma tapers to 0: extremes become white-hot, brighter than SDR white. Chroma is then reduced (fixed-step bisection) until the colour fits [0, 1] (y <= 1) or [0, headroom] (y > 1). Below white the map does not depend on the headroom; above white it does, which is why a table belongs to a (map, headroom) pair.

Version 1 measured chroma from az = bz = 0, which in Jzazbz is close to grey and not grey: with the chroma tapered to nothing, the top of a map was a faint tint that had then to be clipped to the headroom, so the last step of a map could fall in lightness. Its ice map also ran along the edge of the gamut near 15 % of white, where lightness dipped for red-green colour-vision deficiencies. Version 2 measures chroma from the (az, bz) of the grey of the same lightness, so that no chroma is exactly grey and the top is exactly white; and its ice map asks for less chroma at its second node (0.13 against 0.16), which keeps it off that edge. In version 2 lightness rises at every node of every shipped table, for normal vision and for each simulated deficiency (udeep.vision; tested).

Version 3 is version 2 with every value below FLOOR (1e-7 of white) set to zero. Where version 2 holds a channel at the edge of the gamut it leaves the small remainder of the arithmetic, a few hundred-millionths of white, and that remainder depends on which processor-specific routines NumPy and OpenBLAS choose: the generator did not give the same table on every machine (project/performance-scope.md, 4g). With the remainders removed it does, so a version-3 table can be checked by making it again. No colour moves by more than FLOOR.

functionsearch

search(y, name: str='fire', headroom: float=4.0, version: int=VERSION) -> np.ndarray

The generator: the exact colour for each y, by a per-value search. Slow; tables are made from it.

classTable

A colour map as data: below covers light 0..1, above covers 1..headroom; each is nodes x 3, float32.

lookup(self, y) -> np.ndarray

to_bytes(self) -> bytes

digest

to_text(self) -> str

from_text(cls, name: str, headroom: float, text: str) -> Table

functionidentifier

identifier(name: str, version: int=VERSION) -> str

How a file names a map: the name and the version of its definition, such as "fire/3".

functionparse

parse(text: str) -> tuple[str, int] | None

The (name, version) a file's map name stands for, or None if it is not a registered map.

functiongenerate

generate(name: str, headroom: float, nodes: int=NODES, version: int=VERSION) -> Table

Build a table from the generator. For headroom 1 there is nothing above white: that segment is flat.

functiontable

table(name: str, headroom: float, version: int=VERSION) -> Table

The table for a map at a reference headroom: the shipped one for a registered headroom, otherwise one built now at EMBED_NODES nodes, which the file must then carry. Headroom 1 (an SDR figure) uses the lower segment of a shipped table.

functionis_registered

is_registered(headroom: float) -> bool

Whether a table is shipped for this reference headroom: 2, 4, 8 or 16, and 1 (an SDR figure, which uses the lower segment of a shipped table). At any other headroom a file must carry its table.

functioncmap

cmap(y, name: str='fire', headroom: float=4.0, version: int=VERSION) -> np.ndarray

Colours for light y, from the map's table: linear RGB in multiples of SDR white, float64.

udeep.gainmap

src/udeep/gainmap.py

Gain-map JPEG (Ultra HDR v1 layout, the model ISO 21496-1 standardises) for scientific plots.

The primary image is the ordinary SDR plot; a gain map records how much brighter each pixel is in the HDR plot. A display applies the gain in proportion to its own headroom (HDRCapacityMin..Max), so one file shows the SDR plot on SDR screens, part of the gain on a 2x panel, all of it at headroom and beyond -- a static, declared per-display fit.

Spec: developer.android.com/media/platform/hdr-image-format. Deliberate choices for plots: the gain map is full resolution and 3-channel (the spec recommends 1/4 size, which blurs sharp data into halos, and a luminance-only map cannot carry the colour change above white); JPEGs are 4:4:4 so thin lines and text keep their colour. Layout: primary [SOI, APP1 XMP container, APP2 MPF, ...] followed by the gain-map JPEG [SOI, APP1 XMP hdrgm, ...].

functionwrite_gainmap_jpeg

write_gainmap_jpeg(
    sdr,
    hdr,
    path,
    headroom: float,
    quality: int = 95,
    gain_quality: int = 100,
    description: bytes = b'',
    iso: bool = True,
) -> None

sdr: linear RGB in [0, 1] (the SDR plot); hdr: linear RGB in multiples of SDR white.

A pixel the figure does not lift (hdr no greater than sdr in all three channels) is stored with a gain of exactly 1, so GainMapMin is 0 and the gain map is flat wherever the figure is not lifted. No pixel is darkened: a channel that is lower in hdr than in the stored SDR plot keeps the SDR value.

Before the file is written its gain map is decoded again and the figure a reader would show is rebuilt; where that is brighter than the largest value in hdr the gain is lowered and the map coded again, until nothing is over. The cost is time (several coding passes on a coloured figure) and a top of the range that reads a little low. If it cannot be done the file is not written and ValueError is raised: a file is never written that reads above the brightest value it was given, on this module's reader.

The gain map is stored at gain_quality (default 100): its errors are exponentiated, so JPEG ringing at sharp gain edges (a colour bar beside a white background) otherwise pushes values past the headroom. Below 100 it is unreliable on coloured maps (fire, ice: where one SDR channel is small the gain range is wide and each code a large step, with this writer or any other): the passes may not converge, or gain may spread onto unlifted pixels, and the call then raises ValueError; a file that is written may still fail the validator's checks on its pixels. Leave it at 100 unless the figure is grey.

description is an extra rdf:Description for the primary image's XMP packet: the profile's declared metadata, from udeep.metadata.description. iso=True also writes the gain-map metadata in the binary form of ISO 21496-1, beside the Ultra HDR XMP, for decoders that read only that.

functionread_iso_metadata

read_iso_metadata(path) -> dict | None

The gain map's ISO 21496-1 binary metadata, as numbers (log2 for the headrooms, min and max), or None if the file has none. Reads the single-channel, separate-denominator layout that this module writes. Files written before 2026-10-10 have flags 0 (bit 6 not set); they are read the same.

functionread_gainmap_jpeg

read_gainmap_jpeg(path, display_boost: float) -> np.ndarray

Linear RGB (multiples of SDR white) as a display with display_boost headroom shows it.

Read by the Rust core when it is in use (udeep.core), with the JPEG decoder its writer checks files against; otherwise, and for any file the core does not take, by the code below with Pillow's decoder. The two decoders differ by a code or two at some pixels, which is a few per cent of a lifted pixel where one SDR channel is small: a file reads exactly as its writer left it only on the decoder its writer used.

A file that is not a gain-map JPEG, or is damaged, is refused with ValueError, whatever the decoder made of it: files come from anyone, and a caller should have one thing to catch.

udeep.export

src/udeep/export.py

Write HDR (PQ-tagged 16-bit PNG) and SDR (8-bit sRGB PNG) images from linear sRGB.

Input rgb is relative linear sRGB (1.0 = SDR white), shape (H, W, 3), row 0 at the top. The HDR file is BT.2020 primaries, PQ transfer (BT.2100), signalled by a PNG cICP chunk (PNG 3rd edition), which Chromium-based browsers display as HDR.

functionpq_encode

pq_encode(nits) -> np.ndarray

Luminance in cd/m^2 -> PQ signal in [0, 1] (SMPTE ST 2084). Luminance outside 0..10000 is clipped to it.

functionpq_decode

pq_decode(signal) -> np.ndarray

PQ signal in [0, 1] -> luminance in cd/m^2 (SMPTE ST 2084): the inverse of pq_encode.

functionpq16_codes

pq16_codes(rgb, sdr_white_nits: float=REF_WHITE_NITS) -> np.ndarray

The 16-bit codes a PQ PNG stores for linear RGB in multiples of SDR white: to BT.2020, to PQ with white at sdr_white_nits, rounded to 16 bits. A NaN is stored as 0.

functionwrite_png_pq

write_png_pq(rgb, path, sdr_white_nits: float=REF_WHITE_NITS, xmp: bytes | None=None) -> None

A 16-bit PQ PNG of rgb (height x width x 3, linear, multiples of SDR white). PQ is absolute, so 1.0 is written at a luminance: sdr_white_nits, by default BT.2408's reference white of 203 cd/m^2. xmp is an XMP packet (udeep.metadata.to_xmp), stored in an iTXt chunk.

A file that carries a udeep declaration is written with white at 203 cd/m^2 (profile 9.1.1), since the file has nowhere to say what its white was: another value with such a packet is refused (ValueError). Matching a particular screen's SDR white is for the viewer, when it shows the file (profile section 12). A file without a declaration may be written at any white, for one's own screen, and is then read back with the same value.

Written by the Rust core when it is in use (udeep.core): the codes are the same as this function's own, and the compressed bytes differ. read_png_pq reads either.

functionread_png_pq

read_png_pq(path, sdr_white_nits: float=REF_WHITE_NITS) -> np.ndarray

A 16-bit PQ PNG as light: float64, height x width x 3, linear RGB (BT.709 primaries) in multiples of the white written at sdr_white_nits, which is 203 cd/m^2 in any file that carries a declaration (profile 9.1.1). It is what write_png_pq was given, to within the file's 16-bit steps. udeep.read_png_data turns it back into data, whatever the colour map.

ValueError for a PNG of another kind: it must be 16 bits a channel, RGB, not interlaced, and declared BT.2020 PQ in a cICP chunk. Read by the Rust core when it is in use, otherwise by the code here.

functionsrgb_oetf

srgb_oetf(v) -> np.ndarray

Linear light, 1.0 = SDR white -> the sRGB signal in [0, 1] (IEC 61966-2-1), as an 8-bit SDR file stores it once multiplied by 255 and rounded. Light outside 0..1 is clipped to it; NaN stays NaN.

functionwrite_png_sdr

write_png_sdr(rgb, path) -> None

An 8-bit sRGB PNG of rgb (height x width x 3, linear, 1.0 = SDR white). A channel above 1 is clipped to it, and a NaN is stored as 0.

udeep.metadata

src/udeep/metadata.py

The declared metadata of the UDeep Figure Profile (v0.3, section 8): what a file must say about how its data became light, so that the mapping can be inverted and checked.

Declaration      the properties, validated
to_xmp           an XMP packet (the `udeep` namespace) for a JPEG APP1 segment, a PNG iTXt chunk or SVG metadata
read_metadata    the Declaration carried by a file, or None
missing_properties   the properties section 8 requires that a packet does not carry
float_text       how a number is written: the shortest text that reads back to the same float64
declared_light   the declared mapping, data -> light, computed from the declaration alone
caption          a caption sentence generated from the declaration (section 8.2)
max_data_step    the widest range of data values that one stored code stands for (section 7.2.3)
rounding_to_noise  the data criterion (section 7.2.1): that rounding against the stated uncertainty
digest_data, digest_pixels   the digests that tie a file to its data and to its SDR figure of record
digest_sdr       SdrDigest of a fair SDR rendering given as light, as the writers store it in 8 bits
values_pq16, read_png_data   a 16-bit PQ PNG's light, and such a file, read back as data values

The namespace URI resolves to a page listing these properties; it is provisional until version 1.0 (profile section 10, open issue 1).

classDeclaration

What a file declares about how its data became light (profile section 8), checked as it is made.

The fields are the udeep: metadata properties under Python names: range_low is written udeep:RangeLow. Those that define the colours are the fields of a Mapping, which the mapping property gives. The rest say what the data is (data_unit, data_unit_code), why the knee is where it is (knee_rule: "physical", "statistical", "preset" or "percentile", then "; " and a description), how uncertain the data is (noise_model, noise_parameter), how finely the file stores it (max_data_step, rounding_to_noise), which pixels were held (masked, derived_mask, derived_masked), which preset made the figure (preset), whether the file is a data record or a rendering (record), and what ties it to its data and to its SDR figure (data_digest, data_source, sdr_digest). version and image_state are fixed by the profile.

ValueError, with the reason, for a declaration the profile does not allow.

mapping

functionfloat_text

float_text(value: float) -> str

A number as the metadata writes it: the shortest text that reads back to the same float64.

It is Python's repr of the float, without the ".0" that ends a whole number: 4.0 is "4", 0.1 is "0.1", 1700000002.3456 is "1700000002.3456", 1e16 is "1e+16" and 0.00001 is "1e-05". Other languages reproduce it from the shortest digits that read back: with the decimal point after digit number p, the text is plain digits when -4 < p <= 16; otherwise it is the first digit, a point and the rest if there are more, then "e", the sign of the exponent p - 1 and at least two digits of it.

functionto_xmp

to_xmp(d: Declaration) -> bytes

A complete XMP packet holding the declaration as udeep: attributes.

functiondigest_data

digest_data(a) -> str

A digest of an array's values and shape, whatever type it is stored as: "sha256:" and 64 hex digits.

The array is taken as float64, little-endian, in row order; every NaN counts as the same value and -0 as 0. The shape goes in first, so the same numbers in another shape give another digest.

Only real numbers that float64 holds exactly are accepted: floats, and integers no larger than 2 ** 53. Anything else (complex numbers, text, booleans, dates, wider floats) raises ValueError, because the cast would give two different arrays one digest. A masked array is digested as its stored values, hidden ones included; the mask is not part of the digest.

functiondigest_pixels

digest_pixels(rgb8) -> str

A digest of an 8-bit RGB image's pixels (H x W x 3), independent of how the file stores them.

It is taken over the text "udeep-rgb8-1;<height>x<width>;" and then the bytes, row by row. The array must already be unsigned 8-bit: any other type raises ValueError, since casting it would wrap or truncate.

functiondigest_sdr

digest_sdr(sdr) -> str

The SdrDigest (profile 8.6) of a fair SDR rendering given as light: height x width x 3, linear, 0 to 1, as HdrView.sdr holds it. It is taken over the 8-bit sRGB codes the writers store for it (the base of a gain-map JPEG before compression, and udeep.write_png_sdr's file), with a NaN stored as 0.

A file with no SDR base of its own, such as a 16-bit PQ PNG, must declare it (profile 9.1.2); the gain-map JPEG of the same figure then declares the same value.

functiondescription

description(d: Declaration) -> bytes

The declaration as one rdf:Description element, to sit beside others in an existing packet.

The element is read back before it is returned, and ValueError is raised if that gives another declaration: a file must not state a mapping other than the one that made its pixels (profile 6.4).

functionparse

parse(xmp: bytes) -> Declaration | None

The declaration in an XMP packet, or None if it holds none. An invalid declaration raises ValueError: a value out of range or not a number, or a property the record cannot be made without.

A property the packet does not carry takes the record's default, so the result does not show what the file itself states: missing_properties does.

functionmissing_properties

missing_properties(xmp: bytes) -> list[str] | None

The properties profile 8.1 requires of every file that this packet does not carry, by name and in the record's order; None if the packet holds no declaration at all.

functionpacket

packet(data: bytes) -> bytes | None

The XMP packet in a file's bytes that uses the udeep namespace, or None.

functionread_metadata

read_metadata(path) -> Declaration | None

The declaration a file carries (gain-map JPEG, PNG, SVG or a bare packet), or None if it has none. ValueError if it carries one that cannot be read.

functiondeclared_light

declared_light(d: Declaration, x) -> np.ndarray

The declared mapping itself: data values -> light in multiples of SDR white, computed from the declaration alone (profile 6.4). Values outside the declared range are clipped to it.

functioncaption

caption(d: Declaration) -> str

One caption paragraph generated from the declaration (profile 8.2 and 9.4.2).

functiondata_steps

data_steps(
    mapping: Mapping,
    scaled_by: float | None = None,
    samples: int = 1 << 16,
) -> tuple[np.ndarray, np.ndarray]

The stored levels across the declared range: where each run of data values sharing one code starts, and how wide it is, both in data units.

It follows the 8-bit path of udeep.mpl.savefig_hdr: below the knee the code is the 8-bit sRGB colour of the SDR figure; above it, the 8-bit sRGB colour of the HDR figure scaled by scaled_by (default: the headroom). JPEG compression adds error on top of this and is not included. The range is sampled at samples points, so the result is good to about two parts in samples of the declared range.

functiondata_steps_pq16

data_steps_pq16(
    mapping: Mapping,
    sdr_white_nits: float = REF_WHITE_NITS,
    samples: int = 1 << 20,
) -> tuple[np.ndarray, np.ndarray]

The stored levels of a 16-bit PQ PNG (udeep.export.write_png_pq) across the declared range: where each run of data values sharing one code starts, and how wide it is, both in data units.

Unlike the 8-bit path of data_steps there is one code for each colour and no split at the knee. Such a file has tens of thousands of levels, so the range is sampled at about a million points by default. Give the result to rounding_to_noise as steps to see whether the file may be a data record (profile 7.3.1).

functionvalues_pq16

values_pq16(
    mapping: Mapping,
    light,
    sdr_white_nits: float = REF_WHITE_NITS,
    samples: int = 1 << 20,
) -> np.ndarray

The data values that a 16-bit PQ PNG's light stands for (light as udeep.read_png_pq gives it): the declared mapping turned back, through the file's own codes, whatever the colour map.

Each stored level is a run of data values that share one code in all three channels (data_steps_pq16). A pixel is given the middle of the run whose code it holds, and so comes back to within half that run's width, which is at most half of MaxDataStep. The ends of the runs are found to within a millionth of the sampling's spacing. A writer that rounds a sum of three products in a different order can leave a rare code one step from the reference's (udeep.core); a pixel whose code is in no run takes the nearest run that is a step away in each channel at most, and a pixel with none so near is not a colour of this mapping and is NaN.

ValueError when two separate runs share one code: that code would not say which data value it stands for. It has not happened with any map tried (grey, fire and ice, linear over 0 to 4095 at a 4x headroom).

functionread_png_data

read_png_data(path, samples: int=1 << 20) -> tuple[np.ndarray, Declaration]

A 16-bit PQ PNG read back as data: the data values its pixels stand for under the declaration it carries (values_pq16), height x width, and that declaration. A value outside the declared range was stored at the range's end, and a masked pixel holds its fair SDR light instead of its value (profile 6.2a), so neither comes back as it was. The file need not declare itself a data record; if it does not, the values are good only to within its levels, which may be wider than the data's uncertainty.

ValueError if the file is not a 16-bit PQ PNG, carries no declaration, or its colour map is one this reader has no table for.

functionmax_data_step

max_data_step(mapping: Mapping, scaled_by: float | None=None, samples: int=1 << 16) -> float

The widest run of data values that share one stored code, in data units (profile 7.2.3: MaxDataStep).

functionrounding_to_noise

rounding_to_noise(
    mapping: Mapping,
    noise: tuple,
    scaled_by: float | None = None,
    steps: tuple | None = None,
) -> dict | None

The data criterion (profile 7.2.1) for a file's stored levels, or None when the noise is unknown.

For a noise model, each level's rounding error (its width / sqrt(12)) is divided by the data's standard deviation at that level; the limit is K = 0.1. For a declared tolerance, the largest rounding error (half the width) is divided by the tolerance; the limit is 1.

The noise is taken at the point of each level where it is smallest, so a level meets the criterion only if all of it does. Poisson and relative noise grow away from zero, so that point is the one nearest zero, min(max(0, start), start + width): the end of the level nearer zero, or zero itself when the level contains it. At zero the noise is zero and the ratio infinite, so a range that reaches zero cannot meet the criterion (7.2.1). Constant noise and a tolerance are the same everywhere, and are reported at the middle of the level.

Returns the worst ratio (the first, if several levels share it), the data value where it was taken, the limit, whether the whole declared range meets it, and the fraction of the range that does not.

udeep.validate

src/udeep/validate.py

Check a file against its own declaration: the conformance points of the profile (v0.3, section 10).

from udeep.validate import validate
report = validate("figure.jpg")              # what the file alone can show
print(report); report.ok

import udeep.mpl as hp
report = hp.validate(fig, "figure.jpg")      # also compares the pixels with the figure that made it

python -m udeep.validate figure.jpg ...    # exit status 1 if any file fails

Two kinds of file are checked: the gain-map JPEG, which is always a rendering, and the 16-bit PQ PNG (udeep.export.write_png_pq), which may declare itself a data record. For the PNG the check that matters is that claim: the file's levels are worked out again and set against the stated uncertainty (profile 7.3.1), and with the data supplied, the data is read back out of the file, whatever its colour map. A PNG holds no SDR base, so it must name the SDR figure that goes with it (SdrDigest, profile 9.1.2).

A file carries its declared mapping but not its data, so on its own it can show that the declaration is complete and consistent, that the mapping can be inverted, that nothing exceeds the headroom and that the stated precision is right. Whether the pixels are that mapping of the data (points 3 to 5) needs the figure as well.

Files come from anyone (design decision D24), so validate never raises on one: whatever cannot be read or checked becomes a failed check that says what it was.

classCheck

One check made on one file: the conformance point it belongs to, its name, how it came out, and what was found, in words.

classReport

What validate found for one file: the checks in the order they were made, and the declaration the file carries, if one could be read. As text (str(report)) it is a line that says whether the file conforms, and then a line for each check.

ok

status(self, name: str) -> str

add(self, point: str, name: str, status: str, detail: str) -> None

functionvalidate

validate(
    path,
    expected: tuple | None = None,
    data=None,
    sdr=None,
    data_digest: str | None = None,
    sdr_digest: str | None = None,
) -> Report

Check the gain-map JPEG, or the 16-bit PQ PNG, at path. expected is (sdr, hdr): the fair SDR figure and the HDR figure as linear RGB arrays in multiples of SDR white, for the checks that need the pixels (udeep.mpl.validate supplies them from a figure). data is the array the figure was made from and sdr the path of its SDR figure of record: each is compared with the digest the file declares (or pass the digests themselves). For a 16-bit PNG, sdr may also be the gain-map JPEG of the same figure, which is compared by the SdrDigest it declares, and data is also read back out of the file.

It does not raise on a file, whatever the file holds: see the report's checks.

functionmain

main(argv: list[str] | None=None) -> int

The command line, python -m udeep.validate FILE [FILE ...]: prints each file's report. With one file, --sdr FIGURE also checks the SDR figure of record (an 8-bit image; for a 16-bit PNG, also the gain-map JPEG of the same figure). Returns the exit status: 0 if every file conforms, 1 if one does not, 2 for a command line it cannot use.

udeep.scale

src/udeep/scale.py

Data -> normalised position t in [0, 1]. Element-wise; limits are passed in, never inferred.

Float64 (design decision D8), and written against the Python Array API standard (D22): the same code runs on any array that follows it, on a CPU or a GPU.

functionnamespace

namespace(x)

The array library x belongs to, per the Array API standard; NumPy for anything else.

functionas_float

as_float(x)

(library, x as a float64 array of that library). Lists and scalars become NumPy arrays.

functionin_chunks

in_chunks(function, x, tail: tuple=(), dtype=np.float64)

function(x) for a large NumPy array, worked through CHUNK values at a time into one output array of shape x.shape + tail. Each step of the mapping makes several arrays as large as its input on the way to its answer; done whole, those are many times the image. The result is the same, value for value: every step is a function of single values. Small arrays, and arrays of other libraries, go straight through.

dtype is what the output is kept as. The arithmetic is float64 either way; float32 rounds each result once, as it is stored.

functionscale

scale(x, kind: str, lo: float, hi: float, width: float=1.0)

Map x to t in [0, 1]: lo -> 0, hi -> 1, clipped; NaN stays NaN.

kind: "linear", "log" (needs lo > 0; x <= 0 -> 0) or "asinh" (linear within ~width of zero, logarithmic beyond).

functionunscale

unscale(t, kind: str, lo: float, hi: float, width: float=1.0)

The inverse of scale on [0, 1]: the value at position t.

udeep.tone

src/udeep/tone.py

t -> y in multiples of SDR white (t is already log/asinh scaled).

y = t / knee                          for t <= knee   (SDR: 0 .. 1)
y = cubic Hermite from 1 to headroom  for t >  knee   (HDR: 1 .. headroom)

The shoulder starts with the slope of the SDR part (C1: no kink, so no Mach band at the knee) and ends at exit_slope times its secant slope. The default, 1, is the **declared mapping** of the profile (v0.3, 6.3): strictly increasing, so every light level maps back to one data value. With entry slope a and exit slope 1 (in units of the secant) the slope is (3a-3)u^2 + (4-4a)u + a, never below min(a, (4-a)/3) of the secant. exit_slope=0 ends flat at headroom, the roll-off of ITU-R BT.2390's EETF: use it only for display rendering, because it squeezes the top of the data range into the last few percent of light, where it cannot be recovered.

If 1/knee exceeds three times the secant slope, the entry slope is capped there (Fritsch-Carlson: entry and exit slopes both within [0, 3] secants keep a cubic Hermite monotone), at the cost of C1 in that corner case.

knee = 1 gives the full-range SDR plot (y = t); headroom = 1 with knee < 1 gives SDR clipped at the same white -- the fair baseline an HDR rendering must match below white.

functiontone

tone(t, knee: float, headroom: float, exit_slope: float=1.0)

Float64, on any array library that follows the Array API standard (design decisions D8, D22).

functionknee_at_percentile

knee_at_percentile(t, pct: float) -> float

The t below which pct % of the finite values lie; clamped to (0, 1].

udeep.masks

src/udeep/masks.py

Masks: the only place where a pixel's treatment may depend on its neighbours (design decision D2).

The mapping from data to light is pointwise. Anything that needs a neighbourhood runs here, before the mapping, and returns a mask. Pixels in a mask are held at or below SDR white; the mapping never sees their neighbours.

isolated(above)   pixels above the knee with no 3 x 3 block of such pixels around them: the Highlight preset
                  holds these, so that noise cannot sparkle

Every derived mask has a rule name, which the file declares (udeep:DerivedMask), with the number of pixels it holds (udeep:DerivedMasked).

functionopen3

open3(b: np.ndarray) -> np.ndarray

3 x 3 morphological opening of a boolean image (erode, then dilate); outside the image counts as False.

functionisolated

isolated(above) -> np.ndarray

The pixels of above (a boolean image: light above SDR white) that belong to no full 3 x 3 block of it.

functionhold

hold(light, sdr_light, mask)

Apply a mask to mapped light, pointwise: masked pixels take the fair SDR light (at most SDR white).

udeep.jnd

src/udeep/jnd.py

How visible one code-value step is: PQ steps at N bits in units of the DICOM PS3.14 GSDF JND.

The GSDF (Barten model, standard target) gives luminance at JND index j = 1..1023, over 0.05-3993 cd/m2. A PQ step of 1 means one code value changes luminance by one just-noticeable difference for that target. The GSDF target is not the most sensitive pattern: Miller et al. (2013) designed PQ against the minimum of Barten's threshold over all frequencies, which is lower, so these ratios are a lower bound on visibility, not an upper one.

functiongsdf

gsdf(j)

Luminance (cd/m2) at JND index j (DICOM PS3.14, eq. for j = 1..1023).

functionjnd_relative

jnd_relative(nits)

Relative luminance step (dL/L) of one GSDF JND at nits.

functionpq_step_in_jnd

pq_step_in_jnd(nits, bits: int)

Size of one PQ code-value step at nits, in GSDF JNDs, for a bits-bit signal.

udeep.vision

src/udeep/vision.py

Colour-vision deficiency: does a colour map stay readable for people who lack one cone type?

simulate(rgb, kind)     how linear RGB looks under protanopia, deuteranopia or tritanopia
report(table)           per kind: is lightness still strictly increasing along the map, and how much of the
                        map's perceptual length survives

The simulation is that of Machado, Oliveira & Fernandes (2009), "A physiologically-based model for simulation of color vision deficiency", IEEE TVCG 15(6), at full severity: one 3 x 3 matrix per kind, applied to linear RGB with BT.709 primaries. The matrices were copied from the colorspacious package's data (its cvd.py), which gives them as the paper's supplementary data. Being linear, the same matrix serves above SDR white.

What "readable" means here, following the usual advice for scientific maps (Crameri et al. 2020): lightness must keep rising along the map under every simulation, so that order can always be read from lightness alone. The perceptual length that survives is reported, not judged: it says how much finer discrimination is lost.

functionsimulate

simulate(rgb, kind: str) -> np.ndarray

Linear RGB (BT.709 primaries) as seen with the named deficiency; values the display cannot show are clipped at zero.

functionreport

report(table) -> dict

For normal vision and each deficiency: whether lightness (Jz) rises at every node of the table; if not, at how many nodes it falls, where the first is, and the deepest dip below an earlier level as a share of the map's whole lightness range; the share of the map's perceptual length that survives; and the same share over the map's flattest stretch (one of PARTS equal runs of light, in either segment).

functionmain

main() -> None

The command line, python -m udeep.vision: prints report for each current map at each registered headroom.

udeep.color

src/udeep/color.py

Colour spaces for HDR plotting. Element-wise over a trailing axis of 3; float64.

Linear sRGB is *relative*: 1.0 = SDR white = REF_WHITE_NITS (BT.2408 reference white). Jzazbz takes CIE XYZ in absolute cd/m^2: Safdar et al., Opt. Express 25(13), 15131 (2017).

functionsrgb_to_xyz

srgb_to_xyz(rgb) -> np.ndarray

Linear sRGB (BT.709 primaries, D65 white) -> CIE XYZ, in the same unit as it is given.

functionxyz_to_srgb

xyz_to_srgb(xyz) -> np.ndarray

CIE XYZ -> linear sRGB (BT.709 primaries, D65 white): the inverse of srgb_to_xyz. Nothing is clipped, so a colour outside the gamut comes back with a component below zero.

functionsrgb_to_bt2020

srgb_to_bt2020(rgb) -> np.ndarray

Linear sRGB (BT.709 primaries) -> linear RGB with BT.2020 primaries (ITU-R BT.2087), in the same unit.

functionxyz_to_jzazbz

xyz_to_jzazbz(xyz) -> np.ndarray

CIE XYZ (D65 white) in absolute cd/m^2 -> Jzazbz (Safdar et al. 2017): the lightness Jz, then az and bz. A cone response below zero is taken as zero.

functionjzazbz_to_xyz

jzazbz_to_xyz(jab) -> np.ndarray

Jzazbz -> CIE XYZ in absolute cd/m^2: the inverse of xyz_to_jzazbz.

udeep.core

src/udeep/core.py

The Rust core, used from Python wherever it is found.

import udeep.core
udeep.core.enabled()           # True when the core was found and has not been turned off
udeep.core.enable(False)       # use the NumPy and Pillow code instead, from here on

The core (core/, a Rust library with a C interface) is used wherever it is found: it maps and colours an image on every processor core at once (60 to 250 megapixels a second on the development machine against NumPy's 8 to 12, project/performance-scope.md 4d), and it writes and reads gain-map JPEGs. Where it is not found, the NumPy and Pillow code of this package does the same work; udeep.core.enable(False), or ENV["UDEEP_USE_CORE"] = "0", chooses that code even when the core is there.

The two are held to the same golden vectors, and that is the sense in which they are the same: they agree to about one part in 10^12 in the mapping (their logarithms and hyperbolic sines come from different libraries), which is far inside any file's precision but can turn an 8-bit code the other way at a few pixels, and so give a figure another SDR digest. The two JPEG decoders also differ by a code or two at some pixels: a file is held within its headroom exactly on the decoder its writer used, so write and read with the same one.

The library is looked for at ENV["UDEEP_CORE_LIB"], then beside this module (udeep/_lib, where a packaged build would put it), then in core/target/release of a source checkout (cargo build --release in core/).

functionlibrary_path

library_path() -> Path | None

Where the core library is, or None if it is in none of the places looked in.

functionavailable

available() -> bool

Whether a core library with the functions this module needs was found and loaded.

functionfiles

files() -> bool

Whether the core in use can write and read gain-map JPEGs (it was built with its jpeg feature).

functionpng

png() -> bool

Whether the core in use can write and read 16-bit PQ PNGs (it was built with its png feature).

functionenable

enable(on: bool=True) -> None

Map with the core from now on (or, with on=False, with NumPy again). Raises RuntimeError if the core is asked for and is not there.

functionenabled

enabled() -> bool

Whether the core is in use: yes whenever it is there, unless it was turned off (enable(False), or ENV["UDEEP_USE_CORE"] = "0").

functionlight

light(mapping, x, sdr: bool=False, threads: int=0) -> np.ndarray

Data values -> light, as Mapping.light(x); with sdr=True the fair SDR light, as Mapping.light(x, 1.0). threads = 0 uses every processor core; the result is the same on any number. Float64, the shape of x.

functioncolours

colours(mapping, y, dtype=np.float64, threads: int=0) -> np.ndarray

Light -> linear RGB through the mapping's colour map, as Mapping.colours(y, dtype): shape y.shape + (3,). With dtype=np.float32 the colours are worked out in float64 a slab at a time and rounded once as they are kept.

functionwrite_gainmap

write_gainmap(
    sdr: np.ndarray,
    hdr: np.ndarray,
    headroom: float,
    quality: int = 95,
    gain_quality: int = 100,
    description: bytes = b'',
) -> bytes

The bytes of a gain-map JPEG, as udeep.gainmap.write_gainmap_jpeg writes one: sdr and hdr are float64, height x width x 3, already checked for shape and range by the caller. The core holds the file within the brightest value of hdr on its own decoder, which is also the one read_gainmap uses.

functionwrite_png

write_png(rgb: np.ndarray, white_nits: float, xmp: bytes=b'') -> bytes

The bytes of a 16-bit PQ PNG, as udeep.export.write_png_pq writes one: rgb is height x width x 3, linear, in multiples of SDR white. The codes are the reference's; the compressed bytes are the core's own.

functionread_png

read_png(data: bytes, white_nits: float) -> np.ndarray | None

A 16-bit PQ PNG as light: float64, height x width x 3, linear RGB in multiples of the white written at white_nits. None when the bytes are a PNG of another kind, for the caller to say so.

functionlevels_pq16

levels_pq16(
    mapping,
    noise: tuple = ('unknown', None),
    white_nits: float = 203.0,
    samples: int = 1 << 20,
) -> dict

The levels of a 16-bit PQ file of this mapping, as the core works them out: how many there are, the widest in data units, and the data criterion's worst ratio for noise (None when the noise is unknown). The reference is udeep.metadata.data_steps_pq16 with rounding_to_noise.

functionread_gainmap

read_gainmap(data: bytes, display_boost: float, threads: int=0) -> np.ndarray | None

A gain-map JPEG as a display with display_boost of headroom shows it: float32, height x width x 3. None when the core's reader does not take the file (not one of this layout, or a gain map of another size than its base), for the caller to read it another way.

functionwrite_from_renders

write_from_renders(
    base: np.ndarray,
    scaled: np.ndarray,
    black: np.ndarray,
    white: np.ndarray,
    headroom: float,
    quality: int = 95,
    gain_quality: int = 100,
    description: bytes = b'',
) -> tuple[bytes, int]

A figure's gain-map JPEG straight from four 8-bit renders of it (height x width x 3, uint8): the figure as it is; with each HDR image divided by the headroom; and with the HDR images black, and white. Returns the bytes and the number of pixels lifted above the ordinary figure. The rule is that of udeep.mpl._compose, without the two float images that takes.

udeep.gpu

src/udeep/gpu.py

The mapping on a GPU: the reference shader (shaders/mapping.wgsl) and a way to run it and read it back.

uniform_block(mapping)      the shader's parameter block, worked out in float64 and packed as float32
run(mapping, x)             position, light, fair SDR light and colours for an array of values, from the GPU
expected_error(mapping)     how far float32 can be expected to fall from the reference, in units of the tolerances
check(mapping)              ValueError if that is more than a tolerance, or a number does not fit float32

run needs the optional wgpu package (WebGPU for Python, on Vulkan, Metal or DirectX). It exists to hold the shader to the golden vectors and to map large batches; a viewer uses the shader's three functions in its own render pipeline instead. The GPU renders and the CPU reference defines (design decision D15): given the same float32 values, results agree with the float64 reference to 2e-6 x (1 + |value|) in position and light, and to 1e-5 x (1 + |value|) in colour (D8), not exactly. Where float32 cannot do that, uniform_block and run raise ValueError: they do not return a worse answer.

Colour has the wider tolerance because it is judged by what reaches a screen, and because a steep stretch of a colour-map table multiplies the light's error: fire/1 at headroom 2 has one step, seventeen nodes below its top, where blue climbs 19 units per unit of light. 1e-5 is about a fiftieth of half a 10-bit code (0.5 / 1023 = 4.9e-4); the largest difference measured in any accepted mapping is 0.007 of an 8-bit code.

What float32 can hold

The ends of the range and the offset need not be float32 numbers: 273.15..303.15 or 20000.1..20001.1 are held as well as 0..1, because every constant that is subtracted is passed as a pair of float32 (uniform_block). Three things remain, and check refuses each. The positions below are of the knee in the range, for the profile's reference form above the knee (exit slope 1), and are where the rule draws the line, not where the shader fails: run regardless, a mapping just past one of them measured about half its tolerance.

1. A number that does not fit a float32 at all: a size above about 3.4e38, or below about 1.2e-38 and not zero. That covers the ends of the range, its width and the reciprocal of its width, and for a log stretch the ratio of the ends. 2. log and asinh. The function's result is itself a float32, so the position carries a rounding that does not shrink with the range: about 2.2e-7 for a log, and for an asinh between 2.6e-7 of the stretched value (where that is below 1) and 1e-7 of it (above about 4), each divided by the stretched span. A log range must therefore span a ratio of 1.12 at the very least, and more once the curve multiplies the error: with the knee in the middle, 1.25 at headroom 2, 1.4 at 4 and 7.4 at 16. An asinh range must span a ratio of 1.5 where its low end is no more than the width, 1.7 where it is ten widths and 2.1 where it is a hundred (knee in the middle, headroom 4). A knee low in a log or a narrow asinh range is steep enough to break the tolerance: below position 0.012 of a log range of four decades. A linear stretch has no such limit: its error is a fraction of the position. 3. Steepness. The light is out by the position's error times the slope of the curve, and a colour by the light's error times the slope of the colour-map table. So a knee near the top is refused, sooner with more headroom. With a grey map the knee may be as high as

   headroom                              2       4       8       16
   linear                                0.955   0.905   0.84    0.75
   log, four decades                     0.93    0.845   0.745   0.62
   asinh, 0 to 200 widths                0.935   0.85    0.755   0.63

The shipped colour maps, fire/1 and ice/1, are held at every shipped headroom (1, 2, 4, 8 and 16): none is refused outright. At headrooms 4, 8 and 16 their limits are those of the grey map, and at headroom 1 there is none. At headroom 2 the table is steep enough to bring the limit down:

   headroom 2                            linear  log     asinh
   ice/1                                 0.95    0.92    0.92
   fire/1                                0.85    0.75    0.76
   fire/1, exit slope 2                  0.71    0.51    0.53
   fire/1, exit slope 3                  0.565   0.27    0.30

With any colour map, a headroom between 1 and about 1.03 squeezes the whole upper table into a sliver of light, and is refused. A headroom of exactly 1, and a grey map at any headroom, are not.

expected_error is the rule itself: it follows those errors through the chain and check refuses when any is above 1. Its nine constants are measured, on an NVIDIA RTX 5070 Ti through Vulkan, not derived, and other hardware may differ: the tests measure again on whatever GPU they find. On 1800 mappings drawn at random (linear, log and asinh; grey, fire and ice; headrooms 1 to 16) and sampled closely at both ends and just above the knee, 1433 were let through and none of those was out: the largest error was 0.79 of its tolerance (light) and 0.54 (colour), the median 0.17. Of the 367 refused, 203 were out when run regardless and 164 were not: the rule errs towards refusing, and most of all for colour, where it adds up the worst each step can do. Of 20 mappings refused for their colours alone, 3 were out. Two closer looks at the colour limit found nothing out either: 250 accepted mappings whose expected colour error was above 0.6 measured 0.65 at most, and 122 accepted with a colour map at a headroom between 1.005 and 1.1 measured 0.58 at most.

functionuniform_block

uniform_block(mapping: Mapping) -> bytes

The shader's Mapping uniform: 64 bytes, twelve float32 then four uint32, little-endian, in the order of FIELDS (the shader file documents each). Everything is worked out in float64 first.

A number that is subtracted comes as a pair, _hi and _lo: the nearest float32, and what is left of the float64 number after it, already multiplied by the factor the shader applies between the two subtractions. Raises ValueError for a mapping the shader cannot hold to the tolerance (check).

functionexpected_error

expected_error(mapping: Mapping) -> dict

How far the shader's float32 answers can be expected to lie from the float64 reference, for position, light and colours, each as a multiple of its own tolerance: 2e-6 x (1 + |value|) for position and light (TOLERANCE), 1e-5 x (1 + |value|) for colours (COLOUR_TOLERANCE). Each is the largest over the range.

It follows the error through the chain at GRID positions. The position is out by a fraction of itself (POSITION_RELATIVE) and, for log and asinh, by the rounding of the function's argument and of its own float32 result, divided by the stretched span. The light is out by that times the slope of the curve, plus the rounding of the curve's own arithmetic. A colour is out by the light's error at the same place times the slope of the colour-map table there, plus the rounding of the interpolation. The constants are measured, not derived: see the module docstring.

functioncheck

check(mapping: Mapping) -> None

Raise ValueError unless the shader can hold this mapping to its tolerances: every number it needs fits a float32, and no part of expected_error is above 1 (position and light in units of TOLERANCE, colours in units of COLOUR_TOLERANCE). The module docstring says what that rules out.

functionadapter_name

adapter_name() -> str

The GPU that run uses, for reports.

functionrun

run(mapping: Mapping, x) -> dict

Map x (any shape, no NaN) on the GPU. Returns float32 arrays: position, light, sdr_light (the shape of x) and colours (that shape with 3 added).

udeep.golden

src/udeep/golden.py

Golden vectors: the contract every implementation must reproduce (design decision D14).

python -m udeep.golden        # rewrite golden.json after a deliberate change

golden.json is language-neutral. Each case has a kind (what to compute), args, the expected out, and how closely a result must agree: tol = {"rel", "abs"} for numbers (|got - want| <= abs + rel * |want|), or exact for text, digests and bytes. NaN is written as null. The tolerances are the written ones of decision D8: tight where an implementation in float64 must agree with the reference, looser where float32 tables or GPU arithmetic are involved.

Kinds: scale, tone, pq_encode, srgb_oetf, cmap (a table lookup), table (a shipped table's digest), mapping (a parameter record: position, light, fair SDR light, the inverse, colours), levels (stored levels and the data criterion), mask_isolated, digest_data, digest_pixels, xmp (a declaration's serialised form), caption, iso21496 (the gain-map metadata block's bytes).

A levels case gives the noise as [model, parameter]. Its rounding_to_noise is the worst ratio as a file states it, no larger than 1e9: the ratio is unbounded where the noise reaches zero, and JSON has no infinity. at, the data value where the worst ratio was taken, is given for poisson and relative noise, and only where one level is clearly the worst. Elsewhere (constant noise, a tolerance, relative noise on a log stretch) many levels share the worst ratio to within rounding, so which of them is found is not part of the contract and at is left out.

functionmake

make() -> dict

Every golden case, worked out now by this package: the dictionary that golden.json is written from.

functionagree

agree(got, want, case: dict, where: str='') -> list[str]

Differences between a result and the expected output, under the case's own tolerance; empty if none.

functionmain

main() -> None

The command line, python -m udeep.golden: writes golden.json from make.

UDeepCore (Julia)

julia/UDeepCore · its Makie extension is in ext/UDeepCoreMakieExt.jl

The mapping from scientific data to light and colour of the UDeep Figure Profile, for Julia.

value x  ->  position t in [0, 1]  ->  light y, in multiples of SDR white  ->  linear RGB, same unit

This is a thin binding to the project's Rust core (core/, through its C interface), not a second implementation: the arithmetic lives in one place and is held to one set of golden vectors. The library is found at ENV["UDEEP_CORE_LIB"]; or else as the artifact the package was installed with (Artifacts.toml, written for a release); or else in core/target/release of the repository this package sits in (build it with cargo build --release --features jpeg,png).

Arrays are mapped element by element, so any array shape works and its shape is kept. Digests are defined on values in row order (the last index fastest), so a Julia array is walked that way when it is digested.

typeMapping

Mapping(stretch, range_low, range_high, knee, reference_headroom; width = 1, offset = 0, exit_slope = 1)

The mapping a file declares, as one record. stretch is :linear, :log, :asinh or :proportional. The layout is that of the C interface's udeep_mapping: seven Float64, then two UInt32, 64 bytes.

functioncore_version

The core library's version, as a VersionNumber.

functionsdr_light

sdr_light(m, x): data values -> the fair SDR light: the same below the knee, SDR white above it.

functionvalue

value(m, y): light -> data values, the inverse of light.

functionlight

light(m, x; threads = 0)

Data values -> light, in multiples of SDR white: the declared mapping. threads = 0 uses every available core, threads = 1 none but the caller's; the result is the same, bit for bit. An array of fewer than 32768 values is mapped on the caller's thread whatever threads is: starting another would cost more than it saved.

x is any array of real numbers, as for the other functions; the result is an Array{Float64} of its size.

functionlight!

light!(out, m, x; threads = 0): as light, into out.

functioncolours

colours(table, headroom, y)

Light -> linear RGB (BT.709 primaries, multiples of SDR white). table is a colour-map table as the project ships them (Vector{Float32} of 2 * nodes * 3 numbers) or nothing for grey. Returns an array of the shape of y with a last dimension of 3.

functiondigest_data

digest_data(a)

The digest that ties a file to its data (udeep:DataDigest): of the values and shape of a, whatever type it is stored as. Every NaN counts as the same value. An array with no dimensions, such as fill(1.5), has the digest of the one-element vector [1.5]: the Python reference digests a scalar so, and the core follows it.

functiondigest_pixels

digest_pixels(rgb): the digest of an 8-bit RGB image, height x width x 3 (udeep:SdrDigest).

functionsrgb_encode

srgb_encode(linear): linear light -> sRGB signal, both in [0, 1]; values outside are clipped, NaN stays NaN.

functionsrgb_decode

srgb_decode(signal): sRGB signal -> linear light, the inverse of srgb_encode.

functiondigest_sdr

digest_sdr(sdr)

The udeep:SdrDigest of a fair SDR rendering given as light (height x width x 3, linear, 0 to 1: the sdr of an HdrView), taken over the 8-bit sRGB codes the writers store for it, with NaN as 0. A file with no SDR base of its own, such as a 16-bit PQ PNG, must declare it (profile 9.1.2); the gain-map JPEG of the same figure then declares the same value.

functionvalue_at

value_at(m, t): position in [0, 1] -> data values, the inverse of UDeepCore.position inside the range.

functionisolated

isolated(above)

The isolated-3x3 mask rule: the pixels of the boolean image above (light above SDR white) that lie in no full 3 x 3 block of it. The Highlight preset holds these at white, so that noise cannot sparkle.

functioncolour_table

colour_table(name, headroom)

The registered colour-map table for "fire" or "ice" at a registered headroom (2, 4, 8 or 16), as the project ships it; nothing for "grey", which needs none. The name alone is the map's current version (2); "fire/1" names an older one, for reproducing a figure made with it. The tables are read from ENV["UDEEP_TABLES"], or else from src/udeep/tables of the repository this package sits in.

A map at any other headroom needs a table made for that headroom. Only the Python reference makes tables, so this package refuses one rather than draw a map the file could not declare.

typeHdrView

The two renderings of an image, as linear RGB in multiples of SDR white (size(data)..., 3): sdr is the fair SDR plot (the same below the knee, white above it) and hdr the HDR plot. light is the HDR light before colour.

functionhdr_view

hdr_view(data, m; mode = :highlight, cmap = "fire", mask = nothing)

Map an image by the two presets of the profile (section 13), as the Python reference's hdr_view does:

- :highlight: sparse extremes go above SDR white; pixels above the knee that belong to no 3 x 3 block of such pixels are held at white (the derived mask isolated-3x3). - :extend: a second range of real structure goes above white; no area filter.

Pixels where mask is true (saturated, clipped, invalid) are never lifted. The mapping itself is pointwise.

functionlevels

levels(m, table, scaled_by; noise = (:unknown, nothing), samples = 65536)

The stored levels of an 8-bit gain-map rendering of m through table (nothing for grey), written for the headroom scaled_by. Returns (max_data_step, rounding_to_noise): the widest run of data values that share one stored code (udeep:MaxDataStep), and the data criterion's worst ratio of rounding error to the stated uncertainty (udeep:RoundingToNoise, at most 1e9), or nothing when the noise is unknown.

noise is (:constant, sigma), (:poisson, scale), (:relative, fraction), (:tolerance, value) or (:unknown, nothing).

typeDeclaration

Declaration(; mapping, data_unit, knee_rule, max_data_step, ...)

What a file declares about its data and mapping (profile section 8), under the names of the udeep: properties. mapping, data_unit, knee_rule and max_data_step must be given. The profile's version and image state are the core's own.

functiondescription

description(d): the declaration as the rdf:Description element a file carries, exactly as the reference writes it.

functioncaption

caption(d): a caption paragraph generated from the declaration (profile 8.2).

functiongainmap_jpeg

gainmap_jpeg(sdr, hdr, headroom; quality = 95, gain_quality = 100, description = "")

A gain-map JPEG (Ultra HDR layout, with the ISO 21496-1 block) as bytes. sdr and hdr are height x width x 3 linear RGB in multiples of SDR white: the ordinary figure, and the same figure with light above white. An ordinary viewer shows sdr; an HDR one shows hdr, up to headroom. description is the declared metadata (description), or empty for a file that declares nothing and so does not conform to the profile.

A pixel that is not lifted is stored with a gain of exactly 1, and the file never reads above the brightest value of hdr on the core's own decoder: the writer decodes its gain map and lowers it until nothing is over, and throws if it cannot. Keep gain_quality at 100 unless the figure is grey: below it a gain map is unreliable with a colour map ("fire", "ice"), with this writer or any other, and the call may be refused.

Needs a core library built with --features jpeg,png.

functiongainmap_from_renders

gainmap_from_renders(base, scaled, black, white, headroom; quality = 95, gain_quality = 100, description = "")

A figure's gain-map JPEG straight from four 8-bit renders of it, each height x width x 3 (UInt8, sRGB codes): the figure as it is, which becomes the file's SDR image unchanged; the same with each HDR image divided by the headroom; and the same with the HDR images black, and white. A pixel that is black in the third and white in the fourth is made by an HDR image alone, and only such a pixel is lifted (profile 6.11). Returns the file's bytes and the number of pixels lifted above the ordinary figure.

It is what save_hdr uses: the same file as gainmap_jpeg would write from the two float figures, without making them.

functionhdrimage!

hdrimage!(ax, data, m; mode = :highlight, cmap = "fire", mask = nothing, unit = nothing, knee_rule = nothing,
          noise = (:unknown, nothing), source = nothing, unit_code = nothing, kw...)
hdrimage!(ax, x, y, data, m; ...)

Like Makie's image!, for the profile's two presets: draws the fair SDR plot of data under the mapping m, and remembers the HDR one for save_hdr. Returns the plot. data[i, j] is drawn at the i-th place along x and the j-th along y, as image! draws a matrix; x and y are the intervals it spans (default 0 .. size).

unit, knee_rule and noise are what the file will declare (profile section 8): the data's unit ("1" if it has none); why the knee is where it is, as "physical; ...", "statistical; ..." or "preset; ..."; and the noise model with its parameter, such as (:constant, 10.0). source says where the data is kept (a DOI, a URL); the file also records a digest of data. unit_code is the unit as a UCUM code. A NaN in data is drawn black.

Further keywords go to image!. The image must stay opaque and uncovered where it is to go above white.

functionhdrcolorbar!

hdrcolorbar!(position, plot; label = "", n = 256, width = 18)

An HDR colour bar for an hdrimage! plot at a figure position such as fig[1, 2]: ticks in data units, and the value shown at SDR white. Returns its axis.

functionsave_hdr

save_hdr(path, fig; quality = 95, gain_quality = 100, px_per_unit = 2, metadata = true)

Write the figure as a gain-map JPEG: the ordinary figure, plus gain on the hdrimage! plots only. The file carries the profile's declared metadata unless metadata = false; without it the file does not conform.

A pixel is lifted only where an HDR image alone determines it, so text, lines and anything translucent drawn over an image stay as they are in the ordinary figure. The SDR figure the file's digest refers to is the figure as the backend renders it at px_per_unit. Keep gain_quality at 100 with a colour map (see gainmap_jpeg).

functionhdr_declaration

hdr_declaration(fig): what the figure declares (Declaration), as save_hdr would write it, less the digest of the SDR figure.

functionhdr_caption

hdr_caption(fig): a caption paragraph generated from the figure's declaration (profile 8.2).

functionhdr_theme

hdr_theme(name = :dark; ground = nothing, ink = nothing)

A Makie Theme for a figure's furniture on a light or a dark page: the background, and the text, axes and ticks drawn on it. name is :light or :dark; ground and ink replace the background and text colours, to match a particular page. Use it around the code that builds the figure:

with_theme(hdr_theme(:dark)) do
    fig = Figure()
    ...
    save_hdr("figure_dark.jpg", fig)
end

A theme changes nothing about the data: the image, its colours, the mapping and everything the file declares about them are the same in every theme, and so is the data's digest. Only the SDR figure differs, so each theme is its own file with its own SDR digest (profile 8.7). The colours are those of the Python reference's udeep.mpl.theme. A colour given to a plot by hand, such as text!(...; color = :black), is not changed.

functionxmp_packet

xmp_packet(d)

The declaration as a complete XMP packet, as the reference's udeep.metadata.to_xmp writes it: the element of description inside the packet's wrapper. It is what a PNG carries in its iTXt chunk.

functionlevels_pq16

levels_pq16(m, table; white_nits = 203, noise = (:unknown, nothing), samples = 1048576)

The stored levels of a 16-bit PQ PNG of m through table (nothing for grey), with SDR white written at white_nits. Returns (max_data_step, rounding_to_noise, count): the widest run of data values that share one code, the data criterion's worst ratio of rounding error to the stated uncertainty (nothing when the noise is unknown), and how many levels there are. A file whose ratio is 0.1 or less (1 or less for a tolerance) may be declared a data record (profile 7.3.1). noise is as for levels, which does the same for the 8-bit gain-map JPEG.

functionpng_pq

png_pq(rgb; white_nits = 203, declaration = nothing)

A 16-bit PQ PNG as bytes: rgb is height x width x 3, linear RGB in multiples of SDR white (the hdr of an HdrView), and 1.0 is written at white_nits. The file declares BT.2020 primaries and the PQ transfer function in a cICP chunk. With a Declaration it carries that as XMP; without one it declares nothing and does not conform to the profile. A NaN is written as 0.

A file with a declaration is written with white at 203 cd/m² (profile 9.1.1), since it has nowhere to say what its white was; the core refuses another white_nits with one (ArgumentError). The declaration must also name the SDR figure that goes with the file, as sdr_digest (profile 9.1.2). Matching a screen's own SDR white is for the viewer; a file without a declaration may be written at any white, for one's own screen.

This is the form in which a file may be a data record: see levels_pq16 and read_png_pq.

functionread_png_pq

read_png_pq(file; white_nits = 203)

A 16-bit PQ PNG, given as its bytes, as light: height x width x 3, linear RGB in multiples of the white written at white_nits. It is what png_pq was given, to within the file's 16-bit steps. For a grey figure any channel is the light, and value turns it back into data. Throws for a PNG of another kind.