How to benchmark a solver geometry#

This guide shows how to measure the computational throughput of a solver geometry — how many forward() and inverse() operations it can perform per second — using hklpy2.utils.benchmark.

Benchmarking is useful when:

evaluating whether a solver is fast enough for a scanning application,
comparing two solvers for the same geometry, or
detecting a performance regression after a code change.

Note

The benchmark is purely computational. It does not move any motors or communicate with hardware, so it is safe to run on a live diffractometer.

Prerequisites#

You will need hklpy2 ≥ 0.6.0 and hklpy2-solvers installed:

pip install "hklpy2>=0.6.0" hklpy2-solvers

Starting from a saved configuration#

The easiest way to benchmark is to restore a simulator from a saved configuration file using simulator_from_config(). Ready-to-use configurations are provided for each supported geometry:

Solver	Geometry	Configuration file
`diffcalc`	diffcalc_4S_2D	`diffcalc_4s_2d.yml`

Note

Configuration files for ad_hoc solver geometries are not yet provided. You can create your own using the How to use the ad_hoc solver workflow and export().

Download the file for your geometry, save it alongside your script, then run:

import hklpy2

sim = hklpy2.simulator_from_config("diffcalc_4s_2d.yml")
hklpy2.utils.benchmark(sim)

Example output:

Diffractometer benchmark
  solver:     diffcalc
  geometry:   diffcalc_4S_2D
  mode:       4S+2D bisect_eta_fixed nu_fixed
  wavelength: 1.54
  calls:      500

  operation       ops/sec     ms/call status
  ------------ ---------- ----------- ------
  forward()           800       1.250   FAIL
  inverse()          7300       0.137   PASS

  target: 2,000 ops/sec (+/-10%)

The status column compares each operation against a target of 2,000 ops/sec (±10%). A FAIL result means the solver is below that target for this operation.

Note

Some solvers perform iterative constraint solving and return multiple candidate solutions per forward() call. This can result in a FAIL that reflects an upstream library limitation rather than a problem with the solver itself. See the Geometries page for solver-specific notes.

Benchmarking a live diffractometer#

Caution

hklpy2.utils.benchmark runs timing loops directly on the diffractometer object passed to it. Until bluesky/hklpy2#369 is resolved, pass a simulator restored from a snapshot of your live diffractometer’s configuration, rather than the live instrument itself. This guarantees no side effects on motor positions or solver state:

import hklpy2

# Snapshot the live diffractometer, then benchmark the copy.
sim = hklpy2.simulator_from_config(my_diffractometer.configuration)
hklpy2.utils.benchmark(sim)

This page will be updated once the fix is released in hklpy2.

Saving your own configuration#

You can benchmark a diffractometer you have already set up in two ways.

From a file — export the current state using export(), then create a new simulator and benchmark:

my_diffractometer.export("my_config.yml", "benchmark configuration")

sim = hklpy2.simulator_from_config("my_config.yml")
hklpy2.utils.benchmark(sim)

Without a file — pass the configuration property directly, bypassing the file entirely:

import hklpy2

sim = hklpy2.simulator_from_config(my_diffractometer.configuration)
hklpy2.utils.benchmark(sim)

Adjusting the number of calls#

By default, hklpy2.utils.benchmark uses 500 calls per operation. Pass n to change this — more calls give a more stable measurement:

hklpy2.utils.benchmark(sim, n=2000)

Capturing results programmatically#

Pass print=False to suppress the printed report and receive the results as a dict instead:

results = hklpy2.utils.benchmark(sim, print=False)
print(results["forward_ops_per_sec"])
print(results["inverse_ops_per_sec"])

The dict contains:

Key	Description
`solver`	Solver name
`geometry`	Geometry name
`mode`	Current operating mode
`wavelength`	Current wavelength (Å)
`n`	Number of calls measured
`forward_ops_per_sec`	`forward()` throughput (ops/sec)
`forward_ms_per_call`	`forward()` latency (ms/call)
`inverse_ops_per_sec`	`inverse()` throughput (ops/sec)
`inverse_ms_per_call`	`inverse()` latency (ms/call)
`target_ops_per_sec`	Minimum target (2,000 ops/sec)