sweep.SweepManagerBuilder

sweep.SweepManagerBuilder(config)

Builder for SweepManager with flexible configuration.

The builder pattern allows for fluent configuration of SweepManager with sensible defaults and resource ownership tracking.

Examples

>>> manager = (
...     SweepManagerBuilder(config)
...     .with_registry("experiment.duckdb", experiment_name="my_sweep")
...     .with_cli(jar_path=Path("josh.jar"))
...     .build()
... )

Methods

Name	Description
build	Build the SweepManager instance.
with_batch_remote	Pre-configure batch remote dispatch for the built SweepManager.
with_catalog	Configure an optional ProjectCatalog for cross-experiment tracking.
with_cli	Configure the CLI.
with_collision_policy	Configure how batch-remote sweeps handle prior MinIO outputs.
with_defaults	Apply default configuration for simple cases.
with_label	Set a label for this run.
with_registry	Configure the registry.

build

sweep.SweepManagerBuilder.build()

Build the SweepManager instance.

Expands jobs, creates or verifies session, and registers runs.

For adaptive strategies (OptunaStrategy), job expansion is deferred to run time - the JobSet will be empty initially.

Returns

Name	Type	Description
	SweepManager	Configured SweepManager ready for use.

Raises

Name	Type	Description
	ValueError	If session_id provided but hashes don’t match registered runs.

Examples

>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb")
...     .with_cli()
...     .build()
... )

with_batch_remote

sweep.SweepManagerBuilder.with_batch_remote(
    target,
    *,
    no_wait=False,
    poll_interval=10,
    timeout=None,
    auto_ingest=True,
)

Pre-configure batch remote dispatch for the built SweepManager.

After calling this, SweepManager.run() defaults to batch_remote=True with the supplied settings. Callers may still override any individual setting at .run() call time.

Parameters

Name	Type	Description	Default
target	str	Target profile name (loaded from `~/.josh/targets/<name>.json`).	required
no_wait	bool	If True, dispatch and return without polling. Polling is then the caller’s responsibility (or a later `SweepManager.ingest()` call).	`False`
poll_interval	int	Seconds between poll attempts when `no_wait=True`.	`10`
timeout	int \| None	Overall timeout per job in seconds.	`None`
auto_ingest	bool	If True, auto-call `ingest_results()` after each successful batch job.	`True`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb", experiment_name="my_sweep")
...     .with_batch_remote("gke-test")
...     .build()
... )
>>> # Equivalent to manager.run(batch_remote=True, target="gke-test")
>>> results = manager.run()

with_catalog

sweep.SweepManagerBuilder.with_catalog(catalog, *, experiment_name=None)

Configure an optional ProjectCatalog for cross-experiment tracking.

When a catalog is configured, the experiment is automatically registered on build() and its status is updated after run().

Parameters

Name	Type	Description	Default
catalog	Any	A ProjectCatalog instance.	required
experiment_name	str \| None	Human-readable name for the experiment in the catalog.	`None`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> from joshpy.catalog import ProjectCatalog
>>> catalog = ProjectCatalog("project.duckdb")
>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb")
...     .with_catalog(catalog, experiment_name="baseline_dev_fine")
...     .build()
... )

with_cli

sweep.SweepManagerBuilder.with_cli(cli=None, *, jar_path=None, java_path='java')

Configure the CLI.

Parameters

Name	Type	Description	Default
cli	JoshCLI \| None	Existing JoshCLI instance.	`None`
jar_path	Path \| None	Path to josh jar (creates new CLI).	`None`
java_path	str	Path to java executable (default: “java”).	`'java'`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> # Use existing CLI
>>> builder.with_cli(existing_cli)

>>> # Create new CLI with custom jar path
>>> builder.with_cli(jar_path=Path("josh.jar"))

with_collision_policy

sweep.SweepManagerBuilder.with_collision_policy(policy)

Configure how batch-remote sweeps handle prior MinIO outputs.

Parameters

Name	Type	Description	Default
policy	str	One of: - `"fail"` (default) — raise :class:`SweepCollisionError` at `.run()` time if the export path template lacks both `{timestamp}` and `{run_hash}` and any job’s `run_hash` already has runs in the registry. Equivalent to the static check; safest default. - `"pool"` — reconcile by count: list existing outputs and dispatch only the number still needed (`target - have`) at fresh, non-colliding indices via :attr:`BatchRemoteConfig.replicate_start`. Enables “grow a sweep”: first run N=5 succeeds, re-run with N=10 dispatches 5 more. Sparse existing sets are fine — only the count matters. - `"skip"` — idempotent: if any matching output exists, skip the dispatch entirely. Useful for CI reruns.	required

"overwrite" and force=True were removed: overwriting outputs can’t be kept in sync with the registry (ingestion is idempotent per replicate index). To redo a config, :meth:RunRegistry.drop_run it then re-run — the one sanctioned way to clear run data.

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Raises

Name	Type	Description
	ValueError	If `policy` is not one of :data:`COLLISION_POLICIES`.

Examples

>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb")
...     .with_batch_remote("gke-test")
...     .with_collision_policy("pool")
...     .build()
... )

Notes

"pool" and "skip" enumerate prior outputs by listing MinIO and matching the export-path template. The template’s {replicate} slot becomes the integer capture; any other unresolved placeholder (rare in practice — Josh writes one CSV per replicate per export type regardless of which template variables appear) becomes a wildcard. {timestamp} is the one special case: its presence signals per-dispatch isolation, so listing returns empty (no pooling across timestamps).

with_defaults

sweep.SweepManagerBuilder.with_defaults(
    registry=':memory:',
    experiment_name=None,
    jar_path=None,
)

Apply default configuration for simple cases.

Convenience method that sets up both registry and CLI with defaults.

Parameters

Name	Type	Description	Default
registry	str \| Path \| RunRegistry	Registry path or instance (default: in-memory).	`':memory:'`
experiment_name	str \| None	Name for the experiment session.	`None`
jar_path	Path \| None	Path to josh jar file.	`None`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> manager = (
...     SweepManager.builder(config)
...     .with_defaults(registry=":memory:")
...     .build()
... )

with_label

sweep.SweepManagerBuilder.with_label(label, force=False, on_collision=None)

Set a label for this run.

The label is applied at build() time. For a single-job config the bare label is registered as a unique, queryable handle and used to resolve {label} in export paths.

For a multi-job sweep the label becomes a per-job prefix: each job’s {label} resolves to <label>_<run_hash> so export paths stay collision-free. (Without with_label(), sweep jobs default to sweep_<run_hash>.) Per-job sweep labels are not registered as registry handles — use registry.label_run() for that.

Parameters

Name	Type	Description	Default
label	str	Human-readable label for this run.	required
force	bool	If True, reassign the label even if already taken (drops the old label).	`False`
on_collision	str \| None	Collision strategy. `"timestamp"` archives the old label with a timestamp suffix so the bare label always points to the latest run. Mutually exclusive with `force`.	`None`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb")
...     .with_label("baseline")
...     .build()
... )

>>> # Re-run with same label, archive the old one
>>> manager = (
...     SweepManager.builder(config)
...     .with_registry("experiment.duckdb")
...     .with_label("baseline", on_collision="timestamp")
...     .build()
... )

with_registry

sweep.SweepManagerBuilder.with_registry(
    registry_or_path,
    *,
    experiment_name=None,
    session_id=None,
)

Configure the registry.

Parameters

Name	Type	Description	Default
registry_or_path	str \| Path \| RunRegistry	Path to create new registry, or existing instance.	required
experiment_name	str \| None	Name for new session (when creating registry).	`None`
session_id	str \| None	Use existing session ID (to resume a previous session).	`None`

Returns

Name	Type	Description
	SweepManagerBuilder	Self for chaining.

Examples

>>> # Create new registry from path
>>> builder.with_registry("experiment.duckdb", experiment_name="my_sweep")

>>> # Use existing registry
>>> builder.with_registry(existing_registry, session_id="abc-123")