Overview

The Lightcone CLI is an agent-native scientific workflow management system designed to orchestrate the materialization of research artifacts declared in an astra.yaml file while guaranteeing their provenance and reproducibility.

What it doesCopy Anchor Link

• Manages an ASTRA project. lc init sets up astra.yaml, a locked uv environment and a git repository. Only the CLI writes results, each from the recipe astra.yaml declares for it, so every artifact traces back to the code, data and decisions that made it.
• Runs the analysis. lc materialize turns astra.yaml into a graph of tasks and runs each recipe in a sandbox, in parallel where it can, skipping outputs that are already up to date.
• Versions and shares the data. git-annex carries data and results alongside the code, so collaborators clone the project and fetch files with git annex get when they need them.
• Certifies how each result was made. Each output is committed with a manifest of its recipe, inputs and environment, and a run record that can replay it. Declare a license, and the CLI also maintains an RO-Crate ready to archive or deposit.
• Gives your agent compute. lc compute launch starts a Dask cluster, and lc run and lc materialize send it work. The commands are the same on a laptop and on a supercomputer.

Your agent

Starts a cluster with lc compute launch, then sends it work with lc run and lc materialize. The commands are the same on every machine.

Drives the CLI

Lightcone CLI

One execution interfaceStarts a Dask cluster from an offer in ~/.lightcone/compute.yaml, and runs each task on it, wherever the cluster is.

Starts the cluster on

Your machine

A laptop or workstation, with no configuration.provider: local

HPC cluster

Slurm jobs, submitted for you, with no batch scripts.provider: slurm

Kubernetes

Cloud and JupyterHub deployments.Planned

Without a configuration file, the CLI uses your own machine. Configuring compute sets up Slurm offers, and HPC clusters gives ready configurations for known systems.