Command Line Interface

Many core signac functions are — in addition to the Python interface — accessible directly via the $ signac command line interface (CLI).

CLI Overview

The available commands in the CLI can be shown by executing $ signac --help:

$ signac --help
usage: signac [-h] [--debug] [--version] [-v] [-y]
              {init,job,statepoint,diff,document,rm,move,clone,find,view,schema,shell,sync,import,export,update-cache,config}
              ...

signac aids in the management, access and analysis of large-scale
computational investigations.

positional arguments:
  {init,job,statepoint,diff,document,rm,move,clone,find,view,schema,shell,sync,import,export,update-cache,config}

options:
  -h, --help            show this help message and exit
  --debug               Show traceback on error for debugging.
  --version             Display the version number and exit.
  -v, --verbosity       Set level of verbosity.
  -y, --yes             Answer all questions with yes. Useful for scripted
                        interaction.

The commands can be roughly grouped by task, ordered by frequency of use:

Using a project:
Modifying a project:
Sharing and archiving a project:

clone

$ signac clone --help
usage: signac clone [-h] project job_id [job_id ...]

positional arguments:
  project     The directory of the project to clone one or more jobs in.
  job_id      One or more job ids. The corresponding jobs must be initialized.

options:
  -h, --help  show this help message and exit

config

$ signac config --help
usage: signac config [-h] [-g] [-l] [-f] {show,set,verify} ...

positional arguments:
  {show,set,verify}

options:
  -h, --help         show this help message and exit
  -g, --global       Modify the global configuration.
  -l, --local        Modify the local configuration.
  -f, --force        Skip sanity checks when modifying the configuration.

diff

$ signac diff --help
usage: signac diff [-h] [-p [PRETTY]] [-i [INDENT]] [-f FILTER [FILTER ...]]
                   [job_id ...]

Find the difference among job state points.

positional arguments:
  job_id                One or more job ids. The corresponding jobs must be
                        initialized.

options:
  -h, --help            show this help message and exit
  -p [PRETTY], --pretty [PRETTY]
                        Print state point in pretty format. An optional
                        argument to this flag specifies the maximal depth a
                        state point is printed.
  -i [INDENT], --indent [INDENT]
                        Specify the indentation of the JSON formatted state
                        point.
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Limit the diff to jobs matching the filter.

document

$ signac document --help
usage: signac document [-h] [-p [PRETTY]] [-i [INDENT]] [-s]
                       [-f FILTER [FILTER ...]]
                       [job_id ...]

Print the document(s) corresponding to one or more job ids.

positional arguments:
  job_id                One or more job ids. The job corresponding to a job id
                        must be initialized.

options:
  -h, --help            show this help message and exit
  -p [PRETTY], --pretty [PRETTY]
                        Print document in pretty format. An optional argument
                        to this flag specifies the maximal depth a document is
                        printed.
  -i [INDENT], --indent [INDENT]
                        Specify the indentation of the JSON formatted state
                        point.
  -s, --sort            Sort the document keys for output in JSON format.
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Show documents of jobs matching the filter.

export

$ signac export --help
usage: signac export [-h] [--move] [-f FILTER [FILTER ...]]
                     [-j JOB_ID [JOB_ID ...]]
                     target [schema_path]

Export the project data space (or a subset) to a directory, a zipfile, or a
tarball.

positional arguments:
  target                The target to export to. May be a path to a directory,
                        a zipfile, or a tarball.
  schema_path           Specify an optional export path, based on the job
                        state point, e.g., 'foo/{job.sp.foo}'.

options:
  -h, --help            show this help message and exit
  --move                Move data to export target instead of copying. Can
                        only be used when exporting to a directory target.

select:
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Limit the jobs to export to those matching the filter.
  -j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
                        Limit the jobs to export to those matching the
                        provided job ids.

find

$ signac find --help
usage: signac find [-h] [-s [SHOW]] [--sp [SP ...]] [--doc [DOC ...]]
                   [-p [PRETTY]] [-1]
                   [filter ...]

All filter arguments may be provided either directly in JSON encoding or in a simplified form using the query API. For example, the following two bash commands are equivalent:

$ signac find key 42
$ signac find '{"a": 42}'

For the full syntax, see:
https://docs.signac.io/en/latest/query.html#simplified-filter

positional arguments:
  filter                Find jobs matching the filter.

options:
  -h, --help            show this help message and exit
  -s [SHOW], --show [SHOW]
                        Show the state point and document of each job.
                        Equivalent to --sp --doc --pretty 3.
  --sp [SP ...]         Show the state point of each job. Can be passed the
                        list of state point keys to print (if they exist for a
                        given job).
  --doc [DOC ...]       Show the document of each job. Can be passed the list
                        of document keys to print (if they exist for a given
                        job).
  -p [PRETTY], --pretty [PRETTY]
                        Pretty print output when using --sp, --doc, or
                        ---show. Argument is the depth to which keys are
                        printed.
  -1, --one-line        Print output in JSON and on one line.

import

$ signac import --help
usage: signac import [-h] [--move] [--sync] [--sync-interactive]
                     [origin] [schema_path]

Import an existing dataset into this project. Optionally provide a file path
based schema to specify the state point metadata. Providing a path based
schema is only necessary if the data set was not previously exported from a
signac project.

positional arguments:
  origin              The origin to import from. May be a path to a directory,
                      a zipfile, or a tarball. Defaults to the current working
                      directory.
  schema_path         Specify an optional import path, such as
                      'foo/{foo:int}'. Possible type definitions include bool,
                      int, float, and str. The type is assumed to be 'str' if
                      no type is specified.

options:
  -h, --help          show this help message and exit
  --move              Move the data upon import instead of copying. Can only
                      be used when importing from a directory.
  --sync              Attempt recursive synchronization with default
                      arguments.
  --sync-interactive  Synchronize the project with the origin data space
                      interactively.

init

$ signac init --help
usage: signac init [-h]

options:
  -h, --help  show this help message and exit

job

$ signac job --help
usage: signac job [-h] [-w] [-p] [-c] [statepoint]

positional arguments:
  statepoint       The job's state point in JSON format. Omit this argument to
                   read from STDIN.

options:
  -h, --help       show this help message and exit
  -w, --workspace  Print the job's workspace path instead of the job id.
  -p, --path       Print the job's path instead of the job id.
  -c, --create     Create the job directory if necessary.

move

$ signac move --help
usage: signac move [-h] project job_id [job_id ...]

positional arguments:
  project     The destination project directory.
  job_id      One or more job ids. The corresponding jobs must be initialized.

options:
  -h, --help  show this help message and exit

rm

$ signac rm --help
usage: signac rm [-h] [-c] [-i] [-v] job_id [job_id ...]

positional arguments:
  job_id             One or more job ids of jobs to remove.

options:
  -h, --help         show this help message and exit
  -c, --clear        Do not completely remove, but only clear the job.
  -i, --interactive  Request confirmation before attempting to remove/clear
                     each job.
  -v, --verbose      Be verbose when removing/clearing files.

schema

$ signac schema --help
usage: signac schema [-h] [-x] [-t DEPTH] [-p PRECISION] [-r MAX_NUM_RANGE]
                     [-f FILTER [FILTER ...]] [-j JOB_ID [JOB_ID ...]]

options:
  -h, --help            show this help message and exit
  -x, --exclude-const   Exclude state point parameters, which are constant
                        over the complete project data space.
  -t DEPTH, --depth DEPTH
                        A non-zero value will format the schema in a nested
                        representation up to the specified depth. The default
                        is a flat view (depth=0).
  -p PRECISION, --precision PRECISION
                        Round all numerical values up to the given precision.
  -r MAX_NUM_RANGE, --max-num-range MAX_NUM_RANGE
                        The maximum number of entries shown for a value range,
                        defaults to 5.

select:
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Detect schema only for jobs matching the filter.
  -j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
                        Detect schema only for jobs with the given job ids.

shell

$ signac shell --help
usage: signac shell [-h] [-c COMMAND] [-f FILTER [FILTER ...]]
                    [-j JOB_ID [JOB_ID ...]]
                    [file]

positional arguments:
  file                  Execute Python script in file.

options:
  -h, --help            show this help message and exit
  -c COMMAND, --command COMMAND
                        Execute Python program passed as string.

select:
  Specify one or more jobs to preset the `jobs` variable as a generator over
  all job handles associated with the given selection. If the selection
  contains only one job, an additional `job` variable is referencing that
  single job, otherwise it is `None`.

  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Reduce selection to jobs matching the filter.
  -j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
                        Reduce selection to jobs matching the given job ids.

statepoint

$ signac statepoint --help
usage: signac statepoint [-h] [-p [PRETTY]] [-i [INDENT]] [-s] [job_id ...]

Print the state point(s) corresponding to one or more job ids.

positional arguments:
  job_id                One or more job ids. The corresponding jobs must be
                        initialized.

options:
  -h, --help            show this help message and exit
  -p [PRETTY], --pretty [PRETTY]
                        Print state point in pretty format. An optional
                        argument to this flag specifies the maximal depth a
                        state point is printed.
  -i [INDENT], --indent [INDENT]
                        Specify the indentation of the JSON formatted state
                        point.
  -s, --sort            Sort the state point keys for output.

sync

$ signac sync --help
usage: signac sync [-h] [-v] [-a] [-r] [-l] [-p] [-o] [-g] [-t] [-x [EXCLUDE]]
                   [-I] [--size-only] [--round-times] [-n] [-u]
                   [-s {always,never,update,Ask}] [-k KEY] [--all-keys]
                   [--no-keys] [--force] [-m] [--parallel [PARALLEL]]
                   [--stats] [-H] [--json] [-f FILTER [FILTER ...]]
                   [-j JOB_ID [JOB_ID ...]]
                   source [destination]

Use this command to synchronize this project with another project, similar to the synchronization of two directories with `rsync`. Data is always copied from the source to the destination.

For example, the following bash command will synchronize all jobs within this project with jobs in the other project, overwrite files if the source files are newer, and overwrite all conflicting keys in the project and job documents:

$ signac sync /path/to/other/project --update --all-keys

positional arguments:
  source                The directory of the project that this project should
                        be synchronized with.
  destination           Optional: The directory of the project that should be
                        modified for synchronization, defaults to the local
                        project.

options:
  -h, --help            show this help message and exit
  -v, --verbosity       Set level of verbosity.
  --force               Ignore all warnings, just synchronize.
  -m, --merge           Clone all the jobs that are not present in destination
                        from source.
  --parallel [PARALLEL]
                        Use multiple threads for synchronization.You may
                        optionally specify how many threads to use, otherwise
                        all available processing units will be utilized.
  --stats               Provide file transfer statistics.
  -H, --human-readable  Provide statistics with human readable formatting.
  --json                Print statistics in JSON formatting.

copy options:
  -a, --archive         archive mode; equivalent to: '-rltpog'
  -r, --recursive       Do not skip sub-directories, but synchronize
                        recursively.
  -l, --links           Copy symbolic links as symbolic links pointing to the
                        original source.
  -p, --perms           Preserve permissions.
  -o, --owner           Preserve owner.
  -g, --group           Preserve group.
  -t, --times           Preserve file modification times (requires -p).
  -x [EXCLUDE], --exclude [EXCLUDE]
                        Exclude all files matching the given pattern. Exclude
                        all files if this option is provided without any
                        argument.
  -I, --ignore-times    Never rely on file metadata such as the size or the
                        modification time when determining file differences.
  --size-only           Ignore modification times during file comparison.
                        Useful when synchronizing between file systems with
                        different timestamp resolution.
  --round-times         Round modification times during file comparison.
                        Useful when synchronizing between file systems with
                        different timestamp resolution.
  -n, --dry-run         Do not actually execute the synchronization. Increase
                        the output verbosity to see messages about what would
                        potentially happen.
  -u, --update          Skip files with newer modification time stamp.This is
                        a short-cut for: --strategy=update.

sync strategy:
  -s {always,never,update,Ask}, --strategy {always,never,update,Ask}
                        Specify a synchronization strategy, for differing
                        files.
  -k KEY, --key KEY     Specify a regular expression for keys that should be
                        overwritten as part of the project and job document
                        synchronization.
  --all-keys            Overwrite all conflicting keys. Equivalent to
                        `--key='.*'`.
  --no-keys             Never overwrite any conflicting keys.

select:
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Only synchronize jobs matching the filter.
  -j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
                        Only synchronize jobs with the given job ids.

update-cache

$ signac update-cache --help
usage: signac update-cache [-h]

Use this command to update the project's persistent state point cache.

options:
  -h, --help  show this help message and exit

view

$ signac view --help
usage: signac view [-h] [-p [PREFIX]] [-f FILTER [FILTER ...]]
                   [-j JOB_ID [JOB_ID ...]]
                   [path]

Generate a human readable set of paths representing state points in the workspace, e.g. view/param_name_1/param_value_1/param_name_2/param_value_2/job.

The leaf nodes of this directory structure are symlinks (named "job") into the corresponding job directory.

The jobs can be filtered using the query API used by the find command. For the full syntax, see:
https://docs.signac.io/en/latest/query.html#simplified-filter

All positional arguments must be provided before any keyword arguments. In particular, the prefix and path must be specified before arguments such as the filters, like in the following bash command:

$ signac view $PREFIX $VIEW_PATH -f $FILTERS -d $DOC_FILTERS.

positional arguments:
  path                  The path used for the generation of the linked view
                        hierarchy, defaults to '{{auto}}' (see
                        Project.export_to for information on how this is
                        expanded).

options:
  -h, --help            show this help message and exit
  -p [PREFIX], --prefix [PREFIX]
                        The path where the view is to be created. Defaults to
                        view.

select:
  -f FILTER [FILTER ...], --filter FILTER [FILTER ...]
                        Limit the view to jobs matching the filter.
  -j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
                        Limit the view to jobs with these job ids.