Project Command Line Interface¶
Basic Help¶
$ python3 doc-project.py --help
usage: doc-project.py [-h] [-v] [--debug] {status,next,run,submit,exec} ...
positional arguments:
{status,next,run,submit,exec}
options:
-h, --help show this help message and exit
-v, --verbose Increase output verbosity.
--debug This option implies `-vv`.
status¶
$ python3 doc-project.py status --help
usage: doc-project.py status [-h] [-v] [--debug] [-j JOB_ID [JOB_ID ...]]
[-f FILTER [FILTER ...]] [--json] [-d] [-a]
[--only-incomplete-operations] [--stack] [-1]
[-e] [--pretty] [--full] [--no-overview]
[-m OVERVIEW_MAX_LINES] [-p [PARAMETERS ...]]
[--param-max-width PARAM_MAX_WIDTH]
[--eligible-jobs-max-lines ELIGIBLE_JOBS_MAX_LINES]
[--ignore-errors] [--output-format OUTPUT_FORMAT]
[--profile [PROFILE]] [--hide-progress]
[-o OPERATION [OPERATION ...]]
Parallelization of the status command can be controlled by setting the
flow.status_parallelization config value to 'thread' (default), 'none', or
'process'. To do this, execute `signac config set flow.status_parallelization
VALUE`.
options:
-h, --help show this help message and exit
-v, --verbose Increase output verbosity.
--debug This option implies `-vv`.
-j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
Only select jobs that match the given id(s).
-f FILTER [FILTER ...], --filter FILTER [FILTER ...]
Only select jobs that match the given state point
filter.
--ignore-errors Ignore errors that might occur when querying the
scheduler.
--profile [PROFILE] Collect statistics to determine code paths that are
responsible for the majority of runtime required for
status determination. Optionally provide a filename
pattern to select for what files to show result for.
Defaults to the main module. (requires pprofile)
--hide-progress Hide the progress bar
-o OPERATION [OPERATION ...], --operation OPERATION [OPERATION ...]
Select operation or groups that match the given
operation/group name(s). These are interpreted as
regular expressions.
view:
Specify how to format the status display.
--json Do not format the status display, but dump all data
formatted in JSON.
-d, --detailed Show a detailed view of all jobs and their labels and
operations.
-a, --all-operations Show information about all operations, not just active
or eligible ones.
--only-incomplete-operations
Only show information for jobs with incomplete
operations.
--stack Show labels and operations in separate rows.
-1, --one-line Show only one line per job.
-e, --expand Display job labels and job operations in two separate
tables.
--pretty
--full Show all available information (implies --detailed
--all-operations).
--no-overview Do not print an overview.
-m OVERVIEW_MAX_LINES, --overview-max-lines OVERVIEW_MAX_LINES
Limit the number of lines in the overview.
-p [PARAMETERS ...], --parameters [PARAMETERS ...]
Display select parameters of the job state point (with
optional prefix 'sp.') or job document (by using
prefix 'doc.') in the detailed view.
--param-max-width PARAM_MAX_WIDTH
Limit the width of each parameter row.
--eligible-jobs-max-lines ELIGIBLE_JOBS_MAX_LINES
Limit the number of eligible jobs that are shown.
--output-format OUTPUT_FORMAT
Set status output format: terminal, markdown, or html.
run¶
$ python3 doc-project.py run --help
usage: doc-project.py run [-h] [-v] [--debug] [-j JOB_ID [JOB_ID ...]]
[-f FILTER [FILTER ...]]
[-o OPERATION_NAME [OPERATION_NAME ...]] [-n NUM]
[--pretend] [--progress] [--num-passes NUM_PASSES]
[-t TIMEOUT] [--switch-to-project-root]
[-p [PARALLEL]]
[--order {none,by-job,cyclic,random}]
[--ignore-conditions {none,pre,post,all}]
options:
-h, --help show this help message and exit
-v, --verbose Increase output verbosity.
--debug This option implies `-vv`.
job-operation selection:
By default, all eligible operations for all jobs are selected. Use the
options in this group to reduce this selection.
-j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
Only select jobs that match the given id(s).
-f FILTER [FILTER ...], --filter FILTER [FILTER ...]
Only select jobs that match the given state point
filter.
-o OPERATION_NAME [OPERATION_NAME ...], --operation OPERATION_NAME [OPERATION_NAME ...]
Only select operation or groups that match the given
operation/group name(s). These are interpreted as
regular expressions.
-n NUM, --num NUM Limit the total number of operations/groups to be
selected. A group is considered to be one operation
even if it consists of multiple operations.
execution:
--pretend Do not actually execute commands, just show them.
--progress Display a progress bar during execution.
--num-passes NUM_PASSES
Specify how many times a particular job-operation may
be executed within one session (default=1). This is to
prevent accidental infinite loops, where operations
are executed indefinitely, because postconditions were
not properly set. Use -1 to allow for an infinite
number of passes.
-t TIMEOUT, --timeout TIMEOUT
A timeout in seconds after which the execution of one
operation is canceled.
--switch-to-project-root
Temporarily add the current working directory to the
python search path and switch to the root directory
prior to execution.
-p [PARALLEL], --parallel [PARALLEL]
Specify the number of cores to parallelize to.
Defaults to all available processing units.
--order {none,by-job,cyclic,random}
Specify the execution order of operations for each
execution pass.
--ignore-conditions {none,pre,post,all}
Specify conditions to ignore for eligibility check.
submit¶
$ python3 doc-project.py submit --help
usage: doc-project.py submit [-h] [-v] [--debug] [--pretend] [--force]
[--ignore-conditions {none,pre,post,all}]
[--ignore-conditions-on-execution {none,pre,post,all}]
[-j JOB_ID [JOB_ID ...]] [-f FILTER [FILTER ...]]
[-o OPERATION_NAME [OPERATION_NAME ...]] [-n NUM]
[-b [BUNDLE_SIZE]] [-p] [--template TEMPLATE]
[--template-help]
[flags ...]
positional arguments:
flags Flags to be forwarded to the scheduler.
options:
-h, --help show this help message and exit
-v, --verbose Increase output verbosity.
--debug This option implies `-vv`.
--pretend Do not really submit, but print the submission script
to screen.
--force Ignore all warnings and checks, just submit.
--ignore-conditions {none,pre,post,all}
Specify conditions to ignore for eligibility check.
--ignore-conditions-on-execution {none,pre,post,all}
Specify conditions to ignore after submitting. May be
useful for conditions that cannot be checked once
scheduled.
job-operation selection:
By default, all eligible operations for all jobs are selected. Use the
options in this group to reduce this selection.
-j JOB_ID [JOB_ID ...], --job-id JOB_ID [JOB_ID ...]
Only select jobs that match the given id(s).
-f FILTER [FILTER ...], --filter FILTER [FILTER ...]
Only select jobs that match the given state point
filter.
-o OPERATION_NAME [OPERATION_NAME ...], --operation OPERATION_NAME [OPERATION_NAME ...]
Only select operation or groups that match the given
operation/group name(s). These are interpreted as
regular expressions.
-n NUM, --num NUM Limit the total number of operations/groups to be
selected. A group is considered to be one operation
even if it consists of multiple operations.
bundling:
Bundle multiple operations for execution, e.g., to submit them all
together to a cluster job, or execute them in parallel within an execution
script.
-b [BUNDLE_SIZE], --bundle [BUNDLE_SIZE]
Bundle multiple operations for execution in a single
scheduler job. When this option is provided without an
argument, all eligible operations are combined into
one bundle.
-p, --parallel Execute all operations in a single bundle in parallel.
templating:
The execution and submission scripts are always generated from a script
which is by default called 'script.sh' and located within the default
template directory. The system uses a default template if none is
provided. The default template extends from a base template, which may be
different depending on the local compute environment, e.g., 'slurm.sh' for
an environment with SLURM scheduler. The name of the base template is
provided with the 'base_script' template variable.
--template TEMPLATE The name of the template file within the template
directory. The standard template directory is
'${{project_root}}/templates' and can be configured
with the 'template_dir' configuration variable.
Default: 'script.sh'.
--template-help Show information about the template context, including
available variables and filter functions; then exit.
exec¶
$ python3 doc-project.py exec --help
usage: doc-project.py exec [-h] [-v] [--debug] {} [job_id ...]
positional arguments:
{} The operation to execute.
job_id The job ids or aggregate ids in the FlowProject. Defaults to
all jobs and aggregates.
options:
-h, --help show this help message and exit
-v, --verbose Increase output verbosity.
--debug This option implies `-vv`.