1.5 Command Line Interface (CLI)¶
Note: This example is in the process of being updated.
The following section demonstrates how to use the signac command line interface (CLI). The CLI allows you to interact with your data space without python, which may be advantageous in various situations, e.g., for scripting or data exploration.
You will find that for many of the functions introduced earlier there is an equivalent CLI command.
The CLI is accessed via the top-level signac
command. You can get help about the various functions with the -h
or --help
argument.
[1]:
%%sh
signac --help
usage: signac [-h] [--debug] [--version] [-v] [-y]
{init,project,job,statepoint,diff,document,rm,move,clone,index,find,view,schema,shell,sync,import,export,update-cache,config}
...
signac aids in the management, access and analysis of large-scale
computational investigations.
positional arguments:
{init,project,job,statepoint,diff,document,rm,move,clone,index,find,view,schema,shell,sync,import,export,update-cache,config}
optional arguments:
-h, --help show this help message and exit
--debug Show traceback on error for debugging.
--version Display the version number and exit.
-v, --verbosity Set level of verbosity.
-y, --yes Answer all questions with yes. Useful for scripted
interaction.
To interact with a project on the command line, the current working directory needs to be within or below the project’s root directory. Let’s start by reseting the designated project root directory for this section of the tutorial.
[2]:
%rm -rf projects/tutorial/cli
%mkdir -p projects/tutorial/cli
Next we switch the current working directory to the project root directory.
[3]:
%cd projects/tutorial/cli
notebooks/projects/tutorial/cli
Then we initialize the project.
[4]:
%%sh
signac init TutorialCLIProject
Initialized project 'TutorialCLIProject'.
We can verify the project configuration using the signac project
command.
[5]:
%%sh
signac project
signac project --workspace
TutorialCLIProject
notebooks/projects/tutorial/cli/workspace
We access a job by providing the state point on the command line in JSON format †).
†) The JSON format requires double quotes for keys.
[6]:
%%sh
signac job '{"kT": 1.0, "p": 1.0, "N": 1000}'
ee617ad585a90809947709a7a45dda9a
By default this will print the associated job id to STDOUT.
Instead of the job id, we can also get the path to the job’s workspace.
[7]:
%%sh
signac job '{"kT": 1.0, "p": 1.0, "N": 1000}' --workspace
notebooks/projects/tutorial/cli/workspace/ee617ad585a90809947709a7a45dda9a
Please not, that obtaining the path in this way does not necessarily mean that the path exists. However, we can initialize the job and create the workspace using the -c
or --create
argument.
[8]:
%%sh
signac job '{"kT": 1.0, "p": 1.0, "N": 1000}' --create
ee617ad585a90809947709a7a45dda9a
We can use the signac statepoint
command to get the statepoint associated with the initialized job.
[9]:
%%sh
signac statepoint ee617ad585a90809947709a7a45dda9a
{"kT": 1.0, "p": 1.0, "N": 1000}
Usually we will not provide statepoints on the command line, but read them from a file. Let’s create a statepoint file with one statepoint:
[10]:
%%sh
echo '{"kT": 1.0, "p": 0.1, "N": 1000}' > statepoint.txt
cat statepoint.txt
{"kT": 1.0, "p": 0.1, "N": 1000}
We can pipe the content of this file into the signac CLI to get the corresponding job id.
[11]:
%%sh
cat statepoint.txt | signac job
5a6c687f7655319db24de59a2336eff8
We will reproduce the ideal gas project from section 1.1 to generate some data for the following examples.
[12]:
import signac
def V_idg(N, p, kT):
return N * kT / p
project = signac.get_project()
for p in 0.1, 1.0, 10.0:
sp = {"p": p, "kT": 1.0, "N": 1000}
job = project.open_job(sp)
job.document["V"] = V_idg(**sp)
We can use the signac find
command to find all jobs within our project’s workspace.
[13]:
%%sh
signac find
5a6c687f7655319db24de59a2336eff8
5a456c131b0c5897804a4af8e77df5aa
ee617ad585a90809947709a7a45dda9a
Just like with project.find_jobs()
we can provide a filter argument to find a subset of jobs matching the given filter. Here we get all jobs with a pressure of 0.1:
[14]:
%%sh
signac find '{"p": 0.1}'
5a6c687f7655319db24de59a2336eff8
In this example, that is of course only one job.
Similarly, we can also filter based on information in the job document. Here, we find all jobs that have a volume corresponding to a pressure of 1 (volume = 1000*1/1 = 1000).
[15]:
%%sh
signac find --doc-filter '{"V": 1000.0}'
ee617ad585a90809947709a7a45dda9a
Once again, this only returns one job in this case.
We can pipe signac find
results into signac statepoint
with xargs
to resolve the statepoints.
[16]:
%%sh
signac find | xargs signac statepoint
{"p": 0.1, "kT": 1.0, "N": 1000}
{"p": 10.0, "kT": 1.0, "N": 1000}
{"kT": 1.0, "p": 1.0, "N": 1000}
You will have noticed that each time we execute a find operation the data space is indexed anew.
This is no problem for small data spaces, however especially for larger data spaces, where the indexing process may be substantially expensive it’s advantageous to cache the index in a file.
[17]:
%%sh
signac project --index > index.txt
This index file can be used in conjunction with all functions that require a data space index, for example signac find
:
[18]:
%%sh
signac find -i index.txt '{"p": 0.1}'
5a6c687f7655319db24de59a2336eff8
Reading index from file 'index.txt'...
Or for instance when creating a linked view.
[19]:
%%sh
signac view -i index.txt ./view
Reading index from file 'index.txt'...
The signac view
command works exactly like project.create_linked_view
such that the ./view
directory now contains a linked view to the job workspaces.
[20]:
%ls view
p/
This concludes the first chapter of the tutorial. The next chapter introduces a few more advanced topics.