Using ChunkyPipes¶
ChunkyPipes is designed to make running pipelines as painless as possible.
Initializing ChunkyPipes¶
Although ChunkyPipes can run without initializing a hidden directory, doing so makes using ChunkyPipes and running pipelines much easier and more organized. By keeping pipeline files and various configurations in an internal hidden directory structure, ChunkyPipes abstracts out filepath details involved in running pipelines.
$ chunky init
If no argument is provided, ChunkyPipes initializes a hidden directory in the home directory. This is also where
ChunkyPipes looks by default to install, configure, and run pipelines. If the user wishes to initialize a hidden
directory at a location other than the home directory, a path argument may be specified. To change where
ChunkyPipes looks when operating, introduce a CHUNKY_HOME
environment variable and point it to the
directory containing the ChunkyPipes hidden directory.
$ chunky init /path/to/other/place
> Please set a CHUNKY_HOME environment variable to /path/to/other/place
$ export CHUNKY_HOME=/path/to/other/place
Note
The above export
statement will only persist for the life of the terminal session. To introduce a CHUNKY_HOME
environment variable permanently, add the export statement to ~/.bashrc
or the platform equivalent.
Installing Pipelines¶
To install pipelines into ChunkyPipes:
$ chunky install /path/to/ngs-pipeline.py
This will install the pipeline into the ChunkyPipes hidden directory, whether that be the default home directory or
the directory pointed to by CHUNKY_HOME
.
If the pipeline requires any Python package dependencies, chunky install
will prompt the user to install these
dependencies via pip. Though this step is optional, it’s likely that the installed pipeline won’t run without the
declared dependencies.
Warning
chunky install
will feed the developer-provided package name and version directly to pip using the --upgrade
option. The exact version specified will be installed, even if it’s an older version than a package that’s currently
installed. If this behavior isn’t desired, the user can pick and choose dependencies to install by running
chunky show
on the pipeline to get all dependencies and installing those desired.
Note
If the user is not in a virtual environment at the time chunky install
is run, sudo
or the platform
equivalent may need to be prepended to the command in order to install Python packages into system files.
To show a list of installed pipelines:
$ chunky list
This list also shows which pipelines have corresponding configuration files.
Configuring Pipelines¶
To configure a pipeline:
$ chunky configure <pipeline-name>
ChunkyPipes will present an interactive configuration, asking the user for any platform-specific information required by the pipeline.
$ chunky configure ngs-pipeline
> Full path to software1 []: (User enters) /path/to/soft1
> Provide a value for arg1 []: (User enters) 45
> Full path to software2 []: (User enters) /path/to/soft2
If no --location
parameter is given, the configuration file is
stored in the ChunkyPipes hidden directory as a JSON formatted file with the same base
filename as the pipeline. If the user doesn’t provide a --config
parameter when running a pipeline, ChunkyPipes
uses the config file in the hidden directory.
If the user is overwriting an existing configuration, the existing value will be displayed in square brackets
([]
) as a part of the prompt. Leaving this field blank will cause the existing value to be used.
As of version 0.2.0, ChunkyPipes interactive configuration supports TAB-completion for filesystem paths, but not left-and-right character seeking.
Showing Pipelines¶
To show information about a pipeline:
$ chunky show <pipeline-name>
This command will show the pipeline description, arguments, dependencies, configuration dictionary, and the current default configuration, if it exists.
Running Pipelines¶
To run a pipeline:
$ chunky run <pipeline-name or path> [-h] [--config CONFIG] [pipeline_args]
The pipeline can be either the name of an installed pipeline or a path to a Python file containing a properly
formatted Pipeline class. If an installed pipeline name is given without a --config
parameter,
both components will come from the
ChunkyPipes hidden directory. If a path is given, --config
must also be given a value.