Commit b588e96f authored by tchervec's avatar tchervec
Browse files

added flowchart generation config elements

parent c0e5ef43
- Added config elements for generating flowchart
- Use new pyproj syntax without `init:`
- Port code to use `synpp`
......@@ -23,6 +23,24 @@ To clean, simply delete the environment directory (here `myenv`).
The starting point is ``, where some configuration options can be set. Right
now it is not very configurable, but should become more so in the future.
# Generating pipeline flowchart
To generate a json file which represents the graph of the pipeline, you need to specify the output path in the config file.
Simply add `flowchart_path: /path/to/flowchart.json` under the "General pipeline settings".
Then, the flowchart json will be saved at this path next time you run the pipeline until the last specified stage.
To only generate the flowchart without running the pipeline, add `dryrun: true` under the "General pipeline settings".
Setting `dryrun: false` will run the full pipeline.
To convert the json file into an image, you will need to use
[Pipeline Data Flow Plotter](
The full syntax is:
`python3 -j /path/to/flowchart.json -o /path/to/flowchart.png -g`
Type `python3 -h` for further explanations.
# Output
To create a full scenario, add the `` stage to the config file. The
# General pipeline settings
working_directory: /home/tchervec/Documents/data/switzerland/cache
flowchart_path: /home/tchervec/Documents/data/switzerland/flowchart.json
dryrun: true
# Requested stages
- data.microcensus.trips
# - data.microcensus.trips
# -
# - data.statpop.projections.households
# - data.statpop.scaled
- data.statpop.scaled
# These are configuration options that we use in the pipeline
# General pipeline settings
working_directory: /nas/ivtmatsim/scenarios/switzerland/data
flowchart_path: /path/to/flowchart.json
dryrun: false
# Requested stages
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment