Commit b588e96f authored by tchervec's avatar tchervec
Browse files

added flowchart generation config elements

parent c0e5ef43
**3.0.0** **3.0.0**
- Added config elements for generating flowchart
- Use new pyproj syntax without `init:` - Use new pyproj syntax without `init:`
- Port code to use `synpp` - Port code to use `synpp`
......
...@@ -23,6 +23,24 @@ To clean, simply delete the environment directory (here `myenv`). ...@@ -23,6 +23,24 @@ To clean, simply delete the environment directory (here `myenv`).
The starting point is `run.py`, where some configuration options can be set. Right The starting point is `run.py`, where some configuration options can be set. Right
now it is not very configurable, but should become more so in the future. now it is not very configurable, but should become more so in the future.
# Generating pipeline flowchart
To generate a json file which represents the graph of the pipeline, you need to specify the output path in the config file.
Simply add `flowchart_path: /path/to/flowchart.json` under the "General pipeline settings".
Then, the flowchart json will be saved at this path next time you run the pipeline until the last specified stage.
To only generate the flowchart without running the pipeline, add `dryrun: true` under the "General pipeline settings".
Setting `dryrun: false` will run the full pipeline.
To convert the json file into an image, you will need to use
[Pipeline Data Flow Plotter](https://gitlab.ethz.ch/ivt-vpl/populations/pipeline-data-flow-plotter).
The full syntax is:
`python3 visualize_pipeline.py -j /path/to/flowchart.json -o /path/to/flowchart.png -g`
Type `python3 visualize_pipeline.py -h` for further explanations.
# Output # Output
To create a full scenario, add the `matsim.final` stage to the config file. The To create a full scenario, add the `matsim.final` stage to the config file. The
......
# General pipeline settings # General pipeline settings
working_directory: /home/tchervec/Documents/data/switzerland/cache working_directory: /home/tchervec/Documents/data/switzerland/cache
flowchart_path: /home/tchervec/Documents/data/switzerland/flowchart.json
dryrun: true
# Requested stages # Requested stages
run: run:
- data.microcensus.trips # - data.microcensus.trips
# - matsim.run
# - data.statpop.projections.households # - data.statpop.projections.households
# - data.statpop.scaled - data.statpop.scaled
# These are configuration options that we use in the pipeline # These are configuration options that we use in the pipeline
config: config:
......
# General pipeline settings # General pipeline settings
working_directory: /nas/ivtmatsim/scenarios/switzerland/data working_directory: /nas/ivtmatsim/scenarios/switzerland/data
flowchart_path: /path/to/flowchart.json
dryrun: false
# Requested stages # Requested stages
run: run:
......
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment