Commit 8a08f92f authored by clivings's avatar clivings
Browse files

Merge branch 'master' into 'clarissa'

# Conflicts:
#   src/main/java/ch/ethz/matsim/projects/astra_2018_002/RunASTRA2018002.java
parents 16fe3e4f e419fb26
# How to create the partial scenarios
This repository only contains code to run the requested scenarios in the ASTRA 2018/002 project. For the sake of avoiding code duplication, the scenario cutting code is still located in the [baseline_scenario](https://github.com/matsim-eth/baseline_scenario) repository, specifically in the (synpop)[https://github.com/matsim-eth/baseline_scenario/tree/synpop] branch.
The repository can be transformed into a "fat jar" by calling:
```sh
mvn -Pstandalone package
```
The basis for creating a partial scenario is the full Switzerland scenario in arbitrary sample size. How to create such a full Switzerland scenario is explained in the corresponding (Synthetic populaton of Switzerland)[https://gitlab.ethz.ch/ivt-vpl/populations/ch-zh-synpop] repository. This will create a scenario including `switzerland_population.xml.gz`, `switzerland_network.xml.gz` and so forth.
The `baseline_scenario` code can then be used to cut this population and network to a specific diameter. The relevant run script is `ch.ethz.matsim.baseline_scenario.CutScenario`. It has the following parameters:
- *input-config-path*: Path to `switzerland_config.xml`
- *shapefile-path*: Path to a shape file that contains the scenario boundary
- *shapefile-attribute*: Attribute of the shapes contained in the shape file that defines the scenario id
- *shapefile-value*: Defines which value the given attribute should have (see further below)
- *prefix*: Prefix in the filenames of the generated scenario, e.g. `prefix_population.xml.gz`
- *output-path*: Output path (directory) of the new scenario. The directoy must exist.
The concept is therefore as follows: One has to provide a shape file to the script. The shapes (polygons) in that file all have a specific attribute (e.g. `scenario_id`). Then the script will look up the one specific polygon in that file that has the given `shapefile-value` for that attribute, e.g. `zurich_30km` or similar. This way one shape file can contain multiple scenario diameters, which avoids having a shape file for each of the scenarios.
For ASTRA 2018/002, there is already a shape file in `gis/scenarios/scenarios.shp`. It is created from the initial `Gemeinden_fuer_Raumtypen.shp` using the script `gis/create_scenario_shapes.py`. To create a "ländlich ungerichtet" (lu) scenario, one can call, for instance:
```
java -Xmx100G -cp baseline_scenario/baseline_scenario-0.0.1-synpop-SNAPSHOT.jar ch.ethz.matsim.baseline_scenario.CutScenario --input-config-path switzerland_config.xml --shapefile-path /path/to/gis/scenarios/scenarios.shp --shapefile-attribute scenario --shapefile-value lu --prefix lu_ --output-path lu_scenario
```
# How to create a long-distance scenario
To create the long-distance scenario, the script `ch.ethz.matsim.projects.astra_2018_002.long_distance.MakeLongDistanceScenario` is used. It receives a number of parameters:
- *network-path*: Path to the MATSim network file (e.g. `switzerland_network.xml.gz`)
- *population-path*: Path to the MATSim populaton file (e.g. `switzerland_population.xml.gz`)
- *output-path*: Output path of the long-distance population (e.g. `long_distance_population.xml.gz`)
- *osm-types*: `osm:highway` types that are considered as "long distance roads* (optional, default: "motorway,trunk")
- *threads*: Number of threads to use (optional, by default maximum is used)
- *batch-size*: Batch size in parallel processing (optional, default is 100)
The script will do the following:
1. Route every trip of every person using freespeed travel times
2. Find all persons that touch a link with the given `osm-types` at any point during their daily plan
3. Remove all remaining persons (those not interacting with "long distance roads")
4. Write out the reduced population
For a 0.1% sample that script removed around 1/3 of the agents. Remaining steps to verify the process are: https://gitlab.ethz.ch/ivt-vpl/astra_2018_002/milestones/3
import pandas as pd
import geopandas as gpd
import numpy as np
import shapely.geometry as geo
import sys
operating_areas_path = sys.argv[1]
output_path = sys.argv[2]
service_area_attribute_name = sys.argv[3]
waiting_time_zone_attribute_name = sys.argv[4]
hexagon_radius = int(sys.argv[5])
df_operating_areas = gpd.read_file(operating_areas_path)
corners = [
np.array([np.cos(30.0 * np.pi / 180.0), np.sin(30.0 * np.pi / 180.0)]) * hexagon_radius,
np.array([np.cos(90.0 * np.pi / 180.0), np.sin(90.0 * np.pi / 180.0)]) * hexagon_radius,
np.array([np.cos(150.0 * np.pi / 180.0), np.sin(150.0 * np.pi / 180.0)]) * hexagon_radius,
np.array([np.cos(210.0 * np.pi / 180.0), np.sin(210.0 * np.pi / 180.0)]) * hexagon_radius,
np.array([np.cos(270.0 * np.pi / 180.0), np.sin(270.0 * np.pi / 180.0)]) * hexagon_radius,
np.array([np.cos(330.0 * np.pi / 180.0), np.sin(330.0 * np.pi / 180.0)]) * hexagon_radius
]
offset_x = 2.0 * hexagon_radius * np.cos(30.0 * np.pi / 180.0)
t = 2.0 * hexagon_radius * np.cos(30.0 * np.pi / 180.0) / np.sqrt(3.0)
offset_y = hexagon_radius + 0.5 * t
geometry = []
for area_name, area_geometry in zip(df_operating_areas[service_area_attribute_name], df_operating_areas["geometry"]):
min_x, min_y, max_x, max_y = area_geometry.bounds
max_i = int(np.ceil((max_x - min_x) / offset_x)) + 1
max_j = int(np.ceil((max_y - min_y) / offset_y)) + 1
index = 0
for i in range(max_i):
for j in range(max_j):
centroid = np.array([
min_x + offset_x * i + (0.5 * offset_x if j % 2 == 0 else 0.0),
min_y + offset_y * j
])
hexagon = geo.Polygon(corners + centroid)
if hexagon.intersects(area_geometry):
geometry.append((area_name, "%s_%d" % (area_name, index), hexagon))
index += 1
df = pd.DataFrame.from_records(geometry, columns = [service_area_attribute_name, waiting_time_zone_attribute_name, "geometry"])
df = gpd.GeoDataFrame(df, crs = {"init": "EPSG:2056"})
df.to_file(output_path)
import pandas as pd
import geopandas as gpd
df = gpd.read_file("scenarios/Gemeinden_fuer_Raumtypen.shp")
df.crs = {"init": "EPSG:2056"}
df = df[["geometry", "Gebietstyp"]]
scenarios = {
"sa": ("S", "A"),
"lu": ("LU",),
"lg": ("LG",)
}
for scenario, types in scenarios.items():
df.loc[df["Gebietstyp"].isin(types), "scenario"] = scenario
df = df.dissolve(by = "scenario").reset_index()[["scenario", "geometry"]]
df.to_file("scenarios/scenarios.shp")
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
This source diff could not be displayed because it is too large. You can view the blob instead.
ISO-8859-1
\ No newline at end of file
PROJCS["CH1903+_LV95",GEOGCS["GCS_CH1903+",DATUM["D_CH1903+",SPHEROID["Bessel_1841",6377397.155,299.1528128]],PRIMEM["Greenwich",0],UNIT["Degree",0.017453292519943295]],PROJECTION["Hotine_Oblique_Mercator_Azimuth_Center"],PARAMETER["latitude_of_center",46.95240555555556],PARAMETER["longitude_of_center",7.439583333333333],PARAMETER["azimuth",90],PARAMETER["scale_factor",1],PARAMETER["false_easting",2600000],PARAMETER["false_northing",1200000],UNIT["Meter",1]]
\ No newline at end of file
package ch.ethz.matsim.projects.astra_2018_002;
import org.matsim.core.config.ReflectiveConfigGroup;
public class ASTRAConfigGroup extends ReflectiveConfigGroup {
public final static String GROUP_NAME = "astra";
private final static String ENABLE_AVS = "enableAvs";
private final static String TAXI_FLEET_SIZE = "taxiFleetSize";
private final static String POOLED_FLEET_SIZE = "pooledFleetSize";
private final static String SERVICE_AREA_ATTRIBUTE = "serviceAreaAttribute";
private final static String ACTIVE_SERVICE_AREA = "activeServiceArea";
private final static String SERVICE_AREA_SHAPEFILE = "serviceAreaShapefile";
private final static String WAITING_TIME_ZONE_SHAPEFILE = "waitingTimeZoneShapefile";
private final static String WAITING_TIME_ZONE_IDENTIFIER_ATTRIBUTE = "waitingTimeZoneIdentifierAttribute";
private boolean enableAVs = false;
private int taxiFleetSize = 100;
private int pooledFleetSize = 0;
private String serviceAreaAttribute = "OAREA";
private String activeServiceArea = "ZH";
private String serviceAreaShapefile = "service_area.shp";
private String waitingTimeZoneIdentifierAttribute = "WZONE";
private String waitingTimeZoneShapefile = "waiting_time_areas.shp";
public ASTRAConfigGroup() {
super(GROUP_NAME);
}
@StringGetter(ENABLE_AVS)
public boolean getEnableAVs() {
return enableAVs;
}
@StringSetter(ENABLE_AVS)
public void setEnableAVs(boolean enableAVs) {
this.enableAVs = enableAVs;
}
@StringGetter(TAXI_FLEET_SIZE)
public int getTaxiFleetSize() {
return taxiFleetSize;
}
@StringSetter(TAXI_FLEET_SIZE)
public void setTaxiFleetSize(int taxiFleetSize) {
this.taxiFleetSize = taxiFleetSize;
}
@StringGetter(POOLED_FLEET_SIZE)
public int getPooledFleetSize() {
return pooledFleetSize;
}
@StringSetter(POOLED_FLEET_SIZE)
public void setPooledFleetSize(int pooledFleetSize) {
this.pooledFleetSize = pooledFleetSize;
}
@StringGetter(SERVICE_AREA_ATTRIBUTE)
public String getServiceAreaAttribute() {
return serviceAreaAttribute;
}
@StringSetter(SERVICE_AREA_ATTRIBUTE)
public void setServiceAreaAttribute(String serviceAreaAttribute) {
this.serviceAreaAttribute = serviceAreaAttribute;
}
@StringGetter(ACTIVE_SERVICE_AREA)
public String getActiveServiceArea() {
return activeServiceArea;
}
@StringSetter(ACTIVE_SERVICE_AREA)
public void setActiveServiceArea(String activeServiceArea) {
this.activeServiceArea = activeServiceArea;
}
@StringGetter(SERVICE_AREA_SHAPEFILE)
public String getServiceAreaShapefile() {
return serviceAreaShapefile;
}
@StringSetter(SERVICE_AREA_SHAPEFILE)
public void setServiceAreaShapefile(String serviceAreaShapefile) {
this.serviceAreaShapefile = serviceAreaShapefile;
}
@StringGetter(WAITING_TIME_ZONE_SHAPEFILE)
public String getWaitingTimeZoneShapefile() {
return waitingTimeZoneShapefile;
}
@StringSetter(WAITING_TIME_ZONE_SHAPEFILE)
public void setWaitingTimeZoneShapefile(String waitingTimeZoneShapefile) {
this.waitingTimeZoneShapefile = waitingTimeZoneShapefile;
}
@StringGetter(WAITING_TIME_ZONE_IDENTIFIER_ATTRIBUTE)
public String getWaitingTimeZoneIdentifierAttribute() {
return waitingTimeZoneIdentifierAttribute;
}
@StringSetter(WAITING_TIME_ZONE_IDENTIFIER_ATTRIBUTE)
public void setWaitingTimeZoneIdentifierAttribute(String waitingTimeZoneIdentifierAttribute) {
this.waitingTimeZoneIdentifierAttribute = waitingTimeZoneIdentifierAttribute;
}
}
package ch.ethz.matsim.projects.astra_2018_002;
import org.matsim.core.controler.AbstractModule;
import org.matsim.core.controler.corelisteners.PlansScoring;
public class ASTRAModule extends AbstractModule {
@Override
public void install() {
bind(PlansScoring.class).toInstance(new PlansScoring() {
});
}
}
package ch.ethz.matsim.projects.astra_2018_002;
import java.util.ArrayList;
import java.util.Collection;
import org.matsim.contrib.dynagent.run.DynActivityEnginePlugin;
import org.matsim.core.config.Config;
import org.matsim.core.controler.AbstractModule;
import org.matsim.core.mobsim.qsim.AbstractQSimPlugin;
import org.matsim.core.mobsim.qsim.PopulationPlugin;
import org.matsim.core.mobsim.qsim.TeleportationPlugin;
import org.matsim.core.mobsim.qsim.changeeventsengine.NetworkChangeEventsPlugin;
import org.matsim.core.mobsim.qsim.messagequeueengine.MessageQueuePlugin;
import org.matsim.core.mobsim.qsim.qnetsimengine.QNetsimEnginePlugin;
import com.google.inject.Provides;
import com.google.inject.Singleton;
import ch.ethz.matsim.av.framework.AVQSimPlugin;
import ch.ethz.matsim.baseline_scenario.transit.simulation.BaselineTransitPlugin;
import ch.ethz.matsim.projects.astra_2018_002.av.dynamics.AVModesQSimPlugin;
public class ASTRAQSimModule extends AbstractModule {
@Override
public void install() {
}
@Provides
@Singleton
public Collection<AbstractQSimPlugin> provideAbstractQSimPlugins(Config config) {
final Collection<AbstractQSimPlugin> plugins = new ArrayList<>();
plugins.add(new MessageQueuePlugin(config));
plugins.add(new DynActivityEnginePlugin(config));
plugins.add(new QNetsimEnginePlugin(config));
if (config.network().isTimeVariantNetwork()) {
plugins.add(new NetworkChangeEventsPlugin(config));
}
if (config.transit().isUseTransit()) {
plugins.add(new BaselineTransitPlugin(config));
}
plugins.add(new TeleportationPlugin(config));
plugins.add(new PopulationPlugin(config));
plugins.add(new AVQSimPlugin(config));
plugins.add(new AVModesQSimPlugin(config));
return plugins;
}
}
package ch.ethz.matsim.projects.astra_2018_002;
import java.io.File;
import org.matsim.api.core.v01.Scenario;
import org.matsim.contrib.dvrp.run.DvrpConfigGroup;
import org.matsim.contrib.dvrp.trafficmonitoring.DvrpTravelTimeModule;
import org.matsim.core.config.Config;
import org.matsim.core.config.ConfigUtils;
import org.matsim.core.controler.Controler;
import org.matsim.core.scenario.ScenarioUtils;
import ch.ethz.matsim.av.framework.AVConfigGroup;
import ch.ethz.matsim.av.framework.AVModule;
import ch.ethz.matsim.baseline_scenario.config.CommandLine;
import ch.ethz.matsim.baseline_scenario.config.CommandLine.ConfigurationException;
import ch.ethz.matsim.baseline_scenario.traffic.BaselineTrafficModule;
import ch.ethz.matsim.baseline_scenario.transit.BaselineTransitModule;
import ch.ethz.matsim.baseline_scenario.transit.routing.DefaultEnrichedTransitRoute;
import ch.ethz.matsim.baseline_scenario.transit.routing.DefaultEnrichedTransitRouteFactory;
import ch.ethz.matsim.projects.astra_2018_002.av.AVConfigurator;
import ch.ethz.matsim.projects.astra_2018_002.av.AssignPrivatAVs;
import ch.ethz.matsim.projects.astra_2018_002.av.operating_area.OperatingAreaModule;
import ch.ethz.matsim.projects.astra_2018_002.av.operator.OperatorModule;
import ch.ethz.matsim.projects.astra_2018_002.flow_efficiency.FlowEfficiencyConfigurator;
import ch.ethz.matsim.projects.astra_2018_002.mode_choice.SwissDiscreteModeChoiceModule;
import ch.ethz.matsim.projects.astra_2018_002.mode_choice.SwissModeChoiceConfigurator;
......@@ -31,14 +23,21 @@ public class RunASTRA2018002 {
static public void main(String[] args) throws ConfigurationException {
CommandLine cmd = new CommandLine.Builder(args) //
.requireOptions("config-path", "model") //
.allowOptions("use-route-choice", "use-only-significant", "operator-path") //
.allowOptions("use-route-choice", "use-only-significant", "utility-parameters", "crossing-penalty") //
.allowPrefixes(SwissDiscreteModeChoiceModule.COMMAND_LINE_PREFIX) //
.build();
// ASTRAConfigGroup astraConfig = new ASTRAConfigGroup();
// astraConfig.setActiveServiceArea("zurich");
// astraConfig.setPooledFleetSize(0);
// astraConfig.setTaxiFleetSize(100);
// astraConfig.setServiceAreaShapefile("gis/operating_areas.shp");
// astraConfig.setServiceAreaAttribute("OAREA");
// astraConfig.setWaitingTimeZoneShapefile("gis/waiting_time_areas.shp");
// astraConfig.setWaitingTimeZoneIdentifierAttribute("WZONE");
// Load config
Config config = ConfigUtils.loadConfig(cmd.getOptionStrict("config-path"), new DvrpConfigGroup(),
new AVConfigGroup());
Config config = ConfigUtils.loadConfig(cmd.getOptionStrict("config-path"));
// Adjust config
SwissModeChoiceConfigurator.configure(config);
......@@ -59,22 +58,21 @@ public class RunASTRA2018002 {
FlowEfficiencyConfigurator.defineVehicleTypes(scenario);
// Set up controller
Controler controler = new Controler(scenario);
controler.addOverridingModule(new DvrpTravelTimeModule());
controler.addOverridingModule(new SwissRailRaptorModule());
controler.addOverridingModule(new AVModule());
controler.addOverridingModule(new OperatingAreaModule());
controler.addOverridingModule(new OperatorModule(cmd.getOption("operator-path").map(File::new)));
controler.addOverridingModule(new BaselineTransitModule());
// TODO: Consolidate QSim plugins
Controler controller = new Controler(scenario);
controller.addOverridingModule(new SwissRailRaptorModule());
controller.addOverridingModule(new BaselineTransitModule());
controller.addOverridingModule(new ASTRAModule());
controller.addOverridingModule(
new BaselineTrafficModule(cmd.getOption("crossing-penalty").map(Double::parseDouble).orElse(3.0)));
AVConfigurator.configureController(controller);
UtilitySet utilitySet = Enum.valueOf(UtilitySet.class, cmd.getOptionStrict("model"));
boolean useRouteChoice = cmd.getOption("use-route-choice").map(Boolean::parseBoolean).orElse(true);
boolean useOnlySignificant = cmd.getOption("use-only-significant").map(Boolean::parseBoolean).orElse(false);
controler
.addOverridingModule(new SwissDiscreteModeChoiceModule(utilitySet, useRouteChoice, useOnlySignificant));
controller.addOverridingModule(
new SwissDiscreteModeChoiceModule(utilitySet, useRouteChoice, useOnlySignificant, cmd));
// Run
controler.run();
controller.run();
}
}
package ch.ethz.matsim.projects.astra_2018_002.analysis.flow;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.matsim.api.core.v01.Id;
import org.matsim.api.core.v01.events.LinkEnterEvent;
import org.matsim.api.core.v01.events.handler.LinkEnterEventHandler;
import org.matsim.api.core.v01.network.Link;
import org.matsim.vehicles.Vehicle;
import org.matsim.vehicles.Vehicles;
import ch.ethz.matsim.projects.astra_2018_002.flow_efficiency.FlowEfficiencyCalculator;
import ch.ethz.matsim.projects.astra_2018_002.flow_efficiency.FlowEfficiencyConfigurator;
public class FlowListener implements LinkEnterEventHandler {
private final double startTime;
private final double interval;
private final int numberOfLinks;
private final int numberOfBins;
private final Item[][] timeItems;
private final Item[] totalItems;
private final Map<Id<Link>, Integer> linkMap = new HashMap<>();
private final List<Link> links;
private final Set<Id<Link>> linkIds;
private final FlowEfficiencyCalculator flowEfficiencyCalculator;
private final Vehicle CONVENTIONAL_VEHICLE;
private final Vehicle PRIVATE_AV;
private final Vehicle SHARED_AV;
public FlowListener(Vehicles vehicles, Collection<Link> links, double startTime, double endTime, double interval,
FlowEfficiencyCalculator flowEfficiencyCalculator) {
this.startTime = startTime;
this.interval = interval;
this.numberOfBins = (int) Math.ceil((endTime - startTime) / interval);
this.numberOfLinks = links.size();
this.links = links.stream().collect(Collectors.toList());
this.linkIds = links.stream().map(Link::getId).collect(Collectors.toSet());
for (int index = 0; index < links.size(); index++) {
linkMap.put(this.links.get(index).getId(), index);
}
this.CONVENTIONAL_VEHICLE = vehicles.getFactory().createVehicle(Id.createVehicleId("conventional"),
vehicles.getVehicleTypes().get(FlowEfficiencyConfigurator.CONVENTIONAL_VEHICLE_TYPE_ID));
this.PRIVATE_AV = vehicles.getFactory().createVehicle(Id.createVehicleId("private_av"),
vehicles.getVehicleTypes().get(FlowEfficiencyConfigurator.PRIVATE_AV_VEHICLE_TYPE_ID));
this.SHARED_AV = vehicles.getFactory().createVehicle(Id.createVehicleId("shared_av"),
vehicles.getVehicleTypes().get(FlowEfficiencyConfigurator.SHARED_AV_VEHICLE_TYPE_ID));
this.totalItems = new Item[numberOfLinks];
this.timeItems = new Item[numberOfLinks][numberOfBins];
this.flowEfficiencyCalculator = flowEfficiencyCalculator;
for (int linkIndex = 0; linkIndex < numberOfLinks; linkIndex++) {
totalItems[linkIndex] = new Item();
for (int timeIndex = 0; timeIndex < numberOfBins; timeIndex++) {
timeItems[linkIndex][timeIndex] = new Item();
}
}
}
public class Item {
public int numberOfConventionalVehicles;
public int numberOfPrivateAutomatedVehicles;
public int numberOfSharedAutomatedVehicles;
public double numberOfPCUs;
}
@Override
public void handleEvent(LinkEnterEvent event) {
if (linkIds.contains(event.getLinkId())) {
int linkIndex = linkMap.get(event.getLinkId());
Link link = links.get(linkIndex);
Vehicle vehicle = CONVENTIONAL_VEHICLE;
String plainVehicleId = event.getVehicleId().toString();
if (plainVehicleId.startsWith("av_")) {
vehicle = SHARED_AV;
} else if (plainVehicleId.startsWith("prav_")) {
vehicle = PRIVATE_AV;
}
Item totalItem = totalItems[linkIndex];
totalItem.numberOfPCUs += flowEfficiencyCalculator.calculateFlowEfficiency(vehicle, link);
if (vehicle == CONVENTIONAL_VEHICLE) {
totalItem.numberOfConventionalVehicles++;
} else if (vehicle == SHARED_AV) {
totalItem.numberOfPrivateAutomatedVehicles++;
} else {
totalItem.numberOfSharedAutomatedVehicles++;
}
int timeIndex = (int) Math.floor((event.getTime() - startTime) / interval);
if (timeIndex > -1 && timeIndex < numberOfBins) {
Item timeItem = timeItems[linkIndex][timeIndex];
timeItem.numberOfPCUs += flowEfficiencyCalculator.calculateFlowEfficiency(vehicle, link);
if (vehicle == CONVENTIONAL_VEHICLE) {
timeItem.numberOfConventionalVehicles++;
} else if (vehicle == SHARED_AV) {
timeItem.numberOfPrivateAutomatedVehicles++;
} else {
timeItem.numberOfSharedAutomatedVehicles++;
}
}
}
}
public int getNumberOfLinks() {
return numberOfLinks;
}
public int getNumberOfBins() {
return numberOfBins;
}
public Item getTotalItem(Id<Link> linkId) {
int linkIndex = linkMap.get(linkId);
if (linkIndex == -1) {
throw new IllegalStateException();
}
return totalItems[linkIndex];
}
public Item[] getTimeItems(Id<Link> linkId) {
int linkIndex = linkMap.get(linkId);
if (linkIndex == -1) {
throw new IllegalStateException();
}
return timeItems[linkIndex];
}
}
package ch.ethz.matsim.projects.astra_2018_002.analysis.flow;
import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;