To receive notifications about scheduled maintenance, please subscribe to the mailing-list gitlab-operations@sympa.ethz.ch. You can subscribe to the mailing-list at https://sympa.ethz.ch

setup.md 7.65 KB
Newer Older
1
2
3
4
### **Setup**

- [Get permissions](#get-permissions)
- [Local setup](#local-setup)
Xuqian Yan's avatar
Xuqian Yan committed
5
  - [Get the codes](#get-the-codes)
6
  - [Connect with the input database](#connect-with-the-input-database)
Xuqian Yan's avatar
Xuqian Yan committed
7
  - [Prepare software & license](#prepare-software-license)
Xuqian Yan's avatar
Xuqian Yan committed
8
  - [Run Nexus-e](#run-nexus-e)
9
10
11
12
- [Euler setup](#euler-setup)

#### Get permissions

13
14
Before setting up, you need permissions to access the codes and the input database. Email the following people to ask for the corresponding permissions.

Xuqian Yan's avatar
Xuqian Yan committed
15
```Note:: |:warning:| Currently, the codes are only accessible with an ETH account.
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
```

- Xuejiao Han (xuhan@eeh.ee.ethz.ch): 
  - permission for the **DistIv** repository 
  - access to the **PSL server** for the database (server name: itet-psl-s02)
- Jared Garrison (garrison@fen.ethz.ch): 
  - permissions for the **eMark** repository
  - permissions for the **Shared** repository,
  - username and password for the **database** 
- Blazhe Gjorgiev (gblazhe@ethz.ch): 
  - permission for the **Cascades** repository
  - permission for the **Run_Nexuse** repository
- Elena Raycheva (elena.raycheva@esc.ethz.ch):
  - permission for the **CentIv** repository
- Florian Landis  (landisf@ethz.ch):
  - permission for the **GemEl** repository
- Xuqian Yan (xuqian.yan@esc.ethz.ch):
Xuqian Yan's avatar
Xuqian Yan committed
33
  - permission for the overarching **Nexus-e** repository (<https://gitlab.ethz.ch/xyan/nexus-e.git>)
34

35
36
#### Local setup

Xuqian Yan's avatar
Xuqian Yan committed
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
Most the instructions below have been tested both on **Windows** and on **Mac**.

#####  1. Get the codes

The instructions here use **command lines**. If you have a GUI tool for git, you could use the corresponding operations in the GUI tool instead. An example using a GUI tool "GitAhead" can be found in section [Nexus-e repository instructions](nexuse_repo_instruction.html).

- Create a folder where you want to put the codes

- cd to that folder

- Clone the overarching repository with this command:

  ```git clone --recursive https://gitlab.ethz.ch/xyan/nexus-e.git```

  You will be asked for your username and password for the repository and its submodules. Input your ETH username and password.

Xuqian Yan's avatar
Xuqian Yan committed
53
In the end, the folder will look like the screenshot below. Note: You might not see the hidden files whose names start with a dot; the `UserDBInfo.txt` file will be added in the sction [Connect with the input database](#connect-with-the-input-database); the `Results` folder will be automatically generated when you run Nexus-e.
Xuqian Yan's avatar
Xuqian Yan committed
54
55
56
57
58
59

![](images/nexus-e-folder.png)


##### 2. Connect with the input database

60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
- Get access to the database  
Follow the section [Get permissions](#get-permissions) to get access to the database server (PSL server: itet-psl-s02) and your username & password for the database.

- Create a `UserDBInfo.txt` file in the folder where you put the codes as shown in [Get the codes](#get-the-codes). The file should contain four lines:
```
itet-psl-s02
3306
YOUR_DATABASE_USERNAME
YOUR_DATABASE_PASSWORD
```
An example can be found in `Shared/connectionInfoPSL.txt`.

- View the database
  - Install MySQL and MySQL Workbench
  - Connect to ETH VPN
  - Connect to the database with MySQL Workbench
    - Add a new MySQL connection  
    ![](images/mysql-workbench-1.png)
    - Input connection information  
    ![](images/mysql-workbench-2.png)
    - Click the button `Test Connection`. You will be asked to input your database password. Then you will see a seccess message:  
    ![](images/mysql-workbench-3.png)  
    ```eval_rst
Xuqian Yan's avatar
Xuqian Yan committed
83
    |:warning:| By default, everyone has write permission to the database. Therefore, to be safe, always make a local copy of the database before playing with it - i.e., first “Dump” it to a local folder, then import it to your personal MySQL account. These can be done in this window:  
84
85
86
    ```  
    ![](images/mysql-workbench-4.png)

Xuqian Yan's avatar
Xuqian Yan committed
87
88
89

##### 3. Prepare software & license

Xuqian Yan's avatar
Xuqian Yan committed
90
- **Matlab** (2018a or higher)
Xuqian Yan's avatar
Xuqian Yan committed
91
92
93
94

  - Set up database connector

    - Download the **MySQL connector** from [here](https://dev.mysql.com/downloads/connector/j/).  
Xuqian Yan's avatar
Xuqian Yan committed
95
    Or, if you have access to the Nexus-e polybox, use the `mysql-connector-java-8.0.18.zip` provided in the folder `02_Model/02_Model_Setup_Instruction`(**Unzip** the file before the next step).
Xuqian Yan's avatar
Xuqian Yan committed
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111

    - Copy the MySQL connector folder (e.g. mysql-connector-java-8.0.18) into a folder at your preference.  
    We recommend to put it into the **Matlab preferences folder**, which you can find by typing in Matlab Command Window ```prefdir```.

    - Create a **javaclasspath.txt** file in the **Matlab preferences folder**.

    - In the **javaclasspath.txt** file, put the path to the connector .jar file.  
    E.g., On Blazhe's Windows computer the path is `C:\Users\gblazhe\AppData\Roaming\MathWorks\MATLAB\R2018a\mysql-connector-java-8.0.18\mysql-connector-java-8.0.18.jar`.

    - Reload Matlab
    
    - Test whether a database connector is set up successfully. Write the following commands in Matlab (substitue `YOUR_USERNAME` and `YOUR_PASSWORD` with your credentials for the database). If the second command returns `1`, it means success.  
    ```
    conn = database("sys", YOUR_USERNAME, YOUR_PASSWORD, 'Vendor', 'MySQL', 'Server', "itet-psl-s02");
    isopen(conn)
    ```
Xuqian Yan's avatar
Xuqian Yan committed
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
  
- **Python** (3.x)  
  ```eval_rst
  |:warning:| Note: we are now in the phase of transferring from python 2.x to python 3.x. Therefore, it is recommended to use python 3.x locally and identify potential issues for the transfer.

  ```  
  - Python packages: install the required packages listed in `requirements.txt`.
  - Gurobi (9.x): get a free Gurobi academic license <https://www.gurobi.com/academia/academic-program-and-licenses>
  - Python path: If you use Mac or if you want to run Nexus-e in a python virtual environment, you need to configure Python path in Matlab. Otherwise, you could skip this step. 
    - Make a copy of `user_path_example_conf.m` in the folder `Run_Nexuse`. Name the copy as `user_path_conf.m`. (Note: You should only work with the local copy, because the original `xxx_example_conf.m` will be synchronized to git and it shouldn't be changed.)
    - Find out your python path (it varies largely depending on your PC's operating system and how you installed python). Uncomment and adapt the variable `PATH_PYTHON` in `user_path_conf.m` accordingly.
    - If you have a python virtual environment, you could configure it following the instructions in `user_path_conf.m`.

- **GAMS**
  - Download from [here](https://www.gams.com/download/)
  - Set GAMS License
    - Use the `gamslice.txt` provided in polybox `02_Model/02_Model_Setup_Instruction`.
    - Install the license: [here](https://support.gams.com/installation:how_do_i_install_a_license_file) or [here](https://www.youtube.com/watch?v=vSe3YGkUVoc) or (for MacOS) [here](https://www.gams.com/latest/docs/UG_MAC_INSTALL.html).
  - GAMS path
    - If you haven't done so, make a copy of `user_path_example_conf.m` in the folder `Run_Nexuse`. Name the copy as `user_path_conf.m`. (Note: You should only work with the local copy, because the original `xxx_example_conf.m` will be synchronized to git and it shouldn't be changed.)
    - Find out your GAMS path (it varies largely depending on your PC's operating system and the version of GAMS). Uncomment and adapt the variable `PATH_GAMS` in `user_path_conf.m` accordingly.
Xuqian Yan's avatar
Xuqian Yan committed
133
134
135


##### 4. Run Nexus-e
136

Xuqian Yan's avatar
Xuqian Yan committed
137
138
139
140
141
142
- Connect to ETH VPN (in order to connect with the database)
- Open Matlab
- Run the script `run_Nexuse.m` in the `Run_Nexuse` folder
  - You could change the variables `scen`, `tpRes`, and `limDifference` as instructed in the script
  - Consult Blazhe Gjorgiev (gblazhe@ethz.ch) if you want to run other scripts in the `Run_Nexuse` folder.

143
#### Euler setup
Xuqian Yan's avatar
Xuqian Yan committed
144
145
146
147
148
149
##### 1. Join the usergroup

##### 2. Get the codes
##### 3. Prepare software & license
##### 4. Connect with the input database
##### 5. Run Nexus-e
150
151