# Next steps

In this part, you will learn about scenario configuration files and the command line interface.

## Scenario configuration

A midas co-simulation is highly configurable with so-called scenario configuration files. If you have followed this guide from the beginning, you should by now have run your first simulation with midas.

Inspect your current directory and you will find a new directory _outputs and inside that directory a file called midasmv_cfg.yml that contains the full configuration of the scenario you ran before. Note, since the file is created automatically, it is kind of unordered (actually, the keys are ordered alphabetically). If you plan to tinker around with that file, you should rename it. Otherwise, it will be overwritten when you start the midasmv scenario again.

Keep this in mind. However, for the purpose of this guide, we will start with a new and empty configuration.

## Runtime configuration

Remember, we created a runtime configuration in the first step. Now it’s time to have a short look at it. If you decided to use the default configuration, you will find the runtime configuration midas-runtime-conf.yml at

- Linux: ~/.config/midas/

- Windows: C:\Users\%USERNAME%\AppData\Local\OFFIS\midas\

Open it with the text editor of your choice. Most of the stuff configured here is not important for now. Scroll down to bottom of the file, where you find path definitions. This contains three path definitions:

- data_path: This path specifies where the datasets are located. Per default you find this folder at the same location like the runtime configuration. You should not change afterwards. Otherwise, the datasets will be downloaded again (unless you move the old data folder to the new location).

- output_path: This path specifies where output files will be created. We already saw the midasmv_cfg.yml file that was created by the demo scenario. Per default, an output folder _outputs is created in your current working directory. You can change this value to an absolute path if you like or choose a different name. Relative paths will always use the current working directory as base.

- scenario_path: This is the configuration we’re looking for. This path specifies where midas looks for scenario configuration files. Per default, this folder is named midas_scenarios and is located in the same directory like the runtime configuration. It is recommended to change it to any location that has a convenient path for your workflow (especially for Windows, the default location might be too hidden for convenient access), e.g., point it to your Documents folder.

## Creating your own configuration

Now, create an empty text document and rename it to my_first_scenario.yml. Open it and paste the following content:

    my_first_midas:
      modules: [store, powergrid, sndata]
      end: 1*24*60*60

This is the most simple scenario definition you can create. You could run it with `midasctl run my_first_midas` if you like, but we will have a closer look at what happened here.

On the top level you find a key called my_first_midas. You can rename this as you like as long as it is unique. If you ever happen to have the same key twice, the scenario creation will fail. This key is the name that you pass to the midasctl command. As long as the scenario file is located in the scenario folder, passing that name is sufficient.

On the second level, global settings for the scenario are defined. The most important setting is the modules key. It is a list specifying which modules, i.e., which simulators, should be included in the scenario. In this case you have store, powergrid, and sndata (which is the simulator for the Smart Nord dataset).

The second parameter is called end and specifies how many steps of the simulation will be performed. One step corresponds to one (simulated) second. The default step size for all simulators is 900 seconds (15 minutes). Providing a value lower than 900 will result in only the first step simulated. It is recommended (but not necessary) to provide this value as multiplication products since it is more readable. In this example, we want the scenario to be simulated for one day.

## Default values

To allow such a minimalistic configuration, midas provides default values for nearly everything. If you run your previously created scenario, you will find my_first_midas_cfg.yml in the _outputs folder. You can have a look at it and see what was auto-configurated for you. Not all the values there are needed and most values can be left at their defaults.

In our example, the configurations for store, powergrid, and sndata were auto-configurated, i.e., a database file was defined, a grid model was chosen, and the load mapping was provided. The default database file is midas_store.hdf5 and will be saved to the _outputs folder. The default grid is midasmv, which is basically the CIGRE medium voltage grid. The default mapping for sndata is a mapping for midasmv. Simply choosing a different grid will, therefore, fail since you have to provide an appropriate mapping.

For the rest of this tutorial, we assume that you keep the midasmv as grid. However, let’s add the default values to our scenario:

    my_first_midas:
      modules: [store, powergrid, sndata]
      end: 1*24*60*60
      store_params:
        filename: mymidasdb.hdf5
      powergrid_params:
        my_grid:
          gridfile: midasmv
      sndata_params:
        my_grid:
          land_mapping:
            1: [[0, 1.0], [2, 1.0], [3, 2.0], [6, 2.0], [7, 1.0]]
            3: [[2, 1.0], [3, 1.0], [6, 1.0], [7, 1.0]]
            4: [[0, 2.0], [3, 2.0], [7, 1.0]]
            5: [[3, 2.0], [7, 1.0]]
            6: [[0, 2.0], [3, 1.0]]
            7: [[0, 2.0], [2, 1.0], [3, 2.0], [7, 1.0]]
            8: [[0, 1.0], [3, 1.0], [6, 1.0]]
            9: [[2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]
            10: [[0, 2.0], [2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]
            11: [[0, 1.0], [2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]

Just a few more words about that configuration. We added parameters for the three modules (the key scheme is always module name underscore params). The first subkey of the powergrid_params and sndata_params modules is my_grid. This is called the scope of this modules’ simulator (yes, this means you can define different-scoped simulators in a module). For both modules this scope needs to be same to allow midas to connect those configurations. The store_params are an exception here, because we only allow one instance of it. All modules will find the store regardless of their scope.

The land_mapping of sndata seems a bit confusing at the first look. The first keys of the land_mapping represent the grid bus, to which the load should be connected, e.g., `1: [[..], ..]` means connect the following loads to bus 1 of the grid. On the next sublevel, we have a list that contains several smaller lists. Each of these smaller lists represents a time series and is configured by two values: The first one is the ID and the second one a scaling factor, e.g., `[6, 2.0]` means take the time series with ID 6 and scale it with factor 2.

This schema will be used by other modules as well, although sometimes it will look a bit different.

## Adding a different load simulator

Now we will add another load simulator with commercial loads. Those loads have a different profile than households and have their own module, which we first have to add. Simply modify the following line of your configuration:

    modules: [store, powergrid, sndata, comdata]

Although we have a default configuration for this as well, we will add the parameters manually. Add the following lines after the last current line in the file:

    comdata_params:
      my_grid:
        interpolate: true
        randomize_data: true
        noise_factor: 0.2
        mapping:
          13: [[SuperMarket, 0.089]]
          14: [[SmallHotel, 0.022]]

Make sure you get the indentation right. The comdata_params needs to be at the same level like the other _params.

You will recognize the mapping scheme. However, this time we have names as IDs instead of numbers. You also see two new options that we’ve activated.

- interpolate: Most of the datasets have a certain time resolution (e.g., hourly, quarter-hourly). Setting interpolate to true will activate interpolation if the values are accessed with higher frequency than the resolution. Since the commercial datasets have hourly resolution, it makes sense to use this feature here.

- randomize_data: This feature adds a normal distributed random noise to the data from the dataset.

- noise_factor: The default noise is 20 % (`noise_factor: 0.2`) of the datasets’ standard deviation.

All these three options could be activated for the sndata module, as well.

## Simulation results

Before we add the final two modules for this guide, we’ll have a short look at the simulation results. You’ll find them in the _outputs folder. During the simulation, a HDF5 database will be created and saved to mymidasdb.hdf5. Although you can open this file with any HDF5 viewer, the easiest to get some generic results is to use midasctl again:

    midasctl analyze _outputs/mymidsdb.hdf5

This takes a few seconds. Afterwards, you’ll find a new folder _outputs/mymidasdb containing results of the analysis. There is another folder _outputs/mymidasdb/Powergrid_0 that contains a few .png files, one of them is the average voltage magnitude per unit of the buses in our scenario:

## Extending the scenario

As last part of this tutorial, we want to add some distributed energy resources (DER). More precise, we will add Photovoltaic (PV) plants and combined heat and power (CHP) units. They are provided by the pysimmods package that was installed together with midas. Both of them depend on weather information. Therefore, we will add a weather simulator as well.

But first, we do some additional considerations. We have created a basic scenario with loads only. Now, we are going to add some generation units. But what if you want to compare both scenarios afterwards? Do you have to create two files, one with and one without the generators?

Of course not. midas allows to create scenarios that inherit from other scenarios and both can even (but don’t need to) be in the same file! Let’s give it a try and add this to the bottom of your scenario configuration file:

    my_second_midas:
      parent: my_first_midas
      modules: [weather, der]
      start_date: 2017-01-01 00:00:00+0100

A scenario that has the parent key defined, inherits all configurations from the parent scenario. If you change something there, it will be changed here, too. But you are free to overwrite single values. Keys that have a list or a dictionary, will be updated, e.g., the modules key is extended by two values, weather and der. The final configuration will contain all modules from the parent scenario and the modules from this scenario. We now also added a start date as ISO timestring. This is the default value that is already used in the first scenario.

First, we need to configure the weather module. Update the configuration:

    my_second_midas:
      parent: my_first_midas
      modules: [weather, der]
      start_date: 2017-01-01 00:00:00+0100
      weather_params:
        my_weather_station:
          weather_mapping:
            WeatherCurrent: ["interpolate": true]

Like at the grid configuration, we have a custom scope my_weather_station here. This name is required when we a assign the a weather station to our DER models. The weather_mapping allows to define two models, WeatherCurrent and WeatherForecast. The latter will not be used in this tutorial. Additionally, multiple instances can be created, although, most of the time, one instance should be sufficient. Furthermore, interpolation and randomization can be activated for each instance individually.

Next, we need to add the DER models. The module is already loaded, so we only need to add the configuration.

    der_params:
      my_grid_pv:
        grid_name: my_grid
        mapping:
          3: [[PV, 3], [PV, 1]]
          7: [[PV, 1]]
          8: [[PV, 2]]
          14: [[PV, 2], [PV, 2]]
        weather_provider_mapping:
          PV: [my_weather_station, 0]

(Again, make sure you get the indentation right.) Most of the scheme should be common by now. However, some things are different. Instead of relying on the subkey my_grid as scope like at the other simulators, we added a new key-value pair grid_name. Whenever you use a different scope key than the powergrid module, you can provide the correct value with the grid_name key. This means, my_grid_pv does not need to match the correct grid.

This allows you to split the definition or even define multiple simulators for the same grid, e.g., one for PV plants and a second one for CHP. Alternatively, it is still possible to only use a single simulator for both.

The mapping follows the same rules like what we’ve seen before. The new thing here is the weather_provider_mapping. This field defines, which weather station is used as source for weather data. You need to define a mapping for each plant type but in the most simple case, all plants of a type use the same weather data provider.

Next, we add another simulator definition for the CHP models and change the path of the database, so that a different database will be created instead of overwriting the database from my_first_midas. The full configuration file now looks like:

    my_first_midas:
      modules: [store, powergrid, sndata, comdata]
      end: 1*24*60*60
      powergrid_params:
        my_grid:
          gridfile: midasmv
      sndata_params:
        my_grid:
          land_mapping:
            1: [[0, 1.0], [2, 1.0], [3, 2.0], [6, 2.0], [7, 1.0]]
            3: [[2, 1.0], [3, 1.0], [6, 1.0], [7, 1.0]]
            4: [[0, 2.0], [3, 2.0], [7, 1.0]]
            5: [[3, 2.0], [7, 1.0]]
            6: [[0, 2.0], [3, 1.0]]
            7: [[0, 2.0], [2, 1.0], [3, 2.0], [7, 1.0]]
            8: [[0, 1.0], [3, 1.0], [6, 1.0]]
            9: [[2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]
            10: [[0, 2.0], [2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]
            11: [[0, 1.0], [2, 1.0], [3, 1.0], [6, 2.0], [7, 1.0]]
      comdata_params:
        my_grid:
          interpolate: true
          randomize_data: true
          noise_factor: 0.2
          mapping:
            13: [[SuperMarket, 0.089]]
            14: [[SmallHotel, 0.022]]
    my_second_midas:
      parent: my_first_midas
      modules: [weather, der]
      start_date: 2017-01-01 00:00:00+0100
      store_params:
        filename: my_second_midas.hdf5
      weather_params:
        my_weather_station:
          weather_mapping:
            WeatherCurrent: ["interpolate": true]
      der_params:
        my_grid_pv:
          grid_name: my_grid
          sim_name: PysimmodsPV
          mapping:
            3: [[PV, 3], [PV, 1]]
            7: [[PV, 1]]
            8: [[PV, 2]]
            14: [[PV, 2], [PV, 2]]
          weather_provider_mapping:
            PV: [my_weather_station, 0]
        my_grid_chp:
          grid_name: my_grid
          sim_name: PysimmodsCHP
          mapping:
            4: [[CHP, 0.4], [CHP, 0.4]]
            13: [[CHP, 0.4], [CHP, 0.4], [CHP, 0.4]]
          weather_provider_mapping:
            CHP: [my_weather_station, 0]

Let’s run the new scenario: midasctl run my_second_midas.

Once the simulation has finished, we can use the analysis function of midas another time: midasctl analyze _outputs/my_second_midas.hdf5. We will now have a look at another analysis result: the markdown file _outputs/my_second_midas/my_second_midas-Powergrid_0_report.md.

    # Analysis of my_second_midas-Powergrid__0

    ## Summary

    * bus health: 100.00 %
    * active energy sufficiency: 80.76 %

    ## Demand and Supply

    * total active energy demand: 43.50 MWh
    * total active energy supply: 35.13 MWh or about 11.71 full load hours
    * extg. active energy supply: 8.97 MWh
    * total reactive energy demand: 21.07 MVArh
    * total reactive energy supply: -0.55 MVArh
    * extg. reactive energy supply: 15.38 MVArh
    * total apparent energy demand: 48.33 MVAh
    * total apparent energy supply: 35.13 MVAh
    * extg. apparent energy supply: 17.80 MVAh

The file contains a lot more information. Inspect it as you like. You can also use pandoc to convert it to an .odt or .pdf file. Actually, if you have pandoc installed while you execute the midasctl analyze command, this conversion is done automatically.

This concludes this short tutorial. Further information will follow in the near future.