12. Creating a DataStore from numpy arrays
==========================================

The goal of this notebook is to demonstrate how to create a
``DataStore`` from scratch. This can be useful if your device is not
supported or if you would like to integrate the ``dtscalibration``
library in your current routine.

.. code:: ipython3

    import numpy as np
    import os
    import matplotlib.pyplot as plt
    import xarray as xr
    
    from dtscalibration import DataStore, read_silixa_files

For a ``DataStore`` object, a few things are needed:

-  timestamps

-  Stokes signal

-  anti-Stokes signal

-  x (length along fiber)

Let’s grab the data from an existing silixa dataset:

.. code:: ipython3

    filepath = os.path.join('..', '..', 'tests', 'data', 'single_ended')
    
    ds_silixa = read_silixa_files(directory=filepath,
                                  silent=True)

We will get all the numpy arrays from this ``DataStore`` to create a new
one from ‘scratch’.

Let’s start with the most basic data:

.. code:: ipython3

    x = ds_silixa.x.values
    time = ds_silixa.time.values
    ST = ds_silixa.ST.values
    AST = ds_silixa.AST.values

Now this data has to be inserted into an xarray ``Dataset``

.. code:: ipython3

    ds = xr.Dataset()
    ds['x'] = ('x', x)
    ds['time'] = ('time', time)
    ds['ST'] = (['x', 'time'], ST)
    ds['AST'] = (['x', 'time'], AST)

.. code:: ipython3

    ds = DataStore(ds)
    print(ds)


.. parsed-literal::

    <dtscalibration.DataStore>
    Sections:                  ()
    Dimensions:  (time: 3, x: 1461)
    Coordinates:
      * x        (x) float64 -80.74 -80.62 -80.49 -80.36 ... 104.4 104.6 104.7 104.8
      * time     (time) datetime64[ns] 2018-05-04T12:22:17.710000 ... 2018-05-04T12:23:18.716000
    Data variables:
        ST       (x, time) float64 -0.8058 0.4287 -0.513 ... 27.99 27.83 28.81
        AST      (x, time) float64 -0.2459 -0.5932 0.1111 0.3748 ... 36.2 35.7 35.16
    Attributes:
        _sections:  null\n...\n


For calibration, a few more paramaters are needed:

-  acquisition time (for calculating residuals for WLS calibration)

-  reference temperatures

-  a double ended flag

We’ll put these into the custom ``DataStore``:

.. code:: ipython3

    ds['acquisitiontimeFW'] = ds_silixa['acquisitiontimeFW'].values
    ds['temp1'] = ds_silixa['probe1Temperature']
    ds['temp2'] = ds_silixa['probe2Temperature']
    
    ds.attrs['isDoubleEnded'] = '0'

Now we can calibrate the data as usual (ordinary least squares in this
example).

.. code:: ipython3

    ds = ds.sel(x=slice(-30, 101))
    sections = {
                'temp1':    [slice(20, 25.5)],  # warm bath
                'temp2':    [slice(5.5, 15.5)],  # cold bath
                }
    ds.sections = sections
    
    ds.calibration_single_ended(st_label='ST',
                                ast_label='AST',
                                method='ols')
    
    ds.isel(time=0).TMPF.plot()


.. parsed-literal::

    /Users/bfdestombe/Projects/dts-calibration/python-dts-calibration-dev/.tox/docs/lib/python3.7/site-packages/dask/array/core.py:1333: FutureWarning: The `numpy.ndim` function is not implemented by Dask array. You may want to use the da.map_blocks function or something similar to silence this warning. Your code may stop working in a future release.
      FutureWarning,




.. parsed-literal::

    [<matplotlib.lines.Line2D at 0x12344cfd0>]


