Metadata-Version: 2.1
Name: dextrusion
Version: 0.0.9
Summary: DeXtrusion: automatic detection of cell extrusion in epithelial tissu
Author: Gaëlle Letort, Alexis Villars
License: BSD 3-Clause License
        
        Copyright (c) 2022, Gaëlle  LETORT and Alexis Villars
        All rights reserved.
        
        Redistribution and use in source and binary forms, with or without
        modification, are permitted provided that the following conditions are met:
        
        1. Redistributions of source code must retain the above copyright notice, this
           list of conditions and the following disclaimer.
        
        2. Redistributions in binary form must reproduce the above copyright notice,
           this list of conditions and the following disclaimer in the documentation
           and/or other materials provided with the distribution.
        
        3. Neither the name of the copyright holder nor the names of its
           contributors may be used to endorse or promote products derived from
           this software without specific prior written permission.
        
        THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
        AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
        IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
        DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
        FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
        DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
        SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
        CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
        OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
        OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
        
Project-URL: Bug Tracker, https://github.com/Image-Analysis-Hub/dextrusion/issues
Project-URL: Documentation, https://image-analysis-hub.github.io/dextrusion/
Project-URL: Source Code, https://github.com/Image-Analysis-Hub/dextrusion
Project-URL: User Support, https://github.com/Image-Analysis-Hub/dextrusion/issues
Classifier: Framework :: napari
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Topic :: Scientific/Engineering :: Image Processing
Requires-Python: >3.8
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: matplotlib
Requires-Dist: numpy<2
Requires-Dist: opencv-python
Requires-Dist: tifffile>=2022.2.2
Requires-Dist: roifile
Requires-Dist: scikit-image
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: tensorflow<=2.11
Requires-Dist: protobuf==3.19
Requires-Dist: ipython

# DeXtrusion <img src=https://github.com/Image-Analysis-Hub/Dextrusion/blob/main/images/DeX.png caption="DeXtrusion logo"/> 

<img src=https://github.com/Image-Analysis-Hub/Dextrusion/blob/main/images/dextrusion_maps.png caption="DeXtrusion maps" width=150 align="left"/> 
</br></br></br>
<p align="center"><b>Automatic detection of cellular events in epithelia.</b></p>
</br>

DeXtrusion is  a machine learning based python pipeline to detect cell extrusions in epithelial tissues movies. It can also detect cell divisions and SOPs, and can easily be trained to detect other dynamic events. 

This repository contains the code source of the python DeXtrusion library, the trained neural networks (ready to use) with Jupyter notebooks to run, train or retrain DeXtrusion networks, and Fiji macros for visualization/analysis of the results.

* [Presentation](#presentation)
* [Installation](#installation)
* [Usage](#usage)
    * [Troubleshooting](#trouble-shooting)
* [Data](#data)
* [DeXtrusion networks](#dexnets-dextrusion-neural-networks)
* [Fiji macros](#fiji-macros)
* [Jupyter notebooks](#jupyter-notebooks)
* [References](#references)

## Presentation
DeXtrusion takes as input a movie of an epithelium and outputs the **spatio-temporal location of cell extrusion events** or other event as cell divisions. 
The movie is discretized into small overlapping rolling windows which are individually [classified for event detection](#event-classification) by a trained [neural network](#deXNets-deXtrusion-neural-networks). Results are then put together in event [probability map](#event-probability-map) for the whole movie or as [spatio-temporal points](#event-spot-localization) indicating each event. 

<p align="center"> 
    <img src="https://github.com/Image-Analysis-Hub/DeXtrusion/blob/main/images/SequenceExtrusion.png" height="100"/>
Example of the detection of an extrusion event (probability map, red).
 </p>


### Event classification
The movie is discretized into small windows that span all the movie in a rolling windows fashion with overlap between the different windows. 
Each window is classified by a trained neural network as containing or not a cellular event as cell extrusion. The different classes available in the main DeXtrusion network are:
- No event
- Cell extrusion/cell death
- Cell division
- SOPs

It is easy to add or remove a cellular event to this list, but necessitates to train new neural networks for this. Jupyter notebook is available in this repository to do it.

### Event probability map
Each window is associated an event probability which allow to generate an events probability map on the whole movie. This probability maps can be refined to precise spatio-temporal spots for each event.
The results can be visualized by overlaying the initial movie and all the probability maps saved by DeXtrusion in Fiji with the saved by DeXtrusion with the [`deXtrusion_overlayProbabilityMaps.ijm`](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_overlayProbabilityMaps.ijm) macro.

<p align="center">
   <img src="https://github.com/Image-Analysis-Hub/DeXtrusion/blob/main/images/SequenceProbamaps.png" height="200"/>
   
   Probability maps example
</p>

Example of probability maps (green: division, red: extrusion, blue: SOP

### Event spot localization
An event is visible in the probability map as a volume of connected pixels of high probability. To convert the probability map to a list of event, we place the event localization at the center of each of these high probability volumes.
Moreover, to reduce false positive detections, the volumes can be thresholded to keep only big enough volume of high enough probability values. 
The list of spots obtained in this way are saved in ROIS `.zip` file that can be open in Fiji through the `ROIManager` tool. The macro [`deXtrusion_showROIs.ijm`](#https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_showROIs.ijm) allows to directly visualize with Fiji the results saved by DeXtrusion. 


## Installation
DeXtrusion is distributed as a python module, compatible with `python3`. 
You can install it in a virtual environment to be sure to have the required versions.
**DeXtrusion** can be installed either manually through the setup file or with pip:

* DeXtrusion can be directly installed via pip. In your python virtual environment, enter: `pip install dextrusion` to install it.
You should also download the trained neural network [DeXNet](#dexnets-dextrusion-neural-networks) that you want to use from this repository.

* To install manually **DeXtrusion**, clone this github repository, and inside this repository type: `python setup.py install`. 
If you want to install it in a virtual environment, you should have activated it before.
It will install all the necessary dependencies.


### Detailled installation on Linux (Ubuntu)
Here's a step by step command lines to run in a Terminal to install DeXtrusion on Linux from scratch. It is creating a new virtual environment on which DeXtrusion will be installed (not necessary but recommended). Tested with `python 3.8`.

* Installation with `pip`
```sh
python3 -m venv dextrusenv               ## create a virtual environment with python 3 called dextrusenv (you can choose the name)
source dextrusenv/bin/activate           ## activate the virtual environment: now python commands will be runned in that environment
pip install dextrusion                  ## install DeXtrusion, download all the necessary dependencies, can take time
```
DeXtrusion is installed and can be runned with:

```sh
python -m dextrusion 	                  ## run DeXtrusion. Next time to run it again with the same networks, you only have to do this line
```

DeXtrusion needs neural networks to classify each sliding window. To used our trained neural networks (in the DeXNets repository), you have to download and uncompress it. From the command line:

```sh
mkdir DeXNets							## create directory on which neural networks will be put
mkdir DeXNets/notum_ExtSOPDiv			## create directory for the neural networks traind with 4 events available in this repository
cd DeXNets/notum_ExtSOPDiv               ## go inside the desired neural networks directory (here the network trained with 4 events)
wget https://github.com/Image-Analysis-Hub/DeXtrusion/-/blob/main/DeXNets/notum_ExtSOPDiv/notumExtSOPDiv0.zip   ## download the first neural network
wget https://github.com/Image-Analysis-Hub/DeXtrusion/-/blob/main/DeXNets/notum_ExtSOPDiv/notumExtSOPDiv1.zip   ## download the second neural network
unzip notumExtSOPDiv0.zip                ## uncompress it. It is now ready to be used
unzip notumExtSOPDiv1.zip
cd ../..                                 ## go back to main DeXtrusion directory
```
 
* You can also clone this repository and install DeXtrusion from the source code `python setup.py install` instead of using pip installation.


### Detailled installation on Windows
You need to have anaconda installed to run python and create a new virtual environment. On the command prompt, write:
```cmd
$ C:\ProgramData\Anaconda3\Scripts\Activate     ## activate conda (should be the path of your anaconda installation)
$ conda create --name dextrusionenv python=3.8  ## to create virtual environment called dextrusionenv
$ conda activate dextrusionenv                  ## activate the virtual environment that you juste created
$ pip install dextrusion                        ## install DeXtrusion in it
$ python -m dextrusion                          ## Run it
```

### Choose the neural network DeXNets to use
DeXtrusion needs neural networks to classify each sliding window. To used our trained neural networks (in the [DeXNets repository](./DeXNets)), you have to download and uncompress it. 
From DeXtrusion user interface, to select a network to run, you have to go inside its uncompressed directory (click on `Browse` and go there). If you want to use several networks together for the detection for better results (see our paper), the networks folders should be placed within the same directory. Go inside that directy with the interface, and it will use all the networks presents in that directory.

## Usage

DeXtrusion can be used directly within python. 
To run an already trained network and detect cell events, run `python3 -m dextrusion` in your virtual environment. It will open a graphical interface to perform the detection.

We also propose [Jupyter notebooks](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks) in this repository, to use dextrusion options. 
You can find a notebook to perform the detection, train a new neural network, retrain a network or evaluate the performance of the network compared to a manual detection.

### Trouble shooting

Encountered errors/issues and solutions:

#### Internal error copying input tensor

If you get this error message, finishing by `InternalError: Failed copying input tensor from ... ` it is likely to be an `Out of Memory` problem if you are trying to run DeXtrusion with too many windows running in parallel (at once). To change this, decrease the `groupsize` parameter which controls this number.


<details>
<summary>
See full error message
</summary>

```
---------------------------------------------------------------------------
InternalError                             Traceback (most recent call last)
Cell In[7], line 14
     11     print( "Detecting events on movie "+str(image) )
     13 ## Detect events
---> 14 dexter.detect_events_onmovie( image, models\=model, 
     15 cell_diameter\=cell_diameter, extrusion_duration\=extrusion_duration, 
     16 dxy\=shift_xy, dz\=shift_t, 
     17 group_size\=groupsize )
     19 ## Save probability maps to file, to visualize the results
     20 if save_proba:

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\dextrusion\\DeXtrusion.py:375, in DeXtrusion.detect_events_onmovie(self, moviepath, models, cell_diameter, extrusion_duration, dxy, dz, group_size, outfolder)
    373     os.makedirs(self.outpath)
    374 self.outname \= os.path.join(self.outpath, output_name)
--> 375 self.detect_events(img, models, cell_diameter, extrusion_duration, dxy, dz, group_size\=group_size)
    377 if self.verbose:
    378     print("---- Detection finished, took %s minutes ---" % ((time.time() - start_time)/60))

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\dextrusion\\DeXtrusion.py:434, in DeXtrusion.detect_events(self, img, model_paths, cell_diameter, extrusion_duration, dxy, dz, group_size)
    432         print("   Doing windows with shift: "+str(self.shiftxy)+" "+str(self.shiftz)+" and model "+model_paths[cmod])
    433         print("   Number of windows to categorize: "+str(self.nw[4]))
--> 434     do \= list(map(functools.partial(self.do_one_group, img, pred_channels, npred), range(0,self.nw[4],self.group_size) ))
    436 ## divide result map by the number of predictions added in each pixel
    437 self.divide(pred_channels, npred)

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\dextrusion\\DeXtrusion.py:302, in DeXtrusion.do_one_group(self, img, pred_channels, npred, group)
    300 img_group \= np.array(img_group)/255 # normalize
    301 self.model.reset(self.model_path)
--> 302 pred \= self.model.predict_batch(img_group)
    304 # Fill the proba results in the correct place on the pred_channels image
    305 for c in range(self.ncat-1):

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\dextrusion\\Network.py:182, in Network.predict_batch(self, img_batch)
    181 def predict_batch(self, img_batch):
--> 182     return self.model.predict(img_batch)

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\keras\\utils\\traceback_utils.py:67, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     65 except Exception as e:  # pylint: disable=broad-except
     66   filtered_tb \= _process_traceback_frames(e.__traceback__)
---> 67   raise e.with_traceback(filtered_tb) from None
     68 finally:
     69   del filtered_tb

File ~\\AppData\\Local\\miniforge3\\envs\\dextrusion_env\\lib\\site-packages\\tensorflow\\python\\framework\\constant_op.py:102, in convert_to_eager_tensor(value, ctx, dtype)
    100     dtype \= dtypes.as_dtype(dtype).as_datatype_enum
    101 ctx.ensure_initialized()
--> 102 return ops.EagerTensor(value, ctx.device_name, dtype)

InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run _EagerConst: Dst tensor is not initialized. 
```

</details>


#### 


## Data

Data used for the training of our neural networks (raw movies with manual anotation of events) are freely available on Zenodo: [https://doi.org/10.5281/zenodo.7586394](https://zenodo.org/records/7586394). 

In the Zenodo repository, you have movies and the corresponding manually annotated ROI files. The ROIs will have the same name as the movie, followed, by "_cell_*event*.zip", where `event` is the corresponding event that were annotated (death/extrusion, division, sop).

The description of all these movies (microscope used, biological marker, resolution...) are given in the [Supplemental Table S1](https://cob.silverchair-cdn.com/cob/content_public/journal/dev/150/13/10.1242_dev.201747/1/dev201747supp.pdf?Expires=1742676929&Signature=LcovagyqTBBmVv-YTgKCBOhFvT2-oJ1CFMn4GZQm4gZjjP2V5eeI9qCn~DIg6XRzLAoSd1h7q2BLcX8AelIhF3WogXQIMNRJMKBQxXzQmHSS0yDEvRu2aA9CAPRbk1L5P9k8vlPBQ389TraYMSor~2vSGf5sC4cRcZg5sKvCPLaCZpJiZYTZvnx9OYauV0cIFod1N3UZf48nJEXCeCaNx70709OwwIWVPvvjA~ZyutlET4f~ZpZq3vWDcCZsu7aYfDdTjpDAYTiyNf~qodN3N5zBKLmr~8nHY6xgUtQvpLiOyRVWjxOoMjs3M3sYWMP2h-txXug0VWw606cNJX7YAA__&Key-Pair-Id=APKAIE5G5CRDK6RD3PGA) of our publication. The number in the `Movie` column in the table corresponds to the number in the name of the movie in the Zenodo.


## DeXNets: deXtrusion neural networks
DeXtrusion was trained on E-cadherin tagged drosophilia notum but can be easily adapted to other markers/biological conditions. Retraining of DeXtrusion network may be necessary when images are quite different.

In the `deXNets` folder of this repository, we proposed several trained networks:
- `notum_Ext`: trained on drosophilia notum movies with only extrusion events
- `notum_ExtSOP`: trained on drosophilia notum movies with extrusion and SOP events
- `notum_ExtSOPDiv`: trained on drosophilia notum movies with extrusion, division and SOP events
- `notum_all`: trained on drosophilia notum movies with extrusion, division and SOP events, on all our annotated data (training and test). Networks to use by default.

Download them and unzip to be ready to use in DeXtrusion.

### Train/retrain a new DeXNet
If you want to train/retrain a neural network, to add a new event to detect or to improve its performance on new datasets, Jupyter notebooks are proposed in this repository: [Jupyter notebooks](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks). You can just follow step by step the notebook.

If you want to change the architecture of DeXNet, you will have to change it in the `src/dextrusion/Network.py` file. The architecture is defined in the `action_model` function.


## Fiji macros

Fiji macros to prepare data for DeXtrusion neural network training or to handle DeXtrusion results are available in the [`Fiji macros`](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/) folder of this repository.

* [deXtrusion_checkAndClearROIs.ijm](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_checkAndClearROIs.ijm) allows to check manually all the detected ROIs and to keep only the correct ones. 
It shows each event (point ROI) detected by deXtrusion as a pop-up window centered temporally and spatially around the event.
The user is asked wether the detection is correct (if there is an event in the current window) by pressing `y` or `n`.
If yes, the ROI is kept, else it will be removed from the list of ROIs.


* [deXtrusion_scoreROIs_Random.ijm](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_scoreROIs_Random.ijm) allows to score manually the precision of the detected events.
The macro will open randomly a given number of ROIs, and show a window centered temporally and spatially around the event.
The user is asked wether the detection is correct (if it contains the event) by pressing `y` or `n`.
The macro will output the count of correct and false detections and the precision. 
The ROIs file will not be edited and the ROIs left after running this macro should not be saved as it deletes all the ROIs explored to be sure not to draw them twice.

* [deXtrusion_overlayProbabilityMaps.ijm](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_overlayProbabilityMaps.ijm) allows to visualize the probability maps together with the original image.
The user is asked for the original image to visualize and will open the probability maps in the `results` folder where they should be saved by default.
The directory to look in can be changed in the macro. 

* [deXtrusion_showROIs.ijm](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_showROIs.ijm) displays the original movie (chosen by the user) and the ROI file found in the `results` folder. 
The location of the ROI file can be changed in the macro by changing the `resfold` parameter and the event to show (by default extrusion) can be changed in the `type` parameter.

* [deXtrusion_subsetMovieAndRois.ijm](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/ijmacros/deXtrusion_subsetMovieAndRois.ijm) save a temporal subset of a movie, and keep only the corresponding ROIs. 


## Jupyter notebooks
We provide in this repository several jupyter notebooks to run, train/retrain, evaluate DeXtrusion. 
* [detectEventsOnMovie.ipynb](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks/detectEventsOnMovie.ipynb) allows you to run DeXtrusion with the graphical interaface to choose the parameters. 
* [deXtrusion_TrainNetwork.ipynb](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks/deXtrusion_TrainNetwork.ipynb) to train a new DeXNet network with your own data (or using our data publicly available, see [data section](#Data)).
* [deXtrusion_RetrainNetwork.ipynb](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks/deXtrusion_RetrainNetwork.ipynb) to retrain an already trained neural network with new data.
* [deXtrusion_CompareRois.ipynb](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks/deXtrusion_CompareRois.ipynb) to compare two ROIs files (manual detection vs DeXtrusion detection) and gives the score. It can also generate ROIs files of the false detection (false positives and/or false negatives).
* [deXtrusion_testClassification_OnWindows.ipynb](https://github.com/Image-Analysis-Hub/DeXtrusion/raw/main/jupyter_notebooks/deXtrusion_testClassification_OnWindows.ipynb): run the trained neural network (DeXNet) on test data on generated windows, and print the resulting confusion matrix of the classification (DeXtrusion classification compared to the manual classification).

#### Notebooks usage
* If you have Jupyter installed on your computer, to add your virtual environment to the environment Jupyter can see:
```
ipython kernel install --user --name=dextrusionenv
```
You can refer to this [tutorial](https://www.geeksforgeeks.org/using-jupyter-notebook-in-virtual-environment/) for instructions on creating virtual environment and using it with jupyter.

* You can also install Jupyter Lab or Jupyter notebook in your virtual environment if you don't have it installed in your computer:
```sh
pip install jupyterlab
jupyter-lab
```
Refer to [Jupyter](https://jupyter.org/) webpage for more information.

## References
To know more about DeXtrusion, refer to our paper: [Villars, Letort et al. *Development* 2023](https://journals.biologists.com/dev/article/150/13/dev201747/321184/DeXtrusion-automatic-recognition-of-epithelial).

## License
DeXtrusion is distributed open-source under the BSD-3 license, see the license file in this repository.

When you use DeXtrusion source code, neural networks or data for your projects, please cite our paper. 

