Metadata-Version: 2.4
Name: pyfstree
Version: 0.1.0
Summary: Python bindings for fstree - filesystem tree sharing and sync
Author: Robert Andersson
Author-email: Robert Andersson <rbrtandersson@gmail.com>
License: GPL-3.0-or-later
Project-URL: Homepage, https://github.com/srand/fstree
Project-URL: Repository, https://github.com/srand/fstree.git
Project-URL: Issues, https://github.com/srand/fstree/issues
Keywords: filesystem,sync,tree,fstree,merkle
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Developers
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Programming Language :: Python :: 3.9
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Programming Language :: C++
Classifier: Topic :: Software Development :: Libraries :: Python Modules
Classifier: Topic :: System :: Filesystems
Classifier: Topic :: System :: Networking
Requires-Python: >=3.7
Description-Content-Type: text/x-rst
License-File: LICENSE
Requires-Dist: pybind11
Dynamic: author
Dynamic: license-file
Dynamic: requires-python

.. image:: https://github.com/srand/fstree/actions/workflows/build.yml/badge.svg
   :target: https://github.com/srand/fstree/actions/workflows/build.yml
   :alt: Build status


fstree - filesystem tree sharing and sync
=========================================

Fstree a tool to share a filesystem tree over the network and keep it in sync.

It is similar to rsync, but it is designed to be used in a client-server model, where
trees are pushed and pulled between the server and the clients. Merkle trees are used
to efficiently determine what parts of the tree need to be downloaded or uploaded.

Files and directories can be ignored using a `.fstreeignore` file in the root of the tree.
It uses the same syntax as `.gitignore` files.

A simple gRPC protocol is used to communicate between the client and the server.
A reference server implementation is available as the
`robrt/jolt-cache <https://hub.docker.com/r/robrt/jolt-cache>`_
Docker image on Docker Hub.

HTTP remotes are also supported for basic upload and download of objects. This allows
integration with existing HTTP file servers or CDNs. Note that HTTP remotes are 
significantly less efficient than the gRPC server, as they do not support
querying for the presence of multiple objects and trees at once. 


Usage
-----

To start the server, run:

.. code-block::

  docker run -v /path/to/data:/data -p 9090:9090 robrt/jolt-cache -i

To push a tree to the server, run:

.. code-block::

  fstree write-tree-push --remote jolt://localhost:9090 /path/to/data

The steps above can be split into two separate commands:

.. code-block::

  fstree write-tree /path/to/data
  fstree push --remote jolt://localhost:9090 <digest>

To push a tree to a standard HTTP server, run:

.. code-block::

  fstree write-tree-push --remote http://localhost:8080 /path/to/data'

To pull a tree from the server, run:

.. code-block::

  fstree pull-checkout --remote jolt://localhost:9090 <digest> /path/to/data
  
The steps above can be split into two separate commands:

.. code-block::

  fstree pull --remote jolt://localhost:9090 <digest>
  fstree checkout <digest> /path/to/data

To pull a tree from a standard HTTP server, run:

.. code-block::

  fstree pull-checkout --remote http://localhost:8080 <digest> /path/to/data

To list the contents of a tree, run:

.. code-block::

  fstree ls-tree <digest>


Configuration
-------------

Configuration can be passed as command line arguments or environment variables.

The following environment variables are supported:

- ``FSTREE_CACHE``: The directory where the local object cache is stored. Defaults to ``~/.cache/fstree`` on Linux and macOS and ``~/AppData/Local/fstree`` on Windows.
- ``FSTREE_IGNORE``: The relative path to the ignore file. Defaults to ``.fstreeignore`` in the root of the tree.
- ``FSTREE_REMOTE``: The remote address of the server to connect to. Defaults to ``jolt://localhost:9090``.
- ``FSTREE_THREADS``: The number of threads to use for parallel operations. Defaults to the number of CPU cores.

The following command line arguments are supported:

- ``--cache``: See ``FSTREE_CACHE``.
- ``--ignore``: See ``FSTREE_IGNORE``.
- ``--remote``: See ``FSTREE_REMOTE``.
- ``--threads``: See ``FSTREE_THREADS``.


Building
--------

The project can be built using CMake, but developers typically use Jolt as a higher level orchestrator.
To build the project using Jolt, run:

.. code-block::

  pip install jolt
  jolt build fstree

