Metadata-Version: 2.4
Name: sgn-manifold
Version: 0.1.1.post0
Summary: Geometric tools for gravitational wave data analysis
Author-email: Chad Hanna <crh184@psu.edu>
Maintainer-email: Chad Hanna <crh184@psu.edu>
License: GPL-2.0-or-later
Project-URL: Homepage, https://git.ligo.org/chad-hanna/manifold
Project-URL: Documentation, https://git.ligo.org/chad-hanna/manifold
Project-URL: Repository, https://git.ligo.org/chad-hanna/manifold
Project-URL: Bug Tracker, https://git.ligo.org/chad-hanna/manifold/issues
Keywords: gravitational-waves,ligo,data-analysis,geometry,manifold
Classifier: Development Status :: 3 - Alpha
Classifier: Intended Audience :: Science/Research
Classifier: License :: OSI Approved :: GNU General Public License v2 or later (GPLv2+)
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python :: 3
Classifier: Programming Language :: Python :: 3.10
Classifier: Programming Language :: Python :: 3.11
Classifier: Programming Language :: Python :: 3.12
Classifier: Topic :: Scientific/Engineering :: Physics
Classifier: Topic :: Scientific/Engineering :: Astronomy
Requires-Python: >=3.10
Description-Content-Type: text/markdown
Requires-Dist: lalsuite
Requires-Dist: numpy>=1.20.0
Requires-Dist: scipy>=1.7.0
Requires-Dist: matplotlib>=3.3.0
Requires-Dist: h5py>=3.0.0
Requires-Dist: pandas>=1.3.0
Requires-Dist: plotly>=5.0.0
Requires-Dist: astropy>=4.3
Requires-Dist: numdifftools>=0.9.39
Requires-Dist: pyyaml>=5.0.0
Requires-Dist: python-ligo-lw
Requires-Dist: confluent-kafka>=1.7.0
Requires-Dist: tqdm>=4.0.0
Provides-Extra: dev
Requires-Dist: black>=25.0.0; extra == "dev"
Requires-Dist: isort>=5.10.0; extra == "dev"
Requires-Dist: flake8>=4.0.0; extra == "dev"
Requires-Dist: flake8-future-annotations>=1.0.0; extra == "dev"
Requires-Dist: mypy>=0.950; extra == "dev"
Requires-Dist: pre-commit>=2.17.0; extra == "dev"
Requires-Dist: types-PyYAML; extra == "dev"
Requires-Dist: pandas-stubs; extra == "dev"
Requires-Dist: plotly-stubs; extra == "dev"
Provides-Extra: test
Requires-Dist: pytest>=7.0.0; extra == "test"
Requires-Dist: pytest-cov>=3.0.0; extra == "test"
Requires-Dist: pytest-freezer>=0.4.6; extra == "test"
Requires-Dist: pytest-markdown-docs>=0.4.0; extra == "test"
Provides-Extra: docs
Requires-Dist: mkdocs>=1.4.0; extra == "docs"
Requires-Dist: mkdocstrings[python]>=0.19.0; extra == "docs"
Requires-Dist: mkdocs-material>=8.5.0; extra == "docs"
Requires-Dist: pymdown-extensions>=9.0; extra == "docs"
Provides-Extra: strike
Requires-Dist: strike; extra == "strike"
Provides-Extra: dag
Requires-Dist: ezdag; extra == "dag"
Provides-Extra: sgn
Requires-Dist: sgn; extra == "sgn"
Requires-Dist: sgn-ts; extra == "sgn"
Requires-Dist: sgn-ligo; extra == "sgn"
Requires-Dist: strike; extra == "sgn"
Provides-Extra: all
Requires-Dist: sgn-manifold[dag,dev,docs,sgn,strike,test]; extra == "all"

# Manifold 

## Tutorial: Generating a CBC Template Bank

This tutorial walks through the complete process of generating a compact binary coalescence (CBC) template bank using Manifold, from initial seed bank generation through parallel refinement and final testing.

### Prerequisites

You'll need the `O4_projected_psds.xml.gz` file from the `examples` directory. Copy it to your working directory:

```bash
cp examples/O4_projected_psds.xml.gz .
```

### Step 1: Create a Configuration File

Create a file called `bank.yaml` with the following contents. This configuration uses a smaller parameter space for faster execution:

```yaml
# Mass parameters (in solar masses)
m1: [10, 100]        # Primary mass range
m2: [10, 100]        # Secondary mass range
M: [20, 200]         # Total mass range
mc: [8, 200]         # Chirp mass range
q: [1.0, 10.0]       # Mass ratio range (m1/m2)

# Spin parameters
chi: [-0.5, 0.5]     # Effective aligned spin parameter
ns-mass: [0, 3.0]    # Neutron star mass threshold
ns-s1z: [-0.05, 0.05]  # Neutron star spin limit

# Frequency parameters
freq: [10, 512]      # Frequency range [flow, fhigh] in Hz
max-duration: 128    # Maximum signal duration in seconds

# PSD and instrument
psd-xml: O4_projected_psds.xml.gz
instrument: L1       # Options: L1, H1, V1

# Bank generation parameters
mm: 0.03             # Maximum mismatch (3% mismatch = 97% minimum match)
reuse-g-mm: 0.10     # Mismatch for reusing metric calculations
min-coord-vol: 1000  # Minimum coordinate volume for rectangles
min-depth: 7         # Minimum tree depth for placement
approximant: IMRPhenomD

# Processing options
trim: true           # Trim templates outside constraints
verbose: true
```

### Step 2: Generate the Initial Seed Bank

Generate a coarse seed bank with ~1000 templates:

```bash
manifold-cbc-bank --yaml bank.yaml --output-h5 SEED.h5 --max-num-templates 1000
```

You should see output showing the placement of templates, ending with something like:

```
900 <Rectangle [(...), (...), (...)]> 1.0
1000 <Rectangle [(...), (...), (...)]> 1.0
Trimming templates outside constraints...
1024 templates before
987 templates after
Saving bank to SEED.h5...
Bank saved successfully with 987 templates
```

### Step 3: Divide the Seed Bank into Groups

Split the seed bank into groups for parallel refinement (e.g., 10 groups):

```bash
manifold-cbc-bank-group --bank SEED.h5 --num-groups 10
```

This creates:
- `H1L1V1-00000_MANIFOLD_SEED-0-2000000000.h5` through `H1L1V1-00009_MANIFOLD_SEED-0-2000000000.h5` (seed groups)
- `H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json` (manifest file)

### Step 4: Refine Each Seed Group in Parallel

For each seed group, generate a refined bank with much smaller rectangles. This step can be parallelized across multiple cores/machines:

```bash
# Process all 10 groups (can be run in parallel)
for i in $(seq -f "%05g" 0 9); do
    manifold-cbc-bank --yaml bank.yaml \
        --seedbank H1L1V1-${i}_MANIFOLD_SEED-0-2000000000.h5 \
        --output-h5 H1L1V1-${i}_MANIFOLD-0-2000000000.h5 \
        --max-num-templates 1 \
        --min-coord-vol 0.0001
done
```

Each refined bank will have many more templates (~5000-6000 each) covering its portion of the parameter space more finely.

### Step 5: Combine the Refined Banks

Merge all refined banks into a single bank:

```bash
manifold-cbc-bank-add --output-h5 H1L1V1-MANIFOLD_ADD-0-2000000000.h5 \
    H1L1V1-00000_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00001_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00002_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00003_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00004_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00005_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00006_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00007_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00008_MANIFOLD-0-2000000000.h5 \
    H1L1V1-00009_MANIFOLD-0-2000000000.h5
```

### Step 6: Check and Improve Coverage (Parallel)

Check the bank's coverage and split poorly covered templates. This can also be parallelized by processing intervals:

```bash
# Process 10 intervals in parallel (each checks 1/10th of the bank)
for i in $(seq 0 9); do
    manifold-cbc-bank-check-coverage \
        --bank H1L1V1-MANIFOLD_ADD-0-2000000000.h5 \
        --interval-index $i \
        --num-intervals 10 \
        --neighbors 10000 \
        --target-match 0.965 \
        --output-bank H1L1V1-MANIFOLD_COVERAGE_$(printf "%02d" $i)-0-2000000000.h5 \
        --verbose
done
```

**Key parameters:**
- `--interval-index`: Which interval to process (0-based)
- `--num-intervals`: Total number of intervals
- `--neighbors`: Number of neighbors to use as buffer (should be large, e.g., 10000)
- `--target-match`: Minimum match threshold (0.965 = 96.5% match)

Each interval will check its portion of the bank and split templates that don't meet the target match.

### Step 7: Combine Coverage-Improved Banks

Merge all coverage-improved intervals:

```bash
manifold-cbc-bank-add --output-h5 H1L1V1-MANIFOLD-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_00-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_01-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_02-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_03-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_04-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_05-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_06-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_07-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_08-0-2000000000.h5 \
    H1L1V1-MANIFOLD_COVERAGE_09-0-2000000000.h5
```

This is your final template bank!

### Step 8: Test the Bank

Generate test injections to verify bank performance. Again, this can be parallelized:

```bash
# Generate 10 test sets in parallel (100 injections each)
for i in $(seq -f "%02g" 1 10); do
    manifold-cbc-bank-test \
        --bank H1L1V1-MANIFOLD-0-2000000000.h5 \
        --output-h5 ${i}_test.h5 \
        --num-injections 100 \
        --verbose
done
```

### Step 9: Combine and Plot Test Results

Combine all test results:

```bash
manifold-cbc-bank-test-add --output-h5 test.h5 \
    01_test.h5 02_test.h5 03_test.h5 04_test.h5 05_test.h5 \
    06_test.h5 07_test.h5 08_test.h5 09_test.h5 10_test.h5
```

Generate diagnostic plots:

```bash
manifold-cbc-bank-test-plot --test test.h5
```

This creates several plots showing:
- Match vs injection parameters (m1, m2, chirp mass, mass ratio, etc.)
- Mismatch histogram
- 2D match distributions

### Step 10 (Optional): Add Constraint Sets and Template IDs

If you need to prepare the bank for online analysis, add constraint sets and assign template IDs:

```bash
manifold-cbc-bank-constrain \
    --bank H1L1V1-MANIFOLD-0-2000000000.h5 \
    --output-h5 H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --yaml constrain.yaml
```

where `constrain.yaml` contains:

```yaml
m1: [12, 98]
m2: [12, 98]
mc: [10, 90]
q: [1.0, 8.0]
M: [24, 196]
chi: [-0.4, 0.4]
ns-mass: [0, 3.0]
ns-s1z: [-0.04, 0.04]
assign-ids: true
```

### Step 11 (Optional): Generate Mass Model

Create a mass model for the bank:

```bash
manifold-cbc-bank-mass-model \
    --bank H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --output-h5 H1L1V1-MANIFOLD_MASS_MODEL-0-2000000000.h5 \
    --yaml mass_model.yaml
```

where `mass_model.yaml` contains:

```yaml
mass_model:
    model: salpeter
    m_min: 0.8
```

### Step 12 (Optional): Convert to XML

Convert the bank to XML format for use with other tools:

```bash
manifold-cbc-bank-to-xml \
    H1L1V1-MANIFOLD_CONSTRAINED-0-2000000000.h5 \
    --output-xml H1L1V1-MANIFOLD-0-2000000000.xml.gz
```

## Using Make for Automation

You can automate the entire workflow using a Makefile. Create a file called `Makefile`:

```makefile
all: test

# Step 2: Generate seed bank
SEED.h5: bank.yaml O4_projected_psds.xml.gz
	manifold-cbc-bank --yaml bank.yaml --output-h5 $@ --max-num-templates 1000 --min-coord-vol 1000

# Step 3: Divide into groups
H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json: SEED.h5
	manifold-cbc-bank-group --bank SEED.h5 --num-groups 10

# Step 4: Refine each group (pattern rule)
H1L1V1-%_MANIFOLD-0-2000000000.h5: H1L1V1-MANIFOLD_SEED_MANIFEST-0-2000000000.json
	manifold-cbc-bank --yaml bank.yaml --output-h5 $@ \
		--seedbank H1L1V1-$*_MANIFOLD_SEED-0-2000000000.h5 \
		--max-num-templates 1 --min-coord-vol 0.0001

# Step 5: Combine refined banks
H1L1V1-MANIFOLD_ADD-0-2000000000.h5: \
		H1L1V1-00000_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00001_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00002_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00003_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00004_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00005_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00006_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00007_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00008_MANIFOLD-0-2000000000.h5 \
		H1L1V1-00009_MANIFOLD-0-2000000000.h5
	manifold-cbc-bank-add --output-h5 $@ $^

# Step 6: Check coverage for each interval (pattern rule)
H1L1V1-MANIFOLD_COVERAGE_%-0-2000000000.h5: H1L1V1-MANIFOLD_ADD-0-2000000000.h5
	manifold-cbc-bank-check-coverage \
		--bank $< \
		--interval-index $* \
		--num-intervals 10 \
		--neighbors 10000 \
		--target-match 0.965 \
		--output-bank $@ \
		--verbose

# Step 7: Combine coverage-improved banks
H1L1V1-MANIFOLD-0-2000000000.h5: \
		H1L1V1-MANIFOLD_COVERAGE_00-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_01-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_02-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_03-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_04-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_05-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_06-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_07-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_08-0-2000000000.h5 \
		H1L1V1-MANIFOLD_COVERAGE_09-0-2000000000.h5
	manifold-cbc-bank-add --output-h5 $@ $^

# Step 8: Generate test injections (pattern rule)
%_test.h5: H1L1V1-MANIFOLD-0-2000000000.h5
	manifold-cbc-bank-test --bank $< --output-h5 $@ --num-injections 100 --verbose

# Step 9: Combine test results
test.h5: 01_test.h5 02_test.h5 03_test.h5 04_test.h5 05_test.h5 \
         06_test.h5 07_test.h5 08_test.h5 09_test.h5 10_test.h5
	manifold-cbc-bank-test-add --output-h5 $@ $^

# Step 9: Plot test results
test: test.h5
	manifold-cbc-bank-test-plot --test $<

clean:
	rm -rf *.h5 *.png *.json

.PHONY: all test clean
```

Then simply run:

```bash
make -j 10  # Run with up to 10 parallel jobs
```

## Performance Notes

**Parallel Processing:**
- Steps 4, 6, and 8 can be parallelized across multiple cores or machines
- For Step 6 (coverage checking), use a large `--neighbors` value (e.g., 10000) to ensure accurate results
- Small differences in template counts between single and parallel runs are expected and acceptable

**Parameter Space Size:**
- Smaller mass ranges and spin ranges = faster execution
- The tutorial configuration should complete in ~30-60 minutes on a modern machine
- For production banks, use larger ranges (e.g., m1/m2: [3, 200], chi: [-0.95, 0.95])

**Memory Usage:**
- Each refined bank group uses ~1-2 GB of memory
- Coverage checking uses ~2-4 GB per interval
- Testing uses ~500 MB per test set
