Metadata-Version: 2.4
Name: vlm2vec_for_pyserini
Version: 0.1.5
Summary: Fork of the original VLM2Vec repository modified for easy and simple Pyserini integration
Author-email: Ziyan Jiang <ziyanjiang528@gmail.com>, Rui Meng <memray0@gmail.com>, Wenhu Chen <wenhuchen@uwaterloo.ca>, Sahel Sharifymoghaddam <sahel.sharifymoghaddam@uwaterloo.ca>
Project-URL: Homepage, https://github.com/castorini/VLM2Vec-for-Pyserini
Project-URL: Bug Tracker, https://github.com/castorini/VLM2Vec-for-Pyserini/issues
Classifier: Programming Language :: Python :: 3
Classifier: License :: OSI Approved :: Apache Software License
Classifier: Operating System :: OS Independent
Requires-Python: >=3.12
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: accelerate
Requires-Dist: datasets
Requires-Dist: decord
Requires-Dist: einops
Requires-Dist: flash-attn
Requires-Dist: hjson
Requires-Dist: hnswlib
Requires-Dist: huggingface-hub
Requires-Dist: ninja
Requires-Dist: numpy
Requires-Dist: opencv-contrib-python
Requires-Dist: opencv-python
Requires-Dist: peft
Requires-Dist: pillow
Requires-Dist: py-cpuinfo
Requires-Dist: pytrec-eval
Requires-Dist: qwen-vl-utils[decord]==0.0.8
Requires-Dist: ray
Requires-Dist: scikit-image
Requires-Dist: scikit-learn
Requires-Dist: scipy
Requires-Dist: sentencepiece
Requires-Dist: timm
Requires-Dist: torch==2.9.1
Requires-Dist: torchvision
Requires-Dist: tqdm
Requires-Dist: transformers==4.57.6
Requires-Dist: wandb
Requires-Dist: wrapt
Dynamic: license-file

# VLM2Vec (MMEB) for Pyserini

[![PyPI](https://img.shields.io/pypi/v/vlm2vec_for_pyserini?color=brightgreen)](https://pypi.org/project/vlm2vec_for_pyserini/)
[![Downloads](https://static.pepy.tech/personalized-badge/vlm2vec_for_pyserini?period=total&units=international_system&left_color=grey&right_color=brightgreen&left_text=downloads)](https://pepy.tech/project/vlm2vec_for_pyserini)
[![Downloads](https://static.pepy.tech/personalized-badge/vlm2vec_for_pyserini?period=week&units=international_system&left_color=grey&right_color=brightgreen&left_text=downloads/week)](https://pepy.tech/project/vlm2vec_for_pyserini)
[![LICENSE](https://img.shields.io/badge/license-Apache-blue.svg?style=flat)](https://www.apache.org/licenses/LICENSE-2.0)


This repository contains a fork of the original VLM2Vec codebase, modified for easy [Pyserini](https://github.com/castorini/pyserini/) integration and repackaged as a PyPI package.

`current_version = "0.1.5"`

## Supported Datasets and Tasks
All 24 Visual Document Retrieval tasks are supported. This covers `ViDoRE`  `ViDoRE v2`, `VisRAG`, `ViDoSeek`, and `MMLongBench`.

## Supported Models
Any VL models with `qwen2-vl`, `gme`, and `lamra` backbones are supported. This includes `gme-Qwen2-VL-2B/7B-Instruct`, `VLM2Vec/VLM2Vec-V2.0`, `code-kunkun/LamRA-Ret` and more. 

## Installation

Install the package directly from PyPI:

```bash
pip install vlm2vec-for-pyserini
```

Or, install from source:

```bash
git clone https://github.com/castorini/VLM2Vec-for-Pyserini.git
cd VLM2Vec-for-Pyserini
pip install .
```

## Quick Start
Assuming that you have cloned the repository and you are in the root dir:
1. Download the visdoc from HuggingFace and convert the `corpus`, `topics` and `queries` to the format ready for Pyserini:
```bash
bash src/pyserini_integration/prepare_dataset.sh
```
2. Run encoding, indexing, and search. Then evaluation and results aggregation using the following script:
```bash
bash src/pyserini_integration/experiments.sh
```
If you want to use the PyPI package, take a look at `download_visdoc.py`, `save_pyserini_data.py`, and `quick_start_demo.py` files under `src/pyserini_integration/` as sample code.

## Contact
For contact regarding the Pyserini integration section, please email [Sahel Sharifymoghaddam](mailto:sahel.sharifymoghaddam@uwaterloo.ca).

For contact regarding the original VLM2Vec codebase, please email the authors of the original repository.

## Citation

If you use this work with Pyserini, please cite Pyserini in addition to the original VLM2Vec paper:

```bibtex
@INPROCEEDINGS{Lin_etal_SIGIR2021_Pyserini,
   author = "Jimmy Lin and Xueguang Ma and Sheng-Chieh Lin and Jheng-Hong Yang and Ronak Pradeep and Rodrigo Nogueira",
   title = "{Pyserini}: A {Python} Toolkit for Reproducible Information Retrieval Research with Sparse and Dense Representations",
   booktitle = "Proceedings of the 44th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2021)",
   year = 2021,
   pages = "2356--2362",
}

@article{jiang2024vlm2vec,
  title={VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks},
  author={Jiang, Ziyan and Meng, Rui and Yang, Xinyi and Yavuz, Semih and Zhou, Yingbo and Chen, Wenhu},
  journal={arXiv preprint arXiv:2410.05160},
  year={2024}
}

@article{meng2025vlm2vecv2,
  title={VLM2Vec-V2: Advancing Multimodal Embedding for Videos, Images, and Visual Documents},
  author={Rui Meng and Ziyan Jiang and Ye Liu and Mingyi Su and Xinyi Yang and Yuepeng Fu and Can Qin and Zeyuan Chen and Ran Xu and Caiming Xiong and Yingbo Zhou and Wenhu Chen and Semih Yavuz},
  journal={arXiv preprint arXiv:2507.04590},
  year={2025}
}
```

## 📄 License

This project is licensed under the Apache 2.0 License. See the [LICENSE](LICENSE) file for details.
