PDF Utils
=========

The module `ricecooker.utils.pdf` contains helper functions for manipulating PDFs.



PDF splitter
------------
When importing source PDFs like books that is a very long documents (100+ pages),
it is better for Kolibri user experience to split them into multiple shorter PDF
content nodes.

The `PDFParser` class in `ricecooker.utils.pdf` is a wrapper around the `PyPDF2`
library that allows us to split long PDF documents into individual chapters,
based on the information available in the PDF's table of contents.


### Split into chapters

Here is how to split a PDF document located at `pdf_path`, which can be either
a local path or a URL:

    from ricecooker.utils.pdf import PDFParser
    
    pdf_path = '/some/local/doc.pdf' or 'https://somesite.org/some/remote/doc.pdf'
    with PDFParser(pdf_path) as pdfparser:
        chapters = pdfparser.split_chapters()

The output `chapters` is list of dictionaries with `title` and `path` attributes:

    [ 
      {'title':'First chapter',  'path':'downloads/doc/First-chapter.pdf'},
      {'title':'Second chapter', 'path':'downloads/doc/Second-chapter.pdf'},
      ... 
    ]

Use this information to create individual `DocumentNode`s for each PDF and store
them in a `TopicNode` that corresponds to the book:

    from ricecooker.classes import nodes, files

    book_node = nodes.TopicNode(title='Book title', description='Book description')
    for chapter in chapters:
        chapter_node = nodes.DocumentNode(
            title=chapter['title'],
            files=files.DocumentFile(chapter['path']),
            ...
        )
        book_node.add_child(chapter_node)

By default, the split PDFs are saved in the directory `./downloads`. You can customize
where the files are saved by passing the optional argument `directory` when initializing
the `PDFParser` class, e.g., `PDFParser(pdf_path, directory='somedircustomdir')`.


The `split_chapters` method uses `get_toc` method internally to obtain the list
of page ranges for each chapter. Use `pdfparser.get_toc()` to inspect the PDF's
table of contents. The table of contents data returned by the `get_toc` method
has the following format:

    [
      {'title': 'First chapter',  'page_start': 0,  'page_end': 10},
      {'title': 'Second chapter', 'page_start': 10, 'page_end': 20},
      ...
    ]

If the page ranges automatically detected form the PDF's table of contents are
not suitable for the document you're processing, or if the PDF document does not
contain table of contents information, you can manually create the title and 
page range data and pass it as the `jsondata` argument to the `split_chapters()`.

    page_ranges = pdfparser.get_toc()
    # possibly modify/customize page_ranges, or load from a manually created file
    chapters = pdfparser.split_chapters(jsondata=page_ranges)



### Split into chapters and subchapters

By default the `get_toc` will detect only the top-level document structure,
which might not be sufficient to split the document into useful chunks.
You can pass the `subchapters=True` optional argument to the `get_toc()` mothod
to obtain a two-level hierarchy of chapters and subchapter from the PDF's TOC.

For example, if the table of contents of textbook PDF has the following structure:

     Intro
     Part I
        Subchapter 1
        Subchapter 2
     Part II
        Subchapter 21
        Subchapter 22
     Conclusion

then calling `pdfparser.get_toc(subchapters=True)` will return the following
chapter-subchapter tree structure:

    [
      { 'title': 'Part I', 'page_start': 0,  'page_end': 10,
        'children': [
            {'title': 'Subchapter 1',  'page_start': 0,  'page_end': 5},
            {'title': 'Subchapter 2',  'page_start': 5,  'page_end': 10}
         ]},
      { 'title': 'Part II', 'page_start': 10,  'page_end': 20,
        'children': [
            {'title': 'Subchapter 21',  'page_start': 10, 'page_end': 15},
            {'title': 'Subchapter 22',  'page_start': 15,  'page_end': 20}
         ]},
      { 'title': 'Conclusion', 'page_start': 20,  'page_end': 25 }
    ]

Use the `split_subchapters` method to process this tree structure and obtain the
tree of title and paths:


    [
      { 'title': 'Part I',
        'children': [
            {'title': 'Subchapter 1', 'path': '/tmp/0-0-Subchapter-1.pdf'},
            {'title': 'Subchapter 2', 'path': '/tmp/0-1-Subchapter-2.pdf'},
         ]},
      { 'title': 'Part II',
        'children': [
            {'title': 'Subchapter 21', 'path': '/tmp/1-0-Subchapter-21.pdf'},
            {'title': 'Subchapter 22', 'path': '/tmp/1-1-Subchapter-22.pdf'},
         ]},
      { 'title': 'Conclusion', 'path': '/tmp/2-Conclusion.pdf'}
    ]

You'll need to create a `TopicNode` for each chapter that has `children` and
create `DocmentNode`s for each of the children of that chapter.





Accessibility notes
-------------------
Do not use the `PDFParser` for tagged PDFs because splitting and processing loses
the accessibility features of the original PDF document.

