util.postprocess
Copyright (C) 2021 Microsoft Corporation.
Functions
|
Filter out objects below a certain score. |
|
Filter out bounding boxes whose confidence is below the confidence threshold for |
|
Compute the intersection-over-union of two bounding boxes. |
|
Compute the intersection area over box area, for bbox1. |
|
Process the bounding boxes produced by the table structure recognition model |
|
Process the bounding boxes produced by the table structure recognition model into |
|
Apply operations to the detected rows, such as |
|
Apply operations to the detected columns, such as |
|
Non-maxima suppression (NMS) of objects based on shared containment of other objects. |
|
Slot a collection of objects into the container they occupy most (the container which holds the largest fraction of the object). |
|
Put any set of objects in order from high score to low score. |
|
Remove any objects (these can be rows, columns, supercells, etc.) that don't |
|
Extract the text inside a bounding box. |
|
Reduce the set of spans to those that fall within a bounding box. |
|
Test if more than "threshold" fraction of bbox1 overlaps with bbox2. |
|
Convert a collection of page tokens/words/spans into a single text string. |
Put the objects in order from left to right. |
|
Put the objects in order from top to bottom. |
|
|
For every column, align the top and bottom boundaries to the final |
|
For every row, align the left and right boundaries to the final |
|
Apply operations to the detected table structure objects such as |
|
A customizable version of non-maxima suppression (NMS). |
|
Adjust the header boundary to be the convex hull of the rows it intersects |
|
For each supercell, align it to the rows it intersects 50% of the height of, |
|
A NMS scheme for supercells that first attempts to shrink supercells to |
|
Make sure no supercell in the header is below more than one supercell in any row above it. |
|
Assuming the row, column, supercell, and header bounding boxes have |
|
This function resolves overlap between supercells (supercells must be |
Module Contents
- util.postprocess.apply_threshold(objects, threshold)[source]
Filter out objects below a certain score.
- util.postprocess.apply_class_thresholds(bboxes, labels, scores, class_names, class_thresholds)[source]
Filter out bounding boxes whose confidence is below the confidence threshold for its associated class label.
- util.postprocess.iou(bbox1, bbox2)[source]
Compute the intersection-over-union of two bounding boxes.
- util.postprocess.objects_to_cells(table, objects_in_table, tokens_in_table, class_map, class_thresholds)[source]
Process the bounding boxes produced by the table structure recognition model and the token/word/span bounding boxes into table cells.
Also return a confidence score based on how well the text was able to be uniquely slotted into the cells detected by the table model.
- util.postprocess.objects_to_table_structures(table_object, objects_in_table, tokens_in_table, class_names, class_thresholds)[source]
Process the bounding boxes produced by the table structure recognition model into a consistent set of table structures (rows, columns, supercells, headers).
This entails resolving conflicts/overlaps, and ensuring the boxes meet certain alignment conditions (for example: rows should all have the same width, etc.).
- util.postprocess.refine_rows(rows, tokens, score_threshold)[source]
Apply operations to the detected rows, such as thresholding, NMS, and alignment.
- util.postprocess.refine_columns(columns, tokens, score_threshold)[source]
Apply operations to the detected columns, such as thresholding, NMS, and alignment.
- util.postprocess.nms_by_containment(container_objects, package_objects, overlap_threshold=0.5)[source]
Non-maxima suppression (NMS) of objects based on shared containment of other objects.
- util.postprocess.slot_into_containers(container_objects, package_objects, overlap_threshold=0.5, unique_assignment=True, forced_assignment=False)[source]
Slot a collection of objects into the container they occupy most (the container which holds the largest fraction of the object).
- util.postprocess.sort_objects_by_score(objects, reverse=True)[source]
Put any set of objects in order from high score to low score.
- util.postprocess.remove_objects_without_content(page_spans, objects)[source]
Remove any objects (these can be rows, columns, supercells, etc.) that don’t have any text associated with them.
- util.postprocess.extract_text_inside_bbox(spans, bbox)[source]
Extract the text inside a bounding box.
- util.postprocess.get_bbox_span_subset(spans, bbox, threshold=0.5)[source]
Reduce the set of spans to those that fall within a bounding box.
threshold: the fraction of the span that must overlap with the bbox.
- util.postprocess.overlaps(bbox1, bbox2, threshold=0.5)[source]
Test if more than “threshold” fraction of bbox1 overlaps with bbox2.
- util.postprocess.extract_text_from_spans(spans, join_with_space=True, remove_integer_superscripts=True)[source]
Convert a collection of page tokens/words/spans into a single text string.
- util.postprocess.sort_objects_left_to_right(objs)[source]
Put the objects in order from left to right.
- util.postprocess.sort_objects_top_to_bottom(objs)[source]
Put the objects in order from top to bottom.
- util.postprocess.align_columns(columns, bbox)[source]
For every column, align the top and bottom boundaries to the final table bounding box.
- util.postprocess.align_rows(rows, bbox)[source]
For every row, align the left and right boundaries to the final table bounding box.
- util.postprocess.refine_table_structures(table_bbox, table_structures, page_spans, class_thresholds)[source]
Apply operations to the detected table structure objects such as thresholding, NMS, and alignment.
- util.postprocess.nms(objects, match_criteria='object2_overlap', match_threshold=0.05, keep_higher=True)[source]
A customizable version of non-maxima suppression (NMS).
Default behavior: If a lower-confidence object overlaps more than 5% of its area with a higher-confidence object, remove the lower-confidence object.
objects: set of dicts; each object dict must have a ‘bbox’ and a ‘score’ field match_criteria: how to measure how much two objects “overlap” match_threshold: the cutoff for determining that overlap requires suppression of one object keep_higher: if True, keep the object with the higher metric; otherwise, keep the lower
- util.postprocess.align_headers(headers, rows)[source]
Adjust the header boundary to be the convex hull of the rows it intersects at least 50% of the height of.
For now, we are not supporting tables with multiple headers, so we need to eliminate anything besides the top-most header.
- util.postprocess.align_supercells(supercells, rows, columns)[source]
For each supercell, align it to the rows it intersects 50% of the height of, and the columns it intersects 50% of the width of.
Eliminate supercells for which there are no rows and columns it intersects 50% with.
- util.postprocess.nms_supercells(supercells)[source]
A NMS scheme for supercells that first attempts to shrink supercells to resolve overlap.
If two supercells overlap the same (sub)cell, shrink the lower confidence supercell to resolve the overlap. If shrunk supercell is empty, remove it.
- util.postprocess.header_supercell_tree(supercells)[source]
Make sure no supercell in the header is below more than one supercell in any row above it.
The cells in the header form a tree, but a supercell with more than one supercell in a row above it means that some cell has more than one parent, which is not allowed. Eliminate any supercell that would cause this to be violated.
- util.postprocess.table_structure_to_cells(table_structures, table_spans, table_bbox)[source]
Assuming the row, column, supercell, and header bounding boxes have been refined into a set of consistent table structures, process these table structures into table cells.
This is a universal representation format for the table, which can later be exported to Pandas or CSV formats. Classify the cells as header/access cells or data cells based on if they intersect with the header bounding box.
- util.postprocess.remove_supercell_overlap(supercell1, supercell2)[source]
This function resolves overlap between supercells (supercells must be disjoint) by iteratively shrinking supercells by the fewest grid cells necessary to resolve the overlap.
Example: If two supercells overlap at grid cell (R, C), and supercell #1 is less confident than supercell #2, we eliminate either row R from supercell #1 or column C from supercell #1 by comparing the number of columns in row R versus the number of rows in column C. If the number of columns in row R is less than the number of rows in column C, we eliminate row R from supercell #1. This resolves the overlap by removing fewer grid cells from supercell #1 than if we eliminated column C from it.