How do you get bounding boxes from LLM document extraction?
General-purpose vision LLMs like GPT-4 do not reliably return pixel coordinates for the values they extract — ask for a bounding box and you'll often get a plausible-looking but wrong rectangle. To get real coordinates you need a model with an OCR/detection encoder that reports where each value was read. Interfaze returns these as bounds in its precontext metadata.
Last updated 2026-07-20
Why general LLMs can't box reliably
A general model wasn't trained to emit calibrated coordinates, so any box it returns is generated, not measured. The numbers drift — especially on dense, low-contrast, or skewed documents — and you have no way to know when they're wrong.
What returns real boxes
A specialist OCR engine produces bounds as a measured output. Interfaze fuses one into the model and returns per-word bounds — corner coordinates in the original image's pixel space — inside its `precontext` array, alongside the structured answer.
Using the boxes
Scale each box by rendered-width ÷ natural-width and draw it over the source image. To attach a box to a specific field, match the extracted value against the OCR words. This is exactly what the live demo does — hover a field and its region lights up on the document.
FAQ
Can GPT-4 vision return bounding boxes?
It can return numbers that look like boxes, but they aren't reliable — there's no OCR ground truth behind them, so they drift on hard documents.