I am facing an issue with data extraction and data mapping in my Azure Custom Extraction Model.

Hello @test sujeet

You've described five distinct symptoms incomplete extraction on your own training documents, multiple field values merging into one, values landing in the wrong mapped field, degraded results on new documents of the same type, and run-to-run inconsistency on identical fields — from a model trained on 5 documents of a single document type. That set of symptoms points to two separate causes, and I want to be precise about which is which so you don't spend a week relabelling for something that relabelling won't fix.

Root cause : Likely, pending the details requested below). Two factors are almost certainly in play. First, configuration: custom extraction models learn field boundaries from label geometry, and the documented requirement is that the words of a field must be in a consecutive sequence in natural reading order without interleaving with other fields, or in a region that does not cover any other field. With a five-document dataset, a single loosely-drawn label becomes the learned rule. The five-document figure is also the minimum to train a model at all Microsoft's guidance calls for at least five labeled samples per layout variation, and treats digital versus scanned PDFs as distinct variations.
Second, product behaviour: Microsoft has publicly confirmed that "there is a certain probability of incorrect cell merging in tables with a large aspect ratio and small character spacing," that the product team is aware of it, and that there is no ETA for a fix. It was re-confirmed as still reproducible on API version 2024-11-30. I have ruled in the configuration factors as the primary lever because they are unverified in your environment; I have not ruled out the product limitation, and I would expect a residual level of merging to persist even after a clean dataset. I'm not able to confirm which dominates until you send the model type, API version, and a sample analyze response.

Step 1: Confirm the model type in Document Intelligence Studio project settings; if it is custom template, rebuild as custom neural. Why this matters: the template model relies on a consistent visual template and any variance in visual structure degrades accuracy, whereas neural generalizes across formats of the same document type; Microsoft's guidance is to start with neural. Labelling and training are identical between the two, so you do not need to relabel. How to verify: analyze the same problem document against both model IDs and compare the returned values and confidence scores field by field.

Step 2: Audit label geometry on all five training documents, focusing on the fields that merge or misroute. Use shift-select across the words of a span rather than free-drawn region boxes; where two fields genuinely overlap, use region labeling and include at least one training sample with the overlap explicitly labeled a single word may be labeled for at most two fields. Why this matters: overlapping or loosely scoped labels are the direct mechanism behind merged and misrouted values, and with five documents there is no counter-example to dilute the error.

How to verify: re-analyze the corrected document and confirm no field value contains an embedded \n at a point where two fields should have separated; that \n is the reported signature of a merge.

Step 3: Expand the dataset by layout variation, not by raw count. Remove all user-entered values from your documents and compare the shells identical shells are one variation, differing shells are separate variations that each need at least five labeled samples, or separate models composed together. Add five samples per input format as well if you process both digital and scanned PDFs. Why this matters: the model may see documents as dissimilar that look identical to a human, which is precisely why new documents of "the exact same type" extract inconsistently.

How to verify: analyze a held-out document per variation and inspect the document type confidence score; a low score confirms the analyzed document doesn't resemble the training set.

Step 4: Move to API version 2024-11-30 (GA) and instrument on confidence. That version adds field-level word confidence and table, row, and cell-level confidence for custom models. Note that for neural models the estimated accuracy value may render blank in Studio, so confidence scores are the correct measurement instrument. Use the published matrix: high accuracy with low confidence indicates the analyzed document differs from the training dataset and warrants at least five more labeled documents. Why this matters: without confidence telemetry you cannot distinguish a correct extraction from a coincidental one reported case saw >90% confidence on a document pattern the model had never been trained on.

How to verify: log per-field confidence across a batch of at least 20 documents and confirm the incorrect extractions cluster at the low end.

Step 5: Add a deterministic post-processing guard before the data reaches your business logic. Assert the expected field count per document, split values carrying the \n merge signature using content rules (date pattern, currency pattern, keyword anchor), and route sub-threshold fields to human review. Why this matters: this is your protection against the residual merging that steps 1–4 cannot fully eliminate.

How to verify: replay your previously-failing documents through the guard and confirm every known merge is caught and split correctly.

Please treat Step 5 explicitly as a workaround, not a permanent fix. The incorrect-merge behaviour is an acknowledged product limitation with no published ETA, re-confirmed on the latest GA API version by users training on 20+ and 90+ document sets. I do not want to imply a fix date that Microsoft has not committed to.

Thanks,
Manish.

Hello test sujeet,

Greetings! Thanks for raising this question in the Q&A forum.

The pattern you are describing, fields missing, values merging across fields, values landing in the wrong field, and results varying between similar documents, is a classic sign that the training set is too small and does not cover enough variation for the model to generalize reliably. Five documents is well below what Document Intelligence needs for stable extraction, and if your documents have any layout differences at all, the model has not seen enough examples to learn where field boundaries truly are.

Increase your training set size. Microsoft recommends at least five labeled samples per distinct layout variation as an absolute minimum, but for production-grade reliability you generally need considerably more, especially when fields sit close together or have variable-length content. If your document type has any variation in layout, add more labeled examples covering each variation rather than relying on five documents of a single layout.

Confirm you are using a custom neural model, not a template model. Neural models generalize much better across layout variation and are the recommended default. Template models expect a highly consistent visual layout and will produce exactly the kind of inconsistent results you describe if your documents vary even slightly in spacing, alignment, or field position. Check this under your project settings in Document Intelligence Studio.

Review your field labeling for overlap and ambiguity. If two fields are extracting into a single one, or values are landing in the wrong field, open each training document in Document Intelligence Studio and check that label bounding boxes do not overlap and that each label is tightly scoped to only the relevant text. Loosely drawn or overlapping bounding boxes are a common cause of merged or misrouted field values.

Standardize field naming and labeling consistency across all training documents. Make sure the same field is labeled the same way, in the same relative pattern, on every training document. Any inconsistency here, such as labeling a value differently on one document versus another, teaches the model conflicting patterns and directly causes inconsistent extraction on new documents.

Check document image quality and format consistency. Use one clear scan or text-embedded PDF per document rather than mixed photo quality, since OCR errors upstream of extraction will cascade into incorrect field mapping. Confirm that all training documents represent the same document type and are not silently mixing subtly different form versions.

Retrain and validate incrementally. After expanding the dataset and fixing labeling, retrain the model and test against a validation set of documents not used in training. Compare confidence scores per field. Fields with persistently low confidence usually need more labeled examples specifically covering that field's variations rather than more of the same overall document type.

If extraction remains inconsistent after increasing the dataset and confirming clean labeling, this is worth an Azure Support technical ticket referencing your Document Intelligence resource so an engineer can review the training set and model diagnostics directly.

If this answer helps you kindly accept the answer which will help others who have similar questions.

Best Regards,

Jerald Felix.