Extract Data From Documents With Content Understanding Analyzers (AI-103)

The final technical skill group of AI-103 is Extract content from documents. The tool at its centre is Azure Content Understanding in Foundry Tools, which combines OCR, layout analysis and generative field extraction in one multimodal pipeline, and produces clean outputs for agents and RAG.

Why a multimodal pipeline

Real documents mix printed and handwritten text, tables, checkboxes, stamps, photos and multi-page layouts. Extracting reliable data needs several steps:

1. OCR: read the characters, including scans and handwriting
2. Layout analysis: find reading order, paragraphs, headings, tables, figures and selection marks
3. Field extraction: turn content into the values you need, such as InvoiceTotal, DueDate or LineItems[]
4. Normalisation and validation: dates, currencies, totals that add up

Content Understanding runs all of this behind one analyzer call, using generative models deployed in your Foundry resource for the extraction and reasoning steps.

Analyzers and field schemas

An analyzer defines how content is processed and what comes back:

• Prebuilt analyzers for common documents (invoices, receipts, IDs and more) and general analyzers for each modality
• Custom analyzers with your own field schema

{
  "description": "Supplier contract analyzer",
  "baseAnalyzerId": "prebuilt-document",
  "config": { "returnDetails": true, "enableOcr": true, "enableLayout": true },
  "fieldSchema": {
    "fields": {
      "SupplierName": { "type": "string", "method": "extract" },
      "StartDate": { "type": "date", "method": "extract" },
      "AutoRenews": { "type": "boolean", "method": "classify" },
      "ContractType": { "type": "string", "method": "classify", "enum": ["services", "software", "hardware"] },
      "RiskSummary": { "type": "string", "method": "generate", "description": "One sentence on unusual liability or termination terms" },
      "PaymentTerms": { "type": "array", "items": { "type": "object", "properties": {
          "Milestone": { "type": "string" }, "Amount": { "type": "number" } } } }
    }
  }
}

The three field methods:

• Extract: the value is written in the document
• Classify: pick from a fixed list (or true/false)
• Generate: create a value from the content, such as a summary or risk note

Clear field descriptions matter: the model uses them to decide what to return. Property names in the analyzer JSON vary by API version, so check the current reference when you build one.

Markdown output for agents and RAG

Besides fields, Content Understanding returns the document content as Markdown: headings, paragraphs, tables and figure descriptions in reading order. That's exactly what downstream reasoning needs:

• RAG: chunk the Markdown by headings, so chunks keep their context and tables stay intact. It's much better than raw OCR text.
• Agents: pass clean Markdown (or just the extracted fields) to an agent, instead of a PDF it can't read reliably
• Grounding: each field includes source locations (page and bounding region), so apps can highlight where a value came from

This is what the exam means by "clean, grounded representations".

Call an analyzer from Python

from azure.ai.contentunderstanding import ContentUnderstandingClient
from azure.ai.contentunderstanding.models import AnalysisInput

client = ContentUnderstandingClient(endpoint=os.environ["CU_ENDPOINT"], credential=DefaultAzureCredential())

poller = client.begin_analyze(
    analyzer_id="supplier-contract",
    inputs=[AnalysisInput(url=contract_sas_url)],
)
result = poller.result()

doc = result.contents[0]
markdown = doc.markdown                     # clean content for RAG / agents
for name, field in (doc.fields or {}).items():
    if field.confidence is not None and field.confidence < 0.8:
        send_to_review(name, field)         # human-in-the-loop
    else:
        save(name, field.value)

The pattern: create a client → begin_analyze(analyzer_id, inputs) → it's asynchronous, so poll → read markdown and fields with confidence. Over REST you get 202 Accepted plus an Operation-Location URL to poll. The SDK is still evolving, so check the current samples.

Single-task vs pro mode

• Standard (single-task) analyzers: one file in, one schema out. Fast and cheaper, ideal for high-volume forms.
• Pro mode: multi-step reasoning across multiple inputs and reference data. For example, check an invoice against its purchase order and the contract's price list, and flag mismatches. Use it when validation needs cross-document reasoning.

Content Understanding vs alternatives

Production tips

• Set confidence thresholds per field: auto-approve high confidence, review low confidence
• Validate with business rules (line items sum to the total; dates are in range)
• Keep a labelled test set and re-measure accuracy when you change the schema or model
• Protect documents: private endpoints, managed identity, and retention rules for results

Need test documents? Create sample invoices with our invoice generator.

Key takeaways

• Content Understanding = OCR + layout + generative field extraction in one analyzer call.
• Field schema methods: extract, classify, generate, with types including object and array.
• Markdown output plus grounding gives clean, traceable input for RAG and agents.
• begin_analyze → poll → fields with confidence; use pro mode for cross-document reasoning.

Next lesson: the AI-103 cheat sheet.