Handling Complex Nested Objects in API Schemas

Real payloads are rarely flat. Orders carry line items that carry options; comment threads nest replies inside replies; a notification can be an email, an SMS, or a push event behind one field. Modeling these shapes correctly is the difference between a contract that holds and one that fails silently in production. This guide is part of Schema Design & Validation Patterns and walks through deep nesting, arrays of objects, recursive structures, polymorphism with oneOf and discriminator, $ref composition, and how to validate deep payloads without melting your CPU.

The techniques here lean on the OpenAPI Specification Deep Dive for the spec-level constructs and on Runtime Validation with Zod for enforcing the same shapes at the application boundary. Two focused walkthroughs go deeper than this page can: validating deeply nested JSON payloads in Node.js and modeling polymorphic types with oneOf in OpenAPI.

When to Use This Approach

Reach for structured nested modeling and the validation discipline below when:

  • A payload nests three or more levels deep (object → array → object → object) and you need every level checked, not just the root.
  • The same sub-shape (an address, a money amount, a line item) appears in more than one parent and you want a single source of truth.
  • You have a genuinely recursive structure — comment trees, org charts, nested categories, file system nodes.
  • A single field can hold one of several distinct shapes (payment methods, notification channels, content blocks) and you need a tagged union.
  • Validation latency or CI duration is already a problem because of combinatorial oneOf/anyOf evaluation, and you need a deterministic, bounded approach.
  • You accept untrusted client input and must defend against depth-bomb and size-amplification attacks before parsing.

If your payloads are flat or only one level deep, this is over-engineering — model them inline and move on.

Prerequisites

Pin tool versions so examples stay reproducible:

# OpenAPI 3.1 spec authoring + linting
npm install -D @stoplight/spectral-cli@6.11
# Compile-once JSON Schema validation (OpenAPI 3.1 is a JSON Schema dialect)
npm install ajv@8.17 ajv-formats@3.0
# TypeScript-first runtime validation
npm install zod@3.23
# Mock server for contract alignment
npm install -D @stoplight/prism-cli@5

You should be comfortable reading OpenAPI 3.1 components.schemas, understand JSON Schema keywords ($ref, oneOf, discriminator, additionalProperties), and have a Node.js 20+ runtime for the validation examples.

The diagram below shows the two shapes that cause the most trouble: a self-referencing recursive node, and a polymorphic field resolved by a discriminator.

Recursive self-reference and discriminator-resolved polymorphism Left: a Comment node whose children array points back to Comment. Right: a Notification field with a channel discriminator mapping to Email, SMS, and Push subschemas. Recursive $ref oneOf + discriminator Comment id, body, author children: [ ] items: $ref self Comment (same shape) children: [ ] depth bounded at runtime Notification channel: ... Email to, subject SMS phone, text Push deviceId channel maps tag → subschema

Step 1: Decompose the Shape into Reusable $ref Components

The first instinct with a deep payload is to write it as one giant inline object. Resist it. Inline objects duplicate definitions, hide circular references, and make diffs unreadable. Instead, break the payload into named entries under components.schemas and wire them together with $ref. Each shape is defined once; every parent references it.

# openapi.yaml (OpenAPI 3.1)
components:
  schemas:
    Order:
      type: object
      required: [id, items, shippingAddress]
      additionalProperties: false        # reject unknown keys at the root
      properties:
        id:
          type: string
          format: uuid
        items:
          type: array
          minItems: 1
          maxItems: 200                   # bound the array, see Step 2
          items:
            $ref: '#/components/schemas/LineItem'
        shippingAddress:
          $ref: '#/components/schemas/Address'   # reused shape, defined once
    LineItem:
      type: object
      required: [sku, quantity]
      additionalProperties: false        # also strict one level down
      properties:
        sku:
          type: string
        quantity:
          type: integer
          minimum: 1
        options:
          type: array
          maxItems: 20
          items:
            $ref: '#/components/schemas/LineItemOption'
    LineItemOption:
      type: object
      required: [name, value]
      additionalProperties: false
      properties:
        name: { type: string }
        value: { type: string }
    Address:
      type: object
      required: [line1, city, country]
      additionalProperties: false
      properties:
        line1:   { type: string }
        line2:   { type: string }
        city:    { type: string }
        country: { type: string, minLength: 2, maxLength: 2 }

The rationale: Address is referenced by Order here, but in a real spec it will also be referenced by Customer, Invoice, and Warehouse. Defining it once means a change propagates everywhere automatically, and tooling can generate a single Address type. Critically, additionalProperties: false is repeated at every object level — set it only at the root and unknown keys slip silently into LineItem and Address.

When $ref chains become circular by accident (A references B references A through a non-recursive path), resolution can blow up. The companion guide on reusing schemas with OpenAPI components and $ref covers composition and circular-reference hygiene in detail.

The same decomposition maps cleanly onto TypeScript with Zod, where each component becomes a named schema you compose:

import { z } from "zod"; // zod 3.23

const LineItemOption = z.object({
  name: z.string(),
  value: z.string(),
}).strict();                        // .strict() == additionalProperties: false

const LineItem = z.object({
  sku: z.string(),
  quantity: z.number().int().min(1),
  options: z.array(LineItemOption).max(20).optional(),
}).strict();

const Address = z.object({
  line1: z.string(),
  line2: z.string().optional(),
  city: z.string(),
  country: z.string().length(2),
}).strict();

export const Order = z.object({
  id: z.string().uuid(),
  items: z.array(LineItem).min(1).max(200),
  shippingAddress: Address,
}).strict();

export type Order = z.infer<typeof Order>;   // fully typed, nested included

Step 2: Model Arrays of Objects with Bounded Constraints

Arrays of objects are where unbounded payloads sneak in. An attacker (or a buggy client) sends 5 million line items and your service spends seconds deserializing and validating before it can reject them. Always pair items: $ref with minItems and maxItems.

items:
  type: array
  minItems: 1            # an order with zero items is meaningless
  maxItems: 200          # hard ceiling — reject before deep validation
  uniqueItems: false     # set true only if duplicates are genuinely invalid
  items:
    $ref: '#/components/schemas/LineItem'

maxItems is not just a business rule, it is a denial-of-service control. The validator can reject an oversized array in O(1) after counting, instead of validating every element. For arrays of arrays (a matrix, a grid), apply the bound at every level — a 10,000 × 10,000 grid that passes element-level checks still exhausts memory.

In Zod, .min() and .max() on the array do the same job, and superRefine lets you express cross-element rules (for example, no two line items sharing a SKU) without a second pass:

const Items = z.array(LineItem).min(1).max(200).superRefine((arr, ctx) => {
  const seen = new Set<string>();
  for (const [i, item] of arr.entries()) {
    if (seen.has(item.sku)) {
      ctx.addIssue({ code: z.ZodIssueCode.custom, path: [i, "sku"], message: "duplicate SKU" });
    }
    seen.add(item.sku);
  }
});

Step 3: Express Recursion with a Self-Referencing $ref

A comment tree, a category hierarchy, or a nested rule group is genuinely recursive: a node contains children that are nodes. OpenAPI 3.1 and JSON Schema support this directly — the node’s children array references the node schema itself.

components:
  schemas:
    Comment:
      type: object
      required: [id, body]
      additionalProperties: false
      properties:
        id:
          type: string
          format: uuid
        body:
          type: string
          maxLength: 10000
        children:
          type: array
          maxItems: 50                     # bound breadth per level
          items:
            $ref: '#/components/schemas/Comment'   # the recursion

The trap: the schema cannot bound depth. maxItems limits breadth at each level, but a malicious client can still send a comment nested 100,000 levels deep and crash your parser with a stack overflow long before validation runs. Depth must be enforced at runtime, before or during validation.

Zod handles recursion with z.lazy() to defer the self-reference, and an explicit depth guard:

type Comment = {
  id: string;
  body: string;
  children?: Comment[];
};

const Comment: z.ZodType<Comment> = z.lazy(() =>
  z.object({
    id: z.string().uuid(),
    body: z.string().max(10000),
    children: z.array(Comment).max(50).optional(),
  }).strict()
);

// Bound depth BEFORE handing untrusted JSON to the validator.
function depth(node: unknown, level = 0): number {
  if (level > 32) throw new Error("max nesting depth exceeded");
  if (node && typeof node === "object" && "children" in node) {
    const kids = (node as { children?: unknown[] }).children ?? [];
    return Math.max(level, ...kids.map((k) => depth(k, level + 1)));
  }
  return level;
}

The exact middleware to cap depth and array breadth on incoming requests — including the JSON-parse-time guards that stop a depth bomb before Comment.parse ever runs — is covered in validating deeply nested JSON payloads in Node.js.

Step 4: Model Polymorphism with oneOf and a Discriminator

A Notification can be an email, an SMS, or a push event. Each shape is different, but they share a slot. This is a tagged (discriminated) union: use oneOf so the payload must match exactly one branch, and add a discriminator so the validator resolves the branch from a tag field instead of trial-fitting every option.

components:
  schemas:
    Notification:
      oneOf:
        - $ref: '#/components/schemas/EmailNotification'
        - $ref: '#/components/schemas/SmsNotification'
        - $ref: '#/components/schemas/PushNotification'
      discriminator:
        propertyName: channel          # the tag field
        mapping:                        # tag value -> subschema
          email: '#/components/schemas/EmailNotification'
          sms:   '#/components/schemas/SmsNotification'
          push:  '#/components/schemas/PushNotification'
    EmailNotification:
      type: object
      required: [channel, to, subject]
      additionalProperties: false
      properties:
        channel: { type: string, enum: [email] }   # constant tag
        to:      { type: string, format: email }
        subject: { type: string }
    SmsNotification:
      type: object
      required: [channel, phone, text]
      additionalProperties: false
      properties:
        channel: { type: string, enum: [sms] }
        phone:   { type: string }
        text:    { type: string, maxLength: 1600 }
    PushNotification:
      type: object
      required: [channel, deviceId]
      additionalProperties: false
      properties:
        channel:  { type: string, enum: [push] }
        deviceId: { type: string }

Why this matters for performance: without a discriminator, a validator faced with oneOf of N branches tries each branch and reports a tangled error if all fail. With a discriminator it reads channel, jumps to that single subschema, and produces a precise error path. Prefer oneOf over anyOf for unions — anyOf accepts a match against several branches and is both ambiguous and slower. The trade-offs, edge cases, and validator support are explored in modeling polymorphic types with oneOf in OpenAPI.

Zod’s discriminatedUnion is the direct equivalent and is the version you want at runtime, because it is dramatically faster than a plain z.union over object schemas:

const EmailNotification = z.object({
  channel: z.literal("email"),
  to: z.string().email(),
  subject: z.string(),
}).strict();

const SmsNotification = z.object({
  channel: z.literal("sms"),
  phone: z.string(),
  text: z.string().max(1600),
}).strict();

const PushNotification = z.object({
  channel: z.literal("push"),
  deviceId: z.string(),
}).strict();

export const Notification = z.discriminatedUnion("channel", [
  EmailNotification,
  SmsNotification,
  PushNotification,
]);

Step 5: Compile a Performant Runtime Validator

Validating deep payloads is expensive only when you do it wrong. The two rules: compile the validator once, and bound the input before validating. Ajv compiles a JSON Schema into an optimized JavaScript function; recompiling per request throws that work away.

import Ajv from "ajv";          // ajv 8.17
import addFormats from "ajv-formats"; // ajv-formats 3.0
import openapiSchemas from "./schemas.json"; // components.schemas extracted

const ajv = new Ajv({
  allErrors: false,             // stop at first error: faster for gating
  discriminator: true,          // honor OpenAPI discriminator for fast branch
  strict: true,
});
addFormats(ajv);

// Compile ONCE at startup, reuse the function per request.
const validateOrder = ajv.compile(openapiSchemas.Order);

export function checkOrder(payload: unknown) {
  if (!validateOrder(payload)) {
    return { ok: false, errors: validateOrder.errors };
  }
  return { ok: true };
}

discriminator: true makes Ajv use the discriminator keyword to pick the oneOf branch in one step rather than evaluating all of them — the single biggest win for polymorphic payloads. Set allErrors: false in CI gates where you only need pass/fail; flip it to true only where a human needs the full list. If you generate types from the same spec, compile-time type generation from OpenAPI keeps your TypeScript types and runtime validators derived from one source.

Spec / Schema Reference

Keyword / option Applies to Type Default Effect
$ref any schema string (JSON Pointer) — Resolves to another schema; the core of composition and recursion.
additionalProperties object boolean / schema true false rejects unknown keys; set at every level for strict contracts.
minItems / maxItems array integer unbounded Bounds array length; maxItems is a DoS control, not just a rule.
uniqueItems array boolean false Requires all elements distinct; O(n log n) cost on large arrays.
oneOf any schema array of schemas — Payload must match exactly one branch; correct for tagged unions.
anyOf any schema array of schemas — Passes if one or more match; slower and ambiguous for unions.
discriminator.propertyName oneOf/anyOf string — Tag field used to resolve the branch in one step.
discriminator.mapping discriminator object implicit Maps tag values to subschema refs; avoids relying on schema names.
enum (single value) property array — Pins the discriminator tag to a constant per branch.
Ajv discriminator validator boolean false Enables fast discriminator-based oneOf resolution.
Ajv allErrors validator boolean false false is faster (fail fast); true collects every error.
Zod .strict() object method non-strict Equivalent to additionalProperties: false.
Zod z.lazy() schema wrapper — Defers evaluation to allow self-reference (recursion).
Zod z.discriminatedUnion() union factory — Fast tagged union keyed on a literal field.

Verification

Confirm the spec is valid and the nested rules are enforced. First lint, then validate fixtures.

# 1. Lint the spec — catches structural errors and unbounded arrays
npx spectral lint openapi.yaml --ruleset .spectral.yaml
# ✔ No results with a severity of "error" found!

# 2. A valid nested order passes
node -e "const {checkOrder}=require('./dist/validate');console.log(checkOrder(require('./fixtures/order.valid.json')))"
# { ok: true }

# 3. An order with an unknown nested key fails with a precise path
node -e "const {checkOrder}=require('./dist/validate');console.log(JSON.stringify(checkOrder(require('./fixtures/order.extrakey.json')).errors))"
# [{"instancePath":"/items/0","schemaPath":"#/.../additionalProperties","keyword":"additionalProperties","params":{"additionalProperty":"colour"},"message":"must NOT have additional properties"}]

A green Spectral run plus a passing valid fixture and a rejected invalid fixture — with the error path pointing at the exact nested location — is the signal that your nested contract holds. Add this as a CI step so regressions fail the build.

# .github/workflows/schema-gate.yml
jobs:
  validate-nested-contracts:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - name: Lint OpenAPI spec
        run: npx spectral lint openapi.yaml --ruleset .spectral.yaml
      - name: Validate fixtures
        run: npm run test:contracts        # must fail on any invalid fixture

Add a Spectral rule so no future array escapes the bound:

# .spectral.yaml
extends: ["spectral:oas"]
rules:
  arrays-must-bound-size:
    description: Every array must declare maxItems
    given: "$..[?(@.type=='array')]"
    severity: error
    then:
      field: maxItems
      function: defined

Troubleshooting

Symptom Root cause Fix
Maximum call stack size exceeded during validation or JSON parse A recursive $ref payload nested deeper than the runtime stack tolerates — the schema bounds breadth but not depth. Enforce a depth cap before validation (Step 3); reject payloads exceeding ~32 levels. See validating deeply nested JSON payloads in Node.js.
Validation passes but an unknown nested field was accepted additionalProperties: false set only at the root, not on nested LineItem/Address objects. Set additionalProperties: false (or Zod .strict()) at every object level, including those reached via $ref.
oneOf / discriminated union accepts the wrong branch or gives a vague error Missing discriminator, or the tag property is not pinned with a single-value enum/z.literal. Add a discriminator with propertyName + mapping; pin each branch’s tag with enum: [value]. Enable Ajv discriminator: true.
Validation latency spikes on large or polymorphic payloads Validator recompiled per request, anyOf used for a union, or oneOf evaluated without a discriminator. Compile the validator once at startup; switch unions to oneOf + discriminator; cap arrays with maxItems (Step 5).
can't resolve reference / circular $ref error from the toolchain A $ref cycle that is not a deliberate self-reference, or a typo in the JSON Pointer. Verify the pointer target exists; for intended recursion ensure the node references itself directly. See fixing OpenAPI $ref circular reference errors.

Frequently Asked Questions

When should I split a nested object into a separate $ref component?

Split when the shape is reused by two or more parents, when it represents a distinct domain entity, or when it is recursive. A one-off inline object that appears in a single place is fine to leave inline.

What is the difference between oneOf and anyOf for polymorphic objects?

oneOf requires the payload to match exactly one subschema and is the correct choice for a discriminated union. anyOf passes if one or more subschemas match, which is slower to validate and ambiguous for tagged unions, so reserve it for genuinely overlapping shapes.

Do I need a discriminator if I already use oneOf?

No, but you should add one. Without a discriminator the validator must trial every branch; with a discriminator it jumps straight to the matching subschema using the tag field, which is both faster and produces clearer error messages.

How do I model a recursive structure like a comment tree in OpenAPI?

Give the node schema a property whose items $ref back to the node itself. OpenAPI and JSON Schema support self-reference, but you must enforce a depth limit at runtime because the schema alone does not bound recursion.

Why is validating deeply nested payloads slow, and how do I fix it?

Slowness usually comes from unbounded oneOf/anyOf combinations evaluated at every level, or from missing discriminators. Compile your validator once, add discriminators, cap array sizes with maxItems, and bound depth before parsing.

Should I set additionalProperties: false at every nesting level?

Yes for strict internal contracts. Setting it only at the root lets unknown keys slip into nested objects undetected. Relax it deliberately for forward-compatible public APIs where clients may send fields you have not yet modeled.

Related Articles