TOON: Token-Oriented Object Notation

What is TOON?

TOON (Token-Oriented Object Notation) is an ultra-compact data serialization format that eliminates JSON's verbose syntax. Unlike JSON which requires quotes around every key and value, braces for objects, and colons for separators, TOON uses a YAML-like approach with schema declarations and indentation. Data is represented as simple rows without repeated key names.

The format consists of two parts: a schema declaration (e.g., users[2]{id,name,role}:) that defines the structure, followed by data rows (1.Alice,admin) with no quotes, braces, or colons. This eliminates hundreds of tokens for large datasets while remaining human-readable through indentation.

History

TOON was developed in 2023 as a response to the increasing costs of token-based AI APIs. As GPT-4 and Claude pricing became token-dependent, developers sought ways to minimize token usage without sacrificing data structure. TOON emerged as a community-driven specification for token-optimized data serialization.

Key Milestones

• 2023: Initial TOON specification published
• Q1 2024: First Python/JavaScript libraries released
• Q2 2024: Adopted by major AI applications
• Q3 2024: TOON schema registry launched
• 2024: Over 10,000 projects using TOON
• Present: De facto standard for LLM data optimization

Key Features

Core Capabilities

• No Quotes: Eliminates all quotation marks from data
• No Braces/Brackets: Uses indentation instead of {} and []
• No Colons: Schema-based structure without key:value syntax
• Schema Declarations: Format like users[2]{id,name,role}:
• Indentation-Based Nesting: YAML-like structure for nested objects
• Tabular Data Rows: Simple data rows like 1.Alice,admin
• Minimal Punctuation: Only commas, dots, and hyphens needed
• Human-Readable: Clean, structured format despite minimal syntax

TOON vs JSON Example

Standard JSON:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

TOON Format:

users[2]{id,name,role}:
  1.Alice,admin
  2.Bob,user

Key Differences:

• No quotes, braces, or colons in TOON
• Schema declaration users[2]{id,name,role}: defines array of 2 objects with 3 fields
• Data rows are just values: 1.Alice,admin
• Indentation represents nesting (YAML-like)

Common Use Cases

Advantages

• Eliminates 100% of quotes, braces, and colons
• Significant cost savings on AI API calls (50-70% token reduction)
• Maintains data structure through indentation
• Highly readable despite minimal syntax
• Schema-based structure for clarity
• Open specification, no licensing
• Libraries available for major languages
• Perfect for tabular/array data
• Faster transmission over networks
• YAML-like readability with JSON compatibility

Disadvantages

• Indentation-sensitive (like Python/YAML)
• Not as widely supported as JSON
• Learning curve for new developers
• Schema declaration must be precise
• Limited tool support compared to JSON
• Requires understanding of schema syntax
• Not suitable for deeply nested structures
• Relatively new format (emerging standard)

Technical Information

How TOON Works

TOON uses a schema declaration syntax to define structure, followed by data rows. The schema format is name[count]{field1,field2,...}: which declares an array with a specific number of objects and their fields. Data rows follow with just values (no keys), using indentation for nesting and commas for field separation.

Syntax Examples

Simple Object (No Quotes/Braces/Colons):

name: Alice
age: 30
city: Bengaluru

Array with Schema Declaration:

users[2]{id,name,role}:
  1.Alice,admin
  2.Bob,user

Nested Objects with Indentation:

user:
  id: 1
  name: Alice
  profile:
    age: 30
    city: Bengaluru

Array of Objects with Nested Fields:

teams[1]:
  - name: Team Alpha
    members[2]{id,name}:
      1.Alice
      2.Bob

Format Specifications

Common Tools & Libraries

• toon4s (Scala/JVM): Token-Oriented Object Notation for JVM
• ToonNet (.NET/C#): High-performance .NET implementation
• toon-middleware (Node.js): Express middleware for automatic conversion
• toonz (Zig): Zig implementation of TOON parser
• Toonia (Julia): TOON format parser for Julia
• Contextinator (Python): Codebase search with TOON support