Graph Engineering: The Missing Layer After Context Engineering

In a previous article, we examined the shift from prompt engineering to context engineering — how the discipline expanded from crafting single instructions to curating the full set of tokens a model sees at inference time. We introduced a three-layer stack: prompt engineering at the message level, context engineering at the session level, and harness engineering at the system level.

That framework is now incomplete. A fourth layer has emerged, one that operates underneath all three: how knowledge itself is organized.

Prompt engineering controls what you say. Context engineering controls what the model sees. Graph engineering controls how the model's knowledge is structured — turning flat documents into networks of connected facts that agents can traverse, reason over, and build upon.

The One-Liner

Graph engineering structures knowledge as nodes and relationships so AI agents reason over connections, not just text chunks.

Why Flat Context Fails

Standard RAG works like this: split documents into chunks, embed them as vectors, retrieve the top-k most similar chunks to the query, and feed them to the model.

It works until it does not.

The failure mode is predictable. Any question that requires connecting information across multiple documents — what the research calls "multi-hop reasoning" — exposes the limitation. Vector similarity finds chunks that look like the query. It does not find the path between them.

Scenario: You run KYC/AML checks on a bank's customer graph

  Record A: "Customer C-4471 owns Account A-1092"
  Record B: "Account A-1092 sent funds to Account A-2207"
  Record C: "Account A-2207 sent funds back to Account A-1092's owner
             via a third account, closing the loop"
  Record D: "Customer C-4471 is on the sanctions watchlist"

  Question: "Which watch-listed customers are part of a circular
             transaction ring designed to disguise the source of funds?"

  Vector RAG: finds chunks similar to "transaction", "watchlist"
              → returns individual transaction records and compliance
                policy text, never the cycle connecting them

  Graph RAG:  traverses Account →[FROM|TO]→ Account up to 6 hops,
              filters for cycles back to a watch-listed owner
              → surfaces the ring directly

This is a real pattern, not a hypothetical one — Neo4j published a working version of it in 2025: a KYC agent built on the OpenAI Agents SDK, MCP, and a Neo4j graph of customers, accounts, and transactions, with a tool that hunts for exactly this kind of circular-payment ring by walking the account graph up to six hops. The code is open source. Vector search over transaction records can't see the loop; graph traversal finds it directly. (Neo4j: "GraphRAG in action: A simple agent for know-your-customer investigations", GitHub)

Vector search measures surface similarity. Graph traversal measures structural connection. Different problems require different primitives.

The challenge compounds with scale, though for a separate reason. Chroma's 2025 "Context Rot" research tested 18 frontier models and found every one of them got measurably less reliable as input length grew, well before hitting any token ceiling. That's a different problem than missing a relationship — it's about how much you can safely stuff into a single context window — but it points the same direction: loading more raw chunks isn't a substitute for giving the model a structure it can navigate.

What Graph Engineering Is

Graph engineering is the practice of constructing, maintaining, and querying explicit graph structures — nodes, edges, and properties — to organize knowledge for AI systems.

Two domains are converging under this term:

Knowledge Graph RAG — Build a knowledge graph from data, retrieve context by traversing relationships rather than similarity search, and ground model outputs in structured facts.

Agent Workflow Graphs — Structure multi-agent coordination as graphs where nodes represent tasks, agents, or system states, and edges represent communication, dependency, or control flow.

The term was formalized in August 2026 in a paper from multiple research institutions: "Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence" (arXiv:2608.21156). The authors position graph engineering as the successor to prompt engineering, context engineering, harness engineering, and loop engineering — the paradigm needed when individual agent intelligence hits its architectural limits.

Graph engineering at two levels:

  DATA LEVEL                          AGENT LEVEL
  (knowledge for reasoning)           (coordination for execution)

  ┌─────────────────────┐             ┌─────────────────────┐
  │  Entities            │             │  Tasks               │
  │  Relationships       │             │  Agent assignments   │
  │  Properties          │             │  Dependencies        │
  │  Communities         │             │  Communication edges │
  │  Paths               │             │  State transitions   │
  └─────────────────────┘             └─────────────────────┘
         │                                    │
         ▼                                    ▼
  GraphRAG retrieval                  Orchestration graphs
  Multi-hop reasoning                 Parallel execution
  Explainable answers                 Dependency resolution

Data-level graph engineering is the practical application most teams will encounter first. That is where the rest of this article focuses.

The Four Moves

Graph engineering follows a pipeline with four recurring operations:

Each move maps to a tool category. The extraction step builds the graph. The storage step makes it queryable. The retrieval step finds relevant subgraphs. The reasoning step turns paths into answers.

GraphRAG in Practice

The extract-cluster-summarize-search pattern was first described end-to-end in a 2024 Microsoft Research paper (Edge et al.), and it's since become the common blueprint that most GraphRAG tools — Microsoft's own library included — implement in some form. The general process:

┌─────────────────────────────────────────────────────────┐
│                  The GraphRAG Pipeline                    │
│                                                          │
│  1. EXTRACT     LLM reads raw text, pulls entities       │
│                 and relationships into triples            │
│                 (Subject → Predicate → Object)            │
│                                                          │
│  2. CLUSTER     Leiden algorithm groups connected        │
│                 entities into communities                 │
│                 (densely connected subgraphs)             │
│                                                          │
│  3. SUMMARIZE   LLM generates natural language           │
│                 summary of each community                 │
│                 (bottom-up, hierarchical)                 │
│                                                          │
│  4. SEARCH      Query-time: route to appropriate          │
│                 retrieval mode based on question type     │
│                                                          │
└─────────────────────────────────────────────────────────┘

Microsoft's implementation of this pattern supports three query modes that handle different question types:

Other implementations of the same pattern make different trade-offs — Neo4j's GraphRAG tooling, for instance, leans on Cypher traversal and MCP-exposed query tools rather than a fixed set of search modes (more on that below). The underlying idea is consistent across implementations: transform unstructured text into a navigable knowledge structure so query time drops, because the model receives evidence sets that already contain the relevant paths instead of having to search over millions of raw tokens.

Plugins and Tools That Do This Today

Neo4j + LlamaIndex

from llama_index.graph_stores.neo4j import Neo4jPropertyGraphStore
from llama_index.core import PropertyGraphIndex

# Connect to Neo4j graph store
graph_store = Neo4jPropertyGraphStore(
    url="bolt://localhost:7687",
    username="neo4j",
    password="password",
)

# Build property graph index from documents
index = PropertyGraphIndex.from_documents(
    documents,
    kg_extractors=[
        SimpleLLMPathExtractor(llm=llm),
        ImplicitPathExtractor(),
    ],
    graph_store=graph_store,
)

# Query with hybrid retrieval (vector + graph traversal)
query_engine = index.as_query_engine(
    similarity_top_k=5,
    include_text=True,
    path_depth=2,  # traverse 2 hops from matched entities
)

LlamaIndex provides Neo4jPropertyGraphStore for graph persistence, TextToCypherRetriever for natural language to graph queries, and VectorContextRetriever for hybrid search. The property graph index construction handles extraction automatically.

Docs: https://neo4j.com/developer/genai-ecosystem/llamaindex/

Neo4j + LangChain

from langchain_neo4j import GraphCypherQAChain, Neo4jGraph

graph = Neo4jGraph(
    url="bolt://localhost:7687",
    username="neo4j",
    password="password",
)

# Natural language → Cypher → execute → answer
chain = GraphCypherQAChain.from_llm(
    ChatOpenAI(temperature=0, model="gpt-4o"),
    graph=graph,
    verbose=True,
    allow_dangerous_requests=True,
    use_function_response=True,  # pass results as tool output
)

result = chain.invoke({"query": "Which watch-listed customers share an account in a circular transaction ring?"})

LangChain provides GraphCypherQAChain for text-to-Cypher translation, Neo4jVector for hybrid search with metadata filtering, and knowledge graph construction tools via GraphTransformer.

Docs: https://neo4j.com/labs/genai-ecosystem/langchain/

Neo4j Agent Memory

pip install neo4j-agent-memory

from neo4j_agent_memory import MemoryClient, MemorySettings

# Point at a Neo4j instance directly, or swap this for
# nams={"api_key": "nams_..."} to use the hosted service instead
settings = MemorySettings(
    neo4j={"uri": "bolt://localhost:7687", "password": "password"}
)

async with MemoryClient(settings) as client:
    # Short-term: conversation history with semantic search
    await client.short_term.add_message("session-1", "user", "Hello")

    # Long-term: extracted entities and facts (POLE+O model)
    await client.long_term.add_entity("Alice", "PERSON")

    # Pulls relevant short- and long-term context in one call
    context = await client.get_context("hello")

Neo4j Agent Memory provides persistent memory for agents backed by a knowledge graph. Three layers — short-term conversation history, long-term entities and facts, and reasoning traces — live in one graph.

GitHub: https://github.com/neo4j-labs/agent-memory Docs: https://neo4j.com/labs/agent-memory/

Neo4j MCP Server

Any agent framework that supports the Model Context Protocol can connect to Neo4j through a standardized tool interface. The MCP server exposes schema inspection and Cypher execution as callable tools — no framework-specific adapters required.

Supported frameworks: LangChain, LlamaIndex, CrewAI, OpenAI Agents SDK, Google ADK, AWS Strands, Claude Agent SDK, Pydantic AI, Microsoft Agent Framework, Vercel AI SDK, and more.

Docs: https://neo4j.com/labs/genai-ecosystem/agent-frameworks/

Microsoft GraphRAG

pip install graphrag

Open-source Python library implementing the extract-cluster-summarize-search pattern described above. Works with local LLMs or API providers. Produces a knowledge graph directory that can be queried locally or deployed as a service. Worth knowing about as the library that popularized this pattern, even if it's not the only — or necessarily the best-fit — option for your stack.

GitHub: https://github.com/microsoft/graphrag Docs: https://microsoft.github.io/graphrag/

Before and After: Same Question, Different Approach

This example is adapted from a real, open-source implementation: Neo4j published a KYC investigation agent in 2025 that does exactly this kind of ring detection over a graph of customers, accounts, and transactions (source, code).

Vector RAG Approach

User question: "Show me watch-listed customers involved in suspicious
circular transaction rings."

System prompt: Answer the question using the provided documents.
[Chunk 1: Transaction TX-8842: Account A-1092 → Account A-2207, $14,300]
[Chunk 2: Transaction TX-9013: Account A-2207 → Account A-3388, $14,100]
[Chunk 3: Transaction TX-9207: Account A-3388 → Account A-1092, $13,950]
[Chunk 4: Customer C-4471 profile — owns Account A-1092, watchlist: true]
[Chunk 5: AML compliance policy — quarterly review checklist]

Model output: "Here is a summary of recent large transactions..."

Wrong, or at best incomplete. Each transaction chunk looks similar to
"transaction" and "customer," so they surface individually — but nothing
in the embeddings marks A-1092 → A-2207 → A-3388 → A-1092 as a closed loop.
The ring is invisible until you connect the four chunks yourself.

Graph RAG Approach

# Simplified from the real find_customer_rings tool — see source above
# for the full, production version with pagination and risk filters

def find_customer_rings(graph, max_hops=6, watchlist_only=True):
    # Cypher-style traversal: find account cycles that return to
    # their starting point within max_hops
    query = """
        MATCH p = (a:Account)-[:FROM|TO*..$max_hops]->(a)
        WITH p, [n IN nodes(p) WHERE n:Account] AS accounts
        UNWIND accounts AS acct
        MATCH (cust:Customer)-[:OWNS]->(acct)
        WHERE cust.on_watchlist = $watchlist_only
        RETURN DISTINCT cust, collect(DISTINCT acct) AS ring_accounts
    """
    return graph.run(query, max_hops=max_hops, watchlist_only=watchlist_only)
    # → Customer C-4471 (watch-listed), ring: [A-1092, A-2207, A-3388]

Key difference: vector RAG retrieves chunks that match the query's surface. Graph RAG traverses the relationship structure that connects the facts — here, literally walking the account graph until it finds a path back to where it started. The ring was not stated in any single transaction record; it existed in the shape formed by all of them together.

When You Need Graph Engineering

You need graph engineering when:

• Questions require connecting facts across multiple documents or data sources
• Your domain has rich relationships (supply chains, org hierarchies, citation networks, medical ontologies)
• You need explainable answers — every response backed by traversable paths
• Multi-hop reasoning is common, not exceptional
• Existing RAG accuracy plateaus despite adding more data

You can skip it when:

• Tasks are single-document, single-hop
• Domain is flat (FAQ, simple classification)
• Existing vector RAG meets accuracy requirements
• No budget for graph database infrastructure

Hybrid approach (most common in practice):

┌─────────────────────────────────────────────────────────┐
│              Hybrid Retrieval Pipeline                    │
│                                                          │
│  User Query                                              │
│       │                                                  │
│       ├──→ Vector Search (fast, surface-level)           │
│       │       │                                          │
│       │       └──→ top-k chunks by similarity            │
│       │                                                  │
│       ├──→ Graph Traversal (slower, structural)          │
│       │       │                                          │
│       │       └──→ paths and neighbor subgraphs          │
│       │                                                  │
│       └──→ Merge + Rerank                                │
│               │                                          │
│               └──→ unified evidence set → LLM            │
│                                                          │
└─────────────────────────────────────────────────────────┘

Start with vector RAG. Add graph traversal when similarity search stops finding the right connections. Most production systems run both.

The Bottom Line

Prompt engineering is how you ask. Context engineering is what the model sees. Graph engineering is how knowledge is organized.

Each layer builds on the one below it. A clear prompt without good context produces confident wrong answers. Good context without structured knowledge produces plausible but incomplete answers. Structured knowledge without clear prompts and proper context framing produces accurate answers that never reach the model.

The teams building reliable AI systems in 2026 are not choosing between these disciplines. They are combining them: clear prompts that shape the interaction, context engineering that curates what the model sees at each step, and graph engineering that structures the knowledge the model reasons over.

Build the graph once. Query it many ways. Reason over connections, not just text.

References

• arXiv:2608.21156 — "Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence" (Aug 2026)
• arXiv:2603.06007 — "MASFactory: A Graph-centric Framework for Orchestrating LLM-Based Multi-Agent Systems with Vibe Graphing" (Mar 2026)
• arXiv:2606.11560 — "LLMs+Graphs: Toward Graph-Native, Synergistic AI Systems" (Jun 2026)
• Neo4j: "GraphRAG in action: A simple agent for know-your-customer investigations" — https://neo4j.com/blog/developer/graphrag-in-action-know-your-customer/ (code: https://github.com/neo4j-product-examples/graphrag-kyc-agent)
• Chroma Research: "Context Rot: How Increasing Input Tokens Impacts LLM Performance" (2025) — https://research.trychroma.com/context-rot
• Microsoft GraphRAG — https://microsoft.github.io/graphrag/
• Neo4j GenAI Ecosystem — https://neo4j.com/labs/genai-ecosystem/
• Neo4j Agent Memory — https://github.com/neo4j-labs/agent-memory
• LeanRAG (AAAI 2026) — Semantic aggregation for hierarchical graph retrieval
• HyGRAG (WWW 2026) — Context-aware and relation-aware graph RAG
• Youtu-GraphRAG (ICLR 2026) — Unified construction and retrieval via graph schema
• arXiv:2604.01610 — GraphWalk: Tool-based graph traversal for enterprise knowledge graphs

Published: September 2026