Graphify — AST call-graph extraction

For anyone building a code knowledge graph (us included — this is the reference tool behind Atlas U7.1's call-graph hook). Graphify turns a folder of code/docs/media into a queryable graph using a deterministic tree-sitter AST pass plus an LLM layer for the non-code stuff. The interesting part for Atlas is its honesty: it explicitly tags every edge EXTRACTED / INFERRED / AMBIGUOUS, which is exactly the candidate-grade framing we adopted when our own call-graph resolution landed at ~15–19% resolved. Sources grounded: the repo's README + ARCHITECTURE.md, the tree-sitter manual, and the Python static-call-graph literature (PyCG, JARVIS).

The one-sentence idea

Parse every file with tree-sitter into a syntax tree, walk it to emit `nodes` (functions, classes, imports) and `edges` (`calls`, `imports`, `uses`), then do a second pass to *guess* the call edges the first pass couldn't prove — and label each edge by how sure you are. Code never leaves the machine; only non-code inputs (PDFs, images, video) go through an LLM.

What it is

Graphify (safishamsi/graphify) is an AI-coding-assistant "skill" — you run /graphify . inside Claude Code / Codex / Cursor / Gemini CLI and it maps the project into an interconnected graph. Its scope is deliberately broad: it ingests ~36 tree-sitter grammars of source (Python, TS/JS, Go, Rust, Java, C/C++, Ruby, C#, Kotlin, Scala, PHP, Swift, Lua, Zig, Elixir, Verilog, Vue, Svelte, Dart, Salesforce Apex…), plus SQL schemas, Terraform/HCL, Markdown/YAML, Office docs, PDFs and images (via LLM vision), and audio/video (via faster-whisper). "App code + database schema + infrastructure in one graph" is the pitch.

The output is three artifacts: graph.json (the queryable graph), graph.html (interactive viz), and GRAPH_REPORT.md (highlights, "god nodes," surprising connections, and the AMBIGUOUS edges flagged for human review). It can export to Neo4j (Cypher), FalkorDB, GraphML, Obsidian vaults, and markdown wikis, and run as an MCP server so a team shares one graph over HTTP.

The privacy story is the headline: "Code is extracted locally with no API calls (AST via tree-sitter). Everything else goes through your AI assistant's model API." Tree-sitter is a rule-based parser — no model, no network — so a code-only corpus is fully offline and free.

Why it matters

Most "chat with your codebase" tools do blind full-text / embedding retrieval: they chunk files, embed them, and hope cosine similarity surfaces the right span. That loses structure — it can't answer "what calls charge_card?" or "what's the path from UserService to DatabasePool?" because those are graph traversals, not similarity lookups. Graphify's bet is that a structured graph grounded in the AST is a better substrate for an LLM than a pile of chunks: the model reasons over verified relationships, not retrieved text. This is the same thesis as [[graphrag]] (graph-structured retrieval beats flat RAG) and is why it's a sibling to our [[mempalace]] and [[agent-memory-systems]] work — a code graph is just structured memory about a codebase.

It matters to Atlas specifically because Atlas is a cron-refreshed code + knowledge graph of the OpenAlice org, and Graphify is the closest open reference implementation of the AST→call-graph step we built in U7.1.

How it works (the real mechanics)

The pipeline is a flat, stateless function chain (per ARCHITECTURE.md):

detect() → extract() → build_graph() → cluster() → analyze() → report() → export()

No shared state; stages pass plain Python dicts and a NetworkX graph between each other. The data schema is minimal:

// node
{"id": "unique_string", "label": "human name",
 "source_file": "path", "source_location": "L42"}

// edge
{"source": "id_a", "target": "id_b",
 "relation": "calls|imports|uses|...",
 "confidence": "EXTRACTED|INFERRED|AMBIGUOUS"}

1. Parse (tree-sitter). Each file is parsed into a concrete syntax tree. Crucial property, straight from the tree-sitter manual: it "does not resolve names, types, or create symbol tables — it captures the syntactic structure as written." So tree-sitter tells you "there is a call expression whose callee is the identifier `foo` at line 42" — it does not tell you which foo. It's robust (error-recovers on broken syntax) and incremental (re-parses only edited regions), which is what makes the git-hook --watch mode cheap.

2. Walk + collect. The extractor contract is literally: "tree-sitter parse → walk nodes → collect nodes and edges → call-graph second pass for INFERRED calls edges." Function/class/import definitions become nodes; the unambiguous facts — an import statement, a direct same-scope call — become `EXTRACTED` edges.

3. Call-graph second pass (the hard part). Here Graphify tries to connect each call site to its definition. The README/ARCHITECTURE are explicit that this pass is where uncertainty enters: anything resolved by heuristic — matching a callee name against known definitions, co-occurrence in context — is downgraded to `INFERRED`, and genuinely unresolvable / multiply-matched calls become `AMBIGUOUS` and get listed in GRAPH_REPORT.md for a human. Notably, the public docs do not describe a full name-resolution / import-resolution / scope algorithm — because doing that precisely is a research problem (see caveats).

4. Cache. Every file is hashed (SHA256); --update re-extracts only changed files and merges, so re-runs are incremental.

5. Cluster + analyze. Leiden community detection (when python < 3.13) groups the graph into semantic communities; the analysis step surfaces "god nodes" (the most-connected entities — your utils.py and your BaseModel) and bridges.

6. The hybrid (LLM) layer. For non-code inputs there is no AST, so Claude vision / the assistant's model extracts concepts and relationships — and those edges are INFERRED by construction. This is the clean seam in the design: the deterministic layer is provably correct and free; the probabilistic layer is for inputs where determinism is impossible. The two never get confused because the confidence tag travels on every edge.

Confidence is defined precisely (quoting ARCHITECTURE.md):

Key ideas & tradeoffs

• Determinism where you can, LLM where you must. Code → tree-sitter (cheap, private, exact). Media → model (necessary, costs tokens). The boundary is explicit, not blurred. This is the single most reusable design decision.
• Confidence as a first-class edge property. Instead of pretending the graph is ground truth, every edge carries provenance. Downstream consumers can filter by confidence — show only EXTRACTED for a "prove it" view, include INFERRED for recall. We do exactly this in Atlas.
• Syntax tree ≠ semantic graph. Graphify never claims tree-sitter resolves calls. The EXTRACTED/INFERRED split is the admission that the second pass is heuristic. Honesty over false precision.
• Breadth vs depth. 36 languages + media is enormous coverage, but coverage comes from staying at the syntactic layer for all of them — it does not run per-language type inference or points-to analysis. Wide and shallow by design.
• NetworkX + JSON, not a heavy graph engine by default. Simple, hackable, exportable to Neo4j/FalkorDB if you outgrow it. Pragmatic.

Honest caveats & limitations

• Call resolution is candidate-grade — and this is fundamental, not a bug. Resolving a call site to the definition is a hard static-analysis problem, especially in dynamic languages. The literature is blunt: PyCG, the precision-leading academic Python tool, hits ~99% precision but only ~70% recall, and "fails to scale… ran out of memory or exceeded the time limit for programs exceeding 2,000 lines" (arXiv 2305.05949). JARVIS improves recall with flow-sensitive type inference but is still a heavyweight research system. A tree-sitter "name-match" second pass like Graphify's is far simpler than PyCG, so its resolved-call edges are best read as candidates, which is exactly why they're tagged INFERRED, not EXTRACTED.
• Duck typing / dynamic dispatch / reflection defeat static matching. obj.method(), function pointers, getattr, monkeypatching, dependency injection — none are resolvable from syntax alone. These are precisely the cases that become AMBIGUOUS or are silently missed.
• Ghost duplicate nodes. The README admits the graph can contain "duplicate nodes for the same entity (ghost duplicates)," auto-merged at build time in recent versions — i.e. node identity/dedup is heuristic too.
• Visualization ceiling. Graphs >~5000 nodes can exceed browser rendering; fall back to CLI query modes.
• The non-code layer costs tokens and inherits LLM error. Concept edges from images/PDFs are only as good as the model that read them — and they're unverifiable.
• Docs lag the code. The public README/ARCHITECTURE deliberately don't spell out the resolution algorithm; treat any claim about "how exactly it resolves a call" as needing a source read of the implementation, not the docs. (The GoPenAI/Medium write-up that covers it kept redirect-looping behind a login wall and could not be fetched for this article — noted rather than fabricated.)

How it connects to OpenAlice

This is the open reference for the exact step Atlas built in U7.1. Per our memory, U7 indexed ~50k symbols across ~60 repos, but two known gaps remained: Rust impl-block methods (fixed in U7.1 — methods now indexed as kind='method') and a true call graph. U7.1's plan was, verbatim, to "steal graphify's AST call-graph + grep→graph hook" — and Graphify is precisely that prior art.

The connection is one-to-one:

• Our candidate-grade caveat is Graphify's `INFERRED` tag. Atlas resolves only ~15–19% of call edges; the rest are external/ambiguous. That is not a defect to hide — it's the same reality PyCG's 70% recall and Graphify's three-tier confidence model describe. The standing rule in agent memory is to filter call-graph queries by `confidence` (feedback-query-atlas-first), which is mechanically the same as filtering Graphify edges to EXTRACTED vs INFERRED. When an agent runs atlas.code.callers / atlas.code.callees, treat unresolved/low-confidence results as candidates, then grep to confirm.
• Determinism boundary matches ours. Atlas's symbol/import index is the EXTRACTED layer (exact, from parsing); semantic search (/api/v1/search, pgvector) is the INFERRED/concept layer — same split Graphify draws between tree-sitter and the LLM pass.
• The "grep→graph hook" is the right fallback. Because resolution is candidate-grade, both tools converge on the same advice: query the graph first, fall back to grep for the gaps — which is also the org rule (query Atlas before grepping, and the known-gaps line that sends you to grep for unresolved call edges).

If we ever harden Atlas's resolution beyond name-matching, the upgrade path is the PyCG/JARVIS direction: per-function flow-sensitive type inference. That buys recall at a steep scaling cost — worth knowing before promising "full call graph."

See also

[[graphrag]] · [[mempalace]] · [[agent-memory-systems]] · [[model-routing]] · [[fusion-and-llm-councils]] · [[mixture-of-agents]] · [[llm-maintained-wiki]] · [[llm-from-scratch]] · [[microgpt-build-an-llm-from-scratch]]