Research pass mapped retrieval mismatch and missing metadata as major failures for RAG-style systems.
Tool profile · provisional profile
LlamaIndex
Strong ingestion and indexing primitives. It helps produce retrievable knowledge objects, but governance and context assembly remain system responsibilities.
Provisional fit
64/100
Best for: Document/corpus pipelines where connectors, parsing, indexing and query abstractions matter.
Avoid if: you need a fully governed, citation-complete knowledge architecture without adding policy, evidence capture, and review workflow around the tool.
Caution: Chunking and retrieval are not enough. Source authority, freshness, privacy and lifecycle need explicit design.
Model signature: Production primary · corpus scope · Data framework
Layer coverage
Where this tool fits.
This is not a completed review. It is a provisional profile from public positioning plus known failure-mode mapping. Hands-on benchmarks, source snapshots, and citation-bound claims are still required before stronger conclusions.
Production
Curation
Storage
Context Assembly
Governance
Review rule: a tool does not get credit for a layer unless it exposes inspectable behavior, not just a marketing claim.
Evidence notes
What the provisional profile has applied so far.
LlamaIndex is mapped to production because ingestion, parsing and indexing shape what can later be known.
Needs review for chunk provenance, metadata discipline, update/delete workflows and ranking policy.
Review packet
What a complete review must contain.
This page exposes the intended review structure. The current artifact is a profile, not a completed evidence-backed review.
Canonical source
Strengths
Document/corpus pipelines where connectors, parsing, indexing and query abstractions matter.
Limitations
Chunking and retrieval are not enough. Source authority, freshness, privacy and lifecycle need explicit design.
Dimension assessment
Scope, volatility, authority, lifecycle, resource economics, interoperability, and evidence quality must each get a rationale and citations before final scoring.
Open questions
- What can be verified from docs, code, issues, benchmarks, and changelogs?
- Where does the tool fail under stale, contradictory, private, or high-cost knowledge?
- Which claims are vendor claims versus independently observed behavior?
Benchmark critique
No benchmark number is accepted as architectural evidence unless it says which layer it tests and what it misses: lifecycle, scope boundaries, authority, context cost, and governance.
Related systems
Related tools should be connected by evidence-backed edges: competes with, integrates with, implements concept, evaluated by, or has governance gap.
Update history
Provisional profile created. Stale-review detection, source snapshots, and changelog watching are required before this becomes a durable review.