Cognee

Further reading Context windows (Anthropic) · Using the Messages API (Anthropic) · Retrieval-augmented generation (Wikipedia) · Vector database (Wikipedia) · Knowledge graph (Wikipedia)

1. i.

   More is not better

   Bigger context windows sound like the answer, but more context isn't automatically better. As the token count grows, accuracy and recall degrade, which has earned the name context rotThe drop in a model's accuracy and recall as its context grows longer.. Models also use information best when it sits at the start or end of their input, and worst when it is buried in the middle, even models built for long contexts. Think of a desk piled so high you can no longer find the pen.

2. ii.

   Close is not relevant

   Similarity is a rough guide to usefulness. Vector search can miss key facts needed to answer a question, and one fix is to bolt old-fashioned keyword search on top. Plain RAG also fails on questions about a whole collection, such as "What are the main themes?", because answering those means summarising everything, and no single passage holds the answer.

3. iii.

   Right source, wrong story

   Fetching the right document doesn't guarantee the right answer. A model can lift a statement out of its context and reach the wrong conclusion. When sources disagree it may blend outdated and updated facts into one confident, misleading reply. RAG does not stop hallucination, and without specific training a model may answer when it should admit it isn't sure.

Further reading Context windows (Anthropic) · Lost in the Middle: How Language Models Use Long Contexts (arXiv) · Retrieval-augmented generation (Wikipedia) · From Local to Global: A Graph RAG Approach to Query-Focused Summarization (arXiv)

Further reading Cognee - Open-Source Agent Memory Platform (Cognee) · Cognee README (GitHub (topoteretes/cognee))

1. Step 1: Three stores, one memory

   Cognee keeps three kinds of storage side by side. A relational store tracks documents, chunks and where each came from. A vector store holds the embeddings. A graph store holds entities and the relationships between them. The idea is data that is searchable by meaning and connected by relationships at the same time.

2. Step 2: Reading into a graph

   As data comes in, a pipeline splits documents into smaller pieces, uses LLMs to pick out entities and relationships, writes a summary of each chunk and embeds the lot. It can run again as a dataset grows and skips what it has already processed. With no LLM key set, a small local model does the extraction instead.

3. Step 3: Asking the graph

   When an agent asks something, Cognee's recall runs graph retrieval by default rather than plain embedding similarity, and each result is tagged so you can tell where it came from.

4. Step 4: Learning from use

   A separate improve step raises or lowers the importance of parts of the graph, based on feedback on answers that used them. It also moves useful short-term session memory into the permanent graph. An optional check compares newly touched facts with the ones already stored nearby and records each conflict as its own link.

Further reading Core Concepts Overview (Cognee Documentation) · Cognify (Cognee Documentation) · Cognee README (GitHub (topoteretes/cognee)) · Recall (Cognee Documentation) · Improve (Cognee Documentation)

• How do you tell whether a memory works?

  Benchmarks such as BEAM test conversational memory with synthetic long conversations and a language model as the judge. Tuning studies of graph-based retrieval find gains that are consistent but not uniform, varying by dataset and metric, which says as much about the metrics as about the systems.

• Will bigger context windows make memory layers unnecessary?

  So far, longer isn't the same as better: recall degrades as context grows and facts in the middle get lost. That makes curating what goes into the window as important as how much of it there is, which is the job a memory layer is trying to do.

Further reading Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning (arXiv) · Cognee README (GitHub (topoteretes/cognee)) · Context windows (Anthropic) · Lost in the Middle: How Language Models Use Long Contexts (arXiv)

context window

All the text a language model can see at once while writing a reply, including the reply itself.

stateless

Keeping nothing between requests, so each one must bring along everything the model needs.

retrieval-augmented generation

Looking up relevant documents first and pasting them into the prompt before the model answers.

embeddings

Lists of numbers that represent a piece of text so that similar meanings end up close together.

vector database

A database that stores embeddings and finds the ones nearest to a query.

knowledge graph

A store of facts as entities joined by named relationships, like a map of who and what connects to what.

context rot

The drop in a model's accuracy and recall as its context grows longer.

1. 1Context windows · Anthropic
2. 2Using the Messages API · Anthropic
3. 3Retrieval-augmented generation · Wikipedia
4. 4Vector database · Wikipedia
5. 5Knowledge graph · Wikipedia
6. 6Lost in the Middle: How Language Models Use Long Contexts · arXiv
7. 7From Local to Global: A Graph RAG Approach to Query-Focused Summarization · arXiv
8. 8Cognee - Open-Source Agent Memory Platform · Cognee
9. 9Cognee README · GitHub (topoteretes/cognee)
10. 10Core Concepts Overview · Cognee Documentation
11. 11Cognify · Cognee Documentation
12. 12Recall · Cognee Documentation
13. 13Improve · Cognee Documentation
14. 14Optimizing the Interface Between Knowledge Graphs and LLMs for Complex Reasoning · arXiv