Lesson 14 / 25
Managing Conversation History
Keep recent turns verbatim and summarise the rest.
By Rahul Agarwal · 15 years in web development
Sliding window plus summary
Sending the entire conversation every turn eventually overflows the window and grows cost every turn. Common strategies: a sliding window (keep the last N turns verbatim), a running summary of older turns (goals, decisions, facts, open questions), or both. Summaries should be updated incrementally and checked, because a lost detail (an account number, a constraint the user stated) causes confusing errors later. Keep important facts in a structured state (fields, not prose) when your application can.
Long conversations and agents
Multi-turn chats and agents keep adding text; you must decide what to keep, summarise or fetch on demand.
Keeping the last four turns plus a summary, run
I ran this with plain Python 3 (standard library only); the data is made-up example data. Twelve turns become five context items: one summary placeholder for turns 1 to 8 and the last four turns verbatim. In a real system the summary would be written by a model and stored.
turns = [f"turn {i}: user/assistant exchange" for i in range(1, 13)]
keep_last = 4
older, recent = turns[:-keep_last], turns[-keep_last:]
summary = f"[summary of turns 1-{len(older)}: goals, decisions, open questions]"
context = [summary] + recent
print("original turns:", len(turns))
print("sent to model:", len(context), "items")
for line in context:
print(" ", line)
Output:
original turns: 12 sent to model: 5 items [summary of turns 1-8: goals, decisions, open questions] turn 9: user/assistant exchange turn 10: user/assistant exchange turn 11: user/assistant exchange turn 12: user/assistant exchange
Pin user-stated constraints
Store explicit user constraints (budget, language, deadlines) in a pinned state block that is never summarised away.
Quick check: Why not send the full conversation forever?
- It makes answers shorter
- Models cannot read history
- History is illegal to store
- It eventually overflows the window and increases cost every turn
Answer
It eventually overflows the window and increases cost every turn — Window plus summary keeps context bounded.