Ordering Content in a Long Context

Edges are safer than the middle

Studies of long-context use have found that models often use information at the beginning and end of the input better than information in the middle ("lost in the middle"), although the effect varies by model and is shrinking in newer ones. Practical defaults: put stable instructions first, place the most relevant documents at the start or right before the question, and put the question last. When you have many ranked chunks, an edge ordering places the best items at both ends. Test on your model rather than assuming.

Where things sit and how they are marked

Order and clear boundaries change how reliably the model uses each part of the context.

Edge ordering of ranked items, run

I ran this with plain Python 3 (standard library only); the data is made-up example data. Five ranked items are rearranged so the best (A) is first and the second best (B) is last, pushing the weakest (E) to the middle.

ranked = ["A (best)", "B", "C", "D", "E (worst)"]
# put the strongest items at the edges, the weakest in the middle
front, back = [], []
for i, item in enumerate(ranked):
    (front if i % 2 == 0 else back).append(item)
print("ranked order :", ranked)
print("edge ordering:", front + back[::-1])

Output:

ranked order : ['A (best)', 'B', 'C', 'D', 'E (worst)']
edge ordering: ['A (best)', 'C', 'E (worst)', 'D', 'B']

Run a position test

Place one key fact at the start, middle and end of a long context and measure accuracy for each position on your model.

Quick check: Where is a sensible default place for the user question in a long context?

• Repeated in every document
• Hidden in the middle of the documents
• Before the system rules
• At the end, after the documents