The needle in the haystack test is unfortunately not a very good measure of long-context performance. It turns out that searching for isolated pieces of information is a lot less demanding than synthesizing the entire context to solve a task.