I thought pagination was easy... and I thought fixing pagination issues with AI was even easier
Let me tell you the sad story about an engineer who caused an incident because she understood what the code was doing (or so she thought!) but she didn't foresee the consequences 😬.
An incident! You might say. What a novel and interesting idea to make a post about that. To which I say, yeah. Why not? After all, we learn more from our mistakes, but I should warn you, this is not a post about one of these interesting incidents where the root cause and solution ends up being mind blowing and super complex, no, this is less about the tech and more about the people as many of my articles.
Once upon a time, there was a pagination bug
Pagination is one of those things you get familiar with early on, whether you are a FE or a BE Engineer, there's a time where you need to either consume an API with pagination or implement one and I've been on both sides of the coin so I thought it was a relatively straightforward concept and that's because I had only learned about the straightforward kinds of pagination 😅 (mostly offset pagination).
So, picture me troubleshooting an issue inside Elasticsearch, where pagination gets significantly more sophisticated because of scale and how fast data changes under the hood.
Enter: CURSOR PAGINATION.
NOTE: The goal of this post is not to explain the nitty gritty (we have AI now, which probably can explain it better than me 😉) but rather share the high level learnings beyond the technical specifics. I still, however, would like to leave some references as any good writer should do (find those at the end).
Cursor pagination with stable keys
Cursor pagination was new to me and the mistake I made was to rely purely on the context of the problem with my coding agent to try to understand it rather than reading some good old document and ask the agent to explain it outside my context (I only did this after the incident 😭)
Again, I don't want to explain how cursor pagination works but it is necessary to talk about an additional, important aspect: stable keys. When doing cursor pagination with stable keys like alphabetical order or ids, there's also not much magic on it. The anchor point doesn't move so you can reliably step through pages.
The problem: unstable keys
The problem is when you have unstable keys like relevant scores because search scores fluctuate dynamically as new data enters the system and results constantly shift rank between queries. A standard cursor can easily skip items or duplicate them across pages.
So what does a pragmatic engineer in the era of AI do? Of course: ask Claude to fix it and make sure to prompt at the end: Make no mistake! 🚀
The fix: PIT
My coding agent suggested a concept that yet again, I was not familiar with, Point In Time or PIT, which is a fancy way to take a snapshot of your dataset and try to keep the cursor stable so there's less risk of missing items.
“Make no mistake”, is a joke of course, that was not the prompt and I actually spent a good amount of time looking at logs, testing and understanding the solution that the agent was proposing, but that's precisely where you can fall into a trap: just because a solution makes sense on paper doesn't mean it's right or that it fits your system's scale.
Anyway, the code looked clean, made sense on paper, so here I go: I say LGTM, and click the merge button.
The downfall: memory
But as with any other solution, cursor pagination with PIT has trade offs, one of which is memory consumption. You want something, you have to pay for it, and that is precisely the cost I didn't fully account for at scale.
So there I am, merging my fancy PIT fix, and a few hours later finding myself in an incident call trying to figure out why production was complaining.
The interesting part of the incident was that nothing pointed out clearly that my changes were the culprit. Rather we just started getting generic 500 errors and some random ingestion errors.
We weren't 100% sure if my changes were the root cause or just a side effect, but my PR was the prime suspect. We reverted it, watched the errors drop down, and the suspicion was confirmed.
Still, I wanted to get to the bottom of it, so I dug deeper into metrics to cross check node heap pressure and confirm the incident was caused by memory going wild and the cluster trying to protect itself.
The decision in the end was to keep the changes reverted and call it the day.
Safe implementation: If I had to do it again
Of course the incident came with the typical post-mortem meeting and some action items like adding some additional alerts to catch API errors early and the PR remained reverted.
However, if someone asked me to implement cursor pagination with PIT again under real scale, the architectural rule is simple:
- Make PIT configurable. Start with a tiny value and tune according to needs.
- Monitor memory pressure continuously as you adjust parameters.
I remember a fellow engineer on the call pointing out that exact rule: "If that PIT ever comes back, let's make it configurable". A good reminder of how obvious state management trade offs look once you see them in production metrics 🙃.
Problem solved
But hey! You might be wondering, if you reverted your changes, how did you end up fixing the bug? Well, we did... and we did not.
The PR was reverted, we stabilized the system but the customer was still waiting for an answer, so the next day I took the time to try to explain the bug and the incident to my team. Honestly I was still connecting my thoughts and had some gaps.
That conversation was absolutely great. The team started asking the kind of foundational product questions that code alone never answers:
- Do our users actually need PIT consistency guarantees here?
- How many users are realistically affected by score drift on page 3?
- Can we just increase the default page size or adjust the query context instead?
- Why is the user navigating this specific interface this way in the first place?
🤯 So many great questions.
Together, we realized we really didn't need the complexity and cost of a solution like PIT. We agreed on a clean workaround, communicated transparently with the customer, and solved the problem without unnecessary complications.
For me, this was a massive reminder that software engineering is rarely just about technical execution.
The moral of the story
There are so many good reminders from this story, the usual suspects are these:
- Dare to be vulnerable. Don't just admit mistakes, dare to share your half-baked learnings. When you don't fully understand an edge case yet, still bring it to your team. Collaboration beats lone-wolf fixes EVERY. SINGLE. TIME.
- Engineering is about deciding what NOT to build. The best architectural choice is often staying away from complexity, not adding a fancier feature.
- Communicate transparently with users. What do they actually need vs. what we assume they need?
But for me the most important lesson is around how we work with AI, I kind of mentioned earlier in the post but nowadays, code changes are so cheap that it’s so tempting to jump straight to your console and ask your agent to fix or create anything and get that quick dopamine rush to see something get done and move on.
I always find it so interesting that people keep mentioning that in order to work with AI you only need "good judgment", The catch? Judgment isn't free, it comes from exposure and mistakes, judgment gets built continuously so I have no shame to admit that I didn't show good judgment here. I fell for the quick fix because I lacked experience with this specific scale pattern, and my judgment failed me.
That's why for me, no matter how powerful AI gets and how fast it produces code, there's no way to outsource understanding, we can't pretend that the quick fix will always succeed. Sometimes it works, sure, but in many situations we still need time to slow down, read and understand. In order to even exercise judgment, any kind of judgment, we require those three things.
But Nelida! You probably would have made the same mistake without AI. Yes, perhaps, and here’s my take: without AI the issue might still have happened but I argue the changes would have been slimmer. AI removed the friction of actual learning. I only collaborated with my team and read docs because of the incident. Without AI I would have done that despite the incident which would have increased the chances of me realizing and understanding the memory cost. It is sad to think that without the incident there was no learning or at least not as deep.
For that reason I think collaboration these days is one of the most important ways we have to counteract the effects of learning erosion. In these times, we need to make collaboration deliberate, intentional, more than ever. I wish I had explained the bug to my team before or had paired with someone instead of with my agent 🤖.