bug: Potential etcd events lost in PR #12514

[dramenk]

Current Behavior

I have been experiencing resource consumption issues due to etcd compaction errors, which led me to review the solution proposed in PR #12514.

It appears that APISIX fetches the latest global revision A by reading /phantomkey (a non-existent key?) before initiating a watch. If the subsequent watch times out without receiving any events, the next iteration uses the previously fetched revision A as the start_revision.

However, I have a question: If a "silent" error occurs during a watch (with start_revision:B), such as a silent network interruption occurring just before etcd sync the historical events, the watch will fail to get those events. If we then update the start_revision for the next watch to A+1, wouldn't the historical events between B and A be permanently lost? Or am I misunderstanding the current implementation?

Expected Behavior

No response

Error Logs

No response

Steps to Reproduce

null

Environment

• APISIX version (run apisix version):
• Operating system (run uname -a):
• OpenResty / Nginx version (run openresty -V or nginx -V):
• etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
• APISIX Dashboard version, if relevant:
• Plugin runner version, for issues related to plugin runners:
• LuaRocks version, for installation issues (run luarocks --version):

[Baoyuantop]

Hi @dramenk, could you provide the specific meaning and scenario of "silent network interruption"? Currently, this is just a question about the code logic; do you have specific steps to reproduce it?

[dramenk]

Hi @Baoyuantop ,

The following diagram helps clarify my point.

The actual issue I am facing is that etcd compaction triggers a configuration reload, leading to spikes in resource overhead. I came across that PR while searching for a solution.
To clarify, I am not currently experiencing the issue depicted in the diagram; I was just puzzled by the approach taken in the PR, I just read the code and suspect there might be a potential bug.