[discussing with jbr offline; filing this for tracking purposes]

The sync soak tests are triggering an unexpected sequence of events that should not be allowed. The sequence of events is something like this:

• T1: client creates an object, foo, out of view
• T2: client receives compensating write for foo
• T3: client receives another ERASE instruction for foo (for reasons that are not really relevant)
• T4: client updates foo

The operation @ T4 triggers a bad changeset because foo no longer exists. The surprising thing here is that the realm allowed T4 to happen at all.

The soak tests trigger this scenario all the time without the operation @ T3 and realm properly rejects the operation @ T4 which makes me think that the second ERASE may somehow be reviving the object though it should be idempotent.

Sample run: https://mongodb.enterprise.slack.com/files/U01V2DD3LRW/F0774L8HZQR/untitled.txt

CV=2438 -> illegal write that gets compensated
SV=40252 -> compensating write that erases obj
SV=40272 -> second erase obj
CV=2457 -> update on object that should not exist on the client