-
Type: Improvement
-
Resolution: Fixed
-
Priority: Major - P3
-
Affects Version/s: 4.3.0
-
Component/s: Test Failure
-
None
The test documentation.DocumentationSamples.testQueryingArrayValues failed in Evergreen for the build variant tests-jdk8-unsecure__version~4.4_os~linux_topology~replicaset_auth~noauth_ssl~nossl_jdk~jdk8: https://spruce.mongodb.com/task/mongo_java_driver_tests_jdk8_unsecure__version~4.4_os~linux_topology~replicaset_auth~noauth_ssl~nossl_jdk~jdk8_test_patch_d57dfa8af686180239058a06eca838e18bfe5cdd_605277f93e8e863987251298_21_03_17_21_43_22/tests?execution=0&page=0&sortBy=STATUS&sortDir=ASC with the following assertion error
FAILURE: java.lang.AssertionError: expected:<5> but was:<7> (java.lang.AssertionError) java.lang.AssertionError: expected:<5> but was:<7> at org.junit.Assert.fail(Assert.java:89) at org.junit.Assert.failNotEquals(Assert.java:835) at org.junit.Assert.assertEquals(Assert.java:647) at org.junit.Assert.assertEquals(Assert.java:633) at documentation.DocumentationSamples.testQueryingArrayValues(DocumentationSamples.java:277)
This test is for the collection.insertMany code sample on the page https://docs.mongodb.com/manual/tutorial/query-arrays/.
The error happened despite documentation.DocumentationSamples doing collection.drop() synchronously in the @After tearDown method. The test class does the following:
...//some modifications to the collection collection.drop();//in @After tearDown collection.insertMany(asList( Document.parse("{ item: 'journal', qty: 25, tags: ['blank', 'red'], dim_cm: [ 14, 21 ] }"), Document.parse("{ item: 'notebook', qty: 50, tags: ['red', 'blank'], dim_cm: [ 14, 21 ] }"), Document.parse("{ item: 'paper', qty: 100, tags: ['red', 'blank', 'plain'], dim_cm: [ 14, 21 ] }"), Document.parse("{ item: 'planner', qty: 75, tags: ['blank', 'red'], dim_cm: [ 22.85, 30 ] }"), Document.parse("{ item: 'postcard', qty: 45, tags: ['blue'], dim_cm: [ 10, 15.25 ] }") )); //End Example 20 assertEquals(5, collection.countDocuments());
The assertion relies on the following:
- drop is ordered before insertMany (monotonic writes)
- countDocuments observes insertMany (read your writes).
According to https://docs.mongodb.com/manual/core/causal-consistency-read-write-concerns/, RYW is guaranteed only if the majority write and read concerns are used. Thus, the test must use them. However, the lack of RYW in the test cannot explain observing 7 elements in the collection instead of 5, which violates the monotonic writes consistency. The problem here is that "MongoDB provides monotonic write guarantees, by default, for standalone mongod instances and replica set.", and the test failed for a replica set. I do not know how to explain the failure, but I think it is worth thinking about and discussing.