I frequently run two benchmark workloads – Linkbench and the insert benchmark. Secondary indexes are created for both after there is some data in the indexed table. For Linkbench there is one secondary index while in the insert benchmark I use 3 secondary indexes per collection and usually use 8 collections.

For the insert benchmark creating the secondary indexes is done concurrently with one client per collection (so 8 in parallel). There is only one secondary index for Linkbench. 

Creating the secondary index is faster in 4.4 than 4.2 for Linkbench. But for the insert benchmark it is almost 2X faster in 4.2 than 4.4.

My scripts use "indexed rows/second" rather than time to report performance (so larger == faster) and that is ips in the data that my test scripts report. There is a lot of data here but I will explain what I see using the results in this section

• for IO-bound Linkbench indexed rows/s is 247146 and 271190 for 4.2.8 without and with Snappy compression for the database compared to 113216 and 145950 for 4.4.0rc14 without and with Snappy compression.
• 4.4 uses less CPU than 4.2 – see the cpupq column which is CPU per indexed row
• 4.4 databases (see dbgb1) are larger than 4.2 which might explain why 4.4 reads more from storage / indexed row than 4.2 (see rkbpq). But this doesn't explain a 2X perf difference
• rkbps (storage read KB/s from iostat) is almost 2X larger in 4.2 than 4.4 and that might explain the 2X perf difference.
• however from a CPU-bound test where the collections are cached and there are no storage reads, 4.2 is still a lot faster (maybe 1.5X) than 4.4. Results are here

Some performance results from Linkbench are here and the indexed rows/s rate (ips) is better for 4.4 than 4.2

Next steps for me:

• Figure out why 4.2 can read from storage almost 2X faster despite using the same setup via DSI (c3.8xlarge with 20k EBS IOPs)
• Re-learn what the limits are for memory used by create index sorts in 4.2 and 4.4
• upload ftdc