Context

There are some cases where the pooled buffer behavior is unexpected:

• bytes.Buffer is always initialized with at least 64 bytes. If only values smaller than half of that (32 bytes) are marshaled, buffers will never be reused because of the buffer used capacity check (see here and here).
• Anytime a small value is marshaled while marshaling some large values, that buffer is discarded if the small value is less than half the size of the large values. That means marshaling even a few small values will result in significantly reduced buffer reuse (and more allocations).

We should find a pooled buffer management strategy that causes less unexpected churn.

Definition of done

• Implement a pooled buffer management system based on this suggested implementation.
• Add benchmarks that cover a range of use cases, including marshaling a mix of large and small messages.
• If we end up making a wrapper for a pooled buffer, we should require access to the buffer bytes to be through a copy of the byte slice. E.g. users must call Bytes(), which returns a copy of the bytes.
  • Note: It's not clear if this is possible in all places that we use pooled buffers in the Go Driver, but we should try to impose that.
  • Note: Maybe this belongs in a separate ticket?

Pitfalls

• If we do it wrong, we could introduce data race bugs or increase the amount of memory that the Go Driver holds onto (similar to a memory leak).