Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-99151

Avoid unnecessary copying of response data

    • Type: Icon: Improvement Improvement
    • Resolution: Unresolved
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • None
    • Query Optimization
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Currently we're copying the response body into the CurOp as a way to keep it available for the recording, however that may cause unwanted memory pressure issues when in production.

      We need to find a way to use the response's memory space both for responding itself and also for recording.

      A couple options are:

      1. Keep a pointer to the whole message rather than the bson on the QueryData struct
      2. Locate (see parsing logic below) the Bson inside the message and point and own that memory space on the QueryData struct

       

      Suggested parsing logic based on OpMsg:: parse method

       

      BSONObj Message::obj() const {
          // MsgData::ConstView qr = singleData().view2ptr();
          // auto bsonObj = BSONObj(qr.data());
          // bsonObj.shareOwnershipWith(this->sharedBuffer());
          // bsonObj.makeOwned();    BSONObj bsonObj;    const uint32_t flags = OpMsg::flags(*this);
          auto dataSize = this->dataSize() - sizeof(flags);
          BufReader sectionsBuf(singleData().data() + sizeof(flags), dataSize);    while (!sectionsBuf.atEof()) {
              const auto sectionKind = sectionsBuf.read<uint8_t>();
              switch (sectionKind) {
                  case 0: {  // body
                      bsonObj = sectionsBuf.read<Validated<BSONObj>>();
                      break;
                  }            case 1: {  // Section::kDocSequence:
                      // The first 4 bytes are the total size, including themselves.
                      const auto remainingSize =
                          sectionsBuf.read<LittleEndian<int32_t>>() - sizeof(int32_t);
                      BufReader seqBuf(sectionsBuf.skip(remainingSize), remainingSize);
                      seqBuf.readCStr();
                      while (!seqBuf.atEof()) {
                          seqBuf.read<Validated<BSONObj>>();
                      }
                      break;
                  }            case 2: {  // Section::kSecurityToken
                      // if op_msg is parsed by mongoBridge, bridge has a backup check since multitenancy
                      // should be false
                      sectionsBuf.readCStr();
                      break;
                  }            default:
                      // Using uint32_t so we append as a decimal number rather than as a char.
                      uasserted(40432, str::stream() << "Unknown section kind " << uint32_t(sectionKind));
              }
          }    return bsonObj;
      }
      

       

       

            Assignee:
            Unassigned Unassigned
            Reporter:
            c.alonso@mongodb.com Carlos Alonso Pérez
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: