Uploaded image for project: 'Node.js Driver'
  1. Node.js Driver
  2. NODE-5283

Move js logic from mongodb-client-encryption to the driver

XMLWordPrintableJSON

    • 8
    • 3
    • Needed
    • Hide

      Review the ticket description for general accuracy and completeness

      • Bug - Confirm that the bug still exists
      • Task / Feature / Improvement - Ensure every section of the template is filled out and makes sense
      • Build failure - Investigate and confirm the cause of the build failure
      • Spec change - Check whether any more recent changes have been made to the spec that might affect the implementation requirements

      What is the expected behavior?

      • What do the official driver or server docs currently say about this functionality?
        • What should they say?
          • If revisions or additions are needed, mark the ticket as docs changes needed and fill out the doc changes form
      • What do our api or readme docs currently say about this functionality?
        • What should they say?
        • Capture any revisions or additions in the ticket documentation AC
      • If applicable, what does the common drivers spec say? (Note: your kickoff partner should independently review the spec)
        • Are any clarifications or revisions needed?
      • If applicable, what do other drivers do?
        • If there is no common spec, is a common spec needed?
      • What should the behavior be?
      • Update the ticket description and implementation requirements as needed

      Review and address any unknowns explicitly called out in the ticket

      What will be the impact on users?

      • Who will be impacted?
      • Why might users care about this change?
      • Capture relevant detail in the "User Impact" section of the ticket description

      What will be the impact on any downstream projects? (e.g., shell, mongoose)

      • Update follow up requirements and create subtasks for follow up or coordination actions

      What variables affect the feature in question?

      • Server versions
      • Deployment types
      • Auth settings
      • Server and client configuration options
      • Specific apis / api options
      • Runtime or bundler settings
      • Special sequences of operations
      • Any other special conditions

      How should all the identified variables be tested?

      • Identify happy path and error case combinations of variables
        • Given [variables], when [action is performed], [feature] should [behave in the expected way]
      • How will we achieve the necessary coverage for these cases?
        • Automated spec tests?
          • Are there test runner changes required?
          • How up to date are our current tests and runners?
        • New integration or prose tests?
        • Unit tests?
      • Will we need to modify any existing tests?
      • Is there technical debt that will affect the implementation of new or existing tests?
      • Do we have the necessary tooling infrastructure already in place for any new tests?
      • Update test requirements on the ticket to reflect reality
      • Create subtasks for any testing groundwork that can happen independently of the implementation

      What is the scope of the code changes?

      • List the code bases and the areas of each code base that will need changes
      • Is there technical debt in any of these areas that will affect the implementation?
      • Identify any existing adjacent functionality that could be impacted by these changes
        • Is there sufficient existing test coverage for the adjacent functionality?
          • Update ticket test AC and create subtask(s) to cover existing functionality if coverage is missing
      • If multiple libraries are affected, determine the order in which changes need to go in
      • Create subtasks for the implementation (at least one per affected codebase)

      What is the expected impact on performance?

      • Do we have existing performance coverage for the affected areas?
      • Do we need to add new coverage?
        • Update ticket test AC and create subtask(s) as needed

      Consider backport requirements

      • Should this be backported?
      • What would be the cost of a backport?

      Is the metadata of this ticket accurate and complete?

      • Double check the acceptance criteria to ensure it accurately captures the expected behavior, test, and follow-up requirements
      • Double check the documentation requirements
      • Double check the task breakdown to ensure it covers all actionable items in the ticket AC
      Show
      Review the ticket description for general accuracy and completeness Bug - Confirm that the bug still exists Task / Feature / Improvement - Ensure every section of the template is filled out and makes sense Build failure - Investigate and confirm the cause of the build failure Spec change - Check whether any more recent changes have been made to the spec that might affect the implementation requirements What is the expected behavior? What do the official driver or server docs currently say about this functionality? What should they say? If revisions or additions are needed, mark the ticket as docs changes needed and fill out the doc changes form What do our api or readme docs currently say about this functionality? What should they say? Capture any revisions or additions in the ticket documentation AC If applicable, what does the common drivers spec say? (Note: your kickoff partner should independently review the spec) Are any clarifications or revisions needed? If applicable, what do other drivers do? If there is no common spec, is a common spec needed? What should the behavior be? Update the ticket description and implementation requirements as needed Review and address any unknowns explicitly called out in the ticket What will be the impact on users? Who will be impacted? Why might users care about this change? Capture relevant detail in the "User Impact" section of the ticket description What will be the impact on any downstream projects? (e.g., shell, mongoose) Update follow up requirements and create subtasks for follow up or coordination actions What variables affect the feature in question? Server versions Deployment types Auth settings Server and client configuration options Specific apis / api options Runtime or bundler settings Special sequences of operations Any other special conditions How should all the identified variables be tested? Identify happy path and error case combinations of variables Given [variables] , when [action is performed] , [feature] should [behave in the expected way] How will we achieve the necessary coverage for these cases? Automated spec tests? Are there test runner changes required? How up to date are our current tests and runners? New integration or prose tests? Unit tests? Will we need to modify any existing tests? Is there technical debt that will affect the implementation of new or existing tests? Do we have the necessary tooling infrastructure already in place for any new tests? Update test requirements on the ticket to reflect reality Create subtasks for any testing groundwork that can happen independently of the implementation What is the scope of the code changes? List the code bases and the areas of each code base that will need changes Is there technical debt in any of these areas that will affect the implementation? Identify any existing adjacent functionality that could be impacted by these changes Is there sufficient existing test coverage for the adjacent functionality? Update ticket test AC and create subtask(s) to cover existing functionality if coverage is missing If multiple libraries are affected, determine the order in which changes need to go in Create subtasks for the implementation (at least one per affected codebase) What is the expected impact on performance? Do we have existing performance coverage for the affected areas? Do we need to add new coverage? Update ticket test AC and create subtask(s) as needed Consider backport requirements Should this be backported? What would be the cost of a backport? Is the metadata of this ticket accurate and complete? Double check the acceptance criteria to ensure it accurately captures the expected behavior, test, and follow-up requirements Double check the documentation requirements Double check the task breakdown to ensure it covers all actionable items in the ticket AC
    • Needed
    • Hide
      1. What would you like to communicate to the user about this feature?
        1. see below
      2. Would you like the user to see examples of the syntax and/or executable code and its output?
        1. see below
      3. Which versions of the driver/connector does this apply to?
        1. mongodb-client-encryption 6.x+ and mongob 6.x+

      Note: for clarity about which versions are compatible with each driver, we will be releasing `mongodb-client-encryption@6.0.0` even though the latest version is currently `mongodb-client-encryption@2.9.0`.

      The Node team is making a large restructure of our Client Side Field Level Encryption logic in the next major release of the driver and mongodb-client-encryption.  Primarily, we are moving the majority of the code from `mongodb-client-encryption` into the driver.

      This has a few impacts on users

      • There is currently a cross-compatibility matrix between mongodb-client-encryption and the driver.  Starting in version 6.0 of the driver, only versions from the same major will be compatible with each other (i.e., mongodb-client-encryption 2.x will not be compatible with driver 6.0 and vice versa).
      • Users who consume `mongodb-client-encryption` directly already need to install a compatible version of the driver.  This has not changed.  However, starting in v6 users must import from the driver instead of `mongodb-client-encryption`.
      • [notable not affected] Users of AutoEncryption already install a compatible version of `mongodb-client-encryption` with the driver and configure auto encryption though the mongo client.  This has not changed.
      Show
      What would you like to communicate to the user about this feature? see below Would you like the user to see examples of the syntax and/or executable code and its output? see below Which versions of the driver/connector does this apply to? mongodb-client-encryption 6.x+ and mongob 6.x+ Note: for clarity about which versions are compatible with each driver, we will be releasing `mongodb-client-encryption@6.0.0` even though the latest version is currently `mongodb-client-encryption@2.9.0`. The Node team is making a large restructure of our Client Side Field Level Encryption logic in the next major release of the driver and mongodb-client-encryption.  Primarily, we are moving the majority of the code from `mongodb-client-encryption` into the driver. This has a few impacts on users There is currently a cross-compatibility matrix between mongodb-client-encryption and the driver.  Starting in version 6.0 of the driver, only versions from the same major will be compatible with each other (i.e., mongodb-client-encryption 2.x will  not be compatible with driver 6.0 and vice versa). Users who consume `mongodb-client-encryption` directly already need to install a compatible version of the driver.  This has not changed.  However, starting in v6 users must import from the driver instead of `mongodb-client-encryption`. [notable not affected] Users of AutoEncryption already install a compatible version of `mongodb-client-encryption` with the driver and configure auto encryption though the mongo client.  This has not changed.

      Use Case

      As a... node driver developer
      I want... fle bindings javascript logic to live in the driver
      So that... there is less code duplication and testing and releases require less work

      User Impact

      • Starting in driver 6.0.0 and mongodb-client-encryption 6.0.0, FLE logic will live in the driver instead of mongodb-client-encryption. If a user wishes to use FLE 6.0+, the user must
        • install a 6.0+ driver
        • install mongodb-client-encryption 6.0+
        • import the Javascript FLE code from the driver, instead of `mongodb-client-encryption`

      This is different from a pre-6.0 world, where the user was only required to install mongodb-client-encryption.

      Dependencies

      • N/A

      Unknowns

      • N/A

      Open Questions

      Do we want to generate documentation for mongodb-client-encryption after these changes? - No, see doc requirements

      Acceptance Criteria

      Implementation Requirements

      • Add js bindings code to the driver
        • [Subtask] Move JS logic and integrate into test suite 
          • Add @aws-sdk/credential-providers and gcp-metadata as optional peer dependencies in the driver. Ensure that these dependencies are lazy loaded.
            • If `socks` is not a peer optional dependency when doing this work, do nothing.  If `socks` is a peer optional dependency in the driver, make sure `socks` is lazy-loaded into the FLE code that uses it.
          • Move tests from bindings/node/test to driver relevant test directories. Ensure tests pass against the local linked mongodb-client-encryption.
          • Export any code needed from the libmongocrypt bindings needed to get the tests passing in the driver (i.e., class MongoCrypt).  A proper reworking of the bindings will come in a subsequent subtask but we need some restructuring here in order to run the driver's tests.
          • Add rough TS types for any changes to mongodb-client-encryption in deps.ts so that the driver can use it.
        • [Subtask] Clean up bindings in the driver
          • Convert to TS when moving - fix duplicate types.
          • Unify constants between the repos.
          • Make MongoCryptErrors (and subclasses) subclasses of MongoError
        • Release a 6.x pre-release of mongodb-client-encryption for driver to use (note: we plan to bump mongodb-client-encryption to 6.x at the next major so it aligns with the driver’s and bson’s major versions)
      • [Subtask] Remove js bindings code from libmongocrypt
        • Create unit tests in mongodb-client-encryption to test bindings only.
        • Cleanup driver evergreen tasks to no longer run the tests in mongodb-client-encryption.
        • Create public types for the bindings.
        • Cleanup unused dependencies from mongodb-client-encryption (@aws-sdk/credential-providers, gcp-metadata, socks, and mongodb)
      • [Subtask] Export new CSFLE types from the driver.
      • [Subtask] Add driver CI testing against the lowest supported peer dependency range version of the bindings alongside the latest version
      • [Subtask] Export javascript FLE logic from legacy mongodb package and setup driver testing with mongodb-legacy for all FLE logic.

      Testing Requirements

      • Unit tests if/as needed
      • CI test for lowest supported peer dependency range version (see above)

      Documentation Requirements

      The following changes are downstream docs changes (not driver API docs):

      • Adjust FLE documentation for Node to install the driver and mongodb-client-encryption, instead of just mongodb-client-encryption.
      • Adjust existing FLE examples to import explicit encryption code from the driver, instead of from `mongodb-client-encryption`
        • Coordinate with the docs team to ensure any example code is updated.
      • Add a note to mongodb-client-encryption readme stating that this package is not intended to be used in isolation and that we reserve the right to make changes that fall outside of semver
      • Adjust any internal documentation on generating docs and testing (in the driver and libmongocrypt) as necessary

      Follow Up Requirements

      • N/A

        1.
        		 Move JS logic into driver and integrate into test suite NODE-5419 Sub-task Closed Bailey Pearson 6.0.0
        2.
        		 Remove js bindings code from libmongocrypt NODE-5420 Sub-task Closed Bailey Pearson mongodb-client-encryption-6.0.0
        3.
        		 Clean up FLE's kms provider logic NODE-5422 Sub-task Closed Bailey Pearson 6.0.0
        4.
        		 Export FLE types from the driver and ensure FLE logic is included in driver documentation NODE-5423 Sub-task Closed Bailey Pearson 6.0.0
        5.
        		 Create driver CI testing matrix for FLE peer dependency versions NODE-5424 Sub-task Closed Bailey Pearson 6.0.0
        6.
        		 Address documentation requirements NODE-5425 Sub-task Closed Bailey Pearson mongodb-client-encryption-6.0.0
        7.
        		 Copy bindings javascript lib and test source into driver NODE-5426 Sub-task Closed Bailey Pearson 6.0.0
        8.
        		 Add testing to bindings in libmongocrypt NODE-5432 Sub-task Closed Bailey Pearson mongodb-client-encryption-6.0.0
        9.
        		 Investigate failing socks5 fle tests on Node20 and latest NODE-5468 Sub-task Closed Bailey Pearson  
        10.
        		 Clean up statemachine, autoencrypter, crypto callbacks and client encryption NODE-5470 Sub-task Closed Bailey Pearson 6.0.0
        11.
        		 allow explicitly setting options values to undefined in FLE bindings NODE-5516 Sub-task Closed Bailey Pearson mongodb-client-encryption-6.0.0

            Assignee:
            neal.beeken@mongodb.com Neal Beeken
            Reporter:
            daria.pardue@mongodb.com Daria Pardue
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: