Uploaded image for project: 'C# Driver'
  1. C# Driver
  2. CSHARP-4395

C# Driver caches AWS credentials assumed for IAM Role until the first error

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Unknown Unknown
    • 2.19.0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Hide

      DRIVERS-2493:

      Summary of required changes

      • Ensure that AWS credentials fetched from environment variables are handled appropriately.
      • Add tests to verify environment variable handling

      Additional background

      Please see https://github.com/mongodb/specifications/commit/875446db44aade414011731840831f38a6c668dffor the specification change.

      Please see https://github.com/mongodb/mongo-python-driver/commit/ff94b0e3094f6bf08645ff0a491ec9b51f504b53 for a reference implementation in Python.

      Integration test

      Drivers are expected to add integration tests as described in the specification change

      Show
      DRIVERS-2493 : Summary of required changes Ensure that AWS credentials fetched from environment variables are handled appropriately. Add tests to verify environment variable handling Additional background Please see https://github.com/mongodb/specifications/commit/875446db44aade414011731840831f38a6c668dffor the specification change. Please see https://github.com/mongodb/mongo-python-driver/commit/ff94b0e3094f6bf08645ff0a491ec9b51f504b53  for a reference implementation in Python. Integration test Drivers are expected to add integration tests as described in the specification change

      Summary

      Our services deployed to EKS and connect to MongoDB Atlas. We use IAM role authentication, which is not yet natively supported by the driver.

      We assume role to get credentials and set corresponding environment variables for MongoDB Driver to pick up:

      Unable to find source-code formatter for language: csharp. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      using var stsClient = new AmazonSecurityTokenServiceClient();
      
      var assumeRoleResponse = await stsClient.AssumeRoleWithWebIdentityAsync(new AssumeRoleWithWebIdentityRequest
      {
          DurationSeconds = (int) CredentialsLifetime.TotalSeconds,
          RoleArn = roleArn,
          WebIdentityToken = GetWebIdentityToken(),
          RoleSessionName = "default"
      });
      
      var awsCredentials = assumeRoleResponse.Credentials;
      
      Environment.SetEnvironmentVariable("AWS_ACCESS_KEY_ID", awsCredentials.AccessKeyId);
      Environment.SetEnvironmentVariable("AWS_SECRET_ACCESS_KEY", awsCredentials.SecretAccessKey);
      Environment.SetEnvironmentVariable("AWS_SESSION_TOKEN", awsCredentials.SessionToken);
      
      

      Few minutes before the expiration we refresh credentials and reset environment variables. Up until v2.18.0 everything was working fine. New version has introduced credentials caching. This new caching assumes that credentials from environment variables never expire, which results in exception each time the token expires.

      As a workaround we have to manually clear the credentials cache upon refres, which is quite hacky:

      Unable to find source-code formatter for language: csharp. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      private static Action CreateClearAction()
      {
          var authenticators = typeof(MongoAWSAuthenticator).Assembly
              .GetType("MongoDB.Driver.Core.Authentication.External.ExternalCredentialsAuthenticators")!
              .GetProperty("Instance", BindingFlags.Public | BindingFlags.Static)!
              .GetValue(null);
      
          var cache = authenticators!
              .GetType()
              .GetProperty("Aws")!
              .GetValue(authenticators);
      
          var clear = cache!.GetType().GetMethod("Clear");
      
          return (Action)Delegate.CreateDelegate(typeof(Action), cache, clear!);
      }
      
      

      It would be great to have some built-in support for cache invalidation until the driver receives native support for IAM roles. As an idea, the driver could check the cached value of the environment variables and the current value to decide whether the cache has expired.

      Motivation

      Who is the affected end user?

      Users that use IAM Roles authentication in EKS.

      How does this affect the end user?

      User has to retry their operation.

      How likely is it that this problem or use case will occur?

      Easily reproducible.

      If the problem does occur, what are the consequences and how severe are they?

      The problem causes transient errors to our end users.

      Is this issue urgent?

      Not urgent as we implemented a workaround.

      Is this ticket required by a downstream team?

      No.

      Is this ticket only for tests?

      No.

            Assignee:
            dmitry.lukyanov@mongodb.com Dmitry Lukyanov (Inactive)
            Reporter:
            andrii.litvinov@gmail.com Andrii Litvinov
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: