Python AWS Database Encryption SDK for DynamoDB

MPL-python-tests Code style: black Documentation Status

This is the official implementation of the AWS Database Encryption SDK for DynamoDB in Python.

The latest documentation can be found at Read the Docs.

Find the source code on GitHub.

Security

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public GitHub issue.

Getting Started

Required Prerequisites

  • Python 3.11+

  • aws-cryptographic-material-providers 1.10.0+

Installation

Note: If you have not already installed cryptography, you might need to install additional prerequisites as detailed in the cryptography installation guide for your operating system.

$ pip install aws-dbesdk-dynamodb

Concepts

The AWS Database Encryption SDK for DynamoDB (DBESDK-DynamoDB) is available in multiple languages. The concepts in the Python implementation of the DBESDK-DynamoDB are the same as in other languages. For more information on concepts in the DBESDK-DynamoDB, see the README for all languages.

DBESDK-DynamoDB uses cryptographic material providers from the AWS Cryptographic Material Providers Library (MPL). For more information on the MPL, see its README or readthedocs page.

Modules

aws_dbesdk_dynamodb.encrypted.client

High-level helper class to provide an encrypting wrapper for boto3 DynamoDB clients.

aws_dbesdk_dynamodb.encrypted.table

High-level helper class to provide an encrypting wrapper for boto3 DynamoDB tables.

aws_dbesdk_dynamodb.encrypted.item

Class for encrypting and decrypting individual DynamoDB items.

aws_dbesdk_dynamodb.encrypted.resource

High-level helper classes to provide encrypting wrappers for boto3 DynamoDB resources.

aws_dbesdk_dynamodb.encrypted.paginator

High-level helper class to provide an encrypting wrapper for boto3 DynamoDB paginators.

aws_dbesdk_dynamodb.smithygenerated.aws_cryptography_dbencryptionsdk_dynamodb.models

aws_dbesdk_dynamodb.smithygenerated.aws_cryptography_dbencryptionsdk_structuredencryption.models

aws_dbesdk_dynamodb.smithygenerated.aws_cryptography_dbencryptionsdk_dynamodb_itemencryptor.models

aws_dbesdk_dynamodb.smithygenerated.aws_cryptography_dbencryptionsdk_dynamodb_itemencryptor.config

The content below applies to all languages of the AWS DBESDK for DynamoDB.

AWS Database Encryption SDK for DynamoDB

📣 Note: This repository contains the source code and related files for all language implementations of the AWS Database Encryption SDK for DynamoDB. See our supported languages section for more information.

The AWS Database Encryption SDK (DB-ESDK) for DynamoDB is a client-side encryption library that allows you to perform attribute-level encryption, enabling you to encrypt specific attribute values within items before storing them in your DynamoDB table. All encryption and decryption are performed within your application. This lets you protect sensitive data in-transit and at-rest, as data cannot be exposed unless decrypted by your application.

For more details about the design and architecture of the DB-ESDK for DynamoDB, see the AWS Database Encryption SDK Developer Guide.

Security

If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public GitHub issue.

Support Policy

See Support Policy for details on the current support status of all major versions of this library.

Giving Feedback

We need your help in making this SDK great. Please participate in the community and contribute to this effort by submitting issues, participating in discussion forums and submitting pull requests through the following channels:

  • Submit issues - this is the preferred channel to interact with our team

  • Articulate your feature request or upvote existing ones

  • Ask questions on AWS re:Post under AWS Crypto Tools tag

Getting Started

Repository structure

This repository is a top level repository which houses all source code in order to compile this library into different runtimes.

This library is written in Dafny, a formally verifiable programming language that can be compiled into different runtimes. This library is currently ONLY supported in Java and .NET

AWS Integration

You need an Amazon Web Services (AWS) account to use the DB-ESDK for DynamoDB as it’s specifically designed to work with Amazon DynamoDB. Optionally, you can use AWS Key Management Service (AWS KMS) as your main keyring provider.

Supported Languages

  • Java

  • .NET

  • Dafny

  • Rust

  • Python

Contributing

See CONTRIBUTING for more information.

License

This project is licensed under the Apache-2.0 License.

Changelog

3.8.1 (2025-04-01)

This release is available in the following languages:

  • Java

Fixes

  • return plaintext items in UnprocessedItems in BatchWriteIttem (#1642) (7c7c8a1)

Maintenance

3.8.0 (2025-02-05)

This release is available in the following languages:

  • Java

Features

  • SharedCache: Shared Cache for Searchable Encryption (#1476) (46076f8)

  • bump to dafny 4.9.0 and mpl 1.9.0 (#1627)

Fixes

  • ensure algorithmSuite is properly copied in getTableConfig for E… (#1474) (ff3acac)

  • Java: Improve Collection of Errors string (#1478) (562b3ef)

  • SearchableEncryption: disable shared cached (#1507) (ffc67be)

  • SearchableEncryption: respect CMC contract (#1434) (b9333fb)

  • support new input validation (#1518) (12bbfbf)

  • Upgrade .net DDB SDK (8aeed4e)

Maintenance

3.7.0 (2024-09-17)

Features

Fixes

Maintenance

3.6.2 (2024-08-22)

Fixes

Maintenance

3.6.1 (2024-08-12)

Fixes

Maintenance

  • Add examples to examine contents of query error list (#1251) (b5705ee)

  • CI: add smithy diff checker GHA (#1226) (86406f5)

  • deps: bump actions/setup-dotnet from 3 to 4 in /.github/workflows (#1191) (c3b736e)

  • deps: bump aws-actions/configure-aws-credentials (#1190) (becbd0a)

  • deps: bump com.amazonaws:aws-java-sdk-dynamodb (#1230) (3aa25d0)

  • deps: bump dafny-lang/setup-dafny-action in /.github/workflows (#1200) (5284f0b)

  • deps: bump software.amazon.awssdk:bom (#1227) (abd1727)

  • deps: bump software.amazon.awssdk:bom (#1229) (bf3e1c3)

  • deps: bump software.amazon.awssdk:core (#1228) (9c67729)

  • do not add beacons when FORCE_PLAINTEXT_WRITE is used. (#1232) (23c8a18)

  • include bad item keys in query errors (#1244) (07bba8b)

  • update version to snapshot (#1225) (c817b5b)

3.6.0 (2024-07-23)

Features

  • allow indirect attribute names with MultiKeyStore (#1208) (4ab97bc)

Maintenance

3.5.0 (2024-05-30)

Features

  • DynamoDbEncryption: Add GetEncryptedDataKeyDescription operation (#856) (8f8471a)

  • Bump MPL to 1.4 (#1067) (51bbab5). This provides three new KMSConfiguration options when constructing a KeyStore (see https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/use-hierarchical-keyring.html). To KmsKeyArn are added KmsMRKeyArn, Discovery and MrDiscovery.

Maintenance

3.4.0 (2024-04-30)

Notes

.NET

  • #797 (785481c) Enforces User input Constraints at Type Conversion.

Prior to this fix, unset Integers defaulted to 0, and unset Booleans defaulted to false.

Now, all required fields MUST be set or a Runtime Exception will be thrown.

This particularly effects Searchable Encryption’s ConstructorPart, who’s required field previously would have defaulted to false. Any configuration ever created for Searchable Encryption can be re-created with the fix, but they may look different.

Features

  • feat(.NET): Validate user input #797 (785481c)

Maintenance

  • format: enforce Dafny formatting (#865) (dfc0dbd)

  • test: more test vectors (#959) (3ca15af)

  • CI add verify test for test vectors (#897) (6c980e7)

  • CI/CD: add semantic release automation (#949) (3f22abc)

  • deps: bump actions/setup-dotnet from 3 to 4 in /.github/workflows (#943) (f5d9748)

  • deps: bump aws-actions/configure-aws-credentials (#954) (90d7d78)

  • deps(Java): bump io.github.gradle-nexus.publish-plugin (#903) (04c6cc4)

  • deps(Java): bump org.projectlombok:lombok (#838) (56f1cd1)

  • deps: bump rrainn/dynamodb-action in /.github/workflows (#932) (16e4d7b)

  • docs: mention sign_and_include in javadoc for keyid supplier (#966) (2796693)

  • docs: point to the correct readme (#845) (b950b4a)

  • fix: repair json file names (#846) (3ca955a)

  • test(.NET): “dotnet pack” in CI (#851) (75e44d0)

  • test: add tests for attribute names that seem structured (#964) (c4c0886)

  • deps(Java & .NET): Update MPL to 1.3.0 (#972) (3d8acae)

3.2.0 2024-03-20

Features

  • A fourth Crypto Action will be made available : SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT, to join the existing DO_NOTHING, SIGN_ONLY and ENCRYPT_AND_SIGN. SIGN_AND_INCLUDE_IN_ENCRYPTION_CONTEXT behaves like SIGN_ONLY, but also includes the value in the encryption context, making it available to the branch key selector.

  • The Parsed Header, returned from EncryptItem and DecryptItem, now returns two more fields

    • encryptionContext : the full encryption context used for encryption

    • selectorContext : the encryption context as presented to the branch key selector

  • The Java Enhanced Client now supports Single Table Design. When using the DynamoDbEnhancedTableEncryptionConfig builder, one can now specify schemaOnEncrypt multiple times, once for each class being modeled in the table.

  • There was a hard limit of 100 on the size of maps and lists in Items to be encrypted. This limit has been removed.

3.2.0 2024-01-16

Features

  • support for .NET

  • Beacon Styles :

    • PartOnly : save a little bit of space for a beacon that used as part of a Compound Beacon, but never alone

    • AsSet : turn a set of values into a set of beacons, rather than into a single beacon

    • Twinned : calculate beacons for one attribute to be compatible with those from a different attribute

    • TwinnedSet : both AsSet and Twinned

  • Global Parts List : all compound beacons can now share a single list of Parts

  • Test vectors to ensure cross language compatibility

  • explicit error message when searching on a Compound Beacon that could never exist.

  • New APIs : ResolveAttributes and GetVirtualFields to assist in development and debugging.

Fix

  • String compare for client side filtering of Scan and Query results could sometimes produce the wrong result for certain characters.

3.1.2 2023-11-13

Fix

Fixed an issue where, when using the DynamoDbEncryptionInterceptor, an encrypted item in the Attributes field of a DeleteItem, PutItem, or UpdateItem response was passed through unmodified instead of being decrypted.

3.1.1 2023-11-07

Fix

Issue when a DynamoDB Set attribute is marked as SIGN_ONLY in the AWS Database Encryption SDK (DB-ESDK) for DynamoDB.

DB-ESDK for DynamoDB supports SIGN_ONLY and ENCRYPT_AND_SIGN attribute actions. In version 3.1.0 and below, when a Set type is assigned a SIGN_ONLY attribute action, there is a chance that signature validation of the record containing a Set will fail on read, even if the Set attributes contain the same values. The probability of a failure depends on the order of the elements in the Set combined with how DynamoDB returns this data, which is undefined.

This update addresses the issue by ensuring that any Set values are canonicalized in the same order while written to DynamoDB as when read back from DynamoDB.

See: https://github.com/aws/aws-database-encryption-sdk-dynamodb-java/tree/v3.1.1/DecryptWithPermute for additional details for additional details

3.1.0 2023-09-07

Features

  • Support underscores in DynamoDB expression attribute names

Maintenance

  • Upgrade various library dependencies

  • A variety of fixes to the library’s CI and testing

3.0.0 2023-07-24

Features

  • Updates to the AWS Cryptographic Material Providers Library for Java, a pivotal dependency of the this library, introduce Thread Safe Cryptographic Materials Caches (CMCs):

    • Storm Tracking Cache Safe for use in a multi threaded environment, tries to prevent redundant or overly parallel backend calls. See Spec changes for details.

    • Multi Threaded Cache Safe for use in a multi threaded environment, but no extra functionality

  • Examples for using the Enhanced Client via Lombok Annotation and TableSchemaBuilder

  • Detection of ignored DynamoDB Encryption Configuration Tags due to Nested Data Models

  • Multi Threading Example

BREAKING CHANGES

  • Updates to the AWS Cryptographic Material Providers Library for Java, a pivotal dependency of the this library, introduce the following breaking changes:

    • CMCs:

      • Original Cryptographic Materials Cache has been renamed to Single Threaded Cache

      • CreateCryptographicMaterialsCacheInput now ONLY accepts CacheType, which determines which, if any, of the three implemented CMCs will be returned.

      • The DefaultCache is StormTrackingCache

    • CreateAwsKmsHierarchicalKeyringInput:

      • no longer has a maxCacheSize field

      • now has an optional cache field for a CacheType

    • Hierarchical Keyring’s Key Store:

      • The Hierarchical Keyring’s Key Store’s Data Structure has changed. As such, entries persisted in the Key Store with prior versions of this library are NOT compatible. Instead, we recommend Creating a new DynamoDB Table for this version of the Key Store.

      • The Key Store’s CreateKeyInput now takes:

        • An Optional String branchKeyIdentifier

        • An Optional EncryptionContext encryptionContext

          • This encryptionContext will be added to the Encryption Context sent to KMS prefixed with aws-crypto-ec:

      • Creating a Key now also calls KMS:ReEncrypt

      • CreateKeyStore no longer creates a GSI

      • The Encryption Context used with KMS’ GenerateDataKeyWithoutPlaintext no longer includes the discarded GSI’s status.

      • More details about the Key Store’s changes are available in our Specification:

Fix

  • With the Enhanced Client, Identify Only Index attributes for Sign Only, NOT all Key Attributes, such as Auto Generated Last Modified Time Stamp.

Maintenance

  • A variety of fixes to the libraries CI and testing

3.0.0-preview-2 2023-06-09

Fix

  • The AWS SDK Core MUST NOT be depended on directly.

3.0.0-preview-1 2023-06-09

Features

  • Initial release of the AWS Database Encryption SDK. This release is considered a developer preview and is not intended for production use cases.