Amazon DocumentDB

(1)

Amazon DocumentDB

Developer Guide

(2)

Amazon DocumentDB: Developer Guide

Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be aﬃliated with, connected to, or sponsored by Amazon.

(3)

What Is Amazon DocumentDB (with MongoDB Compatibility)

Amazon DocumentDB (with MongoDB compatibility) is a fast, reliable, and fully managed database service. Amazon DocumentDB makes it easy to set up, operate, and scale MongoDB-compatible databases in the cloud. With Amazon DocumentDB, you can run the same application code and use the same drivers and tools that you use with MongoDB.

Before using Amazon DocumentDB, you should review the concepts and features described in How It Works (p. 7). After that, complete the steps in Get Started Guide (p. 31).

Topics

• Overview of Amazon DocumentDB (p. 1)

• Clusters (p. 2)

• Instances (p. 2)

• Regions and Availability Zones (p. 4)

• Amazon DocumentDB Pricing (p. 5)

• Monitoring (p. 5)

• Interfaces (p. 6)

• What's Next? (p. 6)

• Amazon DocumentDB: How It Works (p. 7)

• What is a Document Database? (p. 17)

Overview of Amazon DocumentDB

The following are some high-level features of Amazon DocumentDB:

• Amazon DocumentDB automatically grows the size of your storage volume as your database storage needs grow. Your storage volume grows in increments of 10 GB, up to a maximum of 64 TB. You don't need to provision any excess storage for your cluster to handle future growth.

• With Amazon DocumentDB, you can increase read throughput to support high-volume application requests by creating up to 15 replica instances. Amazon DocumentDB replicas share the same underlying storage, lowering costs and avoiding the need to perform writes at the replica nodes. This capability frees up more processing power to serve read requests and reduces the replica lag time

—often down to single digit milliseconds. You can add replicas in minutes regardless of the storage volume size. Amazon DocumentDB also provides a reader endpoint, so the application can connect without having to track replicas as they are added and removed.

• Amazon DocumentDB lets you scale the compute and memory resources for each of your instances up or down. Compute scaling operations typically complete in a few minutes.

• Amazon DocumentDB runs in Amazon Virtual Private Cloud (Amazon VPC), so you can isolate your database in your own virtual network. You can also conﬁgure ﬁrewall settings to control network access to your cluster.

• Amazon DocumentDB continuously monitors the health of your cluster. On an instance failure, Amazon DocumentDB automatically restarts the instance and associated processes. Amazon DocumentDB doesn't require a crash recovery replay of database redo logs, which greatly reduces restart times. Amazon DocumentDB also isolates the database cache from the database process, enabling the cache to survive an instance restart.

(15)

Clusters

• On instance failure, Amazon DocumentDB automates failover to one of up to 15 Amazon DocumentDB replicas that you create in other Availability Zones. If no replicas have been provisioned and a failure occurs, Amazon DocumentDB tries to create a new Amazon DocumentDB instance automatically.

• The backup capability in Amazon DocumentDB enables point-in-time recovery for your cluster. This feature allows you to restore your cluster to any second during your retention period, up to the last 5 minutes. You can conﬁgure your automatic backup retention period up to 35 days. Automated backups are stored in Amazon Simple Storage Service (Amazon S3), which is designed for 99.999999999%

durability. Amazon DocumentDB backups are automatic, incremental, and continuous, and they have no impact on your cluster performance.

• With Amazon DocumentDB, you can encrypt your databases using keys that you create and control through AWS Key Management Service (AWS KMS). On a database cluster running with Amazon DocumentDB encryption, data stored at rest in the underlying storage is encrypted. The automated backups, snapshots, and replicas in the same cluster are also encrypted.

If you are new to AWS services, use the following resources to learn more:

• AWS oﬀers services for computing, databases, storage, analytics, and other functionality. For an overview of all AWS services, see Cloud Computing with Amazon Web Services.

• AWS provides a number of database services. For guidance on which service is best for your environment, see Databases on AWS.

Clusters

A cluster consists of 0 to 16 instances and a cluster storage volume that manages the data for those instances. All writes are done through the primary instance. All instances (primary and replicas) support reads. The cluster's data is stored in the cluster volume with copies in three diﬀerent Availability Zones.

Instances

An Amazon DocumentDB instance is an isolated database environment in the cloud. An instance can contain multiple user-created databases. You can create and modify an instance using the AWS Management Console or the AWS CLI.

(16)

Instances

The computation and memory capacity of an instance are determined by its instance class. You can select the instance that best meets your needs. If your needs change over time, you can choose a different instance class. For instance class specifications, see Instance Class Specifications (p. 303).

Amazon DocumentDB instances run only in the Amazon VPC environment. Amazon VPC gives you control of your virtual networking environment: You can choose your own IP address range, create subnets, and conﬁgure routing and access control lists (ACLs).

Before you can create Amazon DocumentDB instances, you must create a cluster to contain the instances.

Not all instance classes are supported in every region. The following table shows which instance classes are supported in each region.

Supported instance classes by Region

Region R6G R5 R4 T4G T3

US East (Ohio) SupportedSupported Supported SupportedSupported

US East (N.

Virginia) SupportedSupported Supported SupportedSupported

US West

(Oregon) SupportedSupported Supported SupportedSupported

South America

(São Paulo) SupportedSupported SupportedSupported

Asia Paciﬁc

(Mumbai) SupportedSupported SupportedSupported

Asia Paciﬁc

(Seoul) SupportedSupported SupportedSupported

Asia Paciﬁc

(Sydney) SupportedSupported SupportedSupported

Asia Paciﬁc

(Singapore) SupportedSupported SupportedSupported

Asia Paciﬁc

(Tokyo) SupportedSupported SupportedSupported

Canada (Central) SupportedSupported SupportedSupported

Europe

(Frankfurt) SupportedSupported SupportedSupported

Europe (Ireland) SupportedSupported Supported SupportedSupported

Europe (London) SupportedSupported SupportedSupported

Europe (Paris) SupportedSupported SupportedSupported

China (Ningxia) SupportedSupported SupportedSupported

AWS GovCloud

(US) SupportedSupported Supported

(17)

Regions and AZs

Regions and Availability Zones

Regions and Availability Zones deﬁne the physical locations of your cluster and instances.

Regions

AWS Cloud computing resources are housed in highly available data center facilities in diﬀerent areas of the world (for example, North America, Europe, or Asia). Each data center location is called a Region.

Each AWS Region is designed to be completely isolated from the other AWS Regions. Within each are multiple Availability Zones. By launching your nodes in diﬀerent Availability Zones, you can achieve the greatest possible fault tolerance. The following diagram shows a high-level view of how AWS Regions and Availability Zones work.

Availability Zones

Each AWS Region contains multiple distinct locations called Availability Zones. Each Availability Zone is engineered to be isolated from failures in other Availability Zones, and to provide inexpensive, low- latency network connectivity to other Availability Zones in the same Region. By launching instances for a given cluster in multiple Availability Zones, you can protect your applications from the unlikely event of an Availability Zone failing.

The Amazon DocumentDB architecture separates storage and compute. For the storage layer, Amazon DocumentDB replicates six copies of your data across three AWS Availability Zones. As an example, if you are launching an Amazon DocumentDB cluster in a Region that only supports two Availability Zones, your data storage will be replicated six ways across three Availability Zones but your compute instances will only be available in two Availability Zones.

The following table lists the number of Availability Zones that you can use in a given AWS Region to provision compute instances for your cluster.

Region Name Region Availability Zones (compute)

US East (Ohio) us-east-2 3

US East (N. Virginia) us-east-1 6

US West (Oregon) us-west-2 4

South America (São

Paulo) sa-east-1 3

Asia Paciﬁc (Mumbai) ap-south-1 3

Asia Paciﬁc (Seoul) ap-northeast-2 4

(18)

Pricing

Region Name Region Availability Zones (compute)

Asia Paciﬁc (Singapore) ap-southeast-1 3

Asia Paciﬁc (Sydney) ap-southeast-2 3

Asia Paciﬁc (Tokyo) ap-northeast-1 3

Canada (Central) ca-central-1 3

China (Ningxia) cn-northwest-1 3

Europe (Frankfurt) eu-central-1 3

Europe (Ireland) eu-west-1 3

Europe (London) eu-west-2 3

Europe (Paris) eu-west-3 3

AWS GovCloud (US) us-gov-west-1 3

Amazon DocumentDB Pricing

Amazon DocumentDB clusters are billed based on the following components. Amazon DocumentDB does not currently have a free tier so creating a cluster will incur costs.

• Instance hours (per hour)—Based on the instance class of the instance (for example, db.r5.xlarge).

Pricing is listed on a per-hour basis, but bills are calculated down to the second and show times in decimal form. Amazon DocumentDB usage is billed in one second increments, with a minimum of 10 minutes. For more information, see Managing Instance Classes (p. 299).

• I/O requests (per 1 million requests per month) — Total number of storage I/O requests that you make in a billing cycle.

• Backup storage (per GiB per month) — Backup storage is the storage that is associated with automated database backups and any active database snapshots that you have taken. Increasing your backup retention period or taking additional database snapshots increases the backup storage consumed by your database. Backup storage is metered in GB-months and per second does not apply.

For more information, see Backing Up and Restoring in Amazon DocumentDB (p. 211).

• Data transfer (per GB) — Data transfer in and out of your instance from or to the internet or other AWS Regions.

For detailed information, see Amazon DocumentDB (with MongoDB compatibility) pricing.

Monitoring

There are several ways that you can track the performance and health of an instance. You can use the free Amazon CloudWatch service to monitor the performance and health of an instance. You can ﬁnd performance charts on the Amazon DocumentDB console. You can subscribe to Amazon DocumentDB events to be notiﬁed when changes occur with an instance, snapshot, parameter group, or security group.

For more information, see the following:

• Monitoring Amazon DocumentDB with CloudWatch (p. 400)

(19)

Interfaces

• Logging Amazon DocumentDB API Calls with AWS CloudTrail (p. 412)

Interfaces

There are multiple ways for you to interact with Amazon DocumentDB, including the AWS Management Console and the AWS CLI.

AWS Management Console

The AWS Management Console is a simple web-based user interface. You can manage your clusters and instances from the console with no programming required. To access the Amazon DocumentDB console, sign in to the AWS Management Console and open the Amazon DocumentDB console at https://

console.aws.amazon.com/docdb.

AWS CLI

You can use the AWS Command Line Interface (AWS CLI) to manage your Amazon DocumentDB clusters and instances. With minimal conﬁguration, you can start using all of the functionality provided by the Amazon DocumentDB console from your favorite terminal program.

• To install the AWS CLI, see Installing the AWS Command Line Interface.

• To begin using the AWS CLI for Amazon DocumentDB, see AWS Command Line Interface Reference for Amazon DocumentDB.

The mongo Shell

To connect to your cluster to create, read, update, delete documents in your databases, you can use the mongo shell with Amazon DocumentDB. To download and install the mongo 4.0 shell, see Step 4: Install the mongo shell (p. 40).

MongoDB Drivers

For developing and writing applications against an Amazon DocumentDB cluster, you can also use the MongoDB drivers with Amazon DocumentDB.

What's Next?

The preceding section introduced you to the basic infrastructure components that Amazon DocumentDB oﬀers. What should you do next? Depending upon your circumstances, see one of the following topics to get started.

• Get started with Amazon DocumentDB by creating a cluster and instance using AWS CloudFormation Amazon DocumentDB Quick Start Using AWS CloudFormation (p. 44).

• Get started with Amazon DocumentDB by creating a cluster and instance using the instructions in our Get Started Guide (p. 31).

• Migrate your MongoDB implementation to Amazon DocumentDB using the guidance at Migrating to Amazon DocumentDB (p. 121)

(20)

How It Works

Amazon DocumentDB: How It Works

Amazon DocumentDB (with MongoDB compatibility) is a fully managed, MongoDB-compatible database service. With Amazon DocumentDB, you can run the same application code and use the same drivers and tools that you use with MongoDB. Amazon DocumentDB is compatible with MongoDB 3.6 and 4.0.

Topics

• Amazon DocumentDB Endpoints (p. 8)

• TLS Support (p. 10)

• Amazon DocumentDB Storage (p. 10)

• Amazon DocumentDB Replication (p. 11)

• Amazon DocumentDB Reliability (p. 11)

• Read Preference Options (p. 12)

• TTL Deletes (p. 14)

• Billable Resources (p. 15)

When you use Amazon DocumentDB, you begin by creating a cluster. A cluster consists of zero or more database instances and a cluster volume that manages the data for those instances. An Amazon DocumentDB cluster volume is a virtual database storage volume that spans multiple Availability Zones.

Each Availability Zone has a copy of the cluster data.

An Amazon DocumentDB cluster consists of two components:

• Cluster volume—Uses a cloud-native storage service to replicate data six ways across three Availability Zones, providing highly durable and available storage. An Amazon DocumentDB cluster has exactly one cluster volume, which can store up to 64 TB of data.

• Instances—Provide the processing power for the database, writing data to, and reading data from, the cluster storage volume. An Amazon DocumentDB cluster can have 0–16 instances.

Instances serve one of two roles:

• Primary instance—Supports read and write operations, and performs all the data modiﬁcations to the cluster volume. Each Amazon DocumentDB cluster has one primary instance.

• Replica instance—Supports only read operations. An Amazon DocumentDB cluster can have up to 15 replicas in addition to the primary instance. Having multiple replicas enables you to distribute read workloads. In addition, by placing replicas in separate Availability Zones, you also increase your cluster availability.

The following diagram illustrates the relationship between the cluster volume, the primary instance, and replicas in an Amazon DocumentDB cluster:

(21)

Amazon DocumentDB Endpoints

Cluster instances do not need to be of the same instance class, and they can be provisioned and terminated as desired. This architecture lets you scale your cluster’s compute capacity independently of its storage.

When your application writes data to the primary instance, the primary executes a durable write to the cluster volume. It then replicates the state of that write (not the data) to each active replica. Amazon DocumentDB replicas do not participate in processing writes, and thus Amazon DocumentDB replicas are advantageous for read scaling. Reads from Amazon DocumentDB replicas are eventually consistent with minimal replica lag—usually less than 100 milliseconds after the primary instance writes the data.

Reads from the replicas are guaranteed to be read in the order in which they were written to the primary.

Replica lag varies depending on the rate of data change, and periods of high write activity might increase the replica lag. For more information, see the ReplicationLag metrics at Amazon DocumentDB Metrics (p. 400).

Amazon DocumentDB Endpoints

Amazon DocumentDB provides multiple connection options to serve a wide range of use cases.

To connect to an instance in an Amazon DocumentDB cluster, you specify the instance's endpoint.

An endpoint is a host address and a port number, separated by a colon.

We recommend that you connect to your cluster using the cluster endpoint and in replica set mode (see Connecting to Amazon DocumentDB as a Replica Set (p. 448)) unless you have a speciﬁc use case for connecting to the reader endpoint or an instance endpoint. To route requests to your replicas, choose a driver read preference setting that maximizes read scaling while meeting your application's read consistency requirements. The secondaryPreferred read preference enables replica reads and frees up the primary instance to do more work.

The following endpoints are available from an Amazon DocumentDB cluster.

Cluster Endpoint

The cluster endpoint connects to your cluster’s current primary instance. The cluster endpoint can be used for read and write operations. An Amazon DocumentDB cluster has exactly one cluster endpoint.

The cluster endpoint provides failover support for read and write connections to the cluster. If your cluster’s current primary instance fails, and your cluster has at least one active read replica, the cluster endpoint automatically redirects connection requests to a new primary instance. When connecting to your Amazon DocumentDB cluster, we recommend that you connect to your cluster using the cluster endpoint and in replica set mode (see Connecting to Amazon DocumentDB as a Replica Set (p. 448)).

The following is an example Amazon DocumentDB cluster endpoint:

(22)

Amazon DocumentDB Endpoints

sample-cluster.cluster-123456789012.us-east-1.docdb.amazonaws.com:27017 The following is an example connection string using this cluster endpoint:

mongodb://username:[email protected] east-1.docdb.amazonaws.com:27017

For information about ﬁnding a cluster's endpoints, see Finding a Cluster's Endpoints (p. 365).

Reader Endpoint

The reader endpoint load balances read-only connections across all available replicas in your cluster.

Attempting to perform a write operation over a connection to the reader endpoint results in an error. An Amazon DocumentDB cluster has exactly one reader endpoint.

If the cluster contains only one (primary) instance, the reader endpoint connects to the primary instance.

When you add a replica instance to your Amazon DocumentDB cluster, the reader endpoint opens read- only connections to the new replica after it is active.

The following is an example reader endpoint for an Amazon DocumentDB cluster:

sample-cluster.cluster-ro-123456789012.us-east-1.docdb.amazonaws.com:27017 The following is an example connection string using a reader endpoint:

The reader endpoint load balances read-only connections, not read requests. If some reader endpoint connections are more heavily used than others, your read requests might not be equally balanced among instances in the cluster. It is recommended to distribute requests by connecting to the cluster endpoint as a replica set and utilizing the secondaryPreferred read preference option.

For information about ﬁnding a cluster's endpoints, see Finding a Cluster's Endpoints (p. 365).

Instance Endpoint

An instance endpoint connects to a speciﬁc instance within your cluster. The instance endpoint for the current primary instance can be used for read and write operations. However, attempting to perform write operations to an instance endpoint for a read replica results in an error. An Amazon DocumentDB cluster has one instance endpoint per active instance.

An instance endpoint provides direct control over connections to a speciﬁc instance for scenarios in which the cluster endpoint or reader endpoint might not be appropriate. An example use case is provisioning for a periodic read-only analytics workload. You can provision a larger-than-normal replica instance, connect directly to the new larger instance with its instance endpoint, run the analytics queries, and then terminate the instance. Using the instance endpoint keeps the analytics traﬃc from impacting other cluster instances.

The following is an example instance endpoint for a single instance in an Amazon DocumentDB cluster:

sample-instance.123456789012.us-east-1.docdb.amazonaws.com:27017 The following is an example connection string using this instance endpoint:

(23)

TLS Support

NoteAn instance’s role as primary or replica can change due to a failover event. Your applications should never assume that a particular instance endpoint is the primary instance. We do not recommend connecting to instance endpoints for production applications. Instead, we recommend that you connect to your cluster using the cluster endpoint and in replica set mode (see Connecting to Amazon DocumentDB as a Replica Set (p. 448)). For more advanced control of instance failover priority, see Understanding Amazon DocumentDB Cluster Fault Tolerance (p. 298).

For information about ﬁnding a cluster's endpoints, see Finding an Instance's Endpoint (p. 366).

Replica Set Mode

You can connect to your Amazon DocumentDB cluster endpoint in replica set mode by specifying the replica set name rs0. Connecting in replica set mode provides the ability to specify the Read Concern, Write Concern, and Read Preference options. For more information, see Read Consistency (p. 12).

The following is an example connection string connecting in replica set mode:

mongodb://username:[email protected] east-1.docdb.amazonaws.com:27017/?replicaSet=rs0

When you connect in replica set mode, your Amazon DocumentDB cluster appears to your drivers and clients as a replica set. Instances added and removed from your Amazon DocumentDB cluster are reﬂected automatically in the replica set conﬁguration.

Each Amazon DocumentDB cluster consists of a single replica set with the default name rs0. The replica set name cannot be modiﬁed.

Connecting to the cluster endpoint in replica set mode is the recommended method for general use.

NoteAll instances in an Amazon DocumentDB cluster listen on the same TCP port for connections.

TLS Support

For more details on connecting to Amazon DocumentDB using Transport Layer Security (TLS), see Encrypting Data in Transit (p. 147).

Amazon DocumentDB Storage

Amazon DocumentDB data is stored in a cluster volume, which is a single, virtual volume that uses solid state drives (SSDs). A cluster volume consists of six copies of your data, which are replicated automatically across multiple Availability Zones in a single AWS Region. This replication helps ensure that your data is highly durable, with less possibility of data loss. It also helps ensure that your cluster is more available during a failover because copies of your data already exist in other Availability Zones.

These copies can continue to serve data requests to the instances in your Amazon DocumentDB cluster.

How Data Storage is Billed

Amazon DocumentDB automatically increases the size of a cluster volume as the amount of data increases. An Amazon DocumentDB cluster volume can grow to a maximum size of 64 TiB; however, you are only charged for the space that you use in an Amazon DocumentDB cluster volume. When Amazon DocumentDB data is removed, such as by dropping a table or partition, the overall allocated space remains the same. The free space is reused automatically when data volume increases in the future.

NoteBecause storage costs are based on the storage "high water mark" (the maximum amount that was allocated for the Amazon DocumentDB cluster at any point in time), you can manage costs

(24)

Amazon DocumentDB Replication

by avoiding ETL practices that create large volumes of temporary information, or that load large volumes of new data prior to removing unneeded older data.

If removing data from an Amazon DocumentDB cluster results in a substantial amount of allocated but unused space, resetting the high water mark requires doing a logical data dump and restore to a new cluster, using a tool such as mongodump or mongorestore. Creating and restoring a snapshot does not reduce the allocated storage because the physical layout of the underlying storage remains the same in the restored snapshot.

NoteUsing utilities like mongodump and mongorestore incur I/O charges based on the sizes of the data that is being read and written to the storage volume.

For information about Amazon DocumentDB data storage and I/O pricing, see Amazon DocumentDB (with MongoDB compatibility) pricing and Pricing FAQs.

Amazon DocumentDB Replication

In an Amazon DocumentDB cluster, each replica instance exposes an independent endpoint. These replica endpoints provide read-only access to the data in the cluster volume. They enable you to scale the read workload for your data over multiple replicated instances. They also help improve the performance of data reads and increase the availability of the data in your Amazon DocumentDB cluster.

Amazon DocumentDB replicas are also failover targets and are quickly promoted if the primary instance for your Amazon DocumentDB cluster fails.

Amazon DocumentDB Reliability

Amazon DocumentDB is designed to be reliable, durable, and fault tolerant. (To improve availability, you should conﬁgure your Amazon DocumentDB cluster so that it has multiple replica instances in diﬀerent Availability Zones.) Amazon DocumentDB includes several automatic features that make it a reliable database solution.

Storage Auto-Repair

Amazon DocumentDB maintains multiple copies of your data in three Availability Zones, greatly reducing the chance of losing data due to a storage failure. Amazon DocumentDB automatically detects failures in the cluster volume. When a segment of a cluster volume fails, Amazon DocumentDB immediately repairs the segment. It uses the data from the other volumes that make up the cluster volume to help ensure that the data in the repaired segment is current. As a result, Amazon DocumentDB avoids data loss and reduces the need to perform a point-in-time restore to recover from an instance failure.

Survivable Cache Warming

Amazon DocumentDB manages its page cache in a separate process from the database so that the page cache can survive independently of the database. In the unlikely event of a database failure, the page cache remains in memory. This ensures that the buﬀer pool is warmed with the most current state when the database restarts.

Crash Recovery

Amazon DocumentDB is designed to recover from a crash almost instantaneously, and to continue serving your application data. Amazon DocumentDB performs crash recovery asynchronously on parallel threads so that your database is open and available almost immediately after a crash.

Resource Governance

Amazon DocumentDB safeguards resources that are needed to run critical processes in the service, such as health checks. To do this, and when an instance is experiencing high memory pressure,

(25)

Read Preference Options

Amazon DocumentDB will throttle requests. As a result, some operations may be queued to wait for the memory pressure to subside. If memory pressure continues, queued operations may timeout.

You can monitor whether or not the service throttling operations due to low memory with the following CloudWatch metrics: LowMemThrottleQueueDepth, LowMemThrottleMaxQueueDepth, LowMemNumOperationsThrottled, LowMemNumOperationsTimedOut. For more information, see Monitoring Amazon DocumentDB with CloudWatch. If you see sustained memory pressure on your instance as a result of the LowMem CloudWatch metrics, we advise that you scale-up your instance to provide additional memory for your workload.

Read Preference Options

Amazon DocumentDB uses a cloud-native shared storage service that replicates data six times across three Availability Zones to provide high levels of durability. Amazon DocumentDB does not rely on replicating data to multiple instances to achieve durability. Your cluster’s data is durable whether it contains a single instance or 15 instances.

Write Durability

Amazon DocumentDB uses a unique, distributed, fault-tolerant, self-healing storage system. This system replicates six copies (V=6) of your data across three AWS Availability Zones to provide high availability and durability. When writing data, Amazon DocumentDB ensures that all writes are durably recorded on a majority of nodes before acknowledging the write to the client. If you are running a three- node MongoDB replica set, using a write concern of {w:3, j:true} would yield the best possible conﬁguration when comparing with Amazon DocumentDB.

Writes to an Amazon DocumentDB cluster must be processed by the cluster’s writer instance. Attempting to write to a reader results in an error. An acknowledged write from an Amazon DocumentDB primary instance isdurable, and can't be rolled back. Amazon DocumentDB is highly durable by default and doesn't support a non-durable write option. You can't modify the durability level (that is, write concern).

Amazon DocumentDB ignores w=anything and is eﬀectively w: 3 and j: true. You cannot reduce it.

Because storage and compute are separated in the Amazon DocumentDB architecture, a cluster with a single instance is highly durable. Durability is handled at the storage layer. As a result, an Amazon DocumentDB cluster with a single instance and one with three instances achieve the same level of durability. You can conﬁgure your cluster to your speciﬁc use case while still providing high durability for your data.

Writes to an Amazon DocumentDB cluster are atomic within a single document.

Amazon DocumentDB does not support the wtimeout option and will not return an error if a value is speciﬁed. Writes to the primary Amazon DocumentDB instance are guaranteed not to block indeﬁnitely.

Read Isolation

Reads from an Amazon DocumentDB instance only return data that is durable before the query begins.

Reads never return data modiﬁed after the query begins execution nor are dirty reads possible under any circumstances.

Read Consistency

Data read from an Amazon DocumentDB cluster is durable and will not be rolled back. You can modify the read consistency for Amazon DocumentDB reads by specifying the read preference for the request or connection. Amazon DocumentDB does not support a non-durable read option.

Reads from an Amazon DocumentDB cluster’s primary instance are strongly consistent under normal operating conditions and have read-after-write consistency. If a failover event occurs between the write

(26)

Read Preference Options

and subsequent read, the system can brieﬂy return a read that is not strongly consistent. All reads from a read replica are eventually consistent and return the data in the same order, and often with less than 100 ms replica lag.

Amazon DocumentDB Read Preferences

Amazon DocumentDB supports setting a read preference option only when reading data from the cluster endpoint in replica set mode. Setting a read preference option aﬀects how your MongoDB client or driver routes read requests to instances in your Amazon DocumentDB cluster. You can set read preference options for a speciﬁc query, or as a general option in your MongoDB driver. (Consult your client or driver’s documentation for instructions on how to set a read preference option.)

If your client or driver is not connecting to an Amazon DocumentDB cluster endpoint in replica set mode, the result of specifying a read preference is undeﬁned.

Amazon DocumentDB does not support setting tag sets as a read preference.

Supported Read Preference Options

• primary—Specifying a primary read preference helps ensure that all reads are routed to the cluster’s primary instance. If the primary instance is unavailable, the read operation fails. A primary read preference yields read-after-write consistency and is appropriate for use cases that prioritize read-after-write consistency over high availability and read scaling.

The following example speciﬁes a primary read preference:

db.example.find().readPref('primary')

• primaryPreferred—Specifying a primaryPreferred read preference routes reads to the primary instance under normal operation. If there is a primary failover, the client routes requests to a replica.

A primaryPreferred read preference yields read-after-write consistency during normal operation, and eventually consistent reads during a failover event. A primaryPreferred read preference is appropriate for use cases that prioritize read-after-write consistency over read scaling, but still require high availability.

The following example speciﬁes a primaryPreferred read preference:

db.example.find().readPref('primaryPreferred')

• secondary—Specifying a secondary read preference ensures that reads are only routed to a replica, never the primary instance. If there are no replica instances in a cluster, the read request fails. A secondary read preference yields eventually consistent reads and is appropriate for use cases that prioritize primary instance write throughput over high availability and read-after-write consistency.

The following example speciﬁes a secondary read preference:

db.example.find().readPref('secondary')

• secondaryPreferred—Specifying a secondaryPreferred read preference ensures that reads are routed to a read replica when one or more replicas are active. If there are no active replica instances in a cluster, the read request is routed to the primary instance. A secondaryPreferred read preference yields eventually consistent reads when the read is serviced by a read replica. It yields read-after- write consistency when the read is serviced by the primary instance (barring failover events). A

(27)

TTL Deletes

secondaryPreferred read preference is appropriate for use cases that prioritize read scaling and high availability over read-after-write consistency.

The following example speciﬁes a secondaryPreferred read preference:

db.example.find().readPref('secondaryPreferred')

• nearest—Specifying a nearest read preference routes reads based solely on the measured latency between the client and all instances in the Amazon DocumentDB cluster. A nearest read preference yields eventually consistent reads when the read is serviced by a read replica. It yields read-after-write consistency when the read is serviced by the primary instance (barring failover events). A nearest read preference is appropriate for use cases that prioritize achieving the lowest possible read latency and high availability over read-after-write consistency and read scaling.

The following example speciﬁes a nearest read preference:

db.example.find().readPref('nearest')

High Availability

Amazon DocumentDB supports highly available cluster conﬁgurations by using replicas as failover targets for the primary instance. If the primary instance fails, an Amazon DocumentDB replica is promoted as the new primary, with a brief interruption during which read and write requests made to the primary instance fail with an exception.

If your Amazon DocumentDB cluster doesn't include any replicas, the primary instance is re-created during a failure. However, promoting an Amazon DocumentDB replica is much faster than re-creating the primary instance. So we recommend that you create one or more Amazon DocumentDB replicas as failover targets.

Replicas that are intended for use as failover targets should be of the same instance class as the primary instance. They should be provisioned in diﬀerent Availability Zones from the primary. You can control which replicas are preferred as failover targets. For best practices on conﬁguring Amazon DocumentDB for high availability, see Understanding Amazon DocumentDB Cluster Fault Tolerance (p. 298).

Scaling Reads

Amazon DocumentDB replicas are ideal for read scaling. They are fully dedicated to read operations on your cluster volume, that is, replicas do not process writes. Data replication happens within the cluster volume and not between instances. So each replica’s resources are dedicated to processing your queries, not replicating and writing data.

If your application needs more read capacity, you can add a replica to your cluster quickly (usually in less than ten minutes). If your read capacity requirements diminish, you can remove unneeded replicas. With Amazon DocumentDB replicas, you pay only for the read capacity that you need.

Amazon DocumentDB supports client-side read scaling through the use of Read Preference options. For more information, see Amazon DocumentDB Read Preferences (p. 13).

TTL Deletes

Deletes from a TTL index area achieved via a background process are best effort and are not guaranteed within a specific timeframe. Factors like instance size, instance resource utilization, document size, and overall throughput can affect the timing of a TTL delete.

(28)

Billable Resources

When the TTL monitor deletes your documents, each deletion incurs IO costs, which will increase your bill. If throughput and TTL delete rates increase, you should expect an increase in your bill due to increase IO usage.

When you create a TTL index on an existing collection, you must delete all expired documents before creating the index. The current TTL implementation is optimized for deleting a small fraction of

documents in the collection, which is typical if TTL was enabled on the collection from the start, and may result in higher IOPS than necessary if a large number of documents need to be deleted at one go.

If you do not want to create a TTL index to delete documents, you can instead segment documents into collections based on time, and simply drop those collections when the documents are no longer needed.

For example: you can create one collection per week and drop it without incurring IO costs. This can be signiﬁcantly more cost eﬀective than using a TTL index.

Billable Resources

Identifying Billable Amazon DocumentDB Resources

As a fully managed database service, Amazon DocumentDB charges for instances, storage, I/Os, backups, and data transfer. For more information, see Amazon DocumentDB (with MongoDB compatibility) pricing.

To discover billable resources in your account and potentially delete the resources, you can use the AWS Management Console or AWS CLI.

Using the AWS Management Console

Using the AWS Management Console, you can discover the Amazon DocumentDB clusters, instances, and snapshots that you have provisioned for a given AWS Region.

To discover clusters, instances, and snapshots

1. Sign in to the AWS Management Console, and open the Amazon DocumentDB console at https://

console.aws.amazon.com/docdb.

2. To discover billable resources in a Region other than your default Region, in the upper-right corner of the screen, choose the AWS Region that you want to search.

3. In the navigation pane, choose the type of billable resource that you're interested in: Clusters, Instances, or Snapshots.

4. All your provisioned clusters, instances, or snapshots for the Region are listed in the right pane. You will be charged for clusters, instances, and snapshots.

(29)

Billable Resources

Using the AWS CLI

Using the AWS CLI, you can discover the Amazon DocumentDB clusters, instances, and snapshots that you have provisioned for a given AWS Region.

To discover clusters and instances

The following code lists all your clusters and instances for the speciﬁed Region. If you want to search for clusters and instances in your default Region, you can omit the --region parameter.

Example

For Linux, macOS, or Unix:

aws docdb describe-db-clusters \ --region us-east-1 \

--query 'DBClusters[?Engine==`docdb`]' | \

grep -e "DBClusterIdentifier" -e "DBInstanceIdentifier"

For Windows:

aws docdb describe-db-clusters ^ --region us-east-1 ^

--query 'DBClusters[?Engine==`docdb`]' | ^

grep -e "DBClusterIdentifier" -e "DBInstanceIdentifier"

Output from this operation looks something like the following.

"DBClusterIdentifier": "docdb-2019-01-09-23-55-38",

"DBInstanceIdentifier": "docdb-2019-01-09-23-55-38", "DBInstanceIdentifier": "docdb-2019-01-09-23-55-382",

"DBClusterIdentifier": "sample-cluster",

"DBClusterIdentifier": "sample-cluster2",

To discover snapshots

The following code lists all your snapshots for the speciﬁed Region. If you want to search for snapshots in your default Region, you can omit the --region parameter.

For Linux, macOS, or Unix:

aws docdb describe-db-cluster-snapshots \ --region us-east-1 \

--query 'DBClusterSnapshots[?Engine==`docdb`].[DBClusterSnapshotIdentifier,SnapshotType]'

For Windows:

aws docdb describe-db-cluster-snapshots ^ --region us-east-1 ^

--query 'DBClusterSnapshots[?Engine==`docdb`].[DBClusterSnapshotIdentifier,SnapshotType]'

Output from this operation looks something like the following.

[ [

"rds:docdb-2019-01-09-23-55-38-2019-02-13-00-06", "automated"

], [

(30)

What is a Document Database?

"test-snap", "manual"

] ]

You only need to delete manual snapshots. Automated snapshots are deleted when you delete the cluster.

Deleting Unwanted Billable Resources

To delete a cluster, you must ﬁrst delete all the instances in the cluster.

• To delete instances, see Deleting an Amazon DocumentDB Instance (p. 317).

Important

Even if you delete the instances in a cluster, you are still billed for the storage and backup usage associated with that cluster. To stop all charges, you must also delete your cluster and manual snapshots.

• To delete clusters, see Deleting an Amazon DocumentDB Cluster (p. 292).

• To delete manual snapshots, see Deleting a Cluster Snapshot (p. 241).

What is a Document Database?

Some developers don't think of their data model in terms of normalized rows and columns. Typically, in the application tier, data is represented as a JSON document because it is more intuitive for developers to think of their data model as a document.

The popularity of document databases has grown because they let you persist data in a database by using the same document model format that you use in your application code. Document databases provide powerful and intuitive APIs for ﬂexible and agile development.

Topics

• Document Database Use Cases (p. 17)

• Understanding Documents (p. 18)

• Working with Documents (p. 22)

Document Database Use Cases

Your use case drives whether you need a document database or some other type of database for managing your data. Document databases are useful for workloads that require a ﬂexible schema for fast, iterative development. The following are some examples of use cases for which document databases can provide signiﬁcant advantages:

Topics

• User Proﬁles (p. 17)

• Real-Time Big Data (p. 18)

• Content Management (p. 18)

User Proﬁles

Because document databases have a flexible schema, they can store documents that have different attributes and data values. Document databases are a practical solution to online profiles in which

(31)

Understanding Documents

different users provide different types of information. Using a document database, you can store each user's profile efficiently by storing only the attributes that are specific to each user.

Suppose that a user elects to add or remove information from their proﬁle. In this case, their document could be easily replaced with an updated version that contains any recently added attributes and data or omits any newly omitted attributes and data. Document databases easily manage this level of individuality and ﬂuidity.

Real-Time Big Data

Historically, the ability to extract information from operational data was hampered by the fact that operational databases and analytical databases were maintained in diﬀerent environments—operational and business/reporting respectively. Being able to extract operational information in real time is critical in a highly competitive business environment. By using document databases, a business can store and manage operational data from any source and concurrently feed the data to the BI engine of choice for analysis. There is no requirement to have two environments.

Content Management

To eﬀectively manage content, you must be able to collect and aggregate content from a variety of sources, and then deliver it to the customer. Due to their ﬂexible schema, document databases are perfect for collecting and storing any type of data. You can use them to create and incorporate new types of content, including user-generated content, such as images, comments, and videos.

Understanding Documents

Document databases are used for storing semistructured data as a document—rather than normalizing data across multiple tables, each with a unique and ﬁxed structure, as in a relational database.

Documents stored in a document database use nested key-value pairs to provide the document's structure or schema. However, diﬀerent types of documents can be stored in the same document database, thus meeting the requirement for processing similar data that is in diﬀerent formats. For example, because each document is self-describing, the JSON-encoded documents for an online store that are described in the topic Example Documents in a Document Database (p. 20) can be stored in the same document database.

Topics

• SQL vs. Nonrelational Terminology (p. 18)

• Simple Documents (p. 19)

• Embedded Documents (p. 19)

• Example Documents in a Document Database (p. 20)

• Understanding Normalization in a Document Database (p. 21)

SQL vs. Nonrelational Terminology

The following table compares terminology used by document databases (MongoDB) with terminology used by SQL databases.

SQL MongoDB

Table Collection

Row Document

Column Field

Amazon DocumentDB

Amazon DocumentDB

Developer Guide

Amazon DocumentDB: Developer Guide

Table of Contents

What Is Amazon DocumentDB (with MongoDB Compatibility)

Overview of Amazon DocumentDB

Clusters

Instances

Regions and Availability Zones

Regions

Availability Zones

Amazon DocumentDB Pricing

Monitoring

Interfaces

AWS Management Console

AWS CLI

The mongo Shell

MongoDB Drivers

What's Next?

Amazon DocumentDB: How It Works

Amazon DocumentDB Endpoints

Cluster Endpoint

Reader Endpoint

Instance Endpoint

Replica Set Mode

TLS Support

Amazon DocumentDB Storage

How Data Storage is Billed

Amazon DocumentDB Replication

Amazon DocumentDB Reliability

Storage Auto-Repair

Survivable Cache Warming

Crash Recovery

Resource Governance

Read Preference Options

Write Durability

Read Isolation

Read Consistency

Amazon DocumentDB Read Preferences

High Availability

Scaling Reads

TTL Deletes

Billable Resources

Identifying Billable Amazon DocumentDB Resources

Using the AWS Management Console

Using the AWS CLI

Deleting Unwanted Billable Resources

What is a Document Database?

Document Database Use Cases

User Proﬁles

Real-Time Big Data

Content Management

Understanding Documents

SQL vs. Nonrelational Terminology