AWS Developer: Getting Started with Dynamo DB

Introduction

It is fully managed fast and flexible NoSql Database service available in AWS.

Before starting the course, we must have
- access to aws account
- AWS management console.

It is distributed database under AWS.
NoSQL Database Service: Key-Value store with Document Db attributes.
Fast: Designed for < 10ms response time.

Row in SQL is called as Item in DynamoDB.
Column in SQL is called as Attribute in DynamoDB.
Primary Key in SQL is called as Partition Key (single primary key) or Partition and Sort Key (composite primary key).

You can name dynamo db tables type or service
Ex: PS.dev.store

Scalar Datatypes: 
String
Number
Binary
Boolean true/false
Null : unknown or undefined state.

Set:
Set can be only be of one Scalar type.
String Set - ["Black", "Green", "Red"]
Number Set - [42.2,-29,7.5,3.14]

Note: ordering within the set is not preserved.
Document:
List : list of string or numbers
Map : key value pair

Performance Improvement

Capacity Units:
predefine throughput at table level
two controls which impact throughput
- Read Capacity Units (RCUs)
- Write Capacity Units(WCUs)

RCUs and WCUs can be updated anytime by using AWS management console or api's

Capacity Units and Consistency Models:
Partitions are underlined storage mechanisms in dynamoDB.
Here Table can be divided into multiple partitions to get one of the benefit like Manageability, Performance and Availability. Horizontal partitioning is dividing table by rows based on partition key. Vertical partitioning is dividing table by fields.

Two Modes of reading / writing data:
Strongly consistant
Eventually consistant

1KB (kilo byte) of data is roughly considered as 1 WCU. We can write 1KB data per second.
We can calculate value of WCU as below:
write data rounded up to nearest KB / 1KB
Ex: if we want to write 500Bytes of data in one write then 500Bytes rounded to 1KB, then 1KB/1KB= 1

Strong Consistent mode
1RCU = up to 4KB of data per second.
Eventually Consistent mode
0.5RCU  = up to 4KB of data per second.

For 20KB read operation we need 20KB/4KB = 5 RCUs required in strongly consistent mode.

Partition Key and Sort Key are different. Partition key is for faster access of data and Sort Key is for sorting based on value of field.
Region selection is also important, meaning a table AAAA created on same instance with different regions, then both tables get created and operated separately.

Tables can be created using AWS Management Console as below
Step 1: Login to AWS Management Console

 

 Step 2: Click on create table option

Step 3: Enter values for fields to create table with partition key ( primary key) and sort key 

Step 4: Uncheck 'Use Default Options' to set throughput parameters like WCU and RCU

Tables can also be created using AWS Cloud Formation via AWS Management Console:
 Input would be json file containing structure of table pass/upload it to input to cloud formation.



Table can also be created by using code using AWS SDK  (in javascript / java).
AWS SDK is API tailored to your application language can be accessed here:
https://aws.amazon.com/tools/

DynamoDB JSON - Scalar Data Types:
{ "<AttributeName>":{ "<DataType>:<AttributeValue>}}

Example:
{"First Name":{ "S":"Madhu"}}
{"Employee Id":{ "N":"5029"}}
{"Active":{"B":"true"}}

RCUs = Round up ( Item size / 4KB ) / 2   => for inconsistant DB
RCUs = Round up ( Item size / 4KB )    => for consistant DB

Read APIs:
Always use Query with proper keys instead of Scan. Read APIs by default, all Read API calls are eventually consistent.
- Get Item
  Get an item given the primary key.
- Query
  Get Items within a partition key
- Scan
  Scans every item in the table.
Batch APIs:
- BatchGetItem
One or more Get Item calls in one request.
- BatchWriteItem
 One or more PutItem or DeleteItem calls in one request.
Entire Batch API calls are not atomic.

By configuring DynamoDB Stream Events and Triggers, we can track updates to DB like Create/ update / delete operations on a table.

DynamoDB streams are table based feature, it capture table activity in event form. We can build triggers using Lambda + DynamoDB streams.
Local Secondary Index(LSI):
An index you can create on a table. Maximum of 5LSIs per table. LSI can only be created during creation of table, WCU and RCU shared. It allows same partition key but different sort key. Operates in Consistant and Eventually consistant Modes.
Partition Key: Hash Key
Sort Key: Range Key

Global Secondary Index:
An index you can create on a table and maximum 5 GSIs per table. GSI can be created anytime. GSIs have dedicated WCU and RCU. It allows different partition key and sort key. Operates in Eventually consistent reads only.

TTL (Time to Live): When we perform delete operation, we can generate events using DynamoDB Streams -> pass to Lambda expressions and create record in backup table. The closed job records once deleted, will be moved to backup table. Partition key is limited to 10GB of data, meaning at any point of time we can not fetch more than 10GB of data in single query result.

Auto Scaling: with auto scaling settings we can trigger increase of WCUs and RCUs but it is useful for consistent increase scenarios. With autoscaling within 10mins the WCU capcity will increase to handle high load scenarios.

Monitor real usage with CloudWatch with real metrics and alarms. CloudWatch is available in DynamoDB Management Console. avoid changing RCU/WCUs regularly.

DynamoDB Hard Limits:
Partition key capacity limits Max WCU:1000 Max RCU:3000
We can decrease RCUs / WCUs upto 4 times a day.
Partition Key data limit: 10GB

Item size (including indexes and projections) is 400KB.

Notes: 
Partition Key : Primary Key  or Hash Key
Sort Key : Order by index  field  or Range field.
We can update throughput parameters like WCU and RCU using updateTable API in javascript.
Primary key or partition key is unique and avoid querying cross partition keys.

Here are some important helpful links we found:

Introduction/installing/configuring the CLI:
https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html


Secondary indexes breakdown (Global vs Local indexes comparison):
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/SecondaryIndexes.html


Updating an existing Dynamo DB table using the Command line interface (AWS CLI):
https://docs.aws.amazon.com/cli/latest/reference/dynamodb/update-table.html


Updating an existing Dynamo FB table sing the AWS Java SDK (add or remove indexes, adjust read/writes per unit, etc.):
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/JavaDocumentAPITablesExample.html


Good concise explanation of indexing in Dynamo DB (Global vs local indexes, using the AWS CLI to create tables with a very good, full example included):
NOTE: The included example is for Creating tables, but based on the documentation the Update table command is very similar to this...

https://dzone.com/articles/indexing-in-dynamodb