建置 AWS DynamoDB Local,以及如何匯入線上資料的方向
6 min readJun 21, 2024
前置準備:要有 Ubuntu 或 Linux 環境,裡面要有 Java
在 ubuntu 裡面,要先安裝 Java cli,因為 AWS 文件上寫明啟動 DynamoDB local 需要透過 Java
sudo apt install default-jdk
在 Download DynamoDB local 頁簽下我們直接下載 zip 並解壓縮在 ubuntu 環境下,進到那個資料夾,執行:
java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb
這時候會提示 local DynamoDB 已啟動,並放在 localhost:8000。
然後開第二個的 Terminal / Bash/ PowerShell,去跑
aws dynamodb list-tables --endpoint-url http://localhost:8000
刪除 Table:
aws dynamodb delete-table --table-name MyDB-Local --endpoint-url http://localhost:8000
建立 Table:
aws dynamodb create-table \
--table-name MyDB-Local \
--attribute-definitions AttributeName=pk,AttributeType=S AttributeName=sk,AttributeType=S AttributeName=gs1pk,AttributeType=S AttributeName=gs1sk,AttributeType=S AttributeName=gs2pk,AttributeType=S AttributeName=gs2sk,AttributeType=S \\
--key-schema AttributeName=pk,KeyType=HASH AttributeName=sk,KeyType=RANGE \
--provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5 \
--endpoint-url http://localhost:8000 \
--global-secondary-indexes \
'[{
"IndexName": "gs1",
"KeySchema": [{
"AttributeName": "gs1pk",
"KeyType": "HASH"
},{
"AttributeName": "gs1sk",
"KeyType": "RANGE"
}],
"Projection": {
"ProjectionType": "ALL"
},
"ProvisionedThroughput": {
"ReadCapacityUnits": 5,
"WriteCapacityUnits": 5
}
},{
"IndexName": "gs2",
"KeySchema": [{
"AttributeName": "gs2pk",
"KeyType": "HASH"
},{
"AttributeName": "gs2sk",
"KeyType": "RANGE"
}],
"Projection": {
"ProjectionType": "ALL"
},
"ProvisionedThroughput": {
"ReadCapacityUnits": 5,
"WriteCapacityUnits": 5
}}]'
新增資料:
aws dynamodb put-item \
--table-name MyDB-Local \
--endpoint http://localhost:8000 \
--item \
'{
"pk": {
"S": "AAAA1234"
},
"sk": {
"S": "AAAA1234"
},
"city": {
"S": "Acalanes Ridge"
},
"created": {
"S": "2024-03-14T02:43:59.836Z"
},
"state": {
"S": "California"
},
"timezone": {
"S": "America/Los_Angeles"
},
"updated": {
"S": "2024-03-21T01:56:46.107Z"
},
"zipCode": {
"S": "56457"
}
}'
取得資料
scan:
aws dynamodb scan --table-name MyDB-Local --endpoint http://localhost:8000
取得單一資料
aws dynamodb get-item \
--table-name MyDB-Local \
--endpoint http://localhost:8000 \
--key \
'{
"pk": { "S": "AAAA1234" },
"sk": { "S": "AAAA1234" }
}'
從線上 DynamoDB 匯入資料至 DynamoDB Local
匯出的方法為線上 DynamoDB Export to S3 (DynamoDB JSON),然後直接上 S3 去下載那個壓縮的檔案,可以直接下載這個檔案(但不是真的 valid JSON format)
再來的開發方向是寫迴圈,一個迴圈跑25筆資料給 BatchWriteItem,一行一行帶進來包 PutRecord,成以下連結中的資料格式,
可參考:
https://github.com/AxelTob/Load-multiple-items-AWS-DynamoDB---BatchWriteItem/blob/main/script.py
後續匯入完成時,可以輸入以下筆數計算來看真的匯入了
aws dynamodb scan --table-name MyDB-Local --select "COUNT" --endpoint http://localhost:8000