Nosql

The DynamoDB Scan API accesses every items in a table (or secondary index). It is the equivalent of a select * from query. One of the things I will cover in this blog is how to use Scan API with the DynamoDB Go SDK.

To scan a table, we need some data to begin with! So in the process, I will also go into how to use the Batch API to write bulk data in DynamoDB. You can use the BatchWriteItem API to create or delete items in batches (of twenty five) and it’s possible to you can combine these operations across multiple tables.

Learn with practical code samples

DynamoDB provides a rich set of data types including Strings, Numbers, Sets, Lists, Maps etc. In the Go SDK for DynamoDB, the types package contains Go representations of these data types and the attributevalue module provides functions to work with Go and DynamoDB types.

This blog post will demonstrate how to handle conversions between Go types in your application and DynamoDB. We will start off with simple code snippets to introduce some of the API constructs and wrap up with a example of how to use these Go SDK features in the context of a complete application (including a code walk though).

Earlier, I wrote about a Serverless URL shortener application on AWS using DynamoDB, AWS Lambda and API Gateway.

In this blog post, we will deploy that as a REST API on AWS App Runner and continue to use DynamoDB as the database. AWS App Runner is a compute service that makes it easy to deploy applications from a container image (or source code), manage their scalability, deployment pipelines and more.

With the help of a practical example presented in this blog, you will:

Using AWS Lambda, DynamoDB and API Gateway

This blog post covers how to build a Serverless URL shortener application using Go. It leverages AWS Lambda for business logic, DynamoDB for persistence and API Gateway to provide the HTTP endpoints to access and use the application. The sample application presented in this blog is a trimmed down version of bit.ly or other solutions you may have used or encountered.

It’s structured as follows:

• I will start off with a quick introduction and dive into how to deploy try the solution.
• After that, I will focus on the code itself. This will cover:
  • The part which is used to write the infrastructure (using Go bindings for AWS CDK)
  • And also the core business logic which contains the Lambda function (using Lambda Go support) as well as the DynamoDB operations (using the DynamoDB Go SDK)

In this blog, you will learn:

Replicate DynamoDB data from one table to another

This blog post will help you get quickly started with DynamoDB Streams and AWS Lambda using Go. It will cover how to deploy the entire solution using AWS CDK.

The use case presented here is pretty simple. There are a couple of DynamoDB tables and the goal is to capture the data in one of those tables (also referred to as the source table) and replicate them to another table (also referred to as the target table) so that it can serve different queries. To demonstrate an end-to-end flow, there is also an Amazon API Gateway that front ends a Lambda function which persists data in the source DynamoDB table. Changes in this table will trigger another Lambda function (thanks to DynamoDB Streams) which will finally replicate the data into the target table.

Use change data capture with MSK Connect to sync data between Aurora MySQL and DynamoDB

This is the second part of the blog series which provides a step-by-step walkthrough of data pipelines with Kafka and Kafka Connect. I will be using AWS for demonstration purposes, but the concepts apply to any equivalent options (e.g. running these locally in Docker).

This part will show Change Data Capture in action that let’s you track row-level changes in database tables in response to create, update and delete operations. For example, in MySQL, these change data events are exposed via the MySQL binary log (binlog).

Integrate DynamoDB with MSK and MSK Connect

There are many ways to stitch data pipelines - open source components, managed services, ETL tools, etc. In the Kafka world, Kafka Connect is the tool of choice for “streaming data between Apache Kafka and other systems”. It has an extensive set of pre-built source and sink connectors as well as a common framework for Kafka connectors which standardises integration of other data systems with Kafka and making it simpler to develop your own connectors, should there be a need to do so.

This is the second blog post of this two-part series that uses a practical application to demonstrate how to integrate Redis with AWS Lambda. The first part was about the solution overview, deployment and hopefully you were able to try it out end to end. As promised, the second part (this one) will cover the Infrastructure aspects (IaaC to be specific) which is comprised of three (CDK) Stacks (in the context of a single CDK App).

Run your Go app on AWS App Runner service, integrate it with MemoryDB for Redis and use AWS CDK to package and deploy the app along with infrastructure

AWS App Runner allows you to deploy and run cloud-native applications in a fast, simple, and cost-effective manner. You can choose the programming language of your choice since App Runner can deploy directly from source code (in GitHub for example) or a Docker container image (from private or public repo in ECR) - all this without worrying about provisioning and managing the underlying infrastructure.

Hello and welcome 👋🏼 to this two-part blog series that uses a practical application to demonstrate how to integrate Redis with AWS Lambda. The first part (this one) covers the application - by the end of this blog, you should have the solution deployed, played around with it and in the process, have a good overview of the solution.

The second part is about the Infrastructure (IaaC to be specific) aspects - its mostly centred around AWS CDK along with some code walkthrough.