Aws

The field of machine learning has advanced considerably in recent years, enabling us to tackle complex problems with greater ease and accuracy. However, the process of building and training machine learning models can be a daunting task, requiring significant investments of time, resources, and expertise. This can pose a challenge for many individuals and organisations looking to leverage machine learning to drive innovation and growth.

That’s where pre-trained AI Services come in and allow users to leverage the power of machine learning without having extensive machine learning expertise and needing to build models from scratch - thereby making it more accessible for a wider audience. AWS machine learning services provide ready-made intelligence for your applications and workflows and easily integrate with your applications. And, if you add Serverless to the mix, well, that’s just icing on the cake, because now you can build scalable and cost-effective solutions without having to worry about the backend infrastructure.

In this blog post, you will be using AWS Controllers for Kubernetes on an Amazon EKS cluster to put together a solution wherein data from an Amazon SQS queue is processed by an AWS Lambda function and persisted to a DynamoDB table.

AWS Controllers for Kubernetes (also known as ACK) leverage Kubernetes Custom Resource and Custom Resource Definitions and give you the ability to manage and use AWS services directly from Kubernetes without needing to define resources outside of the cluster. The idea behind ACK is to enable Kubernetes users to describe the desired state of AWS resources using the Kubernetes API and configuration language. ACK will then take care of provisioning and managing the AWS resources to match the desired state. This is achieved by using Service controllers that are responsible for managing the lifecycle of a particular AWS service. Each ACK service controller is packaged into a separate container image that is published in a public repository corresponding to an individual ACK service controller.

This blog post is for folks interested in learning how to use Golang and AWS Lambda to build a serverless solution. You will be using the aws-lambda-go library along with the AWS Go SDK v2 for an application that will process records from an Amazon Kinesis data stream and store them in a DynamoDB table. But that’s not all! You will also use Go bindings for AWS CDK to implement “Infrastructure-as-code” for the entire solution and deploy it with the AWS CDK CLI.

…With the managed runtime for Go

In this blog post you will learn how to run a Go application to AWS App Runner using the Go platform runtime. You will start with an existing Go application on GitHub and deploy it to AWS App Runner. The application is based on the URL shortener application (with some changes) that persists data in DynamoDB.

What’s covered?

• Prerequisites
• Create a GitHub repo for the URL shortener application
• Create a DynamoDB table to store URL information
• Create an IAM role with DynamoDB specific permissions
• Deploy the application to AWS App Runner
• Verify URL shortener functionality
• Clean up
• Conclusion

AWS App Runner can create and manage services based on two types of service sources:

A previous blog post covered how to deploy a Go Lambda function and trigger it in response to events sent to a topic in a MSK Serverless cluster.

This blog will take it a notch further.

• The solution consists of a MSK Serverless cluster, a producer application on AWS App Runner and a consumer application in Kubernetes (EKS) persisting data to DynamoDB.
• The core components (MSK cluster, EKS and DynamoDB) and the producer application will be provisioned using Infrastructure-as-code with AWS CDK.
• Since the consumer application on EKS will interact with both MSK and DynamoDB, you will also need to configure appropriate IAM roles.

All the components in this solution have been written in Go.

A previous blog post covered how to deploy a Go Lambda function and trigger it in response to events sent to a topic in a MSK Serverless cluster.

This blog will take it a notch further.

• The solution consists of a MSK Serverless cluster, a producer application on AWS App Runner and a consumer application in Kubernetes (EKS) persisting data to DynamoDB.
• The core components (MSK cluster, EKS and DynamoDB) and the producer application will be provisioned using Infrastructure-as-code with AWS CDK.
• Since the consumer application on EKS will interact with both MSK and DynamoDB, you will also need to configure appropriate IAM roles.

All the components in this solution have been written in Go.

In this blog post you will learn how to deploy a Go Lambda function and trigger it in response to events sent to a topic in a MSK Serverless cluster.

• Prerequisites
• Infrastructure setup
• Send data to MSK Serverless using producer application
• Configure and deploy the Lambda function
• Verify the integration
• Conclusion

The following topics have been covered:

• How to use the franz-go Go Kafka client to connect to MSK Serverless using IAM authentication.
• Write a Go Lambda function to process data in MSK topic.
• Create the infrastructure - VPC, subnets, MSK cluster, Cloud9 etc.
• Configure Lambda and Cloud9 to access MSK using IAM roles and fine-grained permissions.

Prerequisites

You will need an AWS account, install AWS CLI as well a recent version of Go (1.18 or above).

In this blog post you will learn how to deploy a Go Lambda function and trigger it in response to events sent to a topic in a MSK Serverless cluster.

• Prerequisites
• Infrastructure setup
• Send data to MSK Serverless using producer application
• Configure and deploy the Lambda function
• Verify the integration
• Conclusion

The following topics have been covered:

• How to use the franz-go Go Kafka client to connect to MSK Serverless using IAM authentication.
• Write a Go Lambda function to process data in MSK topic.
• Create the infrastructure - VPC, subnets, MSK cluster, Cloud9 etc.
• Configure Lambda and Cloud9 to access MSK using IAM roles and fine-grained permissions.

Prerequisites

You will need an AWS account, install AWS CLI as well a recent version of Go (1.18 or above).

The DynamoDB Scan API accesses every items in a table (or secondary index). It is the equivalent of a select * from query. One of the things I will cover in this blog is how to use Scan API with the DynamoDB Go SDK.

To scan a table, we need some data to begin with! So in the process, I will also go into how to use the Batch API to write bulk data in DynamoDB. You can use the BatchWriteItem API to create or delete items in batches (of twenty five) and it’s possible to you can combine these operations across multiple tables.

Learn with practical code samples

DynamoDB provides a rich set of data types including Strings, Numbers, Sets, Lists, Maps etc. In the Go SDK for DynamoDB, the types package contains Go representations of these data types and the attributevalue module provides functions to work with Go and DynamoDB types.

This blog post will demonstrate how to handle conversions between Go types in your application and DynamoDB. We will start off with simple code snippets to introduce some of the API constructs and wrap up with a example of how to use these Go SDK features in the context of a complete application (including a code walk though).