Azure Cosmos Db

It’s possible to build seamless device-to-cloud experience with the integrations between Azure and Confluent Cloud. The example provided in this blog post showcases how the following capabilities of Azure and Confluent Cloud are leveraged:

• A fully managed platform for data in motion and self-managed connectors from Confluent
• A secure connection from Confluent to Azure
• A highly distributed and scalable data management solution from Azure
• A limitless analytics service offered by Azure that brings together data integration, enterprise data warehousing, and big data analytics
• A fully managed microservice development platform on Azure

The following diagram provides high-level architecture of the end-to-end solution:

Azure Cosmos DB is a fully managed, elastically scalable and globally distributed database with a multi-model approach, and provides you with the ability to use document, key-value, wide-column, or graph-based data.

We will drill further into the multi-model capabilities and explore the options that are available to store and access data. Hopefully, it can help you make an informed decision on the right API are the right choice.

• Core (SQL) API: Flexibility of a NoSQL document store combined with the power of SQL for querying.
• MongoDB API: Supports the MongoDB wire protocol so that existing MongoDB client continue to work with Azure Cosmos DB as if they are running against an actual MongoDB database.
• Cassandra API: Supports the Cassandra wire protocol so that existing Apache drivers compliant with CQLv4 continue to work with Azure Cosmos DB as if they are running against an actual Cassandra database.
• Gremlin API: Supports graph data with Apache TinkerPop (a graph computing framework) and the Gremlin query language.
• Table API: Provides premium capabilities for applications written for Azure Table storage.

You can read further on the some of the Key Benefits

[Read More]

Azure Cosmos DB is a resource governed system that allows you to execute a certain number of operations per second based on the provisioned throughput you have configured. If clients exceed that limit and consume more request units than what was provisioned, it leads to rate limiting of subsequent requests and exceptions being thrown – they are also referred to as 429 errors.

With the help of a practical example, I’ll demonstrate how to incorporate fault-tolerance in your Go applications by handling and retrying operations affected by these rate limiting errors. To help you follow along, the sample application code for this blog is available on GitHub - it uses the gocql driver for Apache Cassandra.

With traditional architectures, it’s quite hard to counter challenges imposed by real-time streaming data – one such use case is joining streams of data from disparate sources. For example, think about a system that accepts processed orders from customers (real time, high velocity data source) and the requirement is to enrich these “raw” orders with additional customer info such as name, email, location etc. A possible solution is to build a service that fetches customer data for each customer ID from an external system (for example, a database), perform a join (in-memory) and stores the enriched data in another database perhaps (materialized view). This has several problems though and one of them is not being able to keep up (process with low latency) with a high volume data.