I was recently tasked with ensuring that objects could not be accidentally deleted or overwritten in an S3 bucket. In my scenario, the bucket stores deployment artifacts. A build pipeline uploads files to the bucket, and EC2 instances download those same files during bootstrap. This task proved a bit more challenging than I expected, so I’ll do a quick walkthrough of my experience and the options available.

IAM

I initially assumed I could handle this solely with IAM. Folding the AWS recommended principle of least privilege, deny access to the bucket except to the two clients, and create a different IAM…


‘Big Data’ is a huge trend in technology today, and AWS has many offerings that allow you to perform complex data analyses. The goal of this article is to introduce some of the very basics of Big Data in AWS. There are many ways to store, query, and visualize data in AWS, but I’ll focus on just a single configuration that utilizes S3 (storage), Glue (metadata catalog), and Athena (query engine). I’ll introduce each service with the bare minimum amount of information you need to get started and then dive right into building a working example. We’ll upload data to…


You’ve embraced infrastructure as code (IoC) and painstakingly created Terraform or CloudFormation to model your EC2 Instances, AutoScaling Groups, ECS clusters, and everything else. The devops and development teams have committed to only modify and create infrastructure using your chosen IoC tools — but, surprise, some people just won’t comply. Maybe someone gets lazy during feature development or takes a shortcut during incident response. Maybe you’ve outsourced your level one support and you don’t entirely trust that group yet. …


A step by step tutorial on consuming a DynamoDB Stream using AWS Lambda in C#

Event-driven programming is all the rage in the software world today. There’s a lot to be said for building a system with loosely coupled, independently deployable, and easily scalable components. Serverless tools can be leveraged to create some of those components; one AWS, that often means using DynamoDB and Lambda. In this article, we’re going to build a small event-driven system in which DynamoDB is our event source, and Lambda functions are invoked in response to those events. To accomplish this, we’ll use a feature called DynamoDB Streams.

After a DynamoDB Stream is enabled on a table, all modifications to…


If you’ve spent years working with relational databases, getting started with Amazon’s DynamoDB may feel intimidating to you. Traditional relational databases were optimized for storage; DynamoDB is optimized for performance. Tables exist independently; there is no ‘Join’ clause to query records from more than one table at a time. Concepts like normalization (not storing duplicate data) are no longer critical; in fact, sometimes your data model will explicitly duplicate data to improve performance.

The goal of this article is to learn the fundamentals of DynamoDB by diving right in and using it. We will design a DynamoDB schema to track…


Whether its “Hey Siri”, “OK Google”, or “Alexa”, voice user interfaces (VUIs) are growing in popularity by the day. Although the most common way to interact with these assistants is still via a smartphone, they are more frequently being built into all kinds of devices from speakers to televisions to cars. Amazon’s Alexa devices are dominating the smart speaker market, having recently announced that they’ve sold over 100 million devices. Amazon has also built a development kit that allows Alexa to run on low-powered devices with as little as 1MB of RAM. This opens the door to all kinds of…


Picking up from Part2, we’re going add the final component of our application — a message queue. The specifics of this example will seem bit contrived, but the goal is to decouple our publisher and worker, use Docker to launch a RabbitMQ container, and use Docker Compose to orchestrate the system. And of course, go line by line through the changes required to accomplish these tasks.

https://www.cloudamqp.com/blog/2015-05-18-part1-rabbitmq-for-beginners-what-is-rabbitmq.html

Why use a message queue?

Microservices are all the rage in today’s distributed systems. But an unfortunate side effect of breaking down monolithic applications into multiple single-purpose services is high coupling between the new…


Picking up from Part1, we’re going to be containerizing the application we’ve built. Containerization is an approach where an application and its dependencies are packaged together and and run in an isolated environment. A VM (or physical machine) runs the container host, instead of running our applications directly. The container host, in turn, runs our containers. Each container is isolated from the others, and can even run a different operating system than the VM.

In this project, we’ll be using Docker, an open-source project for automating the deployment of applications as portable, self sufficient containers that can run almost anywhere…


The goal of this series is to introduce some of the technologies we use at Trimble MAPS. We’re going to build a web application that will accept HTTP messages from a client, publish them to a RabbitMQ message queue, then consume those messages from the queue. We’ll also use Docker to run each application in a container, and Docker-Compose to orchestrate the launch of all of our containers. If any of those terms are unfamiliar, don’t worry, each one will be explained as we approach it. The first part of this tutorial will focus on .NET Core.

Part 1: .NET…

Matthew Harper

Director, Software Engineering @ Trimble Maps

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store