My journey using AWS

Building digital services and learning how to "cloud" along the way.

About 7 years ago, still in college, I joined a recently seeded startup as their only software engineer. There, I would be in charge of building digital services on top of a hardware product. This implied picking technologies, cloud provider, tools, protocols, everything. This was quite a learning journey in itself, but I want to focus on the cloud provider.

I looked into startup benefits of all major cloud providers, and the AWS startup plan seemed the most sensible choice. A lot of breathing space to experiment, not only regarding time but also credits. I started to build our digital services on AWS and everything went smoothly.

The process required me to make some choices along the way. As expected from an inexperienced developer that is not a psychic, some of them were not optimal. I want to reflect a bit on those.

Take your time selecting your initial region

You can't do much with AWS until you've picked a region to start with. This is not a hard lock-in, you can always build your infrastructure on other regions. In fact, the regional diversity and its coverage are a strong plus in AWS.

Even so, we were starting out in Europe and were not expecting to cover more geographies any time soon. So I picked eu-central-1 (Europe Frankfurt).

In hindsight, I should have taken more time to analyze what this choice entailed. I was naive and had no idea service offer and pricing would vary that much.

For instance, eu-west-1 (Europe Ireland) had consistently lower prices on general purpose instances. The latency difference between Frankfurt and Ireland regions was not relevant for us. Plus, all the services we needed were offered by the Ireland region. Ireland would have been a better choice.

If you want to optimize your costs - and who doesn't - there are several variables to take into consideration. The region is easily one of the most important. I was rich on credits and didn't give this enough attention. When picking a region you're picking a range of services and a price list that you will work with. In this price list, you want to confirm the region's reserved instances pricing, data transfer speeds and additional costs of data transfer between regions. Here is a good rundown on these aspects.

Pricing in AWS is a lot of work in itself, but don't underestimate the savings you can get by going deep on the optimization.

Don't fall into authority bias when talking to AWS engineers

Our initial setup was essentially one RDS and two EC2 instances. The company was not exploding, we were still prototyping the hardware and had a few clients. Given so, there was no foreseeable reason to change anytime soon. Software development is faster than Hardware development, so our digital stack didn't need to grow your usual "startup-fast".

The AWS startup program included support from AWS engineers and cloud architects. They occasionally showed up in our startup hub and were very eager to hear me out. These meetings were not scheduled, but they would occur 2 to 3 times a year, so that they could follow our growth and advise accordingly.

This was good on paper, but I have a few complaints about it. Firstly, the people I met with kept on rotating. This meant that I had to keep pitching our solution, our architecture and retell the previous meetings. I was the interested party, and the potential gains were big, so I acknowledge this annoyance was a small price to pay. Secondly, almost every one of them pushed for integrated solutions using AWS specialized services, such as linking API Gateway, IoT and Lambda to replace everything. Now I get they would want to sell AWS products, but the thing is that the solution wasn't even properly answering my requirements. Our hardware was communicating with our EC2 instance through simple TCP sockets, which was not supported in that setup. I don't know where AWS IoT stands today, but it used to support only MQTT and there wasn't really an API to publish via API Gateway either. I expressed this concern, but it fell on deaf ears. Take into consideration that this was between 2015 and 2017, the Lambda hype was real.

As I didn't really have much experience - and I was listening to AWS cloud architects after all - I went ahead and fiddled with these services. I needed to make sure I understood what they did. I learned a couple of cool things, but it wasn't a fit.

A few months later, I presented this at the following meeting I had with a AWS engineer, who confirmed that the setup would be incompatible with my requirements. I was persistent, because I needed this to make sure I was not insane, and because I wasn't too sure about myself. Soon after that, the startup program ended and meetings were more rare.

Looking back, I feel like the majority of my meetings focused too much on AWS services and their functionalities and too little on scaling what I already had.

Today, we are at the hundreds of hardware devices and we are still using a small EC2 fleet. Since the applications we are building are completely stateless, we are simply putting the instances behind a load balancer and an auto-scaling group. As such, there has been no change in protocols or services.

Some integrated services can be a great fit to complete your skill set

One of the first services I tried out on AWS was Elastic Beanstalk. I didn't like the experience: I felt like I had no control and several things were happening without me realizing. This annoyed me, so I switched to running our applications on EC2 instances.

I never had this urge with RDS. It may be because I'm not as comfortable when it comes to configuring a Postgresql instance, but damn did RDS save me tons of time. I want to remind the reader that for the greater part, I was the only software engineer. This meant that everything that was remotely "digital" or "data related" would be handled by me. As such, RDS freed up time that I could invest in leveraging our applications. Furthermore, it freed up some mental availability for projecting the vision for the company.

There was no need to think about automated backups, storing snapshots, collecting metrics. I felt like I had a fellow engineer by my side.

I assume I feel like this about RDS, because I wouldn't like having to learn how to configure all that trouble. For this reason, I believe people that don't want to configure their application servers will feel the same about Lightsail or Elastic Beanstalk. And that's where these services thrive: you may not be good at something today, but you can delegate it to said service. While it still is important to learn on the way, I now acknowledge that I don't need to "know it all" and be on top of every single detail, which was killing me.

Final remarks

Some of these mistakes will inevitably seem silly to more experienced developers. Hell, even to me now.

Still, I wanted to put this out there for two main reasons. Firstly, being able to write this, means that I have learned: looking back, I can pinpoint my naivety and that will help me not making the same mistakes in the future. And secondly, because this can also help the next 21 year old developer that will be diving into AWS trying to change the world. The service offer is even more overwhelming now, and the naming is getting worse so help will be needed.

Hopefully, when I write another "looking back at my decisions" in a few years, I'll find the naivety in this post. Then, I'll know that I have learned a bit more - yet again.