Building Distributed Systems on the Shoulders of Giants - API Days Live, Australia 2020

I recently spoke about building distributed systems at API Days Australia conference. My talk is titled Building distributed systems on the shoulders of giants. It’s a reference to the popular metaphor about leveraging the knowledge on people who have been here before us.

About API Days Live Australia

On the website it says

APIdays is the leading industry tech and business series of conferences in APIs and the programmable economy. As APIs become mainstream, our world becomes more connected, more automated and more intelligent. APIs are the gateway to data, services, devices and emerging technologies. APIs put power into the hands of developers, citizens and consumers.

This is my 2nd time speaking at the event. Last year I spoke about Microservices. The conference speakers consist of technology leaders from companies like Google, Amazon, Microsoft and Twitter alongside some local experts as well. The conference ran in a virtual format due to COVID19 restrictions but it was very well organised and saw 1000+ people attend.

Building distributed systems on the shoulders of giants

The abstract is as follows.

Developing a distributed system is one of the hardest things you will do as a software developer. You will end up having to deal with topics like network inconsistencies, load balancing and service discovery etc. Only to find that solving these problems require expert level knowledge. What if I told you there was a way to leverage the expertise of industry leaders like Google/Microsoft and benefit from their research so you don’t have to reinvent the wheel? Let’s investigate some tools you can use today to build modern, resilient and scalable microservices. Technologies like Service Fabric, Knative, Istio and DAPR can give you the right foundation to build on top of so you can concentrate on solving the business domain problems and be productive.

Developing a distributed system requires expert knowledge that not every mortal software developer has. This results in most Microservices failing in production scenarios under load or intermittent network conditions. There is a better way than expecting every developer to do these complex calculations. We can leverage the knowledge from experts by using some modern distributed applications runtimes and tools that give us a good foundation. (Standing on the shoulders of giants) This frees us to focus on the domain and solve it better. In this talk I wanted to introduce (compare and contrast) developers to some of these choices.

Recording & Slide deck

If you have any thoughts or comments please leave them here. I would love to hear about your experiences with any of these technologies and how they helped or hindered your progress.