Design Modern Web-Scale Distributed Applications Like a Pro

Learn to design distributed systems for web-scale traffic following industry best practices and insights from real-world architectures

Course Contents (What You’ll Learn)

1.Concurrency in Online Services
Handling Concurrency in Online Services – Part 1 – (Transaction Isolation Levels) (preview)
Handling Concurrency in Online Services – Part 2 – (Database Locks)
Handling Concurrency in Online Services – Part 3 – (Concurrency Control Mechanisms)

2. Handling Database Growth
Managing Database Growth – Part 1 – (Partitioning)
Managing Database Growth – Part 2 – (Partitioning Strategies)
Database Replication
Dealing With Conflicts – Part 1 – (Multi-Master Write Conflicts)
Dealing With Conflicts – Part 2 – (Synchronized Clocks for Ordering Events)
Dealing With Conflicts – Part 3 – (CRDTs and OT)
Leaderless Replication

3. Distributed Transactions
Cross-Shard Distributed Transactions
Non-blocking Distributed Transactions – (Saga Pattern)
Two Phase Commit and Saga – Picking the Right Approach
Application Level Sharding – Tweaking Code for Cross Shard Operations
Cross Shard Transactions with Over a Million QPS – (Real-World Instance)
Horizontally Scaling Relational Database Clusters – (Real-World Instance)
Using a Distributed Database Solution to Handle Queries in a Relational Database Cluster
Distributed SQL
ACID Transactions in NoSQL Databases
Summary

4. Design a Sports News and Streaming Service Like ESPN, Cricbuzz and Hotstar
Overview
Live Streaming (Core Feature in Services Like Twitch, YouTube, Facebook, Hotstar) – Part 1
Live Streaming Service – Part 2 – (Infrastructure, Thundering Herd and Request Coalescing)
Live Streaming Service – Part 3 – (Stream Validation in Real-time) (preview)
Using Kafka as a Buffer in Stream Validation
Kafka Performance
Persisting the Live Stream for Video-On-Demand Service
Getting Our Infrastructure Ready for the Traffic Surge – Part 1 – (No Blind Spots and Injecting Chaos)
Getting Our Infrastructure Ready – Part 2 – (Not Relying on Autoscaling and Other Strategies)
Handling Production Traffic – (Throttling)

5. Video-on-demand Service/Module (Like Netflix, YouTube, HBO Max, Disney+)
Video Delivery Architecture Part 1 – (Designing A Workflow)
Video Delivery Architecture Part 2 – (Workflow Execution)
Video Delivery Architecture Part 3 – (Delivering Content through Edge, Data Estimations)
Designing User Profile with a Relational Data Model
Storing Relationships in a Document-Oriented Database

6. Video/News Feed Service (Like Netflix Video Feed, Facebook News Feed)
Designing the Video Recommendation Feed for the VOD Service Home Page – Part 1 – (Change Data Capture)
Designing the Video Recommendation Feed for the VOD Service Home Page – Part 2 – (Creating a Recommendation Object)

7. Push Notifications Service (Like Facebook Notifications, YouTube Notifications)
Notifying Viewers of the New Content

8. CMS – Content Management Service (Like New York Times, Medium, TechCrunch, Bloomberg, Cricbuzz)
Picking the Right Data Model and Knowledge Graphs (preview)
Designing Content Graph for our CMS – Part 1
Designing Content Graph for our CMS – Part 2
CMS Core Architecture

9. Distributed Content Search Service
Picking the Right Data Store to Power our Application Search
Search Architecture: Designing an Indexing Pipeline
Storing our Data with an Inverted Index
Distributed Search Behind the Scenes
Real-time Search with Immutable Inverted Indices – Part 1
Real-time Search with Immutable Inverted Indices – Part 2

10. User Subscription Payment Service (Like Netflix Payments, Cricbuzz Payments)
Understanding the Online Card Payment Flow
Designing the Distributed Payment Flow – Part 1 – (Saga Pattern)
Designing the Distributed Payment Flow – Part 2 – (Event Sourcing)
Event Ordering with Kafka and Persisting Events
Handling Duplicate Events, Ensuring Idempotency and CQRS Pattern

11. Weaving all the Services Together – Inter Microservice Communication
Communication Between Different Platform Services
Discovering Services in the Network
Laying Up a Service Mesh and SideCar Pattern
Handling the Interviews
Designing Something, You’ve No Clue About – Part 1
Designing Something, You’ve No Clue About – Part 2
Future Content (Coming Soon)
Epilogue
Change log

You’ll have 2 years of access to the course content. Please read the FAQs below before making the purchase.

Who is this Course for?

This course is meant for software developers, architects, product managers, product owners, engineering managers, IT consultants and anyone who wants to understand how large-scale distributed systems like Facebook, Netflix, YouTube, ESPN, New York Times etc. work behind the scenes as well as design these services with an industry-oriented approach. 

The course will be immensely helpful for you if you intend to prepare for a system design interview round for your job, are looking to design a web-scale distributed system at your workplace or want to make technologically informed decisions when working on a large-scale service as an engineering manager or a product owner. 

Will this Course be Helpful In My Career Growth?

This course and the Zero to Software Architect learning track (this course included) is the fastest way possible to master software architecture and the art of designing scalable distributed systems. As opposed to going through tons of articles, videos, books and whitepapers, you’ll find all the needed concepts in this learning track which will save you months, if not years, of your time.  

I’ve spent a decade developing and managing distributed systems for the big guns in the industry and I’ve put that experience in the content. Investing in this course will significantly increase your knowledge in the domain, subsequently increasing your pay scale. I wish I had a similar learning product in the initial years of my career.

Are there Any Prerequisites to this Course?

This course assumes you are well acquainted with web architecture and cloud computing fundamentals. If not, check out the below courses and the Zero to Software Architect learning track:
Web Application and Software Architecture 101 
Cloud Computing 101 – Master the Fundamentals
Throughout this course (Design Modern Web-Scale Distributed Applications Like a Pro), when designing systems, you’ll find references to various concepts explained in my other above-stated courses. The system designs in this course have an industry-oriented approach leveraging cloud design patterns like Saga, CQRS, Throttling, etc. and implementation of other architectural concepts.

Do I Need to Have Coding Knowledge to Understand this Course?

No. Coding knowledge is not a prerequisite for this course. This course contains high-level designs of distributed systems with several illustrations to help you understand the concepts better.

Why is this Course Text-based?

My courses are intentionally text-based. The text format of the course enables you to learn faster than the videos. Learning through reading is way faster than sitting through hours of videos. It is easy to take notes—no pausing the videos to take notes or screenshots of diagrams. You can always go back to specific topics immediately when you need them.

Also, my courses are frequently updated. New information is continually added. It is easy for me to update the text-based content as opposed to re-recording a video every time a tiny update needs to be made. Imagine that. It would be killing.

How Long Do I Have Access to the Course Content?

You will have two years of access to the course content from the date of purchase. 

Why Aren’t You Offering Lifetime Access?

I would be happy to but cannot primarily due to two reasons:

1. I have significant recurring monthly expenses in form of platform fees, hosting charges, marketing, international tax compliance and other maintenance costs. Offering lifetime access would significantly spike the price of the courses. In order to keep the content affordable, I had to time restrict access to the platform.

2. The content of my courses is continually updated besides the new content that is added. The software design and development domain continues to evolve and as it evolves the courses get updated accordingly. With my courses, you’ll stay on top of the latest developments in the domain. And, as you might have figured this demands continual time investment 🙂 

I Have a Question. How Can I Get in Touch?

You can send in your queries at [email protected] You can also send me a message on LinkedIn here.

Course Author

Hello there, I am Shivang. I have industry experience of more than ten years designing and developing scalable web systems, right from idea to production. I’ve designed, developed, and maintained code, as well as worked in the production support for systems receiving millions of hits every single day. 

I’ve worked on large-scale web services for some of the industry giants in several domains, including E-commerce, Fintech, Telecom and Travel & Hospitality. My last job was at HP as a full-stack developer in their Technical Solutions – R&D team.

As an independent consultant, I’ve helped businesses build scalable services with fitting software design and technology. This included helping them improve their development processes resulting in better code quality, maximum test coverage, minimal bottlenecks, less technical debt and fewer bugs.

Here is my LinkedIn profile if you want to read more about my professional experience or want to say hello! Cheers!! 

Zero to Software Architect Learning Track

Check out the Zero to Software Architect learning track. This unique learning track educates you step by step on software architecture, cloud infrastructure and distributed system design, starting right from zero. It takes you right from having no knowledge on the domain to making you a pro in designing web-scale distributed systems like YouTube, Netflix, ESPN and the like.

Student Reviews

Check out what those enrolled are saying about my courses here.

Check Out My Blog

I write about distributed systems, software architecture, cloud, system design and backend engineering in general on my blog scaleyourapp.com Check it out.