Monolith vs Event Driven Architecture with an Example

Monolith vs Event Driven Architecture sounds like a competition, when it's actually a decision. The problem? Everyone "understands" what each pattern does, but it's hard to compare them in a practical scenario. Architecture is hard! But I'll try to make it a very small bit easier, by showing you how to solve the same problem with a Monolithic Architecture and with an Event Driven Architecture. We'll do a full comparison, but first let me start by getting you up to speed on each software architecture pattern.

Introducing Monolithic Architectures

A monolithic architecture involves deploying all application components in a single unit. It's not indicative of spaghetti code or a big ball of mud, you still build separate components with high cohesion. The important part is that, since everything is deployed as a single binary or package, communication between those components is a function or method invocation within the same program.

If you ask ChatGPT, it'll tell you that monolithic applications have difficult scaling. This is simply not true, monolithic applications can and do scale, even to the levels of Facebook and Instagram (both are built with monolithic architectures). The limitation is that you can't scale components individually, there's one single bundle of components, and one lever to pull: How many copies of that single bundle you deploy. When that lever is enough, it's way better than a million levers and a ton of added complexity. However, sometimes (not always) you'll want more levers to pull.

Key Characteristics of Monolithic Architectures

These are the key characteristics of monolithic architectures. Remember that these aren't goals, and a monolithic architecture itself isn't a goal. It's a decision, and these are the consequences of that decision (some good, some bad):

1. Unified Codebase: Monolithic architectures are characterized by a single, unified codebase. All components, including the user interface, business logic, and data access layers, are tightly integrated and are deployed as a single unit.

2. Linear Scalability: Scaling a monolithic application typically means replicating the entire application. It doesn't support independent scaling of individual components.

3. Centralized Management: The development, deployment, and maintenance of a monolithic application are centralized. All updates or changes require redeploying the entire application.

4. Simplicity in Development and Deployment: Initially, monolithic architectures are straightforward to develop, test, deploy, and manage because there's no additional complexity introduced.

5. Single Language and Framework: Monoliths are often built using a single programming language and framework.

6. Reliability Concerns: In a monolithic architecture, a bug or failure in any module can potentially bring down the entire application, which is a significant reliability concern.

7. Complexity Over Time: As the application grows, the monolith can become increasingly complex and unwieldy, making it difficult to implement new features or maintain the system. This isn't a given, and it could be said about any architecture. But with monoliths, modules tend to not be as highly cohesive and low coupled as we'd want. It can definitely be avoided, but it requires skill and paying a lot of attention to this as the application grows, not after ir already grew out of control.

8. Difficult to Scale Development Teams: As the application grows, it becomes challenging to scale the development team effectively due to the interconnected nature of the codebase. Like the previous point, this can be avoided, but it needs skill and attention.

9. Longer Startup Time: Larger monolithic applications may have significantly longer startup times, because it's a big binary. This can be a problem in environments that require rapid scaling or frequent restarts.

Introducing Event Driven Architectures

Event Driven Architectures (EDA) represent a paradigm shift compared to monolithic architectures. You know how much I hate those presumptuous terms: "paradigm shift", "in the world of software development", "a groundbreaking something". Keep up with me though, because this really is a different paradigm. Instead of thinking of a transaction as a series of actions that our software performs, such as doA(), doB(), doC(), in an event driven architecture we think of those actions as responses to things that happen (i.e. events). When a request comes in, doA(). When A is done, doB(). When B is done, doC(). It looks the same, and it could solve the same problem with the same steps (in a bit I'll show you an example of this), but the difference is in who determines that A, B and C should be run.

In a monolith, we'd have a request handler that calls module A to doA(), module B to doB() and module C to doC(), literally moduleA.doA(); moduleB.doB(); moduleC.doC();. In an event driven architecture, Module A is a separate program, that is listening for a request, and when a request comes in it runs doA() and publishes an event saying that A is done. Module B, a separate program from Module A, is listening for the event of A is done, and when it receives that event, responds by running doB() and publishing an event B is done. Module C receives the event B is done and runs doC(). There is no central coordinator, each module listens in on one or a few particular events, and does its thing when it receives those events. This is called choreography, btw.

Key Characteristics of Event Driven Architectures

Just like for monoliths, these characteristics aren't goals, and an event driven architecture isn't a goal. It's a decision, and these are the consequences of that decision (some good, some bad):

1. Decoupled Components: In the EDA pattern, components are loosely coupled. They interact primarily through events, which means changes in one part of the application can be made with minimal impact on others. The key here is that each component doesn't depend on other components or on a central coordinator. A component isn't even aware of other components! All it knows is the events to which it needs to respond and the events it publishes, without knowing where those events come from or who else might be listening.

2. Scalability: Each component can be scaled independently, allowing for more efficient resource usage and handling of varying loads. Remember that each component is its own binary, deployed independently.

3. Asynchronous Communication: Components communicate via asynchronous messaging. This is great for low coupling and retries, but it can make back and forth communication harder (you can't even send a response!).

4. Flexibility in Technology Stack: Different components can be built using different technologies and languages, offering flexibility and ease in integrating new technologies. Honestly, 99% of the times you shouldn't use a different programming language just because you can. But, well, now you can.

5. Complex Event Processing: EDAs are well-suited for scenarios that require complex logic to process events, like real-time analytics.

6. Resilience and Fault Tolerance: The decoupled nature of components means the failure of one component doesn’t necessarily bring down the entire system. That's good!

7. Challenges in Testing and Debugging: Due to the asynchronous and distributed nature of event driven architecture, testing and debugging can be more complex compared to a monolithic architecture. Significantly more complex, even when using tools like AWS X-Ray.

Our Problem: Converting Audio to Structured Text

Let's dive into an example! We need to convert unstructured audio into structured text. Our application will receive an audio file of someone saying how much water they drank, in an unstructured form, with no specific measure unit. We want to capture the amount of water mentioned, convert it to liters, and store it in a DynamoDB table.

We'll use Amazon Transcribe to convert the audio to text, and then we'll send that text to Amazon Bedrock (like ChatGPT, but by AWS) to extract the amount and unit of measure. After that we'll convert the amount to liters and use the AWS SDK to store the value in DynamoDB.

Problem Statement and Requirement Analysis

I hope I painted a good picture of the app. Let's formalize our requirements though, since we know from the Architecting with AWS Lambda: Architecture Design article that requirements will restrict our design.

Requirements:

• The application must accept MP3 files. They need to be stored afterwards, for an undetermined amount of time.

• The files contain the audio of one person speaking in English. There is no significant background noise.

• The audios are about how much water the user drank. They are unstructured. They may or may not include the word "water", but they're always about amount of water drank. Each audio has only one amount. Examples: "3 glasses of water", "I drank nearly half a gallon this morning!", "a half-liter bottle".

• The app must extract the amount of water drank from these audios and convert it to liters. 1 glass = 0.25L, 1 bottle = 0.5L.

• For each audio the app must create an item in DynamoDB with the date and time and the amount of water drank.

I know, it sounds rather arbitrary. Also, DynamoDB isn't a requirement, it's a solution. But this article isn't about requirements gathering, it's about monolith vs event driven architecture.

Solving with an Event Driven Architecture in AWS

I'm starting with an Event Driven Architecture because I've actually seen this architecture for similar problems. We even discussed it in the old article Serverless, event-driven image compressing pipeline with Lambda and S3. Here's the architecture diagram:

Components

• Audio Uploads S3 bucket: Stores the audio files

• Transcription function: Is triggered when a new audio file is uploaded. Uses Amazon Transcribe to convert the audio to text, and stores the text in the Transcriptions S3 bucket.

• Amazon Transcribe: Converts the audio to text.

• Transcriptions S3 bucket: Stores the transcription files

• Interpretation function: Is triggered when a new transcription file is uploaded. Uses Amazon Bedrock to extract the information from the text, converts it to liters, and inserts an item into the Water Consumption Records DynamoDB table

• Amazon Bedrock: Extracts the information from the transcription text

• Water Consumption Records DynamoDB table: Stores the records of water consumption

Flow of Data

This is the order of events in the system, or how the data flows in it:

1. Users upload the MP3 files to the Audio uploads S3 bucket

2. Amazon S3 Event Notifications triggers the Transcription function

3. The Transcription function calls Amazon Transcribe with the audio file

4. Amazon Transcribe returns the text of the audio file

5. The Transcription function stores the text in the Transcriptions S3 bucket

6. Amazon S3 Event Notifications triggers the Interpretation function

7. The Interpretation function calls Amazon Bedrock with the transcription

8. Amazon Bedrock returns the amount and unit of measure captured from the transcription

9. The Interpretation function converts the amount and unit into liters

10. The Interpretation function stores the amount in liters in the Water Consumption Records DynamoDB table

Failures, Retries and Idempotency

Since we're storing the audio files in S3, we can always retry the entire process from the beginning. It wouldn't be automatic though.

As for the individual steps, we can retry them individually any time we want, since we're keeping the intermediate artifact of the transcription file (we should probably delete this after a small amount of time). In fact, since the S3 invocation of AWS Lambda is asynchronous, we can configure our Lambda function to retry processing the event on errors, from 0 (no retries) to 2 retries.

Even if we don't configure retries, since Amazon S3 event notifications are designed to be delivered at least once, the same event may be delivered more than once. We should make sure the entire process is idempotent, meaning it won't process the same event twice.

Advantages and Disadvantages of Event Driven Architecture

The main advantage that we have is that our components are independent and can be reused as they are deployed. We could decide we want to offer the same functionality but for text, or we could swap the Transcription function for a library doing the transcription in our mobile app. All we'd need to do is make sure the mobile app can upload text files to the Transcriptions S3 bucket, and everything would work just fine. If we then want to change the Interpretation function to use ChatGPT's API instead of Bedrock, we can make the change, deploy the new version, and all workflows will be updated.

The main disadvantage is that we're using a lot of events and all of that to replace a controller that would literally look something like this:

const textFile = transcribeAudio(audioFile);
const data = interpretText(textFile);
insertWaterConsumptionRecord(data);

It's so simple! Can you see now how much complexity we're introducing with an Event Driven Architecture? This doesn't mean an event driven architecture is bad, it just means there's a cost.

Solving with a Monolithic Architecture in AWS

Now let's solve the same problem with a monolith. Here's the diagram:

Note that instead of the monolith exposing an API to accept the audio file and store it in S3, I'm exposing the bucket and using S3 event notifications to trigger the monolith. I think this makes it easier to focus on the differences I want to focus on, but you could just use an API. You might say this is an event driven architecture. You might be right! But the difference between this and the event driven architecture solution is how we deploy components.

Components

• Audio Uploads S3 bucket: Stores the audio files

• Audio Processing function: Is triggered when a new audio file is uploaded. Uses Amazon Transcribe to convert the audio to text, then uses Amazon Bedrock to extract the information from the text, converts it to liters, and inserts an item into the Water Consumption Records DynamoDB table

• Amazon Transcribe: Converts the audio to text.

• Amazon Bedrock: Extracts the information from the transcription text

• Water Consumption Records DynamoDB table: Stores the records of water consumption

Tip: We removed the S3 bucket in the middle, and combined the Transcription function and the Interpretation function into the Audio Processing function.

Flow of Data

We don't have events anymore (well, except for the upload of the audio file), but here's how things happen in our monolith:

1. Users upload the MP3 files to the Audio uploads S3 bucket

2. Amazon S3 Event Notifications triggers the Audio Processing function

3. The Audio Processing function calls Amazon Transcribe with the audio file

4. Amazon Transcribe returns the text of the audio file

5. The Audio Processing function calls Amazon Bedrock with the transcription

6. Amazon Bedrock returns the amount and unit of measure captured from the transcription

7. The Audio Processing function converts the amount and unit into liters

8. The Audio Processing function stores the amount in liters in the Water Consumption Records DynamoDB table

Tip: Things don't look so different here!

Failures, Retries and Idempotency

Just like with our Event Driven Architecture solution, we're storing the audio files, so we can retry from the beginning. This was actually a requirement.

We can't retry individual steps though. It's all or nothing, either the audio file gets correctly processed, the text interpreted and the amount of water recorded, or we need to start from scratch by transcribing the audio file again. This isn't such a bad thing in this particular case.

Since we're running our monolith in an AWS Lambda function, we can configure the invocation to retry up to 2 times. If we were using EC2, ECS or EKS, which are more typically associated with a monolith, we could achieve the same effect with an SQS queue.

We also have the problem of Amazon S3 event notifications being delivered at least once, so we still need idempotency.

Advantages and Disadvantages of Monolithic Architecture

The main advantage is simplicity. We're losing on a few things, but nothing important really. As a rule of thumb, if it's a simpler way to tick all your boxes, it's better. Of course, you can't determine this if you don't know all your boxes, which is why requirements are so important.

The main disadvantage is that we can't reuse components as easily. Going back to the scenario I proposed on the Advantages and Disadvantages of Event Driven Architectures, suppose we want to replace the transcription part with a native library, but only on the mobile app; the web app still uploads the audio. With a monolith, we need to expose a different endpoint (maybe REST APIs for a JSON payload in this case), or a different way to invoke it. Alternatively, since the S3 bucket accepts any kinds of file, we could have the mobile app upload a text file, and add to our monolith an if (file.extension === "mp3") transcribe();. It's definitely doable, but it becomes more complex.

Comparing Solutions: Monolithic vs Event Driven Architecture

I did some comparisons while discussing advantages and disadvantages. But let's dive deeper, focusing on a few axes (that's the plural of axis, not of axe).

Complexity Comparison

Clearly, the monolith is simpler!! Except when you try to build a separate workflow using some of the same components, like our example of just interpreting the text without transcribing from audio first. In that case, EDA is simpler. So, which one wins?

My consultant answer: It depends. Actually, consultants answer "it depends, that'll be $1000", but I'll tell you it depends for free, because I like you 😊.

In this case, I'd say the monolith wins. Sure, a new use case that reuses some components is slightly harder to implement, but in this particular case I don't foresee that we'll get a lot of those. Both architectures could grow very complex in different scenarios, but with the scenario that we're dealing with right now, and what we can foresee, I believe the monolith would grow to be less complex than EDA.

Scalability Comparison

The Event Driven Architecture allows us to scale components separately, so it clearly wins on scalability. That's not even a question, event driven architecture will always be more scalable. The question, for all architecture decisions, always, is whether we care about that or not. In this particular case, I can see us caring about it when we implement the use case of only processing text. However, I don't believe the rest of the application (i.e. the code that calls Transcribe) is so big that it slows down the application. Sure, there will be some instances of our application where the entire code and libraries for audio transcription will be deployed and won't be used. That's the equivalent of deploying an app that the AWS SDK for Transcribe but never uses it, so it's just sitting there increasing the size of the deployment package and the time the application takes to start, or the Lambda cold start in our case. All of that is true! But for this particular case, I believe the increased size and start time is too small to care about it.

Note that I'm making my judgment without even knowing how much the size and cold start time increase. I'm implicitly assuming that our app isn't serving billions of users, and I'm guessing the impact will be negligible. You need to be very upfront with assumptions like these. Maybe you don't need to record them, but you need to at least understand your own mental models. In this case, if I'm architecting this, this is my verdict and my reasoning, and I'm open to be proven wrong, but I arbitrarily decree that the burden of the proof is on whomever thinks different than me. Dictator much? Maybe, but that's why you want an architect: Get someone who's right most of the time, listen to them most of the time, prove them wrong every once in a while.

Developer Experience Comparison

For most devs, the monolith is more straightforward. Just the fact that you're triggering it from an upload to S3 instead of having the monolith itself upload the file is already a stretch. The first reason is that event driven architectures really are a paradigm shift, much as I hate sounding like ChatGPT. It's like picking a functional paradigm when all your devs know is object-oriented paradigm. They can definitely learn it, but you're creating some friction. Is that friction worth it? Sometimes it is! Sometimes it's better to go with what people know.

What if your devs know both monoliths and EDAs? For your current devs, DevEx (that's developer experience) might not be impacted. But you need to think about who will maintain the app, how are you going to hire, and how hard each pattern makes it to evolve the app. I think this example is too basic for that, so I'll have to leave you with the questions.

The second reason for most devs to tend to think in monolith is that this particular way of implementing an event driven architecture (which is not the only way) is very cloud native. I'm a solutions architect, I know that S3 event notifications invokes Lambda asynchronously, and that async invocations of Lambda can have automatic retries. Most devs with experience using AWS don't know those fine details. I wouldn't expect them to, it's something very specific. But I will expect that if any of you readers are architecting a solution, you will put yourselves in the shoes of other people (devs or whomever else there is), and you will never assume that they know as much as you do in your area of expertise (after all, I bet most of them don't even read Simple AWS! hint hint).

Cost Comparison

I'm sure you know how this solution is priced: S3 put, S3 storage, Lambda invocations, call to Transcribe, call to Bedrock, DynamoDB put item. I'll make some calculations with the following assumptions:

• 10,000 requests per day = 300,000 requests per month

• 0 retries or failures

• Audio size of 10 KB

• Audio duration of 2 seconds (average)

• Transcript size of 1 KB

• Input and output tokens for Bedrock, 25 per request

• Bedrock uses Jurassic-2 Ultra model

• Lambda with 2048 memory, takes 1 second to wait for Transcribe and 2 seconds to wait for Bedrock plus write to DynamoDB

• DynamoDB On Demand mode

Let's see some numbers:

• Monolithic architecture: S3: $1.69, Lambda: $24.06, DynamoDB: $0.45, Transcribe $240, Bedrock: $282. Total: $548.20/month. Here's the estimate in the pricing calculator.

• Event driven architecture: S3: $1.69 for audios + $1.63 for transcriptions, Lambda: $8.06 for transcription + $16.06 for interpretation, DynamoDB: $0.45, Transcribe $240, Bedrock: $282. Total: $549.89/month. Here's the estimate in the pricing calculator.

Insights:

• The cost is dominated by Transcribe and Bedrock. Unless we optimize this, our architecture doesn't really matter.

• The main cost difference between the monolith and the EDA is for PUT and GET operations for the intermediate bucket and for Lambda invocations (even with the same total execution time, we have twice as many invocations).

• For the intermediate bucket, we could instead use SNS for pub/sub and bring down the cost to $0.16

• We can't bring down the Lambda invocations cost, but we see it doesn't have a big impact. If instead of 300,000 invocations for 1000 ms each we had 3,000,000 invocations of 100 ms each, we'd pay $8.60 instead of $8.06. The real difference would be if we had 256 MB of memory: for 24,000,000 invocations of 100 ms each at 256 MB we'd pay $12.80, nearly 60% more even though we're using the same amount of GB-seconds. But at memory configurations over 1 GB and executions over 1 second, invocations become negligible and execution time dominates.

• Damn, Transcribe and Bedrock are EXPENSIVE!!

When to Choose a Monolithic Architecture?

In general, a monolithic architecture is a good choice when your components aren't heavily reused across several workflows, when you want centralized definitions of the steps for processing data (i.e. orchestration), and when you don't need to retry individual steps in your processing workflow.

Here's the biggest tip though: in 95% of cases, default to a monolith. They're easier to design, easier to understand, easier to develop, easier to deploy, and easier to hire for. Event driven architectures have a lot of benefits, but always check whether you actually care about those benefits. If you don't go monolith.

When to Choose an Event Driven Architecture?

An event driven architecture is a great choice when you need to reuse components across multiple distinct workflows. That's going to dictate that they need to scale separately; if they're only used in one workflow, they don't need to scale independently from the other components in that workflow.

In general, EDAs are more complex to design, understand, develop and deploy. There are a lot of benefits to be gained in exchange for that complexity, but always always always check whether those benefits are important to you.

Conclusion

In conclusion, it depends. You've seen both patterns, with their goods and bads. They're both great, and which is better depends on the problem you're solving.

I did give you a specific problem, so I'll give you my verdict for our example: Monolith is better. We have nothing significant to gain from an event driven architecture here, so let's default to the simplest solution.

A small catch: I'm not actually concerned about the complexities of the event-driven mental model for our simple example. I'm more concerned about the complexities introduced by using so many cloud services and so much cloud native stuff. I can train devs in that, I've done it, I've even been an AWS Authorized Instructor. But isn't it better if I don't have to?