Apache Kafka Guide

Summary

1. Apache Kafka Introduction
2. Apache Kafka Concepts
3. The DTO Pattern
4. References

1. Apache Kafka Introduction

Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.

https://kafka.apache.org/

https://kafka.apache.org/

https://kafka.apache.org/

2. Apache Kafka Concepts

Why do you need Apache Kafka?

I. Event-Driven

Synchronous and Asynchronous Requests

• Car
• E-Commerce
• Alarms
• Monitoring
• MicroServices
• Others Systems

Event Driven Apache Kafka

II. Real-Time

III. Historic Data

• Diferent of RabittMQ

IV) Diference Kafka vs RabittMQ

V) Kafka Key Features

• Platform
• Works in a Distributed Way
• Own Database
• Extremely Fast
• Low Latency
• Use disk instead of memory to process data

VI) Basic Concepts of Apache Kafka

• Topic
• Offset
• Producer
• Consumer
• Cluster
• Broker

Kafka Partition

Kafka Topic Creation

VII) Kafka Cluster

VII) Kafka Replication Factor

VIII) Apache Zookeeper — Service Discovery

What is ZooKeeper?

Apache ZooKeeper plays the very important role in system architecture as it works in the shadow of more exposed Big Data tools, as Apache Spark or Apache Kafka.

In other words, Apache Zookeeper is a distributed, open-source configuration, synchronization service along with naming registry for distributed applications.

What is ZooKeeper

Apache ZooKeeper

3. The DTO Pattern

• Data Transfer Object — DTO Pattern

UserDTO example

DTO Overview

In this tutorial, we’ll discuss the DTO pattern, what it is, how, and when to use them. At the end of it, let’s hope we’ll know how to use it properly.

DTO Overview

The Pattern DTO

DTOs or Data Transfer Objects are objects that carry data between processes in order to reduce the number of methods calls. The pattern was first introduced by Martin Fowler in his book EAA. He explained that the pattern’s main purpose is to reduce roundtrips to the server by batching up multiple parameters in a single call. Therefore reducing the network overhead in such remote operations. Another benefit of this practice is the encapsulation of the serialization’s logic (the mechanism that translates the object structure and data to a specific format that can be stored and transferred). It provides a single point of change in the serialization nuances. It also decouples the domain models from the presentation layer, allowing both to change independently.

How to Use It?

DTOs normally are created as POJOs. They are flat data structures that contain no business logic, only storage, accessors, and eventually methods related to serialization or parsing. The data is mapped from the domain models to the DTOs, normally through a mapper component in the presentation or facade layer. The image below illustrates the interaction between the components:

How do you use DTO ?

When to Use It?

As mentioned in its definition, DTOs come in handy in systems with remote calls, as they help to reduce the number of them. They also help when the domain model is composed of many different objects, and the presentation model needs all their data at once or even reduces roundtrip between client and server. With DTOs, we can build different views from our domain models, allowing us to create other representations of the same domain but optimizing them to the clients’ needs without affecting our domain design. Such flexibility is a powerful tool to solve complex problems.

Use Case

To demonstrate the implementation of the pattern, we’ll use a simple application with two main domain models. In this case, User and Role, to focus on the pattern, let’s look at 2 examples of functionality, user retrieval, and creation of new ones.

DTO vs. Domain

Below is the definition of both models:

public class **User** {

 private String id;  
    private String name;  
    private String password;  
    private List<Role> roles;

 public **User**(String name, String password, List<Role> roles) {  
        this.name = Objects.requireNonNull(name);  
        this.password = this.encrypt(password);  
        this.roles = Objects.requireNonNull(roles);  
    }

 // Getters and Setters

 String **encrypt**(String password) {  
       // encryption logic  
   }  
}

public class **Role** {

 private String id;  
    private String name;

 // Constructors, getters and setters  
}

Now let’s look at the DTOs so that we can compare them with the Domain models. At this moment is important to notice that the DTO represents the model sent from or to the API client. Therefore, the small differences are either to pack together the request sent to the server or optimize the response of the client:

public class **UserDTO** {  
    private String name;  
    private List<String> roles;  

    // standard getters and setters  
}

The DTO above provides only the relevant information to the client, hiding the password, for example, for security reasons. The next one groups all the data necessary to create a user and sends it to the server in a single request. And mentioned before, this optimizes the interactions with the API. See the code below:

public class **UserCreationDTO** {

 private String name;  
    private String password;  
    private List<String> roles;

 // standard getters and setters  
}

4. References

[Apache Kafka
More than 80% of all Fortune 100 companies trust, and use Kafka. Apache Kafka is an open-source distributed event…kafka.apache.org](https://kafka.apache.org/ "https://kafka.apache.org/")

• https://www.baeldung.com/java-dto-pattern