CDC for Inventory Using Debezium

note

This tutorial requires a Database to connect to. If you have access to the Demo environment we will have a database already connected with an integration running.

Introduction#

Using a CDC engine like Debezium along with the Jetic platform, we can easily build data pipelines to bridge traditional data stores and new cloud-native event-driven architectures. There is a low overhead and real time change data capture using Jetic for your CDC needs. In this tutorial you will see how you can create different routes based on what is returned via the Data Capture component.

VisionsCorp’s request#

VisionsCorp has the need to capture data and then depending on what is caputured, to perform different operations. You have been tasked to create these different actions and also protect yourself from any exceptions that may occur from the capture, to be safe. Using the Data Capture based on Debezium VisionsCorp expects their CDC needs to be well satisfied.

Data Capture
Message Router
Log
Mail
Throw Exception
Set Body

Step 1: Using your resource#

Docusaurus

Add the LogisticDemo Database as a dependency
If you have been following this tutorial series you should be familiar with this resource. We created this resource back in the ProductsAPI series, in the Add Dependencies section.
In the first route add the Data Capture component
For CONNECTOR TYPE select the debezium-mysql type
COMPONENT NAME LogisticDemo
For the DATABASE HOSTNAME put your DB URL
For the DATABASE USER put your DB USER
For the DATABASE PASSWORD put your DB PASSWORD
If you have an OFFSET STORAGE FILE, put that
We will be using the DATABASE PORT 3306

Step 2: Routing Messages#

Docusaurus

Drag the Message Router component
We will want to create different options dependent on the data received and then log what is received
We will have 5 different outcomes one for route, when, when, when, and otherwise
After each add a Log and keep the MESSAGE as headers: ${headers}; body: ${body}
After the Log in the route add a Mail component
In the Mail component use the smtp PROTOCOL, PORT 25, and add a TO as your desired email

Step 3: Throwing an Exception#

Docusaurus

In the otherwise lane, put a Throw Exception component
Create a second route with the [+]
Put a On Exception component, for Exeptions add Exception and for the Behavior select Handle the exceptions
Add a Log after the On Exception component
For the MESSAGE put Exception Thrown to track your exception

Step 4: Conditions in Routing#

Docusaurus

Back in the first route lane, under each of the when routes, add a Set Body(3) after the Logs
Click on the first when and add ${header.CamelDebeziumOperation} == 'u' for the CONDITION
In the first Set Body following, add Your item has been Updated
Click on the second when and add ${header.CamelDebeziumOperation} == 'c' for the CONDITION
In the second Set Body following, add Your item has been Created
Click on the third when and add ${header.CamelDebeziumOperation} == 'd' for the CONDITION
In the third Set Body following, add Your item has been Deleted
Run your integration and look at the output. To test this completely, make changes in your database and see which messages fire in Jetic

🎉 Congratulations you have successfully used Debezium to perform CDC in Jetic. You can use Jetic to observe your Database and see what transactions occur from that database.🎉