Stateful stream processing for digital twins: Microservice-based kafka stream dsl

A.B.A. Alaasam; G. Radchenko; A. Tchernykh

doi:10.1109/SIBIRCON48586.2019.8958367

Stateful stream processing for digital twins: Microservice-based kafka stream dsl

A.B.A. Alaasam, G. Radchenko, A. Tchernykh

Research output: Contribution to conference (No Proceedings) › Paper › peer-review

Abstract

Digital Twin is a virtual representation of a technological process or a piece of equipment, that supports monitoring, control and state prediction based on the data, gathered from the sensor networks. To parallelize event processing and produce near-real-Time insights over data streams, Digital Twin should be implemented based on an Event-Driven architecture. The Event-Driven architecture is loosely-coupled by its nature. One of the recent possible solutions for loose coupling system is a Microservice approach, a cohesive and independent process that interacts using messages. Stateless behavior is the nature of the microservice, but on the other hand, the vast majority of stream processing in Digital Twin imply stateful operations. Thus, in this paper, we propose a case-study of the possibility to use Apache Kafka Stream API (Kafka stream DSL) to build stateful microservice for real-Time manufacturing data analysis. Also, in the presented work we discuss the fulfillment of such requirements as fault tolerance, processing latency, and scalability to support the stateful stream processing in Digital Twins implementation. © 2019 IEEE.

Original language	English
Pages	804-809
Number of pages	6
DOIs	https://doi.org/10.1109/SIBIRCON48586.2019.8958367
Publication status	Published - 2019
Event	2019 International Multi-Conference on Engineering, Computer and Information Sciences, - Novosibirsk, Russian Federation Duration: 21 Oct 2019 → 22 Oct 2019 https://2019.sibircon.ieeesiberia.org/

Conference

Conference	2019 International Multi-Conference on Engineering, Computer and Information Sciences,
Abbreviated title	SIBIRCON 2019
Country/Territory	Russian Federation
City	Novosibirsk
Period	21/10/19 → 22/10/19
Internet address	https://2019.sibircon.ieeesiberia.org/

Keywords

Apache Kafka
Digital Twin
Event-Driven
Microservice
Sensors
Stream processing
DSL
Fault tolerance
Network architecture
Sensor networks
Event-driven
Modems

Access to Document

10.1109/SIBIRCON48586.2019.8958367

https://www.scopus.com/inward/record.uri?eid=2-s2.0-85079071242&doi=10.1109%2fSIBIRCON48586.2019.8958367&partnerID=40&md5=5f0481758f430edccf7567e0415b8f25

Cite this

@conference{dfff56b2352f4d2db7a806acebcb997c,

title = "Stateful stream processing for digital twins: Microservice-based kafka stream dsl",

abstract = "Digital Twin is a virtual representation of a technological process or a piece of equipment, that supports monitoring, control and state prediction based on the data, gathered from the sensor networks. To parallelize event processing and produce near-real-Time insights over data streams, Digital Twin should be implemented based on an Event-Driven architecture. The Event-Driven architecture is loosely-coupled by its nature. One of the recent possible solutions for loose coupling system is a Microservice approach, a cohesive and independent process that interacts using messages. Stateless behavior is the nature of the microservice, but on the other hand, the vast majority of stream processing in Digital Twin imply stateful operations. Thus, in this paper, we propose a case-study of the possibility to use Apache Kafka Stream API (Kafka stream DSL) to build stateful microservice for real-Time manufacturing data analysis. Also, in the presented work we discuss the fulfillment of such requirements as fault tolerance, processing latency, and scalability to support the stateful stream processing in Digital Twins implementation. {\textcopyright} 2019 IEEE.",

keywords = "Apache Kafka, Digital Twin, Event-Driven, Microservice, Sensors, Stream processing, DSL, Fault tolerance, Network architecture, Sensor networks, Event-driven, Modems",

author = "A.B.A. Alaasam and G. Radchenko and A. Tchernykh",

note = "Conference code: 156894 Cited By :8 Export Date: 27 August 2021 Funding details: Russian Foundation for Basic Research, RFBR, 18-07-01224 Funding text 1: ACKNOWLEDGMENT The reported study was funded by RFBR, project number 18-07-01224 А.; 2019 International Multi-Conference on Engineering, Computer and Information Sciences, , SIBIRCON 2019 ; Conference date: 21-10-2019 Through 22-10-2019",

year = "2019",

doi = "10.1109/SIBIRCON48586.2019.8958367",

language = "English",

pages = "804--809",

url = "https://2019.sibircon.ieeesiberia.org/",

}

Stateful stream processing for digital twins: Microservice-based kafka stream dsl. / Alaasam, A.B.A.; Radchenko, G.; Tchernykh, A.
2019. 804-809 Paper presented at 2019 International Multi-Conference on Engineering, Computer and Information Sciences, , Novosibirsk, Russian Federation.

Research output: Contribution to conference (No Proceedings) › Paper › peer-review

TY - CONF

T1 - Stateful stream processing for digital twins: Microservice-based kafka stream dsl

AU - Alaasam, A.B.A.

AU - Radchenko, G.

AU - Tchernykh, A.

N1 - Conference code: 156894 Cited By :8 Export Date: 27 August 2021 Funding details: Russian Foundation for Basic Research, RFBR, 18-07-01224 Funding text 1: ACKNOWLEDGMENT The reported study was funded by RFBR, project number 18-07-01224 А.

PY - 2019

Y1 - 2019

N2 - Digital Twin is a virtual representation of a technological process or a piece of equipment, that supports monitoring, control and state prediction based on the data, gathered from the sensor networks. To parallelize event processing and produce near-real-Time insights over data streams, Digital Twin should be implemented based on an Event-Driven architecture. The Event-Driven architecture is loosely-coupled by its nature. One of the recent possible solutions for loose coupling system is a Microservice approach, a cohesive and independent process that interacts using messages. Stateless behavior is the nature of the microservice, but on the other hand, the vast majority of stream processing in Digital Twin imply stateful operations. Thus, in this paper, we propose a case-study of the possibility to use Apache Kafka Stream API (Kafka stream DSL) to build stateful microservice for real-Time manufacturing data analysis. Also, in the presented work we discuss the fulfillment of such requirements as fault tolerance, processing latency, and scalability to support the stateful stream processing in Digital Twins implementation. © 2019 IEEE.

AB - Digital Twin is a virtual representation of a technological process or a piece of equipment, that supports monitoring, control and state prediction based on the data, gathered from the sensor networks. To parallelize event processing and produce near-real-Time insights over data streams, Digital Twin should be implemented based on an Event-Driven architecture. The Event-Driven architecture is loosely-coupled by its nature. One of the recent possible solutions for loose coupling system is a Microservice approach, a cohesive and independent process that interacts using messages. Stateless behavior is the nature of the microservice, but on the other hand, the vast majority of stream processing in Digital Twin imply stateful operations. Thus, in this paper, we propose a case-study of the possibility to use Apache Kafka Stream API (Kafka stream DSL) to build stateful microservice for real-Time manufacturing data analysis. Also, in the presented work we discuss the fulfillment of such requirements as fault tolerance, processing latency, and scalability to support the stateful stream processing in Digital Twins implementation. © 2019 IEEE.

KW - Apache Kafka

KW - Digital Twin

KW - Event-Driven

KW - Microservice

KW - Sensors

KW - Stream processing

KW - DSL

KW - Fault tolerance

KW - Network architecture

KW - Sensor networks

KW - Event-driven

KW - Modems

U2 - 10.1109/SIBIRCON48586.2019.8958367

DO - 10.1109/SIBIRCON48586.2019.8958367

M3 - Paper

SP - 804

EP - 809

T2 - 2019 International Multi-Conference on Engineering, Computer and Information Sciences,

Y2 - 21 October 2019 through 22 October 2019

ER -

Stateful stream processing for digital twins: Microservice-based kafka stream dsl

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this