Título:	A non-intrusive and reactive architecture to support real-time ETL processes in data warehousing environments.
Autoria:	VILELA, F. DE A. TIMES, V. C. BERNARDI, A. C. de C. FREITAS, A. DE P. CIFERRI, R. R.
Afiliação:	FLÁVIO DE ASSIS VILELA, Federal Institute of Goiás; VALÉRIA CESÁRIO TIMES, Federal University of Pernambuco; ALBERTO CARLOS DE CAMPOS BERNARDI, CPPSE; AUGUSTO DE PAULA FREITAS, Federal University of São Carlos; RICARDO RODRIGUES CIFERRI, Federal University of São Carlos.
Ano de publicação:	2023
Referência:	Heliyon, v. 9, n. 5, e15728, may 2023.
Páginas:	25 p.
Conteúdo:	Nowadays, organizations are very interested to gather data for strategic decision-making. Data are disposable in operational sources, which are distributed, heterogeneous, and autonomous. These data are gathered through ETL processes, which occur traditionally in a pre-defined time, that is, once a day, once a week, once a month or in a specific period of time. On the other hand, there are special applications for which data needs to be obtained in a faster way and sometimes even immediately after the data are generated in the operation data sources, such as health systems and digital agriculture. Thus, the conventional ETL process and the disposable techniques are incapable of making the operational data delivered in real-time, providing low latency, high availability, and scalability. As our proposal, we present an innovative architecture, named Data Magnet, to cope with real-time ETL processes. The experimental tests performed in the digital agriculture domain using real and synthetic data showed that our proposal was able to deal in real-time with the ETL process. The Data Magnet provided great performance, showing an almost constant elapsed time for growing data volumes. Besides, Data Magnet provided significant performance gains over the traditional trigger technique.
Palavras-chave:	Data warehouse Real time ETL Data extraction Data loading
Digital Object Identifier:	https://doi.org/10.1016/j.heliyon.2023.e15728
Tipo do material:	Artigo de periódico
Acesso:	openAccess
Aparece nas coleções:	Artigo em periódico indexado (CPPSE)