I have studied Geometry in university, but programming and numerical solutions have never ceased to fascinate me. For now, I've come to conclusion that I like using Mathematics for obtaining more precise answers in data science.
Several years ago at age 38, I left my job as a lecturer/researcher in Computer Science department to face more challenges in data industry.
Although, the data industry is inherently interdisciplinary, broad and constantly evolving, and the ecosystem around it very noisy, I was very lucky to begin from classic BI and step by step found my way towards big data pipeline.
Currently I am head of data team at Avihang which is a software company finding solutions and producing enterprise products in e-health, mostly.
In this talk, we discuss streaming and the real-time data stack as a solution for analyzing massive, unbounded data sets that are increasingly common in many modern businesses in different fields and their need for more timely and accurate answers.
A streaming data pipeline flows data continuously from source to destination as it is generated, making it being processed along the way so they are used when the analytics, application or business process requires an updating data flow for an on-time analysis. This analysis can be descriptive like a data dashboard, diagnostic like monitoring logs, predictive like an online fraud detection system or prescriptive like data process in self-driving cars.
This talk is perfect for developers or architects and will be accessible to technical managers and business decision makers—no prior experience with streaming data systems required. The only technical requirement this talk makes is that the audience should feel comfortable with major concepts in data pipelines.
During the talk, examples of e-health data pipelines plus our experience in setting up a data streaming stack help to explicitly of the subject.