Working in webdev industry since he was 12 years old. After 5 years of day-to-day Rails and React he co-founded a SaaS product company which evolved from an Elixir side project.
Current state-of-art DAG workflow schedulers (AirFlow, Luigi) do not seem to leverage the power of on-demand cloud computing instances. Therefore, Lucjan and his team used Elixir's distributed capabilities to write a powerful, yet simple, scheduler for their data processing pipeline. Lucjan will talk about the various approaches to running external processes in Elixir, monitoring and displaying progress UI in LiveView and testing and the challenges.