Erlang & Elixir programmer since 2013. Member of Erlang Solutions from 2017. Interested in data extraction and artificial intelligence. Author of a high-level web scraping framework for Elixir called Crawly. Co-author of epmdless (allows to connect erlang and elixir nodes without epmd). Co-author of erlang-node-discovery - another erlang clustering application.
In this talk, I will cover a topic of web scraping. Who uses web scraping, why, and the challenges the industry of data extraction is facing. I will make an overview of technologies used to perform data extraction and will show how this process can be enhanced with a help of Crawly, the framework built in Elixir.