Ananya Shandilya

Staff Software Engineer at LinkedIn

Ananya is a Staff Software Engineer at LinkedIn solving problems in the infrastructure observability domain. She has been an Erlang developer for the last 3 years working on the network flow collection and processing system. She is a polyglot developer who can code in Erlang, Golang, Java and Python.

Talk:
Network Observability at LinkedIn

The LinkedIn infrastructure has thousands of services serving millions of requests per second. At this scale, various kinds of data points must be collected, processed and observed to maintain the health of the infrastructure - one of them is network flows.

Our infrastructure components export flows at the rate of 2M packets per second. This talk describes a data collection, processing and storage system for network flow data written in Erlang. It gives an overview of the system’s architecture and some of the interesting challenges we faced while scaling this system.

OBJECTIVE

Share the details about the data processing system we built at LinkedIn using Erlang for collecting and processing high volume and velocity of network flow data

AUDIENCE

Erlang developers / Observability engineers / Backend engineers