ElastiFlow vs. Filebeat and Logstash

Performance

The following results were obtained with the collectors running on a 16-core (AMD EPYC 7302P) server. The data was output to an Elasticsearch cluster consisting of seven data nodes, with three dedicated master nodes.

As much as possible given the options available, batch sizes and the # of workers were configured to comparable and optimal levels.

To provide a "full featured" comparison, the ElastiFlow Unified Flow Collector was tested with all enrichment features enabled. Logstash was tested with the legacy ElastiFlow 4.x pipeline to give it better feature parity. Filebeat relies on Elasticsearch ingest pipelines for anything beyond basic functionality. These pipelines were NOT used. This does give Filebeat a bit of an unfair advantage, however it was still many times slower despite its more favorable conditions.

Throughput	ElastiFlow	Filebeat	Logstash
Flows/second	✅ 78818	➖ 21217	❌ 5205

Network Flow Data Support

Flow Data Support	ElastiFlow	Filebeat	Logstash
Netflow	✅ 1562 IEs 10 Vendors	✅ 474 IEs 1 Vendor	✅ 422 IEs 2 Vendors
IPFIX	✅ 4585 IEs 44 Vendors	✅ 1319 IEs 11 Vendors	✅ 1329 IEs 12 Vendors
sFlow Flows	✅	❌	❌
sFlow Counters	✅	❌	❌
Broadcom IFA	✅	❌	❌
IEs most recently added	✅ 11 July 2022	❌ 1 April 2019	❌ 4 January 2019
SLA for supporting new vendors/devices	✅	❌	❌

Platform Support

Feature	ElastiFlow	Filebeat	Logstash
Elastic Stack	✅	✅	✅
OpenSearch	✅	❌	❌
Apache Kafka	✅	✅	✅
Splunk	✅	❌	❌
Cribl	✅	❌	❌
ClickHouse/Grafana	✅ winter 2022	❌	❌

Features

Feature	ElastiFlow	Filebeat	Logstash
ECS schema support	✅	✅	❌
CODEX schema support	✅	❌	❌
Schema for IEs not covered by ECS	✅ CODEX	❌	❌
Properly handle Netflow v9/IPFIX Templates	✅	✅	❌
Support Netflow v9/IPFIX Option Data	✅	❌	❌
Translation ("subtype" handling) of IE values	✅ 587 translators	❌	➖ **
GeoIP Enrichment	✅	➖ *	➖ **
Autonomous System Enrichment	✅ Maxmind, RiskIQ or flow record	➖ *	➖ **
Reverse DNS IPs to hostname	✅	➖ *	➖ **
User-defined IPs to hostname	✅	❌	➖ **
User-defined Metadata for IPs	✅	❌	➖ **
AS-based include/exclude for DNS resolutions and Metadata	✅	❌	❌
IP Block include/exclude for DNS resolutions and Metadata	✅	❌	❌
Obscure IP addresses and Hostnames	✅	❌	❌
Threat Intelligence Enrichment	✅ RiskIQ	➖ *	➖ *
Microsoft 365 service enrichment	✅ winter 2022	❌	❌
SalesForce service enrichment	✅ winter 2022	❌	❌
Infer Client & Server sides of a conversation	✅	❌	❌
Community ID support	✅	➖ *	❌
Conversation ID support	✅	❌	❌
User-defined Metadata for Interfaces	✅	❌	➖ **
Translate Interface Index values to Interface Names	✅	❌	❌
Translate AppIDs to Application names and attributes	✅	❌	❌
User-defined Application names and attributes	✅	❌	❌
Adjust Bytes/Packets based on Sample Rate	✅	❌	❌
User-defined sample rates per flow exporter	✅	❌	❌
Normalize timestamp values	✅	❌	❌
Normalize percentage values	✅	❌	❌
Normalize byte values	✅	❌	❌
Configurable timestamp precision	✅	❌	❌

* Must be done in an Elasticsearch Ingest Pipeline. This puts additional load on Elasticsearch, which is already the primary limiter of overall throughput.

** Can be achieved using a Logstash pipeline. This is not provided out of the box and must be developed and maintained.

Turnkey Analytics for the Elastic Stack

Feature	ElastiFlow	Filebeat	Logstash
Dashboards	✅ 29	➖ 8	➖ 8
Visualizations	✅ 347	➖ 78	➖ 143
Machine Learning Jobs Security	✅ 84	❌	❌
Machine Learning Jobs Performance	✅ 12	❌	❌
Machine Learning Jobs Availability	✅ 14	❌	❌
Detections Security	✅ 84	❌	❌