Processor

EF_PROCESSOR_POOL_SIZE

Specifies the number of record processors to start. You will need at least one (1) processor for every 2000 records/second. Increasing the number of processors will allow the collector to better handle a high volume of high latency enrichment tasks such as DNS lookups for IP addresses.

:::note While increasing the number of processors can be beneficial, there are diminishing returns at higher processor counts. This is especially true when the number of processors exceeds the number of available CPU threads (real cores + SMT threads) or vCPUs. If you require more than 64 processors, and are using a Standard or Premium License, it may be more beneficial to use multiple collector instances. :::

• Default

  • 4 * the number of license units

EF_PROCESSOR_TRANSLATE_KEEP_IDS

Specifies which identifier values will be included in the final dataset.

• Valid Values

  • none - All identifiers are removed from the final dataset.

  • default - Most identifiers are removed from the final dataset. However some identifiers which are required for common use-cases (e.g. raw protocol port values) are included.

  • all - All identifiers are included in the final dataset.

• Default

  • default

EF_PROCESSOR_DURATION_PRECISION

The desired precision of duration-related values. Values received at a different precision than specified will be converted to the desired precision.

• Valid Values

  • sec - seconds

  • ds - deciseconds

  • cs - centiseconds

  • ms - milliseconds

  • us - microseconds

  • ns - nanoseconds

• Default

  • ms

:::tip For most data sources this should milliseconds (ms) :::

EF_PROCESSOR_TIMESTAMP_PRECISION

The desired precision of timestamp values. Values received at a different precision than specified will be converted to the desired precision.

• Valid Values

  • sec - seconds

  • ds - deciseconds

  • cs - centiseconds

  • ms - milliseconds

  • us - microseconds

  • ns - nanoseconds

• Default

  • ms

:::tip For most data stores, e.g. Elasticsearch, this should milliseconds (ms) :::

EF_PROCESSOR_PERCENT_NORM

The desired representation of percentages. Values received with a different representation than specified will be converted to the desired representation.

• Valid Values

  • 1 - values will be based on a scale of 0-1.

  • 100 - values will be based on a scale of 0-100.

• Default

  • 100

EF_PROCESSOR_KEEP_CPU_TICKS

For telemetry sources which provide CPU usage as timeticks, utilization percentages will be calculated. If this setting is set false the timetick values will be removed from the final dataset. If true they will be kept, in addition to the utilization values.

• Valid Values

  • true, false

• Default

  • false

EF_PROCESSOR_DROP_FIELDS

This setting allows for a comma-separated list of fields that are to be removed from all records. The fields are dropped after all enrichment and PRIOR to the records being sent to the enabled outputs.

:::note The conversion from the default CODEX schema to alternate schemas, e.g. Elastic's ECS or Splunk's CIM, happens within the respective outputs. As fields are drop PRIOR to the outputs, CODEX field names must be used to configure this option. :::

• Valid Values

  • any CODEX-schema field names, comma-separated

• Example

  • flow.export.sysuptime,flow.export.version.ver,flow.start.sysuptime,flow.end.sysuptime,flow.seq_num

• Default

  • ''