Lumu Virtual Appliance Metadata Collection with Logstash
Some enterprises use the Elastic stack (ELK) to collect, index, and analyze logs from multiple devices. If yours is one of those organizations, deploy a Lumu Virtual Appliance and create collectors that will receive data directly from existing Logstash devices to seamlessly integrate your network infrastructure with Lumu while layering Continuous Compromise Assessment.
This article provides you with an example of using the typical ELK stack to collect, index, and analyze logs and how to leverage Logstash devices to forward events to a Lumu Virtual Appliance for Continuous Compromise Assessment.
Logstash configuration
Configure Logstash’s filter and output to forward raw messages from different network devices to the Lumu Virtual Appliance, avoiding the transformation of data that will be pushed into the ELK Stack’s components.
The following is an example of how to add a
clone
block inside your
filter
configuration to preserve the raw event, avoid conflicts between original data parsing and the transformation process, and send raw data to the Lumu VA. The
clone
block must be the first one inside the
filter
block.
- filter { clone { clones => [“raw”] } … }
This example will preserve the original event and create a clone with a field named
type
set to “
raw
”. The next step will be adding an additional output configuration to forward the cloned event to the Lumu Virtual Appliance.
- output { if “raw” in [type] { udp|tcp { host => “<Lumu VA IP/hostname>” port => “<Lumu VA collector port>” codec => line { format => “%{message}” } } } }
This example assumes events do not have a
type
field set to “
raw
”. Please set these conditions based on Logstash’s current configuration for collecting events.
The
line
codec combined with the
format
tag, will preserve the original format of the event received by Logstash (saved in the
message
field).
Create and Configure Virtual Appliance Collectors
To create and configure Virtual Appliance collectors, please consult
our documentation
for details. When configuring the collector’s details in the Virtual Appliance, do not use Syslog encapsulation to avoid event processing malfunctions. Logstash’s forwarding feature will add
header
information that could cause irregular Syslog formatting.
In some cases, you will need to tune the collector configuration manually.
Related Articles
Lumu Virtual Appliance Collectors
To get started with Lumu Virtual Appliances, consult our Introduction to Lumu Virtual Appliances article. The Lumu Virtual Appliance (VA) is a pre-configured lightweight virtual machine solution that collects the network metadata of your entire ...Lumu Virtual Appliance DNS Packets Collectors Catalog
Before attempting this type of implementation, we strongly suggest checking out our Lumu Agent for Windows Server , which can act as a DNS server collector and covers the vast majority of the scenarios of this VA implementation (Windows Server 2016 ...Custom Collector API Integration With Packetbeat and Logstash
Some enterprises may already be using Logstash as part of their pipelines to centralize their logs. In such cases, you have the option to deploy custom collectors to seamlessly integrate your network infrastructure with Lumu while layering Continuous ...Lumu Virtual Appliance DNS Queries Collectors Catalog
In the following table, you will find a complete list of DNS Queries Collectors available for deployment as part of Lumu's Virtual Appliances. Collector Logo Collect DNS Queries with Lumu VA and Infoblox Collect DNS Queries with Lumu VA and Citrix ...Collect FortiGate Firewall Metadata with FortiAnalyzer and Lumu VA
In scenarios where all your FortiGate deployment logs are centralized within a FortiAnalyzer, you can use it to accelerate the deployment of Lumu and forward all firewall logs at once using the FortiAnalyzer data collection capabilities from Lumu. ...