Lumu Virtual Appliance Metadata Collection with Logstash

Lumu Virtual Appliance Metadata Collection with Logstash

Some enterprises use the Elastic stack (ELK) to collect, index, and analyze logs from multiple devices. If yours is one of those organizations, deploy a Lumu Virtual Appliance and create collectors that will receive data directly from existing Logstash devices to seamlessly integrate your network infrastructure with Lumu while layering Continuous Compromise Assessment.

This article provides you with an example of using the typical ELK stack to collect, index, and analyze logs and how to leverage Logstash devices to forward events to a Lumu Virtual Appliance for Continuous Compromise Assessment.
Typical ELK stack deployment with Lumu VATypical ELK stack deployment with Lumu VA

Logstash configuration

Configure Logstash’s filter and output to forward raw messages from different network devices to the Lumu Virtual Appliance, avoiding the transformation of data that will be pushed into the ELK Stack’s components.
The following is an example of how to add a clone block inside your filter configuration to preserve the raw event, avoid conflicts between original data parsing and the transformation process, and send raw data to the Lumu VA. The clone block must be the first one inside the filter block.
  1. filter { clone { clones => [“raw”] } … }
This example will preserve the original event and create a clone with a field named type set to “ raw ”. The next step will be adding an additional output configuration to forward the cloned event to the Lumu Virtual Appliance.
  1. output { if “raw” in [type] { udp|tcp { host => “<Lumu VA IP/hostname>” port => “<Lumu VA collector port>” codec => line { format => “%{message}” } } } }
This example assumes events do not have a type field set to “ raw ”. Please set these conditions based on Logstash’s current configuration for collecting events.
The line codec combined with the format tag, will preserve the original format of the event received by Logstash (saved in the message field).

Create and Configure Virtual Appliance Collectors

To create and configure Virtual Appliance collectors, please consult our documentation  for details. When configuring the collector’s details in the Virtual Appliance, do not use Syslog encapsulation to avoid event processing malfunctions. Logstash’s forwarding feature will add header information that could cause irregular Syslog formatting. 
In some cases, you will need to tune the collector configuration manually.

        • Related Articles

        • Lumu Virtual Appliance Collectors

          To get started with Lumu Virtual Appliances, consult our Introduction to Lumu Virtual Appliances article. The Lumu Virtual Appliance (VA) is a pre-configured lightweight virtual machine solution that collects the network metadata of your entire ...
        • Lumu Virtual Appliance DNS Packets Collectors Catalog

          Before attempting this type of implementation, we strongly suggest checking out our Lumu Agent for Windows Server , which can act as a DNS server collector and covers the vast majority of the scenarios of this VA implementation (Windows Server 2016 ...
        • Lumu Virtual Appliance DNS Queries Collectors Catalog

          In the following table, you will find a complete list of DNS Queries Collectors available for deployment as part of Lumu's Virtual Appliances. Collector Logo Collect DNS Queries with Lumu VA and Infoblox Collect DNS Queries with Lumu VA and Citrix ...
        • Collect MikroTik Firewall Metadata with Lumu VA

          Requirements MikroTik Router OS 6 or newer. Have admin access to create a new Forwarding configuration. Have the most recent version of the Lumu Virtual Appliance installed. These are the general steps you should follow to configure a syslog server ...
        • Custom Collector API Integration With Packetbeat and Logstash

          Some enterprises may already be using Logstash as part of their pipelines to centralize their logs. In such cases, you have the option to deploy custom collectors to seamlessly integrate your network infrastructure with Lumu while layering Continuous ...