NSG flow logs will retire on September 30, 2027. New NSG flow logs can't be created after June 30, 2025.

Requirements

• An active Azure subscription.
  An Azure subscription is required to enable the VNet flow logs. It is important to have access with an administrator user.
• Lumu Custom Collector API configuration.
  A Lumu custom collector ID and client key are required to set up the collection process. Information on how to create a Netflow custom collector in your Lumu portal can be found in Manage Custom Collectors.
• Script host.
  
  A scripting host is required to deploy the integration. This host must have Internet visibility over Lumu Custom Collector endpoints and Microsoft Azure Blob storage. According to the deployment model you select, you will need a host with:
  • A Docker-enabled host and its Docker Compose command
    
  • The host must have at least 6 GB of free RAM to deploy the integration component.
• Script package
  Contact the Lumu support team to request the package needed to deploy the required files.

Set up your Azure subscription to collect virtual network flow logs

Virtual network (VNet) flow logs is a feature of Azure Network Watcher that allows you to log information about IP traffic flowing through a virtual network (VNet). For more information about this feature, please refer to Flow logs for virtual networks in the Azure documentation.

There are two methods available to enable VNet flow logs if Network Security Groups (NSGs) are already deployed in your subnets and network interfaces:

• You can migrate from NSG flow logs to VNet flow logs using the migration script provided by Microsoft Azure.
• You can also remove the NSG flow logs on Network Watcher and subsequently deploy the VNet flow logs from the scratch, as detailed in the Enable VNet flow logs section onward.
  

If you already have an active VNet flow log and you want to use it for the Lumu integration, you can identify the Storage account used for your Vnet flow log and follow this guide from the Get the Connection String for the Storage Account step onward.

If you do not have NSG flow logs configured, you can follow this guide from the Enable VNet flow logs section onward.

Migrate from network security group flow logs to virtual network flow logs

You can consult the official Microsoft Azure guide for migration, which provides a migration script accessible via this link. The following is a brief overview of the procedure, we encourage you to follow the official documentation to avoid any mistakes during the process.

Migration Steps:

• Prerequisites - Install PowerShell 7, Azure PowerShell module, and ensure proper RBAC permissions.
• Generate Script - In Azure portal, go to Network Watcher > Migrate flow logs, select subscriptions/regions, download migration files (script + JSON).
• Run Analysis - Execute MigrationFromNsgToAzureFlowLogging.ps1, choose option 1, provide JSON file path, review analysis report.
• Choose Migration Type - Select option 2 (with aggregation) or 3 (without aggregation) based on your needs.
• Confirm & Execute - Review summary, confirm migration (cannot be reverted after accepting).
• Verify - Check Azure portal to confirm NSG flow logs are disabled and new VNet flow logs are created.
• Clean Up - Delete old NSG flow logs from the Azure Portal once the process finishes successfully.

It is recommended to keep the previous Firewall collector from the Lumu Portal unless the collected data is no longer needed.

Enable VNet flow logs

Follow these instructions to enable VNet flow logs in your Azure subscription:

1. Access the Azure Portal.

2. Go to the search bar at the top of the portal, and enter Subscriptions. Then, select Subscriptions from the search results.

3. Choose your specific subscription (e.g., Main).

4. In the left-hand menu, select Settings > Resource providers.

5. Type insight in the search box at the top of the list to filter it.

6. Locate the provider named microsoft.insights.

Register the Provider

1. Review the Status column for microsoft.insights.

2. If the status is Not Registered or Unregistered, do the following:

• Click on the corresponding row to select the provider.
• Click Register at the top of the list.

Confirm Registration

1. Allow several minutes for the process to complete. Refresh the page if necessary.

2. The status should change to Registered once the process is completed.

To get a detailed step-by-step guide on how to enable this feature, go to: Tutorial: Log network traffic to and from a virtual machine using the Azure portal.

Create the virtual network flow logs

Follow these steps to create the virtual network flow logs:

1. Access the Azure Portal.

2. Go to the search bar at the top of the portal, and type Virtual networks. Then, select Virtual Networks from the search results.

3. Select your preferred virtual network, then click on Monitoring > Virtual network flow logs (1) on the left menu.

4. Then, click + Create (2).

5. Configure the virtual network flow log using the parameters of your Azure subscription.

• Subscription: YOUR-SELECTED-SUBSCRIPTION
• Resource: YOUR-VNet-NAME
• Flow log name: YOUR-FLOWLOG-NAME
• Location: AZURE-REGION
• Storage account: STORAGE-ACCOUNT-NAME
• Retention days: RETENTION-DAYS — follow your company’s retention policy
• Enable traffic analytics: unchecked
  

6. Click Save to finish.

Get the Connection String for the Storage Account

To get the Connection String of your Storage Account do the following:

1. Access the Azure Portal.

2. Go to the search bar at the top, and type Storage account. Then, select Storage account from the search results.

3. Select the storage account name used to create the VNet flow logs.

4. Then, select Security + networking > Access keys (1) in the left-side menu.

5. Reveal the Connection String and save it for later use during the deployment.

Check your VNet flow logs in Azure Portal (Recommended)

After setting your Azure VNet flow logs, it will take some time for Azure to show the files with the stored logs. We recommend you to check the logs to make sure everything is working as expected. You can check the flow logs following these steps:

1. Access the Azure Portal.

2. Go to the search bar at the top, and type Storage account. Then, select Storage account from the search results.

3. Click on the storage account used to store your VNet logs.

4. Select Data storage > Containers (1) on the left menu.

5. In the Containers window, click on the insights-logs-flowlogflowevent container, which will be created by default once you create the VNet flow logs for your subscription. There, you will find your virtual network flow logs.

All the logs are stored in a long-named hierarchy. The actual files with the VNet flow log entries will be stored following the structure y=<YEAR>/m=<MONTH>/d=<DAY>/h=<HOUR>/m=<MINUTE>/macAddress=<MAC ADDRESS>/PT1H.json. Each file stores the VNet flow logs for a particular resource ID inside your virtual network.

Preliminary setup - Lumu portal

The integration set-up process needs you to collect this information from Lumu portal:

• Lumu Defender API key
• Company UUID

Log in to your Lumu portal and run the following procedures to collect this data.

Collect your Lumu Collector Key

To collect the Lumu Collector key, please refer to the Collector key document.

Collect your Lumu Collector ID

To collect the Lumu Custom Collector key, please refer to the Collector ID document.

Deploy the integration

We offer two deployment alternatives:

• Run the deployment via Makefile, using it as a build automation tool.
• Run the deployment using the bare docker compose command.

Whichever alternative you select, you must unpack the integration package shared by our Support team first. Unpack the deployment package provided by Lumu in your preferred path/folder. Keep in mind this location, as it will be required for further configurations. From now on, we will refer to this folder as <app_lumu_root>.

Deploy via Makefile (Recommended)

The integration component can be configured by executing the make config command. Upon successful data entry, a .env file will be generated, containing the specified parameters for immediate use by the integration.

You will be asked to enter the following parameters:

• Lumu Collector Key
• Lumu Collector ID
• Your Storage Account Name
• Your Storage Account Connection String

Execute the following command in the terminal to deploy the integration using the previously provided parameters.

Deploy via Docker Compose (Optional)

The package comes with a file named setup.sh. This is a script file that can be used to set the required files in the integration. You will be asked to enter the following parameters:

• Lumu Collector Key
• Lumu Collector ID
• Your Storage Account Name
• Your Storage Account Connection String

To be able to execute it, run the following command:

Build and Run the components

To run the component use the following command:

Expected results

After running the integration, you will see new events processed by the custom collector you have created.

Troubleshooting

The commands defined in this section will allow you to troubleshoot the operation of your integration. Keep in mind that you must locate yourself in the <app_lumu_root> folder before running any of them.

Deployment via Makefile

The following are the troubleshooting commands for this deployment option:

• Checking integration logs
  Run the following command to check your integration logs for the two components, the downloader and the collector.

make logs

Review the logs for the Downloader component.

make logs-downloader

Review the logs for the Collector component.

make logs-collector

• Check the status of the integration
  Run the following command to check the status of the integration.

make stats

• Stopping the integration
  Run the following command if you need to stop the integration.
  
  make stop
  
  
• Starting the integration
  
  Run the following command to start the integration.
  make start
  
  
• Fixing issues with sudo for Docker
  If you cannot run Docker commands with your current user, run the following command.
  make docker-fix-sudo
  
  
• Reinstalling integration from scratch

Run the following command to reinstall the integration from scratch:

make reset-force

• Collecting and packaging logs for Lumu support

Run the following command to collect and package the integration logs to share them with the Lumu support team. This command will create the support.tar package file that contains relevant information for the Lumu support team.

make support

Deployment via Docker Compose command

For troubleshooting purposes, you can run the following commands:

• Collecting integration logs

docker compose -f docker-compose.yml logs

Known Issues

Java Out Of Memory

Out of Memory (OOM) errors can be caused by high network traffic on your virtual networks. If this occurs, we recommend increasing the RAM capacity of the host machine.

Consider increasing the RAM capabilities of the container and the JVM or migrate to another, higher-performing machine. You can contact the Lumu support team for assistance. This situation may arise when the volume of network recording exceeds the performance capacity of the integration component.

In this situation, non-processed records could remain queued. Consider removing them from the data folder and restarting the integration with the following commands:

if you are experiencing multiple Out of Memory errors, these could be related to a high volume of traffic in your monitored virtual networks. Contact Lumu support if this situation persists to get assistance to increase the computational resources assigned to the integration component. Ensure that you have the required resources in your main host for this process.

Authentication error

You will get the following error when there are authentication issues. Double check the Storage Account Connection String from your Azure Account when entering the parameters of the integration to fix this issue. 

Wrong Storage Account

You will get the following error when you input the wrong Storage Account from Microsoft Azure. Check the Storage account name and retype it.

Wrong Container

You will get the following error when using the wrong container for the integration. Check if the Azure storage account has a container named insights-logs-flowlogflowevent in it. If it doesn’t, verify that you are using the right storage account or check the flow log configuration you did in the Create the virtual network flow logs step.

Connection Error

You will get the following errors when there are connection issues. If that happens, you must check the network status of your host and the Internet connection.

Azure Side Error

API Collector Resolution Error