While no man is an island, your Virtual Private Cloud (VPC) is, except it’s a digital island floating in the ocean of a public cloud offered by a cloud service provider (CSP). The VPC means that everything on your digital island is yours, and none of the CSPs other customers can (or should be able to!) access it. You’ve likely been introduced to the shared security model, a sometimes-confusing way that organizations and their cloud-services providers (CSPs) split security responsibilities. While your CSP manages the ocean and you manage your island, the digital beach where the two overlap can create confusion.
Even more challenging, your organization may have a multi-cloud infrastructure where you manage some deployments in Microsoft Azure and others in Amazon Web Services (AWS) public clouds. Now, you have two VPCs that also communicate with each other, meaning you can easily lose visibility.
If you’re using AWS services, then you can use VPC Flow Logs to help track activity within your environment and across a multi-cloud deployment.
What are VPC Flow Logs in AWS?
Virtual Private Cloud (VPC) Flow Logs for AWS track inbound and outbound network traffic related to network interfaces, subnets, and VPC, including data like:
- Accepted and rejected traffic
- Skipped records when data exceeds internal capacity
- No data during a capture period
- Security group rules
- Network ACL rules
- IPv6 traffic
- TCP flag sequence
- Network Address Translation (NAT) gateway traffic
- Transit gateway traffic
- Service name, traffic path, and low direction
Organizations can create low logs for network interfaces associated with AWS service like:
- Elastic Load Balancing
- Amazon RDS
- Amazon ElastiCache
- Amazon Redshift
- Amazon WorkSpaces
- NAT gateways
- Transit gateways
IT and security teams can publish VPC Flow Logs to any of the following locations:
- CloudWatch
- Amazon Simple Storage Service (S3)
- Kinesis Data Firehose
At which levels can VPC flow logs be created?
Depending on the level of detailed visibility the organization needs, organizations can capture data at three different levels:
- VPC: all operations and activities within the cloud, including all subnets and interfaces, but capturing the data can become expensive
- Subnet: operations and activities for every interface associated with a specific IP address, including all interfaces, to focus on subnets that require additional monitoring, like isolated internal resources
- Elastic Network Interface (ENI): operations and activities specific to an interface, including those supporting AWS service objects connected to VPCs
What is the difference between VPC flow logs and CloudTrail?
Although VPC Flow Logs and CloudTrail both monitor and audit activities in your AWS environment, they serve different purposes leading to different functionalities.
Purpose
The first thing to understand is that you use them for different reasons.
- VPC Flow Logs: network monitoring for performance and security issues
- CloudTrail: AWS service for operational and risk auditing, governance, and compliance
You should keep in mind that CloudTrail is a service while VPC Flow Logs are basically just data.
Data collected
Since they serve different purposes, they focus on different data:
- VPC Flow Logs: IP traffic data like source and destination IP addresses, ports, protocol, and amounts of data transferred
- CloudTrail: AWS account activity taken by API and non-API accounts, including creating, updating, and deleting AWS resources, console sign-ins, security credentials, and resource configuration changes
While CloudTrail gives you information about who does what, VPC Flow Logs tell you where people are doing things on your network.
What are the benefits of VPC Flow Logs?
With visibility into network traffic patterns, VPC Flow Logs have several use cases.
Performance issues
NAT and transit gateway VPC Flow Logs can help you identify network issues that impact performance. For examples, if packet loss occurs when deploying applications across multiple VPCs or between cloud and on-premises networks, VPC Flow Logs can provide information about packet loss arising from:
- A missing route
- The destination silently dropping them
- Packet size exceeding the maximum transmission unit (MTU)
- Expiration of time to live (TTL)
Security
When you know what normal traffic looks like, you can use the VPC Flow Logs to identify suspicious activity or abnormal communication patterns that indicate a potential security incident. For example, if you suddenly see high volumes of HTTPS requests on port 6089 (IAM HTTPS endpoint), it could indicate that attackers have compromised an API.
Network capacity planning
To optimize performance, you need to assess your AWS infrastructure’s usage so that you can identify potential bottlenecks and create cost-savings strategies. VPC Flow Logs give you insights into your network traffic partners and usage trends, enabling you to allocate resources and plan for future capacity needs. For example, if your Accounting department routinely runs reports that cause a spike in activity, you can prevent bottlenecks that slow down other services.
Compliance
Analyzing network traffic flows is critical to demonstrating compliance with industry and regulatory requirements. For example, if you need to comply with region specific data protection laws, like the European Union (EU) General Data Protection Regulation (GDPR), VPC Flow Logs have a “region” field that you can use to create visualizations and documentation to achieve those outcomes.
What are the limitations of VPC Flow Logs?
Although VPC Flow Logs provide valuable insights, they come with limitations. These limitations include:
- Requiring you to include all peered VPCs in your account
- Inability to change low log configuration or record format after initial creation
- Complexities around managing multiple IPv4 addresses when you want to forward low logs to a private IPv4 address
- Potential data leakages if the flow log displays a primary IPv4 address in the dstaddr or srcaddr fields
- Failure to capture all IP traffic, like traffic generated by a Windows instance or Amazon Windows license activation or traffic between an endpoint network interface and a Network Load Balancer network interface
Graylog: Centralized Log Management For AWS or Multi-Cloud Monitoring
With Graylog’s AWS Kinesis/CloudWatch input you can centralize all your AWS VPC Flow Logs and any other provider flow logs for a single source of network traffic truth across your cloud, multi-cloud, or hybrid infrastructure.
Graylog ingests all log data, no matter what service generates it, then applies a standardized data model so that you can correlate and analyze all events. Since your IT operations and security teams share the same information, they can communicate more effectively.
Further, with Graylog’s lightning-fast search capabilities, your security and IT teams can get the answers they need, even when they’re searching terabytes of data. Purpose-built for modern log analytics, Graylog gives you the two-for-one solution necessary to improve performance and reduce cybersecurity risk. Our cloud-native capabilities and out-of-the-box security content give your teams the ability to collaborate effectively, reducing service downtime and alert fatigue.
To learn how Graylog can help you save money and respond more effectively to issues, contact us today.
