VPC Flow Logs

Metadata about every connection attempt

VPC Flow Logs record information about IP traffic going to and from network interfaces in a VPC, a subnet or a single interface. They capture metadata, not payloads: source and destination addresses and ports, protocol number, packet and byte counts, a time window, and whether the traffic was ACCEPTed or REJECTed by security groups or network ACLs. Logs can be delivered to CloudWatch Logs, S3 or Amazon Data Firehose, and queried with tools such as CloudWatch Logs Insights or Athena. Flow logs answer questions like 'is the database receiving connections from the app tier at all?' and 'which addresses are scanning our load balancer?'. They are not real-time packet capture; records are aggregated over a capture window, so expect a delay. Some traffic is not logged, such as queries to the VPC DNS resolver and instance metadata requests. Google Cloud and Azure provide VPC flow logs and NSG or VNet flow logs respectively.

Reading a default-format record

The default format is space-separated. Protocol 6 is TCP. This REJECT means a security group or NACL dropped an app-to-database connection.

record = ("2 123456789012 eni-0a1b2c3d 10.0.1.25 10.0.2.40 49152 5432 6 "
          "10 840 1700000000 1700000060 REJECT OK")

fields = ["version", "account_id", "interface_id", "srcaddr", "dstaddr",
          "srcport", "dstport", "protocol", "packets", "bytes",
          "start", "end", "action", "log_status"]
flow = dict(zip(fields, record.split()))

proto = {"6": "TCP", "17": "UDP", "1": "ICMP"}[flow["protocol"]]
print(f"{flow['srcaddr']} -> {flow['dstaddr']}:{flow['dstport']} {proto} {flow['action']}")

Output:

10.0.1.25 -> 10.0.2.40:5432 TCP REJECT

From packet to answer

Troubleshooting with flow logs follows four steps: traffic hits a network interface, a flow record is written, the records land in a log store, and a query answers your question.

Use REJECT plus direction to locate the block

An inbound REJECT on the destination's interface points to its security group or subnet NACL. An inbound ACCEPT followed by an outbound REJECT for the reply is the classic stateless NACL problem.

Quick check: What do VPC flow logs NOT contain?

• Source and destination ports
• Whether traffic was accepted or rejected
• Byte and packet counts
• The packet payload, such as HTTP bodies