Packet Deduplication

Remove packet duplicates and substantially reduce data traffic volume

Packet Deduplication is a powerful feature that helps network administrators optimize the flow of traffic towards monitoring and security tools by removing redundant duplicate packets.

Bandwidth usage in data centers is constantly increasing, and monitoring technologies are pushed to their limits regarding the volume of data they must process. A large number of duplicate packets can cause complications for visibility architectures.

If a large percentage of the packets supplied to the monitoring tools are duplicates, the effective capabilities of these tools are substantially reduced, as they can become unable to keep up with the volume of network traffic.

How can I get rid of duplicate packets?

The advanced capabilities of our Network Packet Brokers allow them to remove duplicate packets at full line speed before delivering traffic to monitoring tools.

Duplicate data is deleted using both a deduplication time window, and the signature with the following settings:

• IPv4/6 source and destination address;
• IPv4/6 protocol/next header field;
• IPv4 identification;
• TCP/UDP/SCTP source and destination ports (or relative offset in fragmented traffic);
• TCP/UDP/SCTP checksum.

If the same packet is received from numerous VLAN sources, our NPBs are engineered in such a way that the VLAN is always ignored when tapping the same traffic from multiple VLANs ID.

Unlike with other solutions, deduplication is performed at line rate without the need of a co-processor, meaning no delay is created, which is critical in latency-sensitive environments.