Efﬁcient Large Flow Detection over Arbitrary Windows: An Algorithm Exact Outside An Ambiguity Region

(1)

Efficient Large Flow Detection over Arbitrary Windows: An Algorithm Exact Outside An Ambiguity Region

Hao Wu

University of Illinois at Urbana-Champaign

Hsu-Chun Hsiao

Carnegie Mellon University National Taiwan University

Yih-Chun Hu

University of Illinois at Urbana-Champaign

ABSTRACT

Many networking and security applications can benefit from exact detection of large flows over arbitrary windows (i.e.

any possible time window). Existing large flow detectors that only check the average throughput over certain time period cannot detect bursty flows and are therefore easily fooled by attackers. However, no scalable approaches provide exact classification in one pass. To address this chal- lenge, we consider a new model of exactness outside an ambiguity region, which is defined to be a range of bandwidths below a high-bandwidth threshold and above a low-bandwidth threshold. Given this new model, we propose a deterministic algorithm, EARDet, that detects all large flows (including bursty flows) and avoids false accusation against any small flows, regardless of the input traffic distribution. EARDet monitors flows over arbitrary time windows and is built on a frequent items finding algorithm based on average frequency.

Despite its strong properties, EARDet has low storage overhead regardless of input traffic and is surprisingly scalable because it focuses on accurate classification of large flows and small flows only. Our evaluations confirm that existing approaches suffer from high error rates (e.g., misclassifying 1% of small flows as large flows) in the presence of large flows and bursty flows, whereas EARDet can accurately detect both at gigabit line rate using a small amount of memory that fits into on-chip SRAM.

Categories and Subject Descriptors

C.2.3 [Computer-Communication Networks]: Net- work Operations—Network monitoring; C.2.0 [Computer- Communication Networks]: General—Security and pro- tection

Keywords

Large Flow Detection; Flow Classification; Ambiguity Region; Arbitrary Windows

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- tion on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org.

IMC’14,November 5–7, 2014, Vancouver, BC, Canada.

Copyright is held by the owner/author(s). Publication rights licensed to ACM.

ACM 978-1-4503-3213-2/14/11 ...$15.00.

http://dx.doi.org/10.1145/2663716.2663724.

1. INTRODUCTION

Being able to identify large flows¹ over any possible time window (referred to as the arbitrary window model ) is of great importance for a wide variety of networking and security applications such as traffic engineering, accounting, anomaly detection, and Denial-of-Service (DoS) defense.

The arbitrary window model monitors time windows on all time-scales, starting at each instant in time and ending at the current time, and can thus detect bursty flows and outperform models that are based on fixed-length windows. Given the high traffic volumes in today’s Internet, no scalable approaches provide exact classification in one pass.

Thus, we consider a novel model of exactness outside a small ambiguity region, which contains flows that use bandwidth between two configurable thresholds. Our model clas- sifies flows as either large, medium, or small. A flow is defined to be a large flow if there exists a time window in which the bandwidth of the flow exceeds the high-bandwidth threshold. A small flow is defined to be a flow whose volume is consistently lower than a low-bandwidth threshold function over all arbitrary window. The rest are defined as medium flows, or the flows in the ambiguity region. Ex- actness outside an ambiguity region guarantees perfect detection of large flows (including bursty flows) and no false accusation against any small flows. This model is reasonable because it limits the damage caused by large flows and allows existing techniques to handle the medium flows statistically. Prior work [17] [16] adopts a similar concept, yet they can only provide probabilistic bounds outside a region. The ambiguity region between the high-bandwidth and low-bandwidth thresholds allows us to trade the level of exactness for scalability, so that we can maintain state small enough to fit into limited on-chip memory for link-speed update.

The new models of exactness and arbitrary window benefit many applications. For example:

• Detecting various DoS flows: Denial-of-Service (DoS) attacks use a combination of large attack flows and bursty attack flows. Flows that are only bursty are hard to catch due to their low average traffic bandwidth. However, using the arbitrary window model, a detector can instantly detect DoS attempts that use bursty flows [25].

• Bandwidth guarantees: To enforce bandwidth allo- cation, schemes such as IntServ make impractical

1Large flows are flows that consume more than a threshold amount of bandwidth. They are also called elephant flows in the literature.

(2)

assumptions that either every router keeps per-flow state [33] or first-hop routers are trusted to regulate traffic on a per-flow basis [36] on behalf of intermediate routers [34]. Although a scalable and robust approach was proposed [28], it causes collateral damage due to the detection delay. Moreover, it cannot catch bursty flows. Our efficient detector built upon these two new models can help enforce bandwidth limits on flows because it enables fast detection with no false accusation on the legitimate small flows and no missed detection on large flows including bursty flows.

To the best of our knowledge, none of the existing algorithms provide exactness outside an ambiguity region under the arbitrary window model. Prior algorithms monitoring average throughput (e.g., algorithms based on fixed window and sliding window models) cannot detect bursty attacks. For example, in a large-flow detection system that resets state and starts a new measurement interval periodically [17], a large bursty flow can bypass detection by staying lower than the threshold of throughput across this whole interval, or even by deliberately spreading its burst across two consecutive intervals. Although randomization of measurement intervals can mitigate the problem of a burst spread across a measurement interval, randomized algorithms may be unable to provide strong deterministic guarantees.

In addition to lacking exactness and the arbitrary window model, the storage overhead of existing algorithms may grow unboundedly with the size of the input traffic in the presence of malicious inputs. For example, an adversary can perturb his traffic patterns (e.g., by varying the size and timing of packets) so as to cause algorithmic complexity attacks [11]

because many algorithms bound their storage and computational overhead by assuming that the flow sizes follow a certain distribution, such as Zipfian.

To identify large flows over arbitrary windows with low storage overhead, we explore deterministic algorithms with a new model of exactness considering a small ambiguity region. We propose EARDet (Exact-Outside-Ambiguity- Region Detector), a simple, efficient, and no-per-flow-state large-flow detector which is exact outside an ambiguity region regardless of the input traffic distribution. Built on the Misra-Gries algorithm (a two-pass frequent items finding algorithm based on average frequency) [31], EARDet is a one-pass streaming algorithm with simple operations: it only keeps a small array of counters which are increased or decreased as each new packet arrives. A flow is identified as a large flow if its associated counter exceeds a threshold.

Surprisingly, despite EARDet’s strong guarantees, we show in our analysis that EARDet requires extremely small amounts of memory that fit into on-chip SRAM for line- speed packet processing. We discuss implementation details to further demonstrate EARDet’s efficiency. EARDet is highly scalable because it focuses on the accurate classification of large and small flows; unlike prior approaches, it does not aim to estimate flow volumes or identify the medium flows. In addition to our theoretical analysis, we also evaluate EARDet using extensive simulations based on real traffic traces. We demonstrate that existing approaches suffer from high error rates under DoS attacks, whereas EARDet can effectively detect large flows in the face of both flooding and burst DoS attacks [22, 25].

Our main contributions are as follows.

• We propose a deterministic streaming algorithm that is exact outside an ambiguity region regardless of the input traffic. Two novel settings distinguish EARDet from previous work: it monitors flows over arbitrary windows, and it supports exact detection outside an ambiguity region.

• We rigorously prove the two guarantees—catching all large flows and preventing false accusation on any small flows—without making assumptions about the flow size distribution.

• Our numerical analysis shows that EARDet can operate at 40 Gbps high-speed links using only hundreds of bytes of on-chip SRAM, which is substantially smaller than the memory consumption in many existing sys- tems. We also provide guidelines on how to configure EARDetto satisfy application-specific requirements.

• We compare EARDet with two closely related proposals [16, 17] via comparative analysis and extensive simulations based on real and synthetic traffic traces.

The results confirm that these two are vulnerable to attack flows that manipulate the input traffic, while EARDetconsistently catches all large flows without misclassifying small flows.

2. PROBLEM DEFINITION

Our goal is to design an efficient arbitrary-window-based algorithm which is exact outside an ambiguity region. In this section, we present the system model, formulate the large flow problem over arbitrary windows, and summarize our design goals.

2.1 System Model

Flow identifiers. Generally, packets are classified into flows based on the flow identifiers (or flow IDs) derived from the packet header fields.² Because our approach to large- flow detection is generic, we make no assumption on the definition of flow IDs. As in prior traffic monitoring work, we assume flow IDs are unforgeable, which is achievable by ingress filtering [19] and source authentication [4, 24, 27, 32].

Packet streams. Let X be the packet space. We consider a packet stream X = hx1, · · · , xki coming through a link of capacity ρ, where xi ∈ X ∀i = 1 · · · k. Packets in X are processed in sequence by a detection algorithm for identifying large flows. The algorithm can only make one pass over the packet stream due to the high link capacity and limited memory.

For a packet x, we denote by time(x) the time at which the detection algorithm observes the packet, by size(x) the size of the packet, and by fid(x) the flow ID of the packet.

The traffic volume of a flow f during a time window [t1, t2) is defined as vol(f, t1, t2),P

x∈X ,fid(x)=f,t1≤time(x)<t2size(x).

Synopses. A traffic synopsis is a data structure that summarizes flows and can be used to answer queries regard- ing certain flow statistics. When a new packet x arrives, the algorithm updates its traffic synopsis based on x’s flow

2While the flow definitions vary depending on the applications, in most applications a flow consists of packets that share one or more header fields, such as the source IP, destination IP, source port, destination port, and the protocol number.

(3)

Landmark window model (Landmark at 0)

0

Sliding window model (window size=30ns) Arbitrary window model

A B C D B

Examine flows in [0, t) → flow B evades detection

Examine flows in [t-30, t) → flow B evades detection

Examine flows in [s, t) for all t > s ≥ 0 →

flow B is a large flow over [10, 50) and can be detected

10 20 30 40 t=50 (ns)

40 Gbps link congested by 50-Byte packets

Figure 1: In this example, a flow is large if it sends more than 40M bps · w + 500kb for any time window of size w. Although flow B violates the limit over the time window [10, 50), it can only be caught in the arbitrary window model.

ID, size, and arrival time. Formally, a large-flow detection algorithm supports three operations over a synopsis S:

• Init(params) → S0. The initialization operation takes as inputs the large-flow definitions and desired detection accuracy, etc.

• Update(Si−1, xi) → Si. The update operation out- puts an updated synopsis Si by incorporating the new packet xi into the previous synopsis Si−1. For convenience, we denote Update(Si, hxi+1, · · · , xi+ji) = Update(Si+1, hxi+2, · · · , xi+ji) = Si+j.

• Detect(Si, xi) → b ∈ {0, 1}. The detection operation evaluates Sito determine if xibelongs to a large flow.

Time window models. Prior approaches to large-flow detection can be classified into three main categories based on the type of time window they monitor: landmark window [10, 14, 17, 18, 23, 29–31], sliding window [5, 21, 26], and arbitrary window [16].

In the landmark window model, each time window starts at the closest landmark in the past (e.g., a landmark is placed every five seconds) and ends at the current time. In the sliding window model, recent traffic is considered more important than old traffic, so the time window begins at some recent time in the past. The window slides as new packets arrive, such that the measurement incorporates the new packets and excludes the oldest packets. Finally, the arbitrary window model monitors every time window ending at the current time. It is more difficult to evade detection in this model than in others, as illustrated in Figure 1. Note that while the arbitrary window model covers every possible window, flows can still evade detection if the detection algorithm is inaccurate.

2.2 Large-Flow Problem Over Arbitrary Windows

Small, medium, and large flows. A flow f is a large flow if there exists a time window [t1, t2) over which its volume vol(f, t1, t2) exceeds a high-bandwidth threshold function THh(t2 − t1). A flow is a small flow if its volume vol(f, t1, t2) is lower than a low-bandwidth threshold function THℓ(t2− t1) over all possible time windows [t1, t2). The rest are defined as medium flows, i.e. flows in an ambiguity region.

In this paper we define the two threshold functions in the form of leaky bucket descriptors: THh(t) = γht + βh and

Limited Memory Incoming Packet Stream

Detection Algorithm x_i

x_i+1

x_i+2 Report

Report

Figure 2: A general framework for a large-flow- detection algorithm. The detection algorithm pro- cesses incoming flows and keeps limited state in memory. Results may be reported to a remote server for further analysis.

TH_ℓ(t) = γℓt + βℓ, where γh> γℓ> 0 and βh> βℓ> 0.³ Al- though selecting appropriate parameters largely depends on the targeted application, we provide guidelines for selecting these parameters in Section 4.

Exact-outside-ambiguity-region large-flow problem.

As exact solutions are inefficient, we consider a relaxed no- tion of exactness:

Definition 1. Given a packet stream, the exact-outside- ambiguity-region large-flow problem returns a set of flows F such that (1) F contains every large flow, and (2) F does not contain any small flow.

Because the number of large flows can increase indefinitely over time, F may become too large to fit into the limited on-chip memory. Hence, such an algorithm often reports to a remote server with mass storage that keeps a complete copy of F , as illustrated in Figure 2. The algorithm must therefore operate correctly even without a complete copy of F .

A positive is when a flow is added to F , and a negative is when a flow is not added to F . Hence, a False Positive of small flow (FPs) occurs when the detection algorithm wrongly adds a small flow, and a False Negative of large flow (FNℓ) occurs when it fails to include a large flow.

The exact-outside-ambiguity-region large-flow problem is reasonable for two reasons: (1) It confines the damage by large flows and allows existing techniques (e.g., Sample and Hold [17]) to handle the medium flows statistically. (2) Prior work [17] [16] also involves a region similar to our ambiguity region, but they only provide probabilistic bounds outside the region.

2.3 Design Goals

Our main goals are as follows:

Exactness outside an ambiguity region. To achieve exactness outside an ambiguity region in traffic monitoring, we desire a deterministic monitor algorithm which identi- fies every large flow including bursty flow (i.e., no FNℓ)and

3Instead of using leaky bucket descriptors, prior work in the landmark window model often defines the high-bandwidth threshold to be a fraction of the link bandwidth, e.g., γh = 0.01 and βh = 0. However, it is infeasible to adopt this fraction-based definition when it comes to the arbitrary window model because every flow will violate the threshold over a sufficiently small time window (e.g., a window con- taining only one packet).

(4)

protects every small flow (i.e., no FPs) with no assumption on the input traffic or attack pattern. Hence, the high- bandwidth and low-bandwidth thresholds are also called the no-FNℓand no-FPsthresholds in this paper, respectively.

Scalability. Although using per-flow leaky buckets enables exact and instantaneous detection of large flows, keeping per-flow state is impractical due to the large number of flows in the Internet. Hence, the algorithm should require few per-packet operations and maintain small router state that fits in fast yet scarce storage devices (e.g., on-chip SRAM or even registers) regardless of input traffic or attack pattern, such that the detection algorithm can operate at line rate.

Fast detection. Fast detection of large flows minimizes collateral damage. Hence, for a large flow violating the high- bandwidth threshold over [t1, t2), the algorithm should detect the flow no later than t2+ tprocess, where tprocessis the time it takes to process a packet.

3. ALGORITHM

In this section, we first investigate and prove the no- FPs and no-FNℓ relationships between landmark and arbitrary window models because they are useful for construct- ing large-flow algorithms over arbitrary windows. Based on these relationships, we present EARDet, a streaming algorithm that efficiently addresses the large-flow problem over arbitrary windows (as defined in Definition 1) with exactness outside an ambiguity region. Finally, we discuss the implementation and optimization techniques in detail, and numer- ically demonstrate that EARDet can operate at high-speed links while using only hundreds bytes of on-chip SRAM.

It is important to investigate the relationships between landmark and arbitrary window models because they enable us to draw on the rich experience of research on the large- flow problem over landmark windows [10,14,17,18,23,29–31]

for designing arbitrary-window algorithms. Particularly, we are interested in knowing whether and to what extent we can leverage existing landmark-window algorithms to build arbitrary-window ones. The technical contributions of this paper include proving two theorems that shed light toward a systematic approach applying existing landmark-window algorithms to arbitrary-window algorithms.

EARDetleverages the Misra-Gries (MG) algorithm [31], which finds all frequent items in a data stream in one pass but may falsely include non-frequent items. The MG algorithm works over landmark windows in the sense that the landmark is at the beginning of the data stream. The research challenges here include (1) how to preserve MG’s no-FNℓ property (over landmark windows) when porting it to the arbitrary window model, and (2) how to achieve the no-FPs property when processing packets in one pass.

EARDetmodifies the MG algorithm in several novel ways to effectively address the above challenges. Interestingly, despite these simple modifications, we prove that EARDet achieves both no-FPs and no-FNℓproperties over arbitrary windows, thereby providing strong guarantees regardless of input traffic.

3.1 Relationships Between Landmark and Arbitrary Windows

Here is a straightforward yet inefficient solution to the exact-outside-ambiguity-region large-flow problem over arbi-

trary windows: the algorithm divides the problem into multiple sub-problems that each can be handled by a landmark- window algorithm, L. More concretely, let Lⁱ be a copy of L that monitors a time window starting from packet xi

and ending at the current time. For every newly arrived packet xi, the algorithm initiates Lⁱ, and adds xi to the new as well as all previous copies, L¹, L², · · · , Lⁱ. Then the algorithm combines the answers returned by L¹, L², · · · , Lⁱ. This straightforward solution is correct, but requires space linear in the length of the traffic stream, which is pro- hibitively expensive.

To make it more efficient, the key idea is to eliminate redundant copies of L. To show why this is possible, we formally state two relationships between landmark windows and arbitrary windows.

No-FPs relationship. We observe that only one copy of L is needed to achieve the no-FPsproperty over arbitrary window. Specifically, Theorem 2 states that if an algorithm ensures (L1) no FPs in the landmark window model, then it also ensures (A1) no FPsin the arbitrary window model.

Theorem 2. If an algorithm satisfies

• L1: For all t, it never reports a flow whose volume is below γ^′_ℓt + β^′_ℓover time interval [0, t).

then it must also satisfy

• A1: It never reports a flow whose volume is below γℓ(t2−t1)+βℓover time interval [t1, t2) for all t2> t1. when γ^′_ℓ= γℓand β^′_ℓ= βℓ.

Proof sketch: L1 implies A1 because if a flow sends less than γℓ(t2−t1)+βℓfor all intervals [t1, t2), it must also send less than γ_ℓ^′t + β^′_ℓ for all intervals [0, t) when γ_ℓ^′ = γℓ and β_ℓ^′= βℓ.

No-FNℓ relationship. The no-FNℓrelationship is more challenging to prove. We observe that only one copy of L is needed to achieve the no-FNℓproperty if L’s traffic synopsis is “similar” to the initial state throughout the execution of the algorithm, as checking such a synopsis is roughly equiv- alent to checking all of L¹, L², · · · , Lⁱ. In other words, we can keep only one synopsis which somehow approximates the synopsis in each sub-problem.

Formally, we define a distance metric dis(S, S^′) quantify- ing the similarity between two synopses:

dis(S, S^′), min

X,S^′=Update(S,X)tspan(X)

where the time span of a packet sequence is defined as tspan(X) = maxx∈Xtime(x) − minx∈Xtime(x). S is defined as a snapshot of the algorithm’s internal state and thus does not depend on time. The distance between two snapshots is the shortest time it takes to convert one to the other given a link capacity.

Theorem 3 states if an algorithm ensures (L2) no FNℓ

in the landmark window model and (L3) its synopsis is bounded, it also ensures (A2) no FNℓin the arbitrary window model.

Theorem 3. If an algorithm satisfies

• L2: For all t, it reports all flows whose volume exceed γ_h^′t + β_h^′ over time interval [0, t).

• L3: Throughout the execution of L, dis(S0, Si) ≤

∆, where ∆ is a small constant and Si = Update(S0, hx1, · · · , xii).

(5)

then it must also satisfy

• A2: It always reports a flow whose volume exceeds γh(t2 − t1) + βh over time interval [t1, t2) for some t2> t1.

when γh(t2− t1) + βh≥ γ_h^′(t2− t1+ ∆) + β_h^′.

Proof sketch: Let L be an algorithm satisfying L2 and L3.

Let f be a flow that sends more than γh(t2− t1) + βhover some time interval [t1, t2), and t2 is the smallest among all possible values if the flow f violates the spec multiple times.

To prove this No-FNℓrelationship, in the following we show that L can catch any f when γh= γ_h^′ and βh≥ β_h^′ + γh∆, thus satisfying A2 as well.

For convenience, we denote by Xa^b the incoming packet stream between time interval [a, b). Since L satisfies L3 (i.e., the synopsis state of L is always bounded), dis(S0, S^t¹) ≤ ∆ where S^t¹ is its synopsis state at t1.

Based on the definition of the distance function, there exists a packet sequence X^′with a time span less than ∆ and Update(S0, X^′) = S^t¹. In other words, from the algorithm’s perspective, Update(S0, X₀^t²) = Update(S0, X^′kX_t^t₁²),⁴ i.e., the two packet sequences produce identical synopses. As a result, if the algorithm L can detect f in X^′kX_t^t₁² then it can also detect f in X₀^t² because the output of the detection function, Detect, solely depends on the synopsis.

Moreover, by construction, f sends more than γh(t2 − t1) + βh in the new sequence X^′kX_t^t₁², whose time span is t2− t1+ ∆. Therefore, L can detect f in the new sequence because γh(t2− t1) + βh≥ γ_h^′(t2− t1+ ∆) + β_h^′ holds when γh= γ_h^′ and βh≥ β_h^′ + γh∆. Hence, L can also detect f in the original stream, X₀^t².

We note that L1 and A2 contradict each other for any parameter selection: for any γ_ℓ^′, β_ℓ^′, γh, and βh, consider an interval [t1, t2) satisfying t1 = t2− ǫ and t2> ^β^h^−β^′^ℓ_γ^+γ′ ^h^ǫ+1

ℓ .

Then a flow sending γh(t2− t1) + βh+ 1 over [t1, t2) will violate the high bandwidth threshold over [t1, t2) but comply with γ_ℓ^′t2+ β^′_ℓover [0, t2). That is, no algorithm can satisfy (A2, L2, L3) and (A1, L1) at the same time.

The above two theorems can be viewed as guidelines for designing new arbitrary-window algorithms based on existing landmark-window algorithms.

3.2 Algorithm Construction

Several existing landmark-window approaches [10, 14, 17, 23, 29–31] satisfy L2 when β_h^′ is set to zero. Among these approaches, we observe that the MG algorithm [31] can be made to satisfy L2 in a general setting (i.e., β^′_h can be non-zero) as well as L3 with slight modifications. As a result, we choose to leverage the MG algorithm for designing EARDet.

We prove in the next section that EARDet’s design ensures L2 and L3, and therefore achieves the no-FNℓ property (i.e., catching every large flow) based on Theorem 3.

We also prove that EARDet achieves the no-FPs property (i.e., protecting every small flow), whereas the MG algorithm requires a second pass to remove false positives in the landmark window model.

Background of the MG algorithm. We briefly review the MG algorithm, which inspires our design. The MG algorithm finds the exact set of frequent items (defined as items

4k denotes concatenating two packet sequences into one sequence.

Is x blacklisted? Counter update for virtual flows packet x No

Blacklist

counter >

threshold?

Blacklist Update

Yes

Counter update for x

Figure 3: EARDet’s decision diagram.

that appear in a stream of m items for more than_n+1^m times) in two passes with only n counters. This algorithm gener- alizes the Majority algorithm [6, 20], which focuses on the case when n = 2. The same generalization was rediscovered by Demaine et al. [14] and Karp et al. [23].

The MG algorithm assumes an associative array of counters indexed by items. Counters are initialized to zeros. We say that an item is stored if its counter is above zero. For each incoming item e, the MG algorithm works as follows:

(1) If e is stored (i.e., ctr[e] > 0), increase ctr[e] by 1; (2) Else if the number of non-zero counters is less than n, ctr[e] = 1;

and (3) Otherwise, decrease all non-zero counters by 1.

Since there are at most n non-zero counters kept at any time, the storage overhead is O(n). This can be easily ex- tended to items with positive weights. After the first pass, the MG algorithm guarantees that every frequent item is associated with a non-zero count, and a second pass is required to remove falsely included infrequent items.

The correctness of this algorithm can be shown intuitively as follows: Suppose an item e appears more than_n+1^m times, but is not stored at the end. The total count would have been reduced by more than _n+1^m · (n + 1) = m counts during the execution, which is impossible since it is more than the total number of items.

EARDet Overview. Figure 3 illustrates EARDet’s decision diagram for each incoming packet. At a high level, EARDetworks similarly to the MG algorithm except three crucial distinctions:

• Blacklist: EARDet keeps a local blacklist L that stores recently identified large flows. Counters are updated only if the flow ID of the packet is not blacklisted. The main purpose of keeping the blacklist is to avoid increasing a flow’s counter when the counter value has already exceeded a counter threshold, βT H. Additionally, we can avoid spending unnecessary resources on accounting blacklisted flows. We bound the blacklist’s size in Section 3.3.

• Counter threshold: A flow is added to the blacklist if its associated counter value exceeds a threshold βT H. Setting a counter threshold together with blacklisting ensures that counter values are always confined, i.e.,

≤ βT H+ α, where α is the maximum packet size.

• Virtual traffic: In contrast to the frequent-item problem, the large-flow problem has to take the idle time between two consecutive packets into account so as to accurately detect large flows with respect to the link capacity. EARDet handles this by virtually fill- ing the unused bandwidth with virtual traffic. Virtual traffic consists of multiple virtual flows, each of which is crafted purposely to comply with the low-bandwidth threshold, thus avoiding unnecessary alarms.

Algorithm description. Algorithm 1 describes how EARDet works. As the MG algorithm, EARDet keeps

(6)

a 5

b 11

Blacklist: b g 2 a

βTH

5 βTH + α

b 8

g 2

b 3

Blacklist:

a 3

b 9

Blacklist: b

b Blacklist: b e

2

virtual traffic 6

6 a

5 b 8

g 2

Blacklist:

Figure 4: Example of EARDet’s counter update.

n counters, each initialized to zero. Counters are stored in an associative array indexed by flow IDs, and the number of non-zero counters never exceeds n. EARDet counters are increased and decreased by the size of the packets, since a packet of size w can be viewed as w uni-sized items. We denote by C the set of non-zero counters. We discuss how to implement EARDet efficiently in Section 3.3.

Algorithm 1 EARDet

1: Initialization (S← Init(n), Line 8-9) 2: foreach packet x in the stream do

3: ifx’s FID f is not blacklisted (f /∈ L) then 4: Update counters for virtual traffic (Line 18-22) 5: Update counters for x (S← Update(S, x), Line 10-17) 6: ifdetect violation (Detect(S, x) == 1, Line 21-22) then 7: Add f to blacklist (L ← L ∪ {f})

8: Initialization, Init(n)

9: initialize all counters to zeros,L ← ∅, C ← ∅ 10: Update counters for packetx, Update(S, x) 11: ifx’s FID f is kept (f∈ C) then

12: Update f ’s counter by the packet size w (c_f ← cf+ w) 13: else ifless than n counters are kept (|C| < n) then 14: Set f ’s counter to w (c_f← w, C ← C ∪ {f}) 15: else

16: Decrease all counters by d = min{w, minj∈Ccj} 17: Set cf to w− d, and ∀j remove j from C if cj= 0 18: Update counters for virtual traffic between xi and

xi−1

19: Compute the virtual traffic size, v (v = ρt_idle− size(xi−1), and tidle= time(xi)− time(xi−1))

20: For each unit u in the virtual traffic, update counters as if u belongs to a new flow (e.g., unit is 1 byte)

21: Detect violation, Detect(S, x)

22: Return whether x’s flow counter exceeds threshold (c_f >

βT H)

Figure 4 gives an example showing how to update counters when n = 3, βT H= 10, and α = 3, where α is the maximum packet size. First, since there is an empty counter, flow g is added and its counter value becomes 2, the size of the new packet. Then, since flow b is stored already, its counter is increased by 3. The new value of ctr[b] exceeds βT H, and thus flow b is blacklisted. The next flow, e, is not stored yet and there is no empty counter, so all counters are decreased by the packet size. Finally, the virtual traffic is divided into single-unit packets with new flow IDs, resulting in the final state.⁵

Despite EARDet’s simple operations, work remains to prove the no-FPsand no-FNℓproperties and to devise practical parameters. We answer these in Section 4.

5Conceptually, the counter values are updated as follows:

[3, 9, 0] → [3, 9, 1] → [2, 8, 0] → [2, 8, 1] → [1, 7, 0] → [1, 7, 1] → [0, 6, 0]. Section 3.3 discusses techniques to ac- celerate this process.

3.3 Data Structure and Optimization

While EARDet requires very little memory state, its processing delay may be high in a naive implementation where EARDetaccesses every counter for each decrement operation (i.e., Line 16 in Algorithm 1). We now present several optimization techniques to reduce the number of memory accesses and the processing time.

Reducing number of memory accesses. We minimize the number of memory accesses per packet via the following technique. First, we keep counters in a data structure that allows insertion, deletion, and finding the minimum in loga- rithmic time. Data structures such as balanced search trees and heaps can satisfy our requirements. Moreover, counter values are not absolute but relative to a floating ground, cground. Hence, the decrement operation, which requires decreasing all counters previously, can now be achieved by elevating the floating ground. The detection function becomes cf− cground> βBF.

The increment operation on Line 12 takes O(1) time using an associative array. Adding a value to an empty counter as described on Line 14 takes O(log n) time because we have to insert the counter to the data structure. To prevent counter overflow, EARDet periodically resets the floating ground to zero and deducts all counters accordingly.

Efficient counter update for virtual traffic. Virtual traffic ensures accurate accounting of unused bandwidth, but efficient implementation is needed to handle virtual traffic at line speed. We now discuss how to efficiently update EARDet’s counters for virtual traffic. Particularly, we aim to divide virtual traffic into multiple virtual flows in a way to minimize the time to process such virtual flows. The only constraint is that each virtual flow should comply with the low-bandwidth threshold to avoid triggering false alarms.

As Line 20 of Algorithm 1 shows, for each unit u of the virtual traffic, EARDet updates its counters (i.e., Update(S, u)) as if the traffic unit belongs to a new flow.

We can minimize the number of updates by maximizing the unit size. To avoid false detection, the maximum size per unit is βT Hbytes. As βT Hmust be larger than the minimum packet size (i.e., 40 bytes) for practical use, the overhead of using βT H-byte virtual flows is bounded by the worst-case scenario where the link is congested by minimum-sized packets.

We can further optimize this task based on the following observation: Once all counters become empty, they should stay empty until the next real packet comes. Furthermore, since the maximum counter value is βBF+ α, counters will all be empty if the size of the virtual flow ≥ (βBF+ α) · n.

(A tighter condition is if the virtual traffic size ≥ (maxjcj) ∗ n −P

jcj, but this requires keeping track of the sum of all counters.) In other words, EARDet can simply reset all counters to zeros and avoid any update if the virtual traffic size exceeds a certain threshold.

Counter implementation. For efficiency, counters are implemented as integers (e.g., in bytes) rather than non- integer numbers. While packet sizes are always multiples of bytes, the size of virtual traffic may be non-integer, which introduces biases on EARDet’s guarantees. For example, given a 800Mbps link and a nanosecond time precision at the router, the size of a 1-ns virtual traffic is 0.1 bytes.

(7)

We bound such biases with a slightly modified algorithm that adjusts virtual traffic. Let us denote by {v1, v2, · · · } the sizes of a sequence of virtual traffic and by {v^′₁, v^′₂, · · · } the adjusted sizes. We maintain an extra field called “carryover”, co, which keeps the amount of uncounted virtual traffic. co is initialized to zero, and we ensure that −0.5 ≤ co < 0.5 for all time. Virtual flows are adjusted such that v^′_i← [vi+ coi] and coi+1← coi+ vi− v_i^′where coiis the value of co before proceeding vi. By construction, v_i^′s are all integers, and for any a, b, |Pb

avi−Pb

av_i^′| = |cob+1− coa| ≤ 1. In other words, the adjusted virtual traffic differs from the original one by at most 1 unit for any time interval. Consequently, the modified algorithm guarantees to catch flows violating THh(t) = γht + (βh+ 1) and guarantees not to catch any flow conforming to THℓ(t) = γℓt + (βh− 1).

Bounding the blacklist. EARDet keeps in memory not only counters but also a local blacklist storing recently detected large flows.

We propose a simple mechanism to bound |L| and thus prevent algorithmic complexity attacks to overflow the blacklist as follows. Since the blacklist’s main purpose is to avoid increasing a flow’s counter when the counter value has already exceeded a counter threshold, once a counter value has dropped below the threshold, the flow can be removed from the blacklist. In this way, the size of L will always be bounded by n because only flows that are currently stored can be blacklisted.

The key observation here is that flows that are not currently stored can be removed from the blacklist because such removal will not affect EARDet’s no-FNℓand no-FPsguar- antees, as in EARDet whether a flow will be caught or not does not depend on other flows’ behavior. The detector can periodically report the current blacklist to a remote administrator, such that the administrator keeps a complete list of detected large flows, while the detector maintains a small blacklist that helps avoid increasing a flow’s counter when the counter value has already exceeded βT H. The only tradeoff of this mechanism is that EARDet may spend unnecessary resources on accounting flows that have been identified as large flows.

Parallelizing EARDet. A common way to reduce processing time is parallelization. EARDet can be parallelized at both the algorithm and instruction levels. At the algorithm level, we can randomly distribute the flows (thus the workload) among multiple copies of EARDet. At the instruction level, we can access and update multiple counters in parallel using multi-port SRAM when the operations are order insensitive. We note that special-purpose SRAMs (i.e., multi-port SRAMs) can support multiple read/write simultaneously [38].

3.4 Storage and Computational Complexity

Given the above optimization techniques, we analyze EARDet’s storage and computational overhead.

To operate at line rate on OC-768 (40 Gbps) high-speed links, a typical 3.2 GHz processor has to process 40 million medium-sized (1000 bits) packets per second, which means the per-packet processing time should be at most 32 ns or 76 CPU cycles. In this analysis, we consider the following memory model for commodity routers: CPU has 32 KB L1 cache, 256 KB L2 cache, 20 MB L3 cache, and gigabytes main DRAM memory. Accessing L1, L2, and L3 caches

takes 4, 12, and 30 CPU cycles, respectively; accessing the main memory is as slow as 300 cycles.

Storage complexity. EARDet keeps an extremely small traffic synopsis, a blacklist of detected flows, and storage of flow ID keys (i.e. unique keys consist of IP address and port number) for flow identifier. The synopsis consists of n counters and a constant number of additional variables for optimization such as the floating ground. In IPv4, a flow ID key is 48 bits, and for IPv6, it is 144 bits. By imple- menting the tree map using a red-black tree [8], each counter needs store only one flow ID key. In most applications the synopsis and flow ID keys will be small enough to fit en- tirely in the router’s L1 cache. For instance, in IPv4, using 100 32-bit counters and 100 48-bit key variables requires only 960 bytes; even in IPv6, the same case requires only 2200 bytes, which can fit in the L1 cache. (Guided by our analysis in Appendix A, we use 100 counters as a represen- tative example.) If the number of counters is much larger than 100 in some configuration, we can also use L2 cache with some performance cost. In practice, a special-purpose device with more fast memory can also be installed for large flow detection. Moreover, we can flexibly tune the counter size to further reduce the memory requirement at the cost of a wider ambiguity region between the no-FPs and no-FNℓ

thresholds.

Computational complexity. For each packet, EARDetlooks up and updates one or more counters, and adjusts the internal data structure (e.g., a heap) of counters.

In EARDet, locating and updating a counter requires an average of O(1) memory access in a hash-map-based associative memory. Adjusting the data structure of n counters requires O(log n) memory accesses.

Since EARDet’s state is small enough to fit into the L1 cache as we discussed, the per-packet processing time can be as low as tens of nanoseconds, which is suitable for processing packets at 40Gbps high-speed links. Even when all of EARDet’s data is in the L2 cache, EARDet can still process packets at line speed on 13Gbps.

4. ANALYSIS

In this section, we prove the no-FPs and no-FNℓproper- ties. Furthermore, we analyze the incubation period of large flows, discuss EARDet’s tradeoffs, and present practical guidelines for EARDet configuration. Finally, we compare EARDet with closely related proposals [16, 17] to demonstrate that it outperforms prior work in terms of both efficiency and detection accuracy.

We consider a network link with a capacity of ρ, and a EARDet detector with n counters. The counter threshold is βT H. Once the value of a counter exceeds βT H, the associated flow will be judged as a large flow and cut off immediately. Hence, the maximum value of each counter is βT H+ α, where α is the maximum packet size. Table 1 summarizes the notations used in this section. We will discuss the relationship among parameters and how to set them in the Section 4.6.

4.1 Large-Flow False-Negative Analysis

Theorem 4. No-FNℓ property. EARDet detects every flow violating the high-bandwidth threshold T Hh(t) =

(8)

Table 1: Table of Notations.

Network management parameters:

ρ , Rate of link capacity α , Maximum packet size

tupincb , Upper bound of tincbfor any large flows T Hℓ , Low-bandwidth threshold

T Hh , High-bandwidth threshold

γℓ, βℓ , Rate and burst for low-bandwidth threshold γh , Rate for high-bandwidth threshold

Tunable parameters:

n , Number of counters in EARDet βT H , Threshold of counters(> βℓ) Parameters that depend on tunable parameters:

βh , Burst for high-bandwidth threshold β∆ , βT H− βℓ

Other notation:

R(t1, t2) , Average flow rate in [t1, t2) t_incb , Incubation period of large flows RN F N , No-FN_ℓrate

RN F P , No-FPsrate

γht+βhover a time window of length t, when γh≥ RN F N =

ρ

n+1 and βh≥ α + 2βT H.

Proof sketch: Firstly, we prove that EARDet satisfies L3 in Theorem 3. According to Algorithm 1, the maximum value of each counter ciis βT H+ α, and there are at most n non-zero counters at any time. Also, given any valid synopsis S = {ci} we can construct a packet stream X consisting of ci bytes for flow i and no space between packets, and by construction S = Update(S0, X). Combining the above two arguments and the definition of the distance function, we conclude that dis(S0, S) ≤ ^{size(X )}_ρ ≤ ^(β^{T H}_ρ^+α)n. That is, setting ∆ = ^(β^{T H}_ρ^+α)n satisfies L3.

Next we prove that EARDet satisfies L2 in Theorem 3 as well when setting γ_h^′ = RN F N = _n+1^ρ and β_h^′ = βT H. We prove by contradiction and assume there were a flow f violating γ_h^′ + β_h^′ in the landmark window model at time t but not being detected (i.e., cf < βT H). This assumption implies that more than γ_h^′t + β_h^′ − βT H amount of flow f would have been canceled out⁶ during the decrement step, or equivalently, more than (γ_h^′t + β^′_h− βT H) · (n + 1) = γ_h^′t · (n + 1) = ρt amount of traffic would have been canceled out. This statement, however, contradicts the setting where the maximum traffic for t units of time is ρt. Thus, f cannot escape from EARDet, and L2 is satisfied by EARDet.

Based on Theorem 3, we conclude that EARDet satisfies A2 when γh = γ_h^′ = _n+1^ρ and βh ≥ β_h^′ + γh∆ = βT H + _n+1^ρ ^(β^{T H}_ρ^+α)n = βT H+ _n+1ⁿ (βT H + α). In partic- ular, EARDet catches every flow violating the threshold T Hh(t) = γht + βhwhen γh≥ RN F N and βh≥ α + 2βT H. That is, EARDet catches all large flows in the arbitrary window model.

4.2 Small-Flow False-Positive Analysis

As discussed in Section 3.1, no algorithm can satisfy A2 in Theorem 3 and L1 in Theorem 2 at the same time. Hence, rather than applying Theorem 2, we have to take a different approach in proving the no-FPsproperty.

6A packet byte is canceled out if it does not contribute to the corresponding counter.

To analyze EARDet’s no-FPsproperty, we consider how EARDetincreases and decreases its counter values. Firstly, let us examine all cases based on the types of incoming flows.

We say a flow is old if it is stored in the counters currently;

otherwise the flow is new.

1. When the incoming flows are virtual flows and there are l empty counters, in a time window t, the decrement is _l+1^ρ t on all counters, and the increment is 0.

(l = 0, 1, 2, 3, ..., n)

2. When the incoming flows are new real flows and there is no empty counter, in a time window t, the decrement is ρt on all counters and the increment is 0 (which is the same as the first case when l = 0).

3. When the incoming flows are old real flows, or new real flows and there are some empty counters, in time interval t, the decrement is 0 and the increment is ρ t on one counter.

Thus, in the first and second cases, when there are l empty counters in the detector, the decrement is always_l+1^ρ t in the interval of t; and in the third case, the increment is always ρ t on one counter in the interval of t. Finally, the increment and decrement cannot happen at the same time.

Lemma 5. For any small flow f that complies with the low-bandwidth threshold (i.e., T Hℓ(t) = γℓt + βℓ), once the flow f is associated to a counter at t1, this counter will be always lower than βT H after time t1+ tβℓ, if the counter is occupied by the same flow as the flow f , where tβ_ℓ =

(n−1)α+(n+1)βℓ [1−(n+1)γℓ/ρ]ρ .

Due to space limitations, the detailed proofs are in our technical report [37].

Theorem 6. No-FPs property. EARDet will not catch any flow complying with the low-bandwidth threshold T Hℓ(t) = γt + βℓ for all time windows of length t, when 0 < βℓ < βT H, γℓ < RN F P, where RN F P =

β∆

(n−1)α+(n+1)βℓ+(n+1)β∆ · ρ.

Proof sketch: According to Lemma 5, to avoid catching a small flow f , the counter should be smaller than βT H before tβℓ. Hence, we choose a γℓto achieve γℓtβℓ+ βℓ< βT H. Then, (n−1)α+(n+1)βℓ

[1−(n+1)γ_ℓ/ρ]ρ <^β^{T H}_γ^−β^ℓ

ℓ ,

⇔ γℓ< β∆

(n− 1)α + (n + 1)βℓ+ (n + 1)β∆· ρ (1) The theorem is proved.

Interestingly, Theorem 6 shows that γℓapproaches_n+1^ρ as β∆increases, but cannot go beyond _n+1^ρ .

4.3 Relationship between Low-Bandwidth and High-Bandwidth Thresholds

Before the discussion, let us define two concepts:

Rate Gap: The ratio between γhand γℓ(i.e. γh/γℓ);

Burst Gap: The ratio between βhand βℓ(i.e. βh/βℓ).

Based on Theorems 4 and 6, the minimum rate gap is:

(γh/γℓ)min=^R_R^{N F N}

N F P =(n−1)α+(n+1)(β_ℓ+β_∆) β_∆(n+1)

Given β∆= βT H− βℓand n + 1= n. = n − 1, we get. (γh/γℓ)min .

= 1 + 2α/β + 2

βh/βℓ− (α/βℓ+ 2) (2)

(9)

Thus, the minimum possible rate gap (γh/γℓ)minis mainly influenced by the burst gap βh/βℓ. Equation (2) tells us a) βh/βℓ cannot be lower than α/βℓ+ 2. b) EARDet only needs a low βh/βℓto achieve small enough (γh/γℓ)min. For example, to achieve (γh/γℓ)min= 10, we only need βh/βℓ= 2.53. c) (γh/γℓ)min cannot be lower than 1. (γh/γℓ)min

approaches to 1 as βh/βℓgrows.

4.4 Incubation Period of Large Flows

To define the incubation period, we first consider a large flow that violates the high-bandwidth threshold over [t1, t2), and the packet at ta triggers the detection. Because of EARDet’s no-FNℓ property, ta ≤ t2. The incubation period is defined as ta− t1, representing the time duration for which the large flow remains under the radar. We bound the incubation period as follows.

Theorem 7. For the flow f which violates T Hh(t) over some time window [t1, t2), if its average rate R(t1, ta) is larger than Ratk in time interval of [t1, ta) (Ratk is a constant rate larger than RN F N = _n+1^ρ ), then f ’s incubation period is bounded by

tincb< α + 2β_{T H}

Ratk−_n+1^ρ . (3)

Proof sketch: Because R(t1, ta) > Ratk, intuitively the tincb of flow with average rate of R(t1, ta) must be shorter than the t^′_incbof flow with rate of Ratk. That is, tincb< t^′_incb. Assume a flow f^′with rate Ratkwill violate T Hh(t) over time window [t^′1, t^′2), then

R_atk(t^′₂− t^′1) = ρ

n + 1(t^′₂− t^′1) + α + 2β_{T H}

⇒ tincb< t^′_incb= t^′_a− t^′1≤ t^′2− t^′1= α + 2βT H

Ratk−_n+1^ρ (4) Thus, the theorem is proved.

From Theorem 7, the bound of the incubation period decreases as Ratkincreases. In other words, if Ratkis fixed, the bound of the incubation period decreases with increasing n, which implies we can reduce the upper bound by adding extra counters. To guarantee detection of flows whose rate is over Ratk (Ratk > _n+1^ρ ), the minimum number of counters is _R^ρ

atk− 1, and the upper bound on the incubation period can be lowered significantly by adding a few counters. The details will be discussed in Section 4.6.

4.5 Tradeoff Analysis

We discuss three tradeoffs in EARDet: (1) memory consumption (i.e., the number of counters) vs. the rate gap, (2) the rate gap and burst gap, and (3) the rate gap and the upper bound on the incubation time.

First, since the rate gap can be expressed as γh/γℓ >

RN F N/γℓ = ^ρ/γ_n+1^ℓ, we can see that the rate gap decreases with increasing n. Second, Equation (2) shows that the minimum rate gap γh/γℓis mainly influenced by βh/βℓ, namely the burst gap, and the minimum rate gap decreases as the burst gap increases. Finally, Theorem 7 shows that a large burst gap results in a long incubation period. Hence, a small rate gap results in a big burst gap, and a high incubation period.

4.6 How To Engineer The Detector

To engineer our detector, we first identify parameters that are often known or given in a priori. Usually, users want a detector for a specific link capacity, ρ, to protect small flows which comply with the low-bandwidth threshold:

T Hℓ(t) = γℓt + βℓ, and to detect attack flows that violate the high-bandwidth threshold: T Hh(t) = γht + βh. How- ever, as discussed in Section 4.5, there is a tradeoff between the rate gap and burst gap, so their requirements cannot be both fulfilled. Thus, we choose to satisfy the rate requirement of γh, and then set βh according to γhbecause it is more important to limit the flow rate than the burst size.

Furthermore, since we want to minimize the incubation period of large flows, there is an upper bound of the incubation period, tupincb.

We set βh = α + 2βT H and γh > _n+1^ρ to guarantee no FNℓaccording to Theorem 4. Since βT H= βℓ+ β∆, we only need to decide the number of counters n and β∆. Hence, the problem can be simplified as follows. Given ρ, γℓ, βℓ, γh, α, and tupincb, we aim to calculate n and β∆such that the parameters satisfy the constraints in Theorems 4, 6 and 7.

The detailed solution and analysis are in Appendix A. The detailed analysis shows that the tunable parameters depend on only the thresholds and are independent of the input traffic.

5. EVALUATION

To evaluate EARDet, we compare EARDet with two closely related proposals, which we refer to as FMF [17] and AMF [16]. Both of our theoretical comparison and trace- based simulations demonstrate that EARDet performs bet- ter than prior work in terms of both exactness⁷ outside an ambiguity region and efficiency. The simulation results using real and synthetic traffic traces are consistent with the analysis in the theoretical evaluation.

5.1 Theoretical Comparison

Multistage filters. Fixed-window-based Multistage Fil- ters (FMF) identify large flows in a fixed measurement interval. A FMF consists of parallel stages, each of which is an array of counters initialized to zeros at the beginning of a measurement interval. Each stage is assigned a hash function that maps a packet’s flow identifier to a counter in the stage. For each incoming packet, its flow identifier is hashed to one counter in each hash stage, and the counter value increments by the size of the packet. A flow is considered a large flow if all of its corresponding counters exceed a pre-specified threshold.

Arbitrary-window-based Multistage Filters (AMF) identify large flows over arbitrary windows. To work in the arbitrary window model, AMF replaces each counter in FMF with a leaky bucket of a bucket size u and a drain rate r. A flow is considered a large flow if the corresponding leaky buckets are all violated.

Performance Comparison. Table 2 presents a numerical comparison of EARDet, FMF and AMF, where the high-bandwidth rate is 1% of the link capacity, and the low- bandwidth rate is 0.1% of link capacity. The results of FMF and AMF are derived based on the authors’ original analysis

7The comparison in this section uses our definition of exactness.