Network Simulation and Testing-General System Analysis The Internet Evaluating the Internet

1

Network Simulation and Testing

Polly Huang EE NTU

http://cc.ee.ntu.edu.tw/~phuang phuang@cc.ee.ntu.edu.tw

Today

General System Analysis

The Internet

3

The Engineering Cycle

• For a running system

– Monitor the usage

– Characterize the

workload

– Predict for the future

– Revise original design

– Instrument the changes

– And back to the top

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

Chicken or Egg

• But where did it all

start?

– It depends

– The Internet case?

– The alternatives

selection

– But it’s really just

experts’ intuition

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

5

Sounds Easy…

Yah, easy to talk about it…

Monitoring the Usage

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

• Monitor the usage

– Measurement methodology – Measurement tools

• Characterize the workload

• Predict for the future

• Revise original design

• Instrument the changes

7

Characterizing the Workload

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

• Monitor the usage

• Characterize the workload

– Modeling the measured data

– The model needs to remain valid for data from taken at different time/location

• Predict for the future

• Revise original design

• Instrument the changes

Predicting the Performance

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

• Monitor the usage

• Characterize the workload

• Predict for the future

– Anticipate the user access pattern, demand increase – Evaluate the existing’s

capacity

• Revise original design

• Instrument the changes

9

Designing the Alternatives

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

• Monitor the usage

• Characterize the workload

• Predict for the future

• Revise original design

– If the current system won’t live up to the challenge, how can it be changed… – Effective solutions

Instrumenting the Changes

System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection

General

System

• Monitor the usage

• Characterize the

workload

• Predict for the future

• Revise original design

• Instrument the

changes

11

If This is the Telephone Network

• Monitor the usage

– The big players place monitors all over the

places in their own networks to collect data

• Characterize the workload

– Fit the collected data to the well-known models

– Human voice is Poisson

Telephone Network (cont)

• Predict for the future

– Queuing theory:

• Safe to supply 1 bandwidth for a call of average rate 1

 1 +2 bandwidth for calls of average rate 1 and 2

– Linear programming:

• Given the max tolerable blocking rate, max the profit

• Revise original design

– Mostly infrastructure-ral

– I.e., rearranging or adding switches&cables

• Instrument the changes

13

No Real Difficulties

Is the data network as profitable?

15

Today

General System Analysis

The Internet

Internet

: Basic Components

• Think the

postal system

• Nodes

– End hosts

and less number of

routers

– Homes

and local/remote

post offices

• Links

– Connecting

nodes (Access net, Ethernet, T1,

T3, OC3, OC12, etc)

17

Internet:

Basic Constructions

• Packets

– Destined to

IP addresses

(129.132.66.28)

– Destined to

postal addresses

(1, Sec. 4 Roosevelt Rd.)

• Protocols

– Packets sent with

TCP

(reliable)

– Packets sent with

registered mail with confirmation

– But

no congestion control

Internet Protocol Stack

• Application:

supporting network

applications

– FTP, SMTP, HTTP

• Transport:

host-host data transfer

– TCP, UDP

• Network:

routing of datagrams from

source to destination

– IP (addressing, routing, forwarding)

• Link:

data transfer between neighboring

network elements

– Error Checking, MAC, Ethernet

application

transport

network

link

physical

19

Data Network Research

(Side-Bar)

• Application

– HTTP evolution

– Web caching

• Transport

– TCP evolution

• Network

– Unicast routing

– Multicast routing

• QoS

– Problem Detection (loss,

congestion)

– Control (end-to-end,

router-assisted)

• Mobile and Wireless

– Isolating drops due to bit error

for TCP

– Routing for dynamic networks

– Handling the hidden terminal

problem

• P2P

– Distributed directory for

effective searching

Internet Protocol Stack

(Back to the Topic)

• Application:

supporting network

applications

– FTP, SMTP, HTTP

• Transport:

host-host data transfer

– TCP, UDP

• Network:

routing of datagrams from

source to destination

– IP (addressing, routing, forwarding)

• Link:

data transfer between neighboring

network elements

– Error Checking, MAC, Ethernet

application

transport

network

link

physical

21

Physical communication

application transport network link physical application transport network link physical application transport network link physical application transport network link physical network link physical data data

The Network Core

• Mesh of interconnected

routers

– Routers under the same

administration are deemed

within one Autonomous System

(or domain)

– Backbone ASs vs. edge ASs

• Data sent thru net in discrete

“chunks”

– Packet switching

23

Internet: The Network

• The Global Internet consists of

Autonomous Systems

(AS)

interconnected with each other:

– Stub AS: small corporation: one connection to other AS’s

– Multihomed AS: large corporation (no transit): multiple

connections to other AS’s

– Transit AS: provider, hooking many AS’s together

• Two-level routing:

– Intra-AS: administrator responsible for choice of routing

algorithm within network

Internet AS Hierarchy

Inter-AS border (exterior gateway) routers

25

Circuit-Switched Network

The Telephone Network

Internet: The Traffic

Differences:

 packets as low-level component

 multiple kinds of traffic

27

Let’s come back to this question:

Today

General System Analysis

The Internet

29

What If This is the Internet

• Monitor the usage

– The big players place monitors all over the places in

their own networks to collect data

– Would this give you representative data?

• Characterize the workload

– Fit the collected data to the well-known models

– Human voice is Poisson

Internet (cont)

• Predict for the future

– Queuing theory:

• Save to supply 1 bandwidth for a call of average rate 1

 1 +2 bandwidth for calls of average rate 1 and 2

– Average, a good measure? Does traffic add up?

• Revise original design

– Mostly infrastructure-ral

– Still infrastructure-ral?

• Instrument the changes

– Have full authority to change

31

For the Most of These Questions

Repeat The Engineering Cycle

• For the Internet

– Monitor the usage

• Passive and active measurement

– Characterize the workload

• Traffic, topology, routing errors, access pattern modeling

– Predict for the future

• Scalable simulation & testing tools

– Revise original design

• Protocol and Infrastructure

– Instrument the changes

Scalable Packet-level Simulation Internet Instrumentation (IETF) Reliable Measurement Internet Characterization Structure & Design Decision

The

Internet

33

Relevance to This Course

• Objective

– We know something better than others do

– We do the right experiments so the results will

be convincing

• Requirements

– Representative (or best known) workload

– Trusted (most used) tools

Workload

• Traffic

– Packet-level characteristics

– Correlation to protocol and user behavior

– Know better how to generate

traffic

for your experiments

• Topology

– Router/domain-level connectivity

– Correlation to routing

35

Workload

• Dynamic

– Packets: drop and delay

– Routing: policy and instability

– Know better how to generate

error

for your

experiments

Case Studies

• Performance evaluation in three major types

– 1. Understanding a protocol & coming up with the best

configuration

• Compare performance varying parameters

– 2. Coming up with a good protocol design choice

• Compare many mechanisms for the same purpose

– 3. Coming up with a model/theory for the performance

of a commonly-used protocol

37

Tools

• ns-2

– About the most popular in the research community

– Platform for cross-examination

• tcpdump

– Not the only one but the most efficient one

– Also the most popular one in the research community

• dummynet

– Not the only one

The Useful Theory

Statistics

39

Keyword: Heavy-Tailed

• It turned out computer processes tend to be

heavy-tailed or power-law distributed!

– CPU time consumed by Unix processes

– Size of Unix files

– Size of compressed video frames

– Size of FTP bursts

– Telnet packet interarrivals

– Size of Web items

Illustrated

y = a – b x

linear

y = a e

-bx

, b>0 exponential

heavy-tailed

y = a / x

b

, b>0

41

Review Some Statistics

• Density vs. Distribution

• Poisson

• Exponential

• Pareto

• Self-similarity…

43

Density vs. Distribution

• Density is the probability of certain events

to happen

– f(x)

• Distribution is usually referred to as the

accumulative density

– f(0)+f(dz)+f(2*dz)+…+f(x)

– F(x) = 

0->x

f(z) dz

Exponential

• # of time units between events

45

Poisson

• # of events per time unit

47

Pareto

• One of the heavy-tailed distributions

49

Distinguishing Them

• Density

• Log density

51 Log Density Log-Log Density Density

Now, what’s with

self-Similarity…

53

Teletraffic vs. Data traffic

• Teletraffic

• Data traffic

Exponential

_{Heavy tailed}

Exponential

Exp

• High variability (bursty)

• Long-range dependency

• Self-similarity

• High variability (bursty)

• Long-range dependency

• Self-similarity

• Performance problem every

1-2 hours (network lags!)

• Profitable business?

• Performance problem every

1-2 hours (network lags!)

• Profitable business?

10ms

100ms

1s

10s

100s

55

Self-similarity?

• Distributions of

#packets/unit

look alike in

different time scale

Wavelet Analysis

• FFT - frequency decomposition d

_j

• WT - frequency and time decomposition d

_j,k

 

k

(d

j,k2

) / N

j



E

j

• E

_j

= 2

j(2H-1)

C (The magic!!)

• log

₂

E

_j

= (2H-1)

j + log

₂

C

57

Self-similar

’Shape' of self-similarity

59

Evaluation Methodology

• Math

– Pen and papers

– Economical

– Gives you the average

• Simulation

– Few computers and simulation software

– Affordable

– Gives you the behavior or distribution

• Implementation

– Many computers and system software

– Costly

Which should you use?

Depends on what you care for the

problem in hand!

61

Assumptions

• It’s OK to leave out details

• But

– You need to be clear what details you leave out.

– You need to argue it is OK to leave those

details out for now.

– And you are working on including those details

and the results will be available in the future.

63

Traffic Papers

• V. Paxson, and S. Floyd, Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, Vol. 3 No. 3, pp. 226-244, June 1995

• W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, On the Self-Similar Nature of Ethernet Traffic. IEEE/ACM Transactions on Networking, Vol. 2, No. 1, pp. 1-15, Feb. 1995

• M. E. Crovella and A. Bestavros, Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes. IEEE/ACM Transactions on Networking, Vol 5, No. 6, pp. 835-846, December 1997

• Anja Feldmann; Anna C. Gilbert; Polly Huang; Walter Willinger, Dynamics of IP traffic: A study of the role of variability and the impact of control. In the Proceeding of SIGCOMM '99, Cambridge, Massachusetts, September 1999

Topology Papers

• E. W. Zegura, K. Calvert and M. J. Donahoo. A Quantitative Comparison of Graph-based Models for Internet Topology. IEEE/ACM Transactions on Networking, December 1997. • M. Faloutsos, P. Faloutsos and C. Faloutsos. On power-law

relationships of the Internet topology. Proceedings of Sigcomm 1999. • H. Tangmunarunkit, R. Govindan, S. Jamin, S. Shenker, W. Willinger.

Network Topology Generators: Degree-Based vs. Structural. Proceedings of Sigcomm 2002.

• D. Vukadinovic, P. Huang, T. Erlebach. On the Spectrum and

Structure of Internet Topology Graphs. In the proceedings of I2CS 2002.

65

Dynamics Papers

• Hongsuda Tangmunarunkit, Ramesh Govindan, and Scott Shenker. Internet path inflation due to policy routing. In Proceedings of the SPIE ITCom, pages 188-195, Denver, CO, USA, August 2001. SPIE • Lixin Gao. On inferring automonous system relationships in the

internet. ACM/IEEE Transactions on Networking, 9(6):733-745, December 2001

• Vern Paxson. End-to-end internet packet dynamics. ACM/IEEE Transactions on Networking, 7(3):277-292, June 1999

• Craig Labovitz, G. Robert Malan, Farnam Jahanian. Internet Routing Instability. ACM/IEEE Transactions on Networking, 6(5):515-528, October 1998

Case Study Papers

• M. Christiansen, K. Jeffay, D. Ott, and F. Donelson Smith. Tuning RED for web traffic. In ACM SIGCOMM2000, August 2000

• J. Broch, D. Maltz, D. Johnson, Y. Hu, J. Jetcheva, A Performance Comparison of Multi-Hop Wireless Ad Hoc Network Routing

Protocols. In the Proceedings of the Fourth Annual ACM/IEEE International Conference on Mobile Computing and Networking (MobiCom'98)

• J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. Modeling TCP throughput: A simple model and its empirical validation. In

Proceedings of the ACM SIGCOMM Conference, pages 303-314, Vancouver, Canada, September 1998. ACM