1
Network Simulation and Testing
Polly Huang EE NTU
http://cc.ee.ntu.edu.tw/~phuang phuang@cc.ee.ntu.edu.tw
Today
General System Analysis
The Internet
3
The Engineering Cycle
• For a running system
– Monitor the usage
– Characterize the
workload
– Predict for the future
– Revise original design
– Instrument the changes
– And back to the top
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection
General
System
Chicken or Egg
• But where did it all
start?
– It depends
– The Internet case?
– The alternatives
selection
– But it’s really just
experts’ intuition
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives SelectionGeneral
System
5
Sounds Easy…
Yah, easy to talk about it…
Monitoring the Usage
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives SelectionGeneral
System
• Monitor the usage
– Measurement methodology – Measurement tools
• Characterize the workload
• Predict for the future
• Revise original design
• Instrument the changes
7
Characterizing the Workload
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection
General
System
• Monitor the usage
• Characterize the workload
– Modeling the measured data
– The model needs to remain valid for data from taken at different time/location
• Predict for the future
• Revise original design
• Instrument the changes
Predicting the Performance
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives SelectionGeneral
System
• Monitor the usage
• Characterize the workload
• Predict for the future
– Anticipate the user access pattern, demand increase – Evaluate the existing’s
capacity
• Revise original design
• Instrument the changes
9
Designing the Alternatives
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives Selection
General
System
• Monitor the usage
• Characterize the workload
• Predict for the future
• Revise original design
– If the current system won’t live up to the challenge, how can it be changed… – Effective solutions
Instrumenting the Changes
System Instrumentation Usage Monitoring Workload Characterization Performance Prediction Alternatives SelectionGeneral
System
• Monitor the usage
• Characterize the
workload
• Predict for the future
• Revise original design
• Instrument the
changes
11
If This is the Telephone Network
• Monitor the usage
– The big players place monitors all over the
places in their own networks to collect data
• Characterize the workload
– Fit the collected data to the well-known models
– Human voice is Poisson
Telephone Network (cont)
• Predict for the future
– Queuing theory:
• Safe to supply 1 bandwidth for a call of average rate 1
1 +2 bandwidth for calls of average rate 1 and 2
– Linear programming:
• Given the max tolerable blocking rate, max the profit
• Revise original design
– Mostly infrastructure-ral
– I.e., rearranging or adding switches&cables
• Instrument the changes
13
No Real Difficulties
Is the data network as profitable?
15
Today
General System Analysis
The Internet
Internet
: Basic Components
• Think the
postal system
• Nodes
– End hosts
and less number of
routers
– Homes
and local/remote
post offices
• Links
– Connecting
nodes (Access net, Ethernet, T1,
T3, OC3, OC12, etc)
17
Internet:
Basic Constructions
• Packets
– Destined to
IP addresses
(129.132.66.28)
– Destined to
postal addresses
(1, Sec. 4 Roosevelt Rd.)
• Protocols
– Packets sent with
TCP
(reliable)
– Packets sent with
registered mail with confirmation
– But
no congestion control
Internet Protocol Stack
• Application:
supporting network
applications
– FTP, SMTP, HTTP
• Transport:
host-host data transfer
– TCP, UDP
• Network:
routing of datagrams from
source to destination
– IP (addressing, routing, forwarding)
• Link:
data transfer between neighboring
network elements
– Error Checking, MAC, Ethernet
application
transport
network
link
physical
19
Data Network Research
(Side-Bar)
• Application
– HTTP evolution
– Web caching
• Transport
– TCP evolution
• Network
– Unicast routing
– Multicast routing
• QoS
– Problem Detection (loss,
congestion)
– Control (end-to-end,
router-assisted)
• Mobile and Wireless
– Isolating drops due to bit error
for TCP
– Routing for dynamic networks
– Handling the hidden terminal
problem
• P2P
– Distributed directory for
effective searching
Internet Protocol Stack
(Back to the Topic)
• Application:
supporting network
applications
– FTP, SMTP, HTTP
• Transport:
host-host data transfer
– TCP, UDP
• Network:
routing of datagrams from
source to destination
– IP (addressing, routing, forwarding)
• Link:
data transfer between neighboring
network elements
– Error Checking, MAC, Ethernet
application
transport
network
link
physical
21
Physical communication
application transport network link physical application transport network link physical application transport network link physical application transport network link physical network link physical data dataThe Network Core
• Mesh of interconnected
routers
– Routers under the same
administration are deemed
within one Autonomous System
(or domain)
– Backbone ASs vs. edge ASs
• Data sent thru net in discrete
“chunks”
– Packet switching
23
Internet: The Network
• The Global Internet consists of
Autonomous Systems
(AS)
interconnected with each other:
– Stub AS: small corporation: one connection to other AS’s
– Multihomed AS: large corporation (no transit): multiple
connections to other AS’s
– Transit AS: provider, hooking many AS’s together
• Two-level routing:
– Intra-AS: administrator responsible for choice of routing
algorithm within network
Internet AS Hierarchy
Inter-AS border (exterior gateway) routers
25
Circuit-Switched Network
The Telephone Network
Internet: The Traffic
Differences:
packets as low-level component
multiple kinds of traffic
27
Let’s come back to this question:
Today
General System Analysis
The Internet
29
What If This is the Internet
• Monitor the usage
– The big players place monitors all over the places in
their own networks to collect data
– Would this give you representative data?
• Characterize the workload
– Fit the collected data to the well-known models
– Human voice is Poisson
Internet (cont)
• Predict for the future
– Queuing theory:
• Save to supply 1 bandwidth for a call of average rate 1
1 +2 bandwidth for calls of average rate 1 and 2
– Average, a good measure? Does traffic add up?
• Revise original design
– Mostly infrastructure-ral
– Still infrastructure-ral?
• Instrument the changes
– Have full authority to change
31
For the Most of These Questions
Repeat The Engineering Cycle
• For the Internet
– Monitor the usage
• Passive and active measurement
– Characterize the workload
• Traffic, topology, routing errors, access pattern modeling
– Predict for the future
• Scalable simulation & testing tools
– Revise original design
• Protocol and Infrastructure
– Instrument the changes
Scalable Packet-level Simulation Internet Instrumentation (IETF) Reliable Measurement Internet Characterization Structure & Design Decision
The
Internet
33
Relevance to This Course
• Objective
– We know something better than others do
– We do the right experiments so the results will
be convincing
• Requirements
– Representative (or best known) workload
– Trusted (most used) tools
Workload
• Traffic
– Packet-level characteristics
– Correlation to protocol and user behavior
– Know better how to generate
traffic
for your experiments
• Topology
– Router/domain-level connectivity
– Correlation to routing
35
Workload
• Dynamic
– Packets: drop and delay
– Routing: policy and instability
– Know better how to generate
error
for your
experiments
Case Studies
• Performance evaluation in three major types
– 1. Understanding a protocol & coming up with the best
configuration
• Compare performance varying parameters
– 2. Coming up with a good protocol design choice
• Compare many mechanisms for the same purpose
– 3. Coming up with a model/theory for the performance
of a commonly-used protocol
37
Tools
• ns-2
– About the most popular in the research community
– Platform for cross-examination
• tcpdump
– Not the only one but the most efficient one
– Also the most popular one in the research community
• dummynet
– Not the only one
The Useful Theory
Statistics
39
Keyword: Heavy-Tailed
• It turned out computer processes tend to be
heavy-tailed or power-law distributed!
– CPU time consumed by Unix processes
– Size of Unix files
– Size of compressed video frames
– Size of FTP bursts
– Telnet packet interarrivals
– Size of Web items
Illustrated
y = a – b x
linear
y = a e
-bx, b>0 exponential
heavy-tailed
y = a / x
b, b>0
41
Review Some Statistics
• Density vs. Distribution
• Poisson
• Exponential
• Pareto
• Self-similarity…
43
Density vs. Distribution
• Density is the probability of certain events
to happen
– f(x)
• Distribution is usually referred to as the
accumulative density
– f(0)+f(dz)+f(2*dz)+…+f(x)
– F(x) =
0->xf(z) dz
Exponential
• # of time units between events
45
Poisson
• # of events per time unit
47
Pareto
• One of the heavy-tailed distributions
49
Distinguishing Them
• Density
• Log density
51 Log Density Log-Log Density Density
Now, what’s with
self-Similarity…
53
Teletraffic vs. Data traffic
• Teletraffic
• Data traffic
Exponential
Heavy tailed
Exponential
Exp
• High variability (bursty)
• Long-range dependency
• Self-similarity
• High variability (bursty)
• Long-range dependency
• Self-similarity
• Performance problem every
1-2 hours (network lags!)
• Profitable business?
• Performance problem every
1-2 hours (network lags!)
• Profitable business?
10ms
100ms
1s
10s
100s
55
Self-similarity?
• Distributions of
#packets/unit
look alike in
different time scale
Wavelet Analysis
• FFT - frequency decomposition d
j• WT - frequency and time decomposition d
j,k
k(d
j,k2) / N
j
E
j• E
j= 2
j(2H-1)C (The magic!!)
• log
2E
j= (2H-1)
j + log
2C
57
Self-similar
’Shape' of self-similarity
59
Evaluation Methodology
• Math
– Pen and papers
– Economical
– Gives you the average
• Simulation
– Few computers and simulation software
– Affordable
– Gives you the behavior or distribution
• Implementation
– Many computers and system software
– Costly
Which should you use?
Depends on what you care for the
problem in hand!
61
Assumptions
• It’s OK to leave out details
• But
– You need to be clear what details you leave out.
– You need to argue it is OK to leave those
details out for now.
– And you are working on including those details
and the results will be available in the future.
63
Traffic Papers
• V. Paxson, and S. Floyd, Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, Vol. 3 No. 3, pp. 226-244, June 1995
• W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, On the Self-Similar Nature of Ethernet Traffic. IEEE/ACM Transactions on Networking, Vol. 2, No. 1, pp. 1-15, Feb. 1995
• M. E. Crovella and A. Bestavros, Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes. IEEE/ACM Transactions on Networking, Vol 5, No. 6, pp. 835-846, December 1997
• Anja Feldmann; Anna C. Gilbert; Polly Huang; Walter Willinger, Dynamics of IP traffic: A study of the role of variability and the impact of control. In the Proceeding of SIGCOMM '99, Cambridge, Massachusetts, September 1999
Topology Papers
• E. W. Zegura, K. Calvert and M. J. Donahoo. A Quantitative Comparison of Graph-based Models for Internet Topology. IEEE/ACM Transactions on Networking, December 1997. • M. Faloutsos, P. Faloutsos and C. Faloutsos. On power-law
relationships of the Internet topology. Proceedings of Sigcomm 1999. • H. Tangmunarunkit, R. Govindan, S. Jamin, S. Shenker, W. Willinger.
Network Topology Generators: Degree-Based vs. Structural. Proceedings of Sigcomm 2002.
• D. Vukadinovic, P. Huang, T. Erlebach. On the Spectrum and
Structure of Internet Topology Graphs. In the proceedings of I2CS 2002.
65
Dynamics Papers
• Hongsuda Tangmunarunkit, Ramesh Govindan, and Scott Shenker. Internet path inflation due to policy routing. In Proceedings of the SPIE ITCom, pages 188-195, Denver, CO, USA, August 2001. SPIE • Lixin Gao. On inferring automonous system relationships in the
internet. ACM/IEEE Transactions on Networking, 9(6):733-745, December 2001
• Vern Paxson. End-to-end internet packet dynamics. ACM/IEEE Transactions on Networking, 7(3):277-292, June 1999
• Craig Labovitz, G. Robert Malan, Farnam Jahanian. Internet Routing Instability. ACM/IEEE Transactions on Networking, 6(5):515-528, October 1998
Case Study Papers
• M. Christiansen, K. Jeffay, D. Ott, and F. Donelson Smith. Tuning RED for web traffic. In ACM SIGCOMM2000, August 2000
• J. Broch, D. Maltz, D. Johnson, Y. Hu, J. Jetcheva, A Performance Comparison of Multi-Hop Wireless Ad Hoc Network Routing
Protocols. In the Proceedings of the Fourth Annual ACM/IEEE International Conference on Mobile Computing and Networking (MobiCom'98)
• J. Padhye, V. Firoiu, D. Towsley, and J. Kurose. Modeling TCP throughput: A simple model and its empirical validation. In
Proceedings of the ACM SIGCOMM Conference, pages 303-314, Vancouver, Canada, September 1998. ACM