Part I: Porting the NCTUns Network Simulator to Linux
4. Porting to Linux
4.1 User-Level Components
4.3.6 Processing Kernel Timers and Kernel Events
In a simulation, the kernel may generate two kinds of kernel event – the tunnel check event and the kernel timeout event. In section 4.3.2, we already explain why we need to generate a tunnel check event and where we generate this kind of events. In this section, we will clearly explain how a kernel timeout event helps us to trigger a kernel software timer which is based on the virtual time (We call this kind of timer a
virtual-time timer and the original timer is called a real-time timer).
4.3.6.1 Maintain Virtual-time Timers
In the original Linux kernel, all software timers (real-time timers) are maintained in a global structure tvecs[]. In order to manage real-time timers and virtual-time timers individually, we create another global structure, callwheel, to store virtual-time timers. When the kernel wants to schedule a timer, we should first know whether the
timer is a virtual-time timer or not. If not, the timer should be inserted to tvecs[] and triggered by the original kernel triggering mechanism. If yes, the timer should be inserted to the callwheel and triggered by the simulation engine. As such, the original kernel function internal_add_timer(), which is used to add a timer structure into tvecs[], should be modified like the following code segment:
static inline void internal_add_timer(struct timer_list *timer) {
unsigned long expires = timer-> expires;
unsigned long idx = expires – timer_jiffies;
struct list_head *vec; virtual-time virtual-timer into the
callwheel static inline void internal_add_timer(struct timer_list *timer) {
unsigned long expires = timer-> expires;
unsigned long idx = expires – timer_jiffies;
struct list_head *vec; virtual-time virtual-timer into the
callwheel
The added kernel function check_if_nctuns_timer() is used to check whether a timer is based on the virtual time. If yes, it will return a value that is larger than zero.
Its implementation is shown as below:
extern void tcp_write_timer(unsigned long);
extern void tcp_delack _timer(unsigned long);
extern void tcp_k eepalive_timer(unsigned long);
extern void it_real_fn(unsigned long);
extern void process_timeout(unsigned long);
int check _if_nctuns_timer(struct timer_list *timer) {
void (*fn)(unsigned long);
fn = timer->function;
if (fn==tcp_write_timer || fn==tcp_delack _timer || fn==tcp_k eepalive_timer) { struct sock *sk = (struct sock *)timer->data;
if (sk ->nodeID)
return sk ->nodeID;
}
if (fn==it_real_fn || fn==process_timeout) {
struct task _struct *p = (struct task _struct *)timer->data;
if (p->p_node)
return p->p_node;
} }
extern void tcp_write_timer(unsigned long);
extern void tcp_delack _timer(unsigned long);
extern void tcp_k eepalive_timer(unsigned long);
extern void it_real_fn(unsigned long);
extern void process_timeout(unsigned long);
int check _if_nctuns_timer(struct timer_list *timer) {
void (*fn)(unsigned long);
fn = timer->function;
if (fn==tcp_write_timer || fn==tcp_delack _timer || fn==tcp_k eepalive_timer) { struct sock *sk = (struct sock *)timer->data;
if (sk ->nodeID)
return sk ->nodeID;
}
if (fn==it_real_fn || fn==process_timeout) {
struct task _struct *p = (struct task _struct *)timer->data;
if (p->p_node)
unsigned long expires;
unsigned long data;
void (*function)(unsigned long);
}
The function field of the timer list structure contains the address of the function to be executed when the timer expires. In check_if_nctuns_timer(), we only care about five kinds of timer structure whose function filed points to the following functions: tcp_write_timer(), tcp_delack_timer(), tcp_keepalive_timer(), it_real_fn(), and process_timeout(). This is because only these five kinds of kernel timer will be used by the NCTUns 1.0 network simulator. At the same time, if the sk->nodeID or
p->p_node in check_if_nctuns_timer() is larger than zero, it means that the timer
belongs to a simulated node and should be inserted into the callwheel.The nctuns_callout_helper() is used to insert the virtual-time timer into the callwheel and insert a kernel timeout event into the tunnel interface t0e0. The
following shows its pseudo code:
#define callwheelsize 8192
#define callwheelmask (callwheelsize-1) timerbuck et callwheel[callwheelsize];
void nctuns_callout_helper(struct timer_list *timer, long us) {
1. if t0e0 output queue is full fatal error!
2. else
2.1 timeout = timer->expires*1000 + us;
2.2 insert timer to callwheel[timeout & callwheelmask ] 2.3 insert a k ernel timeout event into t0e0
with expire time = timeout 3. wak e up the simulation engine
}
#define callwheelsize 8192
#define callwheelmask (callwheelsize-1) timerbuck et callwheel[callwheelsize];
void nctuns_callout_helper(struct timer_list *timer, long us) {
1. if t0e0 output queue is full fatal error!
2. else
2.1 timeout = timer->expires*1000 + us;
2.2 insert timer to callwheel[timeout & callwheelmask ] 2.3 insert a k ernel timeout event into t0e0
with expire time = timeout 3. wak e up the simulation engine
}
Because we set the frequency of timer interrupt to 1000 HZ (once every 1 milli-second), the jiffies, a global variable which stores the number of happened timer interrupts, will be based on milli-second. As such, the unit of expires field of a timer structure will be milli-second. In other words, if we want to schedule a software timer, the shortest time interval will be 1 milli-second. But 1 milli-second is too large for us, we want a shorter time interval such as 1 micro-second. Therefore, in nctuns_callout_helper(), there is an argument us to bring in the micro-second information.
After inserting the kernel timeout event into the tunnel interface t0e0, we have to wake up the simulation engine to receive the event. Then, the simulation engine will read the event from t0e0 and insert the event to its event queue.
4.3.6.2 Kernel Timeout Event Triggering
When the event scheduler (in the simulation engine) processes a kernel timeout
event, it will use a user-defined system call (system call 263) to trigger the corresponding kernel timer. Inside the kernel, we implement a kernel function
nctuns_callback() to do this job:
void nctuns_callback (void) {
1. timeout = NCTUNS _ticks_to_us
2. timer = callwheel[timeout & callwheelmask ] 3. while timer
if timer->expires == NCTUNS_tick s callback timer->function timer = timer->next
}
In nctuns_callout_helper(), the unit of the hash key (timeout) is micro-second.
Therefore, in nctuns_callback(), the hash key (timeout) should also be based on micro-second. As in the previous discussion, NCTUNS_ticks_to_us will turn the current virtual time into micro-second. However, because the expires field of a timer structure is based on milli-second, we need to compare the timer->expires with NCTUNS_ticks rather than NCTUNS_ticks_to_us when in step 3 of nctuns_callback().