myEvalvid-NT (myEvalvid Network Trace)

 

[Motivation]

1)    Previous studies often use publicly available real video traces to evaluate their proposed network mechanisms in a simulation environment. Results are usually presented using different performance metrics, such as the packet/frame loss rate, and packet/frame jitter. Nevertheless, packet loss or jitter rates are network performance metrics and may be insufficient to adequately rate the perceived quality by a (human) end user.

 

2)    How to best simulate and evaluate the performance of video quality delivery in a simulated network environment is a recursive open issue in network simulation forums.

 

3)    Many studies have adopted the MPEG4 traffic traces provided at http://www-tkn.ee.tu-berlin.de/research/trace/ltvt.html. But to the best of my knowledge, no tool-set is publicly available to perform a comprehensive video delivered quality evaluation using these traffic traces in network simulation environments.

 

[What I have done]

Refer to Evalvid system, and I developed myEvalvid-NT.

 

[Overview of myEvalvid-NT]

 

 

In this framework, we can use the network traffic trace download at http://www-tkn.ee.tu-berlin.de/research/trace/ltvt.html.

       myEvalvid: It is employed to extract the frame type and the frame size of the video trace file. Then it fragments the video frames into smaller segments and sends these segments to the lower UDP layer at the appropriate time according to the user settings specified in the simulation script file.

       my_UDP: Essentially, my_UDP is an extension of the UDP agent. This new agent allows users to specify the output file name of the sender trace file and it records the timestamp of each transmitted packet, the packet id, and the packet payload size.

       myEvalvid_Sink: myEvalvid_Sink is the receiving agent for the fragmented video frame packets sent by MyUDP. This agent also records the timestamp, packet ID, and payload size of each received packet in the user specified file.

       The fraction of decodable frame rate: Standard MPEG encoders generate three distinct types of frames, namely I, P, and B frames. Due to the hierarchical structure of MPEG, I frames are more important than P frames, and in turn P frames are more important than B frames. Therefore, a frame is considered decodable if, and only if, all the fragmented packets of this frame and the other packets that this frame depends on are completely received. Thus, the decodable frame rate (Q) is defined as the number of decodable frames over the total number of frames sent by a video source. The larger the Q value, the better the video quality perceived by the end user.

 

[Steps]

1.     Download the trace. (Take the StarWarsIV: high quality trace as an example)

 

2.     Open it and remove the first two lines

 

3.     change the path to myevalvid-NT

 

4.     Run the simulation script. (A video is transmitted from a wireless node 0 to another wireless node 1. The wireless channel adopts random uniform error model.

 

proc getopt {argc argv} {

       global opt

        lappend optlist nn

        for {set i 0} {$i < $argc} {incr i} {

             set opt($i) [lindex $argv $i]

       }

}

 

getopt $argc $argv

 

#loss_model: 0 for uniform distribution, 1 for GE model

#in this example, we focus on random uniform error model.

#therefore, only opt(3) is used to indicate the error rate.

set loss_model  $opt(0)

set pGG $opt(1)

set pBB $opt(2)

set pG $opt(3)

set pB $opt(4)

 

 

#===================================

#     Simulation parameters setup

#===================================

set val(chan)   Channel/WirelessChannel    ;# channel type

set val(prop)   Propagation/TwoRayGround   ;# radio-propagation model

set val(netif)  Phy/WirelessPhy            ;# network interface type

set val(mac)    Mac/802_11                ;# MAC type

set val(ifq)    Queue/DropTail/PriQueue   ;# interface queue type

set val(ll)     LL                         ;# link layer type

set val(ant)    Antenna/OmniAntenna        ;# antenna model

set val(ifqlen) 50                         ;# max packet in ifq

set val(nn)     2                          ;# number of mobilenodes

set val(rp)     AODV                       ;# routing protocol

set val(x)      500                        ;# X dimension of topography

set val(y)      500                        ;# Y dimension of topography

set val(stop)   5000.0                       ;# time of simulation end

Mac/802_11 set dataRate_  2Mb

Mac/802_11 set basicRate_ 1Mb

 

#===================================

#        Initialization       

#===================================

#Create a ns simulator

set ns [new Simulator]

 

#Setup topography object

set topo       [new Topography]

$topo load_flatgrid $val(x) $val(y)

create-god $val(nn)

 

#Open the NS trace file

set tracefile [open out.tr w]

$ns trace-all $tracefile

 

set chan [new $val(chan)];#Create wireless channel

 

#===================================

#     Mobile node parameter setup

#===================================

$ns node-config -adhocRouting  $val(rp) \

                -llType        $val(ll) \

                -macType       $val(mac) \

                -ifqType       $val(ifq) \

                -ifqLen        $val(ifqlen) \

                -antType       $val(ant) \

                -propType      $val(prop) \

                -phyType       $val(netif) \

                -channel       $chan \

                -topoInstance  $topo \

                -agentTrace    ON \

                -routerTrace   OFF \

                -macTrace      ON \

                -movementTrace OFF

 

#===================================

#        Nodes Definition       

#===================================

#Create 2 nodes

set n0 [$ns node]

$n0 set X_ 200

$n0 set Y_ 400

$n0 set Z_ 0.0

$ns initial_node_pos $n0 20

set n1 [$ns node]

$n1 set X_ 300

$n1 set Y_ 400

$n1 set Z_ 0.0

$ns initial_node_pos $n1 20

 

 

set n0if_ [$n0 set netif_(0)]

$n0if_ set-error-level $pGG $pBB $pG $pB $loss_model

 

set n1if_ [$n1 set netif_(0)]

$n1if_ set-error-level $pGG $pBB $pG $pB $loss_model

 

#===================================================

set max_fragmented_size 1024

#8 bytes:UDP header, 12 bytes: RTP header

set packetSize [expr $max_fragmented_size+20]  

 

set src_udp1 [new Agent/my_UDP]

$src_udp1 set packetSize_ $packetSize

$src_udp1 set_filename wireless_sd

set dst_udp1 [new Agent/myEvalvid_Sink]

$ns attach-agent $n0 $src_udp1

$ns attach-agent $n1 $dst_udp1

$ns connect $src_udp1 $dst_udp1

$dst_udp1 set_filename wireless_rd

 

set original_file_name Verbose_StarWarsIV.dat

set trace_file_name video1.dat

set original_file_id [open $original_file_name r]

set trace_file_id [open $trace_file_name w]

 

set frame_count 0

 

while {[eof $original_file_id] == 0} {

 

    gets $original_file_id current_line

    scan $current_line "%d%s%d%d" seq_ frametype_ nexttime_ length_

 

    #1000/25 = 40ms = 40000 us

    set time [expr 1000*40]

 

    if { $frametype_ == "I" } {

      set type_v 1

    }     

 

    if { $frametype_ == "P" } {

      set type_v 2

    }     

 

    if { $frametype_ == "B" } {

      set type_v 3

    }     

 

    puts $trace_file_id "$time $length_ $type_v $seq_ $max_fragmented_size"

    incr frame_count

}

 

close $original_file_id

close $trace_file_id

set end_sim_time [expr 1.0 * 40 * ($frame_count)  / 1000]

puts "$end_sim_time"

 

set trace_file [new Tracefile]

$trace_file filename $trace_file_name

set video1 [new Application/Traffic/myEvalvid]

$video1 attach-agent $src_udp1

$video1 attach-tracefile $trace_file

 

$ns at 0.5         "$video1 start"

$ns at $val(stop)  "$video1 stop"

 

#===================================

#        Termination       

#===================================

#Define a 'finish' procedure

proc finish {} {

    global ns tracefile namfile src_udp1 dst_udp1

    $ns flush-trace

    close $tracefile

    $src_udp1 closefile

    $dst_udp1 closefile

    puts "simulation completed"

    exit 0

}

for {set i 0} {$i < $val(nn) } { incr i } {

    $ns at $val(stop) "\$n$i reset"

}

$ns at $val(stop) "finish"

$ns at $val(stop) "puts \"done\" ; $ns halt"

$ns run

 

Set the wireless channel error rate to 0.1.

 

5.     Before evaluating the delivered video quality, we have to convert the trace file format.

 

6.     Evaluate the trace files.

From the evaluation results, we can see the number of lost I frame packets is 9, lost P frame packets is 6, and lost B frame packets is 16.

The number of lost I frames is 4, lost P frames is 5, and lost B frame is 13.

And the fraction of the decodable frame rate is 0.999744

 

7.     Change the error rate to 0.4 (repeat the simulation again).

We can see that when the channel error rate is higher, the packet/frame loss frame rate is also higher.

And the fraction of decodable frame rate is getting lower.

 

p.s. If you refer to this work, please add the following publication in the reference.

 

C. H. Ke, C. K. Shieh, W. S. Hwang, A. Ziviani, “An Evaluation Framework for More Realistic Simulations of MPEG Video Transmission”, Journal of  Information Science and Engineering, vol. 24, no. 2, pp.425-440, March 2008 (SCI)

 

Last updated: 2012/04/19

 

Dr. Chih-Heng Ke

Department of Computer Science and Information Engineering, National Quemoy University, Taiwan

Email: smallko@gmail.com