How to do H.264 SVC transmission simulations? (Part 2: combined scalability: temporal + quality)

 

Introduction

  In this example, I will provide an example of video that is coded with temporal and quality scalability. Then this video is transmitted over IEEE 802.11 and IEEE 802.11e wireless networks for video quality comparison. However, some file processing functions are not well written. So the provided codes are only useful in this example. I will work hard to provide more general tools that can handle other coding parameters cases.

 

Steps

1.      Please read http://csie.nqu.edu.tw/smallko/ns2/svc.htm first.

2.      Download the related files from here.

3.      Use the following files to encode the YUV video.

(main.cfg)

OutputFile         test.264

FrameRate          30.0

FramesToBeEncoded  300

CgsSnrRefinement   1

EncodeKeyPictures  1

MGSControl         1

 

GOPSize       8

BaseLayerMode 0

 

SearchMode    4

SearchRange   32

NumLayers     2

 

LayerCfg      layer0.cfg

LayerCfg      layer1.cfg

 

(layer0.cfg)

InputFile   foreman_cif.yuv

SourceWidth  352

SourceHeight 288

FrameRateIn  30

FrameRateOut 30

 

QP           15

 

(layer1.cfg)

InputFile   foreman_cif.yuv

SourceWidth  352

SourceHeight 288

FrameRateIn  30

FrameRateOut 30

InterLayerPred 1

 

QP           30

 

1

 

4.      Decode the test.264 and record the decoding process into original_decoderoutput.txt. We need some information in it.

2

 

5.      Use the JSVM BitStreamExtractor to generate the original NALU trace file (originaltrace.txt)

3

 

6.       Use f-nstamp to add the frame-number into the originaltrace.txt. (It will generate originaltrace-frameno.txt)

4

 

7.       Prepare the sending trace needed by SVEF with the aid of prepare_sendtrace.awk. (Only useful for this example. For other cases, the codes need to be modified.)

#This file is prepare_sendtrace.awk

BEGIN{

  time=500; # It means that this SVC video starts transmission at 0.5 second.

            # If you want to start transmission at 1.0 second, change the value to 1000.

  i=0;

}

{

    if($6=="SliceData"){

      printf("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%ld\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,time);

      if(i==0) {

         i=1;

      } else if(i==1)

      {

         i=2;

      } else {

         time+=1000.0/30.0;

         i=0;

      }

    } else {

      printf("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n",$1,$2,$3,$4,$5,$6,$7,$8,$9,0);

    }

}

END{

}

 

5

 

8.       Prepare the NS2 sending trace needed by myEvalSVC agent with the aid of prepare_ns2sendtrace.awk. (only useful for this example. For other cases, the codes need to be modified.)  

BEGIN{

  i=0;

  FrameSize=0;

  time=0.0;

}

{

  StartPos=$1;

  Length=$2;

  LId=$3;

  TId=$4;

  QId=$5;

  PacketType=$6;

  Discardable=$7;

  Truncatable=$8;

  FrameNo=$9;

  SentTime=$10;

  if(i==0 && PacketType=="SliceData") {

    FrameSize=(Length+12);

    i=1;

  }else if (i==1 && PacketType=="SliceData") {

    FrameSize+=(Length+12);

    printf("%f\t%s\t%s\t%s\t%s\t%s\n", time, FrameSize, LId, TId, QId, FrameNo);

    i=2;

    FrameSize=0;

  }else if (i==2 && PacketType=="SliceData") {

    FrameSize+=(Length+12);

    printf("%f\t%s\t%s\t%s\t%s\t%s\n", time, FrameSize, LId, TId, QId, FrameNo);

    time+=1.0/30.0;

    i=0;

    FrameSize=0;

  }

}

END{

 

}

   

    6

 

9.      NS2 Simulation (test_svc.tcl 0: for 802.11 simulation ; test_svc 1: for 802.11e simulation)

proc getopt {argc argv} {

        global opt

        lappend optlist nn

        for {set i 0} {$i < $argc} {incr i} {

                set opt($i) [lindex $argv $i]

        }

}

 

getopt $argc $argv

 

#opt(0)-> 0:dcf 1:edcf

 

#===================================

#     Simulation parameters setup

#===================================

set val(chan)   Channel/WirelessChannel    ;# channel type

set val(prop)   Propagation/TwoRayGround   ;# radio-propagation model

set val(netif)  Phy/WirelessPhy            ;# network interface type

if {$opt(0) > 0} {

      set val(mac)            Mac/802_11e                ;# MAC type

        set val(ifq)            Queue/DTail/PriQ       ;# interface queue type

        Mac/802_11e set dataRate_  5.5Mb

        Mac/802_11e set basicRate_ 1Mb

} else {

      set val(mac)            Mac/802_11                ;# MAC type

      set val(ifq)            Queue/DropTail/PriQueue   ;# interface queue type

      Mac/802_11 set dataRate_  5.5Mb

        Mac/802_11 set basicRate_ 1Mb

}

 

set val(ll)     LL                         ;# link layer type

set val(ant)    Antenna/OmniAntenna        ;# antenna model

set val(ifqlen) 50                         ;# max packet in ifq

set val(nn)     3                          ;# number of mobilenodes

set val(rp)     DSDV                       ;# routing protocol

set val(x)      400                        ;# X dimension of topography

set val(y)      500                        ;# Y dimension of topography

set val(stop)   50.0                       ;# time of simulation end

 

#===================================

#        Initialization       

#===================================

#Create a ns simulator

set ns [new Simulator]

 

#Setup topography object

set topo       [new Topography]

$topo load_flatgrid $val(x) $val(y)

create-god $val(nn)

 

#Open the NS trace file

set tracefile [open out.tr w]

$ns trace-all $tracefile

 

set chan [new $val(chan)];#Create wireless channel

 

#===================================

#     Mobile node parameter setup

#===================================

$ns node-config -adhocRouting  $val(rp) \

                -llType        $val(ll) \

                -macType       $val(mac) \

                -ifqType       $val(ifq) \

                -ifqLen        $val(ifqlen) \

                -antType       $val(ant) \

                -propType      $val(prop) \

                -phyType       $val(netif) \

                -channel       $chan \

                -topoInstance  $topo \

                -agentTrace    OFF \

                -routerTrace   OFF \

                -macTrace      OFF \

                -movementTrace OFF

 

#===================================

#        Nodes Definition       

#===================================

#Create 3 nodes

set n0 [$ns node]

$n0 set X_ 200

$n0 set Y_ 400

$n0 set Z_ 0.0

$ns initial_node_pos $n0 20

set n1 [$ns node]

$n1 set X_ 300

$n1 set Y_ 400

$n1 set Z_ 0.0

$ns initial_node_pos $n1 20

set n2 [$ns node]

$n2 set X_ 200

$n2 set Y_ 350

$n2 set Z_ 0.0

$ns initial_node_pos $n2 20

 

#建立影像的傳輸

#===================================================

 

#設定把原本每一個畫面,切割成最大為多少size的封包

set max_fragmented_size 1480

# 20: IP header length

set packetSize [expr $max_fragmented_size+20]

 

set src_udp1 [new Agent/UDP]

$src_udp1 set packetSize_ $packetSize

set dst_udp1 [new Agent/myEvalSVC_Sink]

$ns attach-agent $n0 $src_udp1

$ns attach-agent $n1 $dst_udp1

$ns connect $src_udp1 $dst_udp1

$dst_udp1 set_filename rd

 

set original_file_name ns2send

set trace_file_name video1.dat

set original_file_id [open $original_file_name r]

set trace_file_id [open $trace_file_name w]

 

set pre_time 0

 

while {[eof $original_file_id] == 0} {

    gets $original_file_id current_line

    

    scan $current_line "%f%d%d%d%d%d" t_ size_ lid_ tid_ qid_ fno_

    set time [expr int(($t_ - $pre_time)*1000000.0)]

   

    set prio_p 1

    #only useful in 802.11e. In 802.11, all packets are into the same queue.

    #map all the video packets to video queue.

#However, you can use different values for prio_p according to tid and qid

 

    puts  $trace_file_id "$time $size_ $lid_ $tid_ $qid_ $fno_ $prio_p $max_fragmented_size"

    set pre_time $t_

}

 

close $original_file_id

close $trace_file_id

 

set trace_file [new Tracefile]

$trace_file filename $trace_file_name

set video1 [new Application/Traffic/myEvalSVC]

$video1 attach-agent $src_udp1

$video1 attach-tracefile $trace_file

 

if {$opt(0) > 0} {

 set n0_ifq [$n0 set ifq_(0)]

 

 $ns at 0.0 "record_qlen"

 set fqlen [open queuelength.txt w]

 

 proc record_qlen {} {

        global ns n0_ifq fqlen

        set time 0.01

        set now [$ns now]

        set qlen0 0

        set qlen1 0

        set qlen2 0

        set qlen3 0

        puts $fqlen "$now [$n0_ifq set qlen0]  [$n0_ifq set qlen1]  [$n0_ifq set qlen2]  [$n0_ifq set qlen3]  "

        $ns at [expr $now+$time] "record_qlen"

 }

}

 

#建立best effort的傳輸

#===================================================

set src_udp2 [new Agent/UDP]

$src_udp2 set packetSize_   1500

$src_udp2 set prio_ 2

set dst_udp2 [new Agent/Null]

$ns attach-agent $n0 $src_udp2

$ns attach-agent $n1 $dst_udp2

$ns connect $src_udp2 $dst_udp2

set traffic0 [new Application/Traffic/CBR]

$traffic0 set  packetSize_   1000

$traffic0 set         rate_              0.2Mb

$traffic0 set         random_                true

$traffic0 attach-agent $src_udp2

 

#建立background的傳輸

#===================================================

set src_udp3 [new Agent/UDP]

$src_udp3 set packetSize_  1500

$src_udp3 set prio_ 3

set dst_udp3 [new Agent/Null]

$ns attach-agent $n0 $src_udp3

$ns attach-agent $n2 $dst_udp3

$ns connect $src_udp3 $dst_udp3

set traffic1 [new Application/Traffic/CBR]

$traffic1 set  packetSize_   1000

$traffic1 set         rate_              0.2Mb

$traffic1 set         random_                true

$traffic1 attach-agent $src_udp3

 

$ns at 0.1 "$traffic0 start"

$ns at 0.1 "$traffic1 start"

$ns at 0.5 "$video1 start"

 

$ns at 30.0 "$video1 stop"

$ns at 30.0 "$traffic0 stop"

$ns at 30.0 "$traffic1 stop"

 

#===================================

#        Termination       

#===================================

#Define a 'finish' procedure

proc finish {} {

    global ns tracefile namfile

    $ns flush-trace

    close $tracefile

    exit 0

}

for {set i 0} {$i < $val(nn) } { incr i } {

    $ns at $val(stop) "\$n$i reset"

}

$ns at $val(stop) "finish"

$ns at $val(stop) "puts \"done\" ; $ns halt"

$ns run

 

10.   Do the simulation. (for 802.11 simulation)

7

 

11.  Post Processing. Convert the rd file to the format required for SVEF file. (prepare_receivedtrace1.awk)

BEGIN{

 i=0;

 m=1;

}

{

  time=$1

  frameno=$2

  len=$3

  lid=$4

  tid=$5

  qid=$6

  sendtime=$7

 

  FrameNo[i]=frameno;

  FrameLen[i]=len; 

  FrameLid[i]=lid;

  FrameTid[i]=tid;

  FrameQid[i]=qid;

  FrameRcvTime[i]=time;

  i++;

}

END{

   for(j=0;j<i;j++)

     myFrame[j]=-1;

 

   for(j=0;j<i;j++){

     if(myFrame[j]!=-1)

       continue;

    

     myFrame[j]=m;

     myFrameNo[m]=FrameNo[j];

     myFrameLen[m]+=FrameLen[j];

     myFrameRcvTime[m]=FrameRcvTime[j];

     myFrameLid[m]=FrameLid[j];

     myFrameTid[m]=FrameTid[j];

     myFrameQid[m]=FrameQid[j];

    

     for(k=j+1;k<i;k++){

       if(FrameNo[k]==FrameNo[j] && FrameLid[k]==FrameLid[j] && FrameTid[k]==FrameTid[j] && FrameQid[k]==FrameQid[j]){

         myFrame[k]=m;

         myFrameLen[m]+=FrameLen[k];

         myFrameRcvTime[m]=FrameRcvTime[k];

       } else{

         if(myFrame[k]<=0)

           myFrame[k]=-1;

       }

     }

     m++;

   }

   

   for(j=1;j<m;j++)

     printf("%f\t%d\t%d\t%d\t%d\t%d\n", myFrameRcvTime[j], myFrameLen[j], myFrameLid[j], myFrameTid[j], myFrameQid[j], myFrameNo[j]);

}

 

8

 

(prepare_receivedtrace2.c)

#include <stdio.h>

#include <stdlib.h>

#include <string.h>

#define  MaxNo    5000  // assume that the max number of video is 1000

 

int main(int n, char *cl[])

{

  FILE *f1, *f2, *f3;

  int i,j;

  int IsFrameOK[MaxNo];

  double SndTime[MaxNo], RcvTime[MaxNo];

  double SndTime_, RcvTime_; 

  int SndSize[MaxNo], SndLid[MaxNo], SndTid[MaxNo], SndQid[MaxNo], SndFNo[MaxNo];

  int RcvSize[MaxNo], RcvLid[MaxNo], RcvTid[MaxNo], RcvQid[MaxNo], RcvFNo[MaxNo];

  int SndSize_, SndLid_, SndTid_, SndQid_, SndFrameNo_;

  int RcvSize_, RcvLid_, RcvTid_, RcvQid_, RcvFrameNo_;

  int MaxFrameRecord=0, m=0, p, myA, myB;

  char startpos[20], type[20], yorn1[20], yorn2[20];

  int  len, lid, tid, qid, no, time;

 

 

  for(i=0;i<MaxNo;i++)

    IsFrameOK[i]=-1;

 

  if(n!=4)

  {

   puts("prepare_receivedtrace2 <ns2send> <ns2received> <originaltrace-frameno.txt");

   return EXIT_FAILURE;

  }

 

  if ((f1 = fopen(cl[1],"r")) == 0) goto A;

  if ((f2 = fopen(cl[2],"r")) == 0) goto B;

  if ((f3 = fopen(cl[3],"r")) == 0) goto B;

 

  while(!feof(f1)){

    fscanf(f1, "%lf%d%d%d%d%d", &SndTime_, &SndSize_, &SndLid_, &SndTid_, &SndQid_, &SndFrameNo_);

    //printf("%lf\t%d\t%d\t%d\t%d\t%d\n", SndTime_, SndSize_, SndLid_, SndTid_, SndQid_, SndFrameNo_);

    SndTime[MaxFrameRecord]=SndTime_;

    SndSize[MaxFrameRecord]=SndSize_;

    SndLid[MaxFrameRecord]=SndLid_;

    SndTid[MaxFrameRecord]=SndTid_;

    SndQid[MaxFrameRecord]=SndQid_;

    SndFNo[MaxFrameRecord]=SndFrameNo_;

    MaxFrameRecord++;

  }

 

  while(!feof(f2)){

    fscanf(f2, "%lf%d%d%d%d%d", &RcvTime_, &RcvSize_, &RcvLid_, &RcvTid_, &RcvQid_, &RcvFrameNo_);

    //printf("%lf\t%d\t%d\t%d\t%d\t%d\n", RcvTime_, RcvSize_, RcvLid_, RcvTid_, RcvQid_, RcvFrameNo_);

    RcvTime[m]=RcvTime_;

    RcvSize[m]=RcvSize_;

    RcvLid[m]=RcvLid_;

    RcvTid[m]=RcvTid_;

    RcvQid[m]=RcvQid_; 

    RcvFNo[m]=RcvFrameNo_;

    m++;

  }

 

  i=0;

  while(!feof(f3)){

    fscanf(f3, "%s%d%d%d%d%s%s%s%d%d", startpos, &len, &lid, &tid, &qid, type, yorn1, yorn2, &no, &time);

    i++;

  } 

 

  //printf("num of line: %d\n", i);

 

  rewind(f3);

  for(j=1;j<i;j++){

    fscanf(f3, "%s%d%d%d%d%s%s%s%d%d", startpos, &len, &lid, &tid, &qid, type, yorn1, yorn2, &no, &time);

    //printf("%s %d %d %d %d %s %s %s %d %d\n", startpos, len, lid, tid, qid, type, yorn1, yorn2, no, time);

    if(strcmp(type, "SliceData")==0){

     

      for(p=0;p<MaxFrameRecord;p++) {

        if(SndFNo[p]==no && SndLid[p]==lid && SndTid[p]==tid && SndQid[p]==qid){

          myA=p;

          break;

        }

      }

     

      for(p=0;p<m;p++) {

        if(RcvFNo[p]==no && RcvLid[p]==lid && RcvTid[p]==tid && RcvQid[p]==qid){

          myB=p;

          break;

        }

      }

 

      if(RcvSize[myB]>=SndSize[myA]){

          printf("%s  %6d  %3d  %3d  %3d     %s  %10s  %10s  %8d  %8ld\n", startpos, len, lid, tid, qid, type, yorn1, yorn2, no, (long)(RcvTime[myB]*1000));

      }

    }

  }

 

 

  return 0;     

 

A: fprintf(stderr, " Error opening <ns2send>.\n"); goto X;

B: fprintf(stderr, " Error opening <ns2received>.\n"); goto X;

C: fprintf(stderr, " Error opening <originaltrace-frameno.txt>.\n"); goto X;

 

X: return EXIT_FAILURE;

}

 

9

 

12.  Based on originaltrace-frameno.txt and received file, nalufilter will discard too late frames and frames that cannot be decoded due to frame dependencies.

10

 

13.  The current version of JSVM (9.19.8) cannot decode video streams affected by out of order, corrupted, or missing NALUs. Therefore, SVEF uses filtered packet trace file to extract the corresponding packets in original h.264 video file by means of BitStreamExtractorStatic. 

11

………………………………………………………………………………………………………………………

12

 

14.   Decode the test-filtered.264 file.

13

 

15.  We need the same number of video frames when we want to calculate the PSNR of original YUV and receiving YUV file. So we need to conceal the missing frames by copying the previous frame.

14

 

16.   Now you can compare the PSNR.

15

 

17.  Redo the simulation for 802.11e scenario.

16

 

18.  Repeat all the steps after step 10 for post-processing.

 

Last modified: 2012/10/25

 

Contact Information

Dr. Chih-Heng Ke

Department of Computer Science and Information Engineering, National Quemoy University, Taiwan

Email: smallko@gmail.com