Subject: problems with multicast feed involving MCI tunnels

From: [email-masked] (Andrew Patrick)
Date: Tue, 2 Jul 96 14:23:11 EDT

I am having a great deal of trouble with a tunnel setup through MCI. 
Repeated contact with the MCI people has not solved it, so I thought
I would ask here if anyone had seen something similar.

The main symptom is that MCI seems to drop all packets for my site,
even though the SDR information does come through.  We can see the
sessions and start the tools, but we never get any data.  At times,
this site will appear in the list of participants for a session that
is joined (viewable from another net location), and then soon
disappear.  It is as though all traffic to and from this site is
being pruned repeatedly.  Based on the mtrace output I have been able
to collect (see below), it does not look like any local problems are
causing this.

When this tunnel was first established it did work, but after about 1
week of operation we got these problems.

The target machine is www.radio.cbc.ca (159.33.1.50), a Sun
SPARCstation 10 running Solaris 2.5.  The multicast kernal patches have
been applied, and we are running mrouted 3.8:

127.0.0.1 (localhost) [version 3.8,prune,genid,mtrace]:
  159.33.1.50 -> 0.0.0.0 (local) [1/1/querier/leaf]
  159.33.1.50 -> 205.207.90.6 (TorConc6.ftn.net) [1/32/tunnel]
  
The mrouted process seems to be running normally:

vifs_with_neighbors = 1
[This host is a leaf]
 
Virtual Interface Table
Vif  Name  Local-Address                               M  Thr  Rate   Flags
 0    le0  159.33.1.50     subnet: 159.33.1/24         1   1      0   querier le
af
                           groups: 224.2.230.64   
                                   224.2.127.254  
                                   224.0.0.2      
                                   224.0.0.4      
                         pkts in : 0
                         pkts out: 8357
 
 1    le0  159.33.1.50     tunnel: 205.207.90.6        1  32    500  
                            peers: 205.207.90.6 (11.1) (0x1a)
                         pkts in : 182234
                         pkts out: 3546

FTN.net is our local ISP, and they have a tunnel connection to MCI:

205.207.90.6 (TorConc6.ftn.net) [version 11.1,prune,mtrace]:
  207.0.180.6 -> 0.0.0.0 (local) [1/0/pim/querier/leaf]
  205.207.90.6 -> 204.70.98.45 (dec3800-1-fddi-1.NorthRoyalton.mci.net) [1/31/tunnel/querier]
  205.207.90.6 -> 159.33.1.50 (www.radio.cbc.ca) [1/31/tunnel/querier]

FTN is also not able to see any multicast data for any sessions that
they join.  

An mtrace for the NASA video stream looks like this:

mtrace -t 127 -l 128.102.84.134 224.2.230.64
Mtrace from 128.102.84.134 to 159.33.1.50 via group 224.2.230.64
Querying full reverse path... 
  0  www.radio.cbc.ca (159.33.1.50)
 -1  www.radio.cbc.ca (159.33.1.50)  DVMRP  thresh^ 1  
 -2  TorConc6.ftn.net (205.207.90.6)  DVMRP  thresh^ 31  
 -3  dec3800-1-fddi-1.NorthRoyalton.mci.net (204.70.98.45)  DVMRP  thresh^ 64  
 -4  dec3800-1-fddi-1.WillowSprings.mci.net (204.70.104.45)  DVMRP  thresh^ 1  
 -5  dec3800-2-fddi-1.Denver.mci.net (204.70.152.77)  DVMRP  thresh^ 1  
 -6  dec3800-2-fddi-1.SanFrancisco.mci.net (204.70.158.77)  DVMRP  thresh^ 1  
 -7  mbone.nsi.nasa.gov (192.203.230.241)  DVMRP  thresh^ 64  
 -8  mbone2.nsi.nasa.gov (192.203.230.242)  DVMRP  thresh^ 1  
 -9  arcbone.arc.nasa.gov (128.102.84.150)  DVMRP  thresh^ 32  
-10  shuttle-bone.arc.nasa.gov (128.102.84.134)
Round trip time 140 ms

...

Waiting to accumulate statistics... Results after 60 seconds:
 
  Source        Response Dest    Packet Statistics For     Only For Traffic
128.102.84.134  159.33.1.50      All Multicast Traffic     From 128.102.84.134
     |       __/  rtt  161 ms    Lost/Sent = Pct  Rate       To 224.2.230.64
     v      /     hop  107 s     ---------------------     --------------------
128.102.84.150  arcbone.arc.nasa.gov 
     |     ^      ttl   32        0/377  =  0%  37 pps     0/377  =  0%  37 pps
     v     |      hop -107 s      0/2361 =  0%  39 pps     0/2357 =  0%  39 pps
192.203.230.242 mbone2.nsi.nasa.gov 
     |     ^      ttl   33        0/457  =  0%  45 pps     0/377  =  0%  37 pps
     v     |      hop -785 ms     0/2861 =  0%  47 pps     0/2357 =  0%  39 pps
192.203.230.241 mbone.nsi.nasa.gov 
     |     ^      ttl   64        5/470  =  1%  47 pps     3/377  =  1%  37 pps
     v     |      hop  171 s     20/2938 =  1%  48 pps    10/2357 =  0%  39 pps
204.70.158.61  
204.70.158.77   dec3800-2-fddi-1.SanFrancisco.mci.net 
     |     ^      ttl   65      380/544  = 70%   0 pps   374/374  =100%   0 pps
     v     |      hop  134 s   2377/3402 = 70%   0 pps  2347/2347 =100%   0 pps
204.70.152.61  
204.70.152.77   dec3800-2-fddi-1.Denver.mci.net 
     |     ^      ttl   66       16/179  =  9%   0 pps     0/0    = --%   0 pps
     v     |      hop  208 s     90/1116 =  8%   0 pps     0/0    = --%   0 pps
204.70.104.29  
204.70.104.45   dec3800-1-fddi-1.WillowSprings.mci.net 
     |     ^      ttl   67        1/162  =  1%   0 pps     0/0    = --%   0 pps
     v     |      hop  132 s      1/1005 =  0%   0 pps     0/0    = --%   0 pps
204.70.98.29   
204.70.98.45    dec3800-1-fddi-1.NorthRoyalton.mci.net 
     |     ^      ttl   68        1/196  =  1%   0 pps     0/0    = --%   0 pps
     v     |      hop  -71 s     -2/1098 =  0%   0 pps     0/0    = --%   0 pps
205.207.90.6    TorConc6.ftn.net 
     |     ^      ttl   69        0/4    = --%   0 pps     0/0    = --%   0 pps
     v     |      hop  -49 s      0/40   =  0%   0 pps     0/0    = --%   0 pps
159.33.1.50     www.radio.cbc.ca 
     |      \__   ttl   70          6            0 pps       0            0 pps
     v         \  hop  -15 ms       31           0 pps       0            0 pps
159.33.1.50     159.33.1.50
  Receiver      Query Source

You can see that all the video data is being lost at MCI's SanFrancisco
machine.  Last week we were seeing the same problems, but then the
100% loss was at the NorthRoyalton machine.

Where should we be looking for problems and solutions?

Thanks

-- 
           Andrew Patrick, Ph.D. <[email-masked]>
               Networks Services & Interfaces Laboratory
                     Communications Research Centre
                       http://debra.dgbt.doc.ca/