Subject: Re: bbnplanet and mci, big packet loss rate...

From: John Hawkinson <[email-masked]>
Date: Fri, 14 Jun 1996 16:58:47 -0400 (EDT)

> Message-Id: <[email-masked]>
> To: [email-masked]
> Subject: bbnplanet and mci, big packet loss rate...
> From: Amancio Hasty <[email-masked]>

This sort of thing is really better scoped to [email-masked],
so I've directed replies there. It's not clear to me that it's
even appropriate there, but...

> Date: Fri, 14 Jun 1996 13:38:00 -0500
> From: Jim Lowe <[email-masked]>
> Subject: Re: Where is the Parking Lot?

Where, indeed? It is almost certainly hard to tell anything definitive
based on the statistics for all groups and all sources.  The
statistics for the source and group you specified are all zero,
suggesting that either something is terribly wrong, or that the mtrace
was done from the wrong source/group pair.

It would also be nice to know when this mtrace was taken. Was it
indeed at 13:38 this afternoon (CDT), or was it at some other point?

Your Subject: suggests your concern is over the packet loss. I can
speak to the two hops with what is ostensibly the largest packet loss
(mbone3 to cambridge1-mbone1 and cambridge1-mbone1 to dec2.wor).

cambridge1-mbone1.bbnplanet.net is a cisco and seems to periodically
have problems with mtrace (particularly in statistics for all groups)
statistics. We and cisco are aware of both of these things and they're
slowly getting better (or so we'd like to think :-)).

In your mtrace below, something is quite peculiar wherein the hop
which ostensibly sees the great loss also sees an order of magnitude
more traffic than the other paths (compare pps numbers and sent
packets). I'm not sure what the explanation for this is, but
I don't a loop is very likely, so I'd suspect confused mtrace results.

In particular, if I source traffic across the link that's tagged
at 39% loss in your mtrace:

---cut
[all-purpose-gunk!jhawk] ~> mtrace -g 204.70.64.61 199.94.220.184 204.70.64.61 224.2.18.70
* Mtrace from 199.94.220.184 to 204.70.64.61 via group 224.2.18.70
Querying full reverse path... 
  0  dec3800-2-fddi-0.WestOrange.mci.net (204.70.64.61)
 -1  dec3800-2-fddi-0.WestOrange.mci.net (204.70.64.61)  DVMRP  thresh^ 1
                                                         Not forwarding
 -2  cambridge1-mbone1.bbnplanet.net (199.94.207.2)  DVMRP  thresh^ 32  
 -3  all-purpose-gunk.near.net (199.94.220.184)  DVMRP  thresh^ 16  
 -4  all-purpose-gunk.near.net (199.94.220.184)
Round trip time 14 ms

Waiting to accumulate statistics... Results after 9 seconds:

  Source        Response Dest    Packet Statistics For     Only For Traffic
199.94.220.184  199.94.220.184   All Multicast Traffic     From 199.94.220.184
     v       __/  rtt   14 ms    Lost/Sent = Pct  Rate       To 224.2.18.70
199.94.220.184  all-purpose-gunk.near.net 
     v     ^      ttl   16        0/1152 =  0% 128 pps     0/1138 =  0% 126 pps
199.94.207.2    cambridge1-mbone1.bbnplanet.net 
     v     ^      ttl   32      198/1495 = 13% 166 pps     2/1138 =  0% 126 pps
204.70.64.61    dec3800-2-fddi-0.WestOrange.mci.net Not forwarding
     v      \__   ttl   33          1            0 pps       1136       126 pps
204.70.64.61    199.94.220.184
  Receiver      Query Source
---cut

You'll see that there's actually no loss at a rate of 126pps, which
your "FreeBSD Lounge" video session should certainly not be
hitting. (Note that the pps from cambridge1-mbone-1 is still higher
than you might expect, I'll pursue that at some other juncture; the 0
pps from MCI is due to an mtrace client bug that will be fixed in the
next release).

On the BBN/UUnet hop, we see a similar effect:

---cut
[all-purpose-gunk!jhawk] ~> mtrace -g 137.39.229.98 199.94.220.184 137.39.229.98 224.2.18.70
Mtrace from 199.94.220.184 to 137.39.229.98 via group 224.2.18.70
Querying full reverse path... 
  0  MBONE3.UU.NET (137.39.229.98)
 -1  MBONE3.UU.NET (137.39.229.98)  DVMRP  thresh^ 1  Not forwarding
 -2  cambridge1-mbone1.bbnplanet.net (199.94.207.2)  DVMRP  thresh^ 32  
 -3  all-purpose-gunk.near.net (199.94.220.184)  DVMRP  thresh^ 16  
 -4  all-purpose-gunk.near.net (199.94.220.184)
Round trip time 31 ms

Waiting to accumulate statistics... Results after 10 seconds:

  Source        Response Dest    Packet Statistics For     Only For Traffic
199.94.220.184  199.94.220.184   All Multicast Traffic     From 199.94.220.184
     v       __/  rtt   26 ms    Lost/Sent = Pct  Rate       To 224.2.18.70
199.94.220.184  all-purpose-gunk.near.net 
     v     ^      ttl   16        2/1281 =  0% 128 pps     2/1264 =  0% 126 pps
199.94.207.2    cambridge1-mbone1.bbnplanet.net 
     v     ^      ttl   32      377/1740 = 22% 174 pps    -4/1262 =  0% 126 pps
137.39.229.98   MBONE3.UU.NET  Not forwarding
     v      \__   ttl   33          0            0 pps       1266       126 pps
137.39.229.98   199.94.220.184
  Receiver      Query Source
---cut

So, to reiterate, while the numbers may be misleading and ought to be
fixed, there is no real operational problem here, or at least, not one
that *I* can see.

A more disturbing issue is the path between UWM and star-gate, which
is pretty odd. In this particular case, the route you are seeing
is actually following an multicast default route:

---cut
[all-purpose-gunk!jhawk] ~> mtrace -s rah.star-gate.com
Mtrace from 204.188.121.18 to 199.94.220.184 via group 224.2.0.1
Querying full reverse path... 
  0  all-purpose-gunk.near.net (199.94.220.184)
 -1  all-purpose-gunk.near.net (199.94.220.184)  DVMRP  thresh^ 1  
 -2  cambridge1-mbone1.bbnplanet.net (199.94.207.2)  DVMRP  thresh^ 16  
 -3  MBONE3.UU.NET (137.39.229.98)  DVMRP  thresh^ 64  
 -4  pulp.otec.com (206.64.4.69)  DVMRP  thresh^ 64  Next router no mtrace
 -5  gw1.otec.com (206.64.4.65) [cisco 10.3] doesn't support mtrace
 -6  rah.star-gate.com (204.188.121.18)
Round trip time 348 ms

[all-purpose-gunk!jhawk] ~> mtrace -s 0.0.0.1
Mtrace from 0.0.0.1 to 199.94.220.184 via group 224.2.0.1
Querying full reverse path... 
  0  all-purpose-gunk.near.net (199.94.220.184)
 -1  all-purpose-gunk.near.net (199.94.220.184)  DVMRP  thresh^ 1  
 -2  cambridge1-mbone1.bbnplanet.net (199.94.207.2)  DVMRP  thresh^ 16  
 -3  MBONE3.UU.NET (137.39.229.98)  DVMRP  thresh^ 64  
 -4  pulp.otec.com (206.64.4.69)  DVMRP  thresh^ 64  Next router no mtrace
 -5  gw1.otec.com (206.64.4.65) [cisco 10.3] doesn't support mtrace
Round trip time 36 ms
---cut

And this is because there is no DVMRP route for rah.star-gate.com.  As
always, I would suggest that you contact your network provider for
resolution of these issues -- you're paying them to help you.

I will admit, however, that there are an abundance of bizarre paths
which mbone traffic does take, and a lot of it ought to be fixed,
and to do that will take some bit of coordination between providers.
Steve D., do you want to pipe up and say something now?

--jhawk					BBN Planet
  John Hawkinson			Network Engineering

Here's the original mtrace you mailed to [email-masked].
---cut
(miller-genuine-draft) 56# mtrace rah.star-gate.com 224.2.100.102
Mtrace from 204.188.121.18 to 129.89.9.110 via group 224.2.100.102
Querying full reverse path... 
  0  miller-genuine-draft.cs.uwm.edu (129.89.9.110)
 -1  alumni.cs.uwm.edu (129.89.9.30)  DVMRP  thresh^ 1  
 -2  dgx-fddi6-0-0.chicago.cic.net (131.103.1.21)  DVMRP  thresh^ 32  
 -3  dec3800-1-fddi-1.WillowSprings.mci.net (204.70.104.45)  DVMRP  thresh^ 32  
 -4  dec3800-2-fddi-1.Denver.mci.net (204.70.152.77)  DVMRP  thresh^ 1  
 -5  dec3800-2-fddi-0.WestOrange.mci.net (204.70.64.61)  DVMRP  thresh^ 1  
 -6  cambridge1-mbone1.bbnplanet.net (199.94.207.2)  DVMRP  thresh^ 32  
 -7  MBONE3.UU.NET (137.39.229.98)  DVMRP  thresh^ 64  
 -8  pulp.otec.com (206.64.4.69)  DVMRP  thresh^ 64  Next router no mtrace
 -9  gw1.otec.com (206.64.4.65) [cisco 10.3] doesn't support mtrace
-10  rah.star-gate.com (204.188.121.18)
Round trip time 152 ms
	  
Waiting to accumulate statistics... Results after 10 seconds:
	  
  Source        Response Dest    Packet Statistics For     Only For Traffic
204.188.121.18  129.89.9.110     All Multicast Traffic     From 204.188.121.18
     v       __/  rtt  128 ms    Lost/Sent = Pct  Rate       To 224.2.100.102
206.64.4.69     pulp.otec.com  
     v     ^      ttl   64        0/0    = --%   0 pps     0/0    = --%   0 pps
137.39.229.98   MBONE3.UU.NET  
     v     ^      ttl   65        3/20   = 15%   2 pps     0/0    = --%   0 pps
199.94.207.2    cambridge1-mbone1.bbnplanet.net 
     v     ^      ttl   66      536/1392 = 39% 139 pps     0/0    = --%   0 pps
204.70.64.61    dec3800-2-fddi-0.WestOrange.mci.net 
     v     ^      ttl   67       -1/850  =  0%   0 pps     0/0    = --%   0 pps
204.70.152.77   dec3800-2-fddi-1.Denver.mci.net 
     v     ^      ttl   68        8/351  =  2%   0 pps     0/0    = --%   0 pps
204.70.104.29  
204.70.104.45   dec3800-1-fddi-1.WillowSprings.mci.net 
     v     ^      ttl   69        0/1111 =  0%   0 pps     0/0    = --%   0 pps
131.103.1.21    dgx-fddi6-0-0.chicago.cic.net 
     v     ^      ttl   70        0/82   =  0%   8 pps     0/0    = --%   0 pps
129.89.9.30     alumni.cs.uwm.edu 
     v      \__   ttl   71          12           0 pps       0            0 pps
129.89.9.110    129.89.9.110
  Receiver      Query Source
	  
---cut

---

From: Amancio Hasty <[email-masked]>
Date: Mon, 17 Jun 1996 13:27:09 -0700

The video session is a 10kb video session that we started on the
FreeBSD Lounge mostly to help out debugging the mbone tools on FreeBSD.

Just run sdr, select FreeBSD Lounge and then select the video channel.
You should see a video of a parking lot at UWM.

Quite often, I get a 20 percent drop on my mbone sessions due to 
the mbone connection from TLG to MCI, since the mbone path from TLG
to mci is not the best in the world;however, it has been reported to TLG
and hopefully the tunnel situation from TLG and MCI will improve in the
near future.  Now my current problem of not getting any video I can't
easily explain from the output of mtrace.

{hasty} date
Mon Jun 17 13:25:41 PDT 1996
{hasty} mtrace 129.89.9.110 224.2.100.102
Mtrace from 129.89.9.110 to 204.188.121.18 via group 224.2.100.102
Querying full reverse path... 
  0  rah.star-gate.com (204.188.121.18)
 -1  rah.star-gate.com (204.188.121.18)  DVMRP  thresh^ 1  
 -2  toad.com (140.174.2.1)  DVMRP  thresh^ 1  
 -3  dec3800-2-fddi-1.SanFrancisco.mci.net (204.70.158.77)  DVMRP  thresh^ 64  
 -4  dec3800-2-fddi-0.Denver.mci.net (204.70.152.61)  DVMRP  thresh^ 1  
 -5  dec3800-1-fddi-0.WillowSprings.mci.net (204.70.104.29)  DVMRP  thresh^ 1  
 -6  dgx-fddi6-0-0.chicago.cic.net (131.103.1.21)  DVMRP  thresh^ 32  
 -7  alumni.cs.uwm.edu (129.89.9.30)  DVMRP  thresh^ 32  
 -8  miller-genuine-draft.cs.uwm.edu (129.89.9.110)
Round trip time 339 ms

  Source        Response Dest    Packet Statistics For     Only For Traffic
129.89.9.110    204.188.121.18   All Multicast Traffic     From 129.89.9.110
     v       __/  rtt  292 ms    Lost/Sent = Pct  Rate       To 224.2.100.102
129.89.9.30     alumni.cs.uwm.edu 
     v     ^      ttl   32       -2/29   = -6%   0 pps     0/0    = --%   0 pps
131.103.1.21    dgx-fddi6-0-0.chicago.cic.net 
     v     ^      ttl   33       -1/31   = -2%   3 pps     0/0    = --%   0 pps
204.70.104.45  
204.70.104.29   dec3800-1-fddi-0.WillowSprings.mci.net 
     v     ^      ttl   34       33/146  = 23%   0 pps     0/0    = --%   0 pps
204.70.152.77  
204.70.152.61   dec3800-2-fddi-0.Denver.mci.net 
     v     ^      ttl   35        1/195  =  1%   0 pps     0/0    = --%   0 pps
204.70.158.77   dec3800-2-fddi-1.SanFrancisco.mci.net 
     v     ^      ttl   64       12/50   = 24%   0 pps     0/0    = --%   0 pps
140.174.2.1     toad.com       
     v     ^      ttl   65        0/14   =  0%   1 pps     0/0    = --%   0 pps
204.188.121.18  rah.star-gate.com 
     v      \__   ttl   66          6            0 pps       0            0 pps
204.188.121.18  204.188.121.18
  Receiver      Query Source

	Tnks,
	Amancio

---

From: Amancio Hasty <[email-masked]>
Date: Mon, 17 Jun 1996 13:39:37 -0700

Ooops, the correct mtrace for the FreeBSD Video channel to my system is:

{hasty} date
Mon Jun 17 13:38:48 PDT 1996
{hasty} mtrace 129.89.9.63 224.2.100.102
Mtrace from 129.89.9.63 to 204.188.121.18 via group 224.2.100.102
Querying full reverse path... 
  0  rah.star-gate.com (204.188.121.18)
 -1  rah.star-gate.com (204.188.121.18)  DVMRP  thresh^ 1  
 -2  toad.com (140.174.2.1)  DVMRP  thresh^ 1  
 -3  dec3800-2-fddi-1.SanFrancisco.mci.net (204.70.158.77)  DVMRP  thresh^ 64  
 -4  dec3800-2-fddi-0.Denver.mci.net (204.70.152.61)  DVMRP  thresh^ 1  
 -5  dec3800-1-fddi-0.WillowSprings.mci.net (204.70.104.29)  DVMRP  thresh^ 1  
 -6  dgx-fddi6-0-0.chicago.cic.net (131.103.1.21)  DVMRP  thresh^ 32  
 -7  alumni.cs.uwm.edu (129.89.9.30)  DVMRP  thresh^ 32  
 -8  homestead.cs.uwm.edu (129.89.9.63)
Round trip time 308 ms

  Source        Response Dest    Packet Statistics For     Only For Traffic
129.89.9.63     204.188.121.18   All Multicast Traffic     From 129.89.9.63
     v       __/  rtt  301 ms    Lost/Sent = Pct  Rate       To 224.2.100.102
129.89.9.30     alumni.cs.uwm.edu 
     v     ^      ttl   32        1/43   =  2%   0 pps     1/43   =  2%   0 pps
131.103.1.21    dgx-fddi6-0-0.chicago.cic.net 
     v     ^      ttl   33        0/42   =  0%   4 pps     0/42   =  0%   4 pps
204.70.104.45  
204.70.104.29   dec3800-1-fddi-0.WillowSprings.mci.net 
     v     ^      ttl   34       43/159  = 27%   0 pps    42/42   =100%   0 pps
204.70.152.77  
204.70.152.61   dec3800-2-fddi-0.Denver.mci.net 
     v     ^      ttl   35        0/215  =  0%   0 pps     0/0    = --%   0 pps
204.70.158.77   dec3800-2-fddi-1.SanFrancisco.mci.net 
     v     ^      ttl   64        7/32   = 22%   0 pps     0/0    = --%   0 pps
140.174.2.1     toad.com       
     v     ^      ttl   65        1/10   = --%   1 pps     0/0    = --%   0 pps
204.188.121.18  rah.star-gate.com 
     v      \__   ttl   66          4            0 pps       0            0 pps
204.188.121.18  204.188.121.18
  Receiver      Query Source