Subject: How to get multi-homed ATM/Ethernet sparcs talking?

From: "Paul Harrington" <[email-masked]>
Date: Wed, 05 Feb 1997 21:33:33 +0000

Existing setup:
24 bits subnet masks, sparc 5 and Ultra-1 machines, all running SunOS
5.5 with 

a number of sparcs on subnet 138.251.199 with ethernet
a number of sparcs on subnet 138.251.192 with ethernet

router (138.251.199.76/138.251.192.70) is a two-interface sparc
running mrouted 3.8:
	138.251.192.70 -> 138.251.63.1 (tweed.st-and.ac.uk) [1/16/tunnel]

a single sparc (134.36.156.19) running mrouted 3.8:

This works.

New setup: as before except in addition there is a Fore SBA200
card in most machines (except our tunnel machine!) and the card is
configured for both Fore-IP and classical IP operation. All interfaces
running Fore-IP are on subnet 138.251.221, all interfaces running
Classical IP are on subnet 138.251.220 (including the machine on ether
134.36.156.19)

Goals:
	'local' MBONE traffic to go onto an ATM subnet.

	138.251.199.76/138.251.192.70 to run a conf-less mrouted and
	_work_ with a cisco living on subnet 138.251.192 which speaks
	PIM to a boarder router which then speaks DVMRP to the rest of
	the MBONE (This is not my juristiction so I hope I have stated
	the case correctly)

With no mrouted running, I only get connectivity between machines on
the same ethernet. Does Solaris send mcast packets out of all
interfaces marked multicast or does it have to have received an IGMP
'join' on that interface first?

I then tried running conf-less mrouted on _one_ of the ATM machines of
the group that shares an ether and another mrouted on a machines that
doesn't share an ether. This works OK-ish. Connectivity between ATM
machines on different ethernets but machines on the same ether used
ether rather than ATM. I guess that in the case of a tie it just picks
the first interface?

Then mrouted.conf with
	phyint fa0 metric 1 threshold 1
	phyint qaa0 metric 1 threshold 1
	phyint le0 metric 2 threshold 2

... actually, I have tried with ethernets disabled, one mrouted,
different metrics and each configuration 'tested' by whistling a few
notes of 'Sweet Georgia Brown' into a vat window and trying to see it
appear in a tcpdump xterm. Five hours later, I am inclined to admit
that I don't know the answer.

pjjH

---

From: Toerless Eckert <[email-masked]>
Date: Wed, 5 Feb 1997 23:13:02 +0100 (MET)

> a single sparc (134.36.156.19) running mrouted 3.8:

How does this sparc fit into your scenario ?

Without going into much detail of your configuration, some basics:

1. Classical IP:
   Forget about classical-IP subnets. They can't do neither broadcast
   nor multicast. When configuring a mrouter it's best to disable them
   via the "phyint" command.

2. Fore-IP:
   Fore-IP subnets do make nearly perfect IP-multicast subnets.
   Actually their behaviour is better then that of a common IGMP driven
   shared medium:

   - A single PTMP SVC is built on demand for every transmitter with
     connections to joining and leaving receiving members built and
     closed down in a matter of milliseconds.
   
   - To run multicast on Fore-IP think of it as just another ethernet
     and configure appropriately.

   - Be warned, as of now there's at least one open bug in Fore-IP and
     multicast. Namely, it only works for the lowest 23 bit multicast address
     space (i.e.: 224.0.0.0 .. 224.127.255.255). Yes ;-)), guess what's
     the problem.

3. BASIC (Beginners All-purpose Simple Installation Checks):
   for your network (and these should be available
   from your next step www.mbone.* server:)

   - Plug  your subnets together as you like, but try to avoid
     redundancy in both unicast and multicast router setup.

   - Run mrouted on all the actual unicast routers and set the dvmrp
     metrics like you set the unicast metrics. Ignore classical-ip
     subnets and try to avoid LANE-Networks (they will work, but multicast
     is broadcast on them).
   
   - Don't use no tunnels except for connecting to the outside world.

   - If a unicast router cannot handle multicast put a workstation in
     parallel that can. Parallel means thaat it should connect exactly to
     the same interfaces as the unicast router.

Hope that helps for the beginning. As for actual debugging of your setup,
well, that's a different problem.

Toerless

---

From: [email-masked] (Erik Van Riper)
Date: Wed, 5 Feb 1997 14:40:55 -0800 (PST)

Any ideas how to get off this list?  (mbone-na).  Email to
owner-mbone-na seems to go unanswered.  Perhaps there is a trick?

Thanks!
-- 
Erik Van Riper (EV34)                    Systems / Network Administrator
Midway Home Entertainment Inc.                     San Diego, California
(619) 658 9500 (x110)                    
Go player.

---

From: Bill Fenner <[email-masked]>
Date: Wed, 5 Feb 1997 15:13:40 PST

In message <9702052133.AA22668@tamdhu> you write:
>Connectivity between ATM

Your question is really about configuring multi-homed hosts, and not
about configuring mrouted.

As it says in the multicast README:

| ESTABLISHING A DEFAULT MULTICAST INTERFACE
| 
| Selection of the default multicast interface is controlled via the kernel
| (unicast) routing table.  If there is no multicast route in the table, all
| multicasts will, by default, be sent on the interface associated with the
| default gateway.  If that interface does not support multicast, attempts to
| send will receive an ENETUNREACH error.
| 
| A route may be added for a particular multicast address or for all multicast
| addresses, to direct them to a different default interface.  For example,
| to specify that multicast datagrams addressed to 224.0.1.3 should, by default,
| be sent on the interface with local address 36.2.0.8, use the following:
| 
| 	/etc/route add 224.0.1.3 36.2.0.8 0
| 
| To set the default for all multicast addresses, other than those with
| individual routes, to be the interface with local address 36.11.0.1, use:
| 
| 	/etc/route add 224.0.0.0 36.11.0.1 0
| 
| If you point a multicast route at an interface that does not support
| multicasting, an attempt to multicast via that route will receive an
| ENETUNREACH error.
| 
| If needed, these commands normally would be added to the /etc/rc.ip or
| /etc/rc.local file, to take effect every time the system is booted.

Thus, you probably want to say "route add 224.0.0.0 <atm interface> 0"
on all of the multi-homed machines to make them use their atm interface
for multicast by default, and then make sure you run at least one mrouter
so that multicast from the atm network is routed.

  Bill

---

From: [email-masked]
Date: Wed, 5 Feb 1997 15:38:40 -0800 (PST)

Sicne I have been running IP multicast in an ATM/ethernet environment for a
while with a Sparc Ultra/Solaris 2.5, here is what I can tell you:

1) When mrouted is not running, the Sparcs will send multicast traffic out
whatever interface is defined as the "default" multicast interface. Do a
"netstat -nr" and look for the interface that is using the 224.0.0.0 route.
This will probably be the ethernet interface since that is usually the first
interface you will have configured. 

2) Classical IP over ATM (aka CLIP) is a non-broadcast/non-multicast
environment, so you can't run IP multicast over it (yet - until we get MARS).
The IP-multicast-3.5 extensions for Solaris 2.5 tag the CLIP interfaces
(qaa0-3) as multicast, which is NOT correct. You will have to use the ForeIP
interface (fa0) to run native IP multicast. On the non-mrouted machines, set
the default multicast route to the IP address used by the fa0 interface. 

3) Normally, you I can set up unicast tunnels over CLIP connections, however I
have never been able to get that to work on Solaris 2.5 - only SunOS 4.1.4 -
with the IP mc-3.5 extensions and Fore ATM cards.

4) Be aware that ForeIP looks at the entire ForeIP cloud as one giant LAN. The
implication of this is that your multicast traffic will be visible ANYWHERE in
the ForeIP cloud, even when you set the TTL to 1. This is not a problem if you
are using ForeIP only locally, however if you have any virtual private network
(VPN) WAN connections that are using ForeThought PNNI, your multicast traffic
will also go out these links, and probably break them, because ForeIP multicast
doesn't scale well into a WAN environment. 
     For that reason, I'm using LANE instead for my multicast traffic over ATM,
even through it hammers the Broadcast/Unknown server (BUS). Hint: don't even
TRY using a Fore switch for LANE services if you are going to pump multicast
traffic over it, it won't hack it. I'm using a Cisco 7505 for my LANE server
instead.

Mark Ganzer          Naval Command, Control & Ocean Surveillance Center,
[email-masked]      RDT&E Div (NRaD), Code 4123,  San Diego, CA
Ph: (619) 553-1186   FAX: (619) 553-4808

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 6 Feb 1997 01:14:34 +0100 (MET)

> the ForeIP cloud, even when you set the TTL to 1. This is not a problem if you

It is true that ForeIP is one giant LAN. It is not true that
Multicast with Fore-IP will cause unnecessary traffic on those machines
that do not want to receive a specific group. Of course, trying
to limit traffic by TTL or different IP-subnetworks
within one SPANS cloud will fail.

> doesn't scale well into a WAN environment. 

I don't know what's "not scaling in a WAN environment" for ForeIP.
It simply behaves as described in the paragraphs above. I cannot imagine
a better way of getting IP multicasting into an ATM environment. I do
not need TTL-limitations for ip-multicast if i do have working
pruning with timeouts in msec ranges for every receiver.

> traffic over it, it won't hack it. I'm using a Cisco 7505 for my LANE server

Or to be more exact: Don't run a BUS on a Fore-ASX series CPU modul
if you expect heavy multicast (aka broadcast) traffic. The BUS in
the current 4.0.2 Fore software is worth about 1000 pps or 1 MByte/sec
throughput at best. See datacom report from a few month ago for this.
They promised to improve on that throughput but i havn't followed this
very much. On the other hand, there's a lot of equipment that shares this
unfortunate behaviour of bad throughput for a bad protocol. I.e.: never try a
Cisco-4500 for BUS.

---

From: [email-masked]
Date: Wed, 5 Feb 1997 16:52:04 -0800 (PST)

> > 4) Be aware that ForeIP looks at the entire ForeIP cloud as one giant
> > LAN. The implication of this is that your multicast traffic will be
> > visible ANYWHERE in the ForeIP cloud, even when you set the TTL to 1. This
> > is not a problem if you 
> 

My main issue there was that there are times you want to keep local multicast
traffic LOCAL. If you live in a large interconnected Fore ATM internetwork (our
site connects into AAI, which connects into ATDnet, which connects into NREN -
all running Fore PNNI and forming one large ForeIP cloud), you CAN'T limit the
scoping if the traffic.

I don't want to turn this list into a forum about problems with ForeIP
multicast. Also, my knowledge of the WAN problems is second-hand so I can't
speak authoritatively (I HAVE tried IP multicast over ForeIP locally, however).
I just wanted to urge caution if your ForeIP cloud extends "off-campus". It
will probably not be any problem in an isolated campus environment.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 6 Feb 1997 02:07:57 +0100 (MET)

> will probably not be any problem in an isolated campus environment.

Well, we only had it for a short time between two universities, so my
experience in practical terms are a bit limited to, but the only thing that
you are missing is "policy access control" with fore-IP, i.e.: that
endless mess of LAN-E configuration stuff which spells "security broke
connectivity". That's if you ask me. I do think that one Fore-IP network
in the backbone between sites would make up one great backbone VLAN
for multicast. Only problem is that it's proprietary and not adopted by
anyone beside fore. Well, only LAN-E and Classical-IP frustratrion speaking
here..

By the way:
   Is there a useful mailing list for fore related topics anyhow ?

---

From: Petri Helenius <[email-masked]>
Date: Thu, 6 Feb 1997 09:02:54 +0200 (EET)

> unfortunate behaviour of bad throughput for a bad protocol. I.e.: never try a
I usually do the performance shooting towards Cisco, not the other way
around, but I do think that C4500 is adequate for average networks BUS
and even a busy multicast network BUS. That's what we run for a long
time and now it's a 4700-M (because of memory limitations in the 4500)
and performs as a BUS/LES for 10 ELANs and runs happily:
CPU utilization for five seconds: 11%/9%; one minute: 11%; five minutes: 10%

As for the ASX, I've seen the symptoms. That's what you get on not
having a SAR chip on board.

Pete

---

From: Dino Farinacci <[email-masked]>
Date: Thu, 6 Feb 1997 09:22:39 -0800

>> unfortunate behaviour of bad throughput for a bad protocol. I.e.: never try a

    Don't use a box that hasn't been design specifically to be a LANE BUS.
    I don't know if such a beast exists.

Dino