
         PVM version 3.4:  Parallel Virtual Machine System
               University of Tennessee, Knoxville TN.
           Oak Ridge National Laboratory, Oak Ridge TN.
                   Emory University, Atlanta GA.
      Authors:  J. J. Dongarra, G. E. Fagg, G. A. Geist,
                 J. A. Kohl, R. J. Manchek, P. Mucci,
         P. M. Papadopoulos, S. L. Scott, and V. S. Sunderam
                   (C) 1997 All Rights Reserved

                              NOTICE

 Permission to use, copy, modify, and distribute this software and
 its documentation for any purpose and without fee is hereby granted
 provided that the above copyright notice appear in all copies and
 that both the copyright notice and this permission notice appear in
 supporting documentation.

 Neither the Institutions (Emory University, Oak Ridge National
 Laboratory, and University of Tennessee) nor the Authors make any
 representations about the suitability of this software for any
 purpose.  This software is provided ``as is'' without express or
 implied warranty.

 PVM version 3 was funded in part by the U.S. Department of Energy,
 the National Science Foundation and the State of Tennessee.

________________________________________________________________________

WHAT IS PVM?

PVM is a software system that enables a collection of heterogeneous
computers to be used as a coherent and flexible concurrent computational
resource, or a "parallel virtual machine".

The individual computers may be shared- or local-memory multiprocessors,
vector supercomputers, specialized graphics engines, or scalar
workstations and PCs, that may be interconnected by a variety of
networks, such as Ethernet or FDDI.

PVM consists of a run-time environment and library for message-passing,
task and resource management, and fault notification.  While PVM will
not automatically make a commerical software package run faster, it
*does* provide a powerful set of functions for manually parallelizing
an existing source program, or for writing new parallel / distributed
programs.

The PVM software must be specifically installed on every machine that
is to be used in your "virtual machine".  There is no "automatic"
installation of executables onto remote machines in PVM, although
simply copying the pvm3/lib and pvm3/bin directories to another
*similar* machine (and setting $PVM_ROOT and $PVM_ARCH - see below)
is sufficient for running PVM programs.  Compiling or building PVM
programs requires the full PVM installation.

User programs written in C, C++ or Fortran can access PVM through
provided library routines.

________________________________________________________________________

UNPACKING

This distribution contains source code, some simple examples, and
run-time support for PVM version 3.  The documentation for PVM can
be obtained separately from Netlib.  To get a list of available
subsets, send e-mail to "netlib@ORNL.GOV" with the subject:
"send index from pvm3".

The files in the source distribution unpack into a "pvm3" directory.
The pvm3 directory can reside in either a private or shared disk area.
Installations for multiple machine architectures can coexist in the
same shared filesystem because compiled files are placed in different
subdirectories named for each architecture ($PVM_ARCH).

Some of the more important directories are:

 Directory   Contains
 ---------------------------------------------------------------
 bin/$PVM_ARCH/  PVM user program executables (examples & your programs)
 conf/           Make configuration files for all PVM architectures
 console/        Source for the pvm console
 doc/            Miscellaneous documentation
 examples/       Example PVM programs source files
 gexamples/      More example PVM programs - for group library
 hoster/         An example "hoster" program
 include/        Header files for PVM programs
 lib/            Generic system executables (scripts, rc file stubs)
 lib/$PVM_ARCH/  System executables (pvmd, console, etc.)
 libfpvm/        Source for the libfpvm Fortran library
 man/man[13]/    Online manual pages (nroff format)
 misc/           Some miscellaneous PVM examples and utilities
 patches/        Patch files and instructions, as they are released
 pvmgs/          Source for the libgpvm library and group nameserver
 rm/             An example resource manager for PVM
 shmd/           A special daemon for shared memory systems (*MP)
 src/            Source for the libpvm library and pvmd daemon
 src/$PVM_ARCH/  Additional source code for specific machines
 tasker/         An example "tasker" program for PVM
 tracer/         An example "tracer" program for PVM
 xep/            An example interactive X-Window program


BUILDING AND INSTALLING

To build the full PVM software, you will need to have a "make" utility,
a C or C++ compiler, and a Fortran compiler installed on your system.

Before building or running PVM, you must set the environment variable
"PVM_ROOT" to the path where PVM resides, i.e. the path of the
directory with this "Readme" file.  This can be in a private area,
for example $HOME/pvm3, or a public one, such as /usr/local/pvm3.

If your shell is csh, add a line such as:

    setenv PVM_ROOT $HOME/pvm3

to your .cshrc file.  If you use a shell that reads .profile, such as sh
or ksh, or .bashrc for bash, add the following lines to that file:

    PVM_ROOT=$HOME/pvm3
    export PVM_ROOT

The use of this variable and others is explained more fully in the
pvm_intro man page.

You can also include an appropriate shell startup file stub to set
other PVM environment variables and to add PVM directories to your
execution path.  Inert the matching stub file, pvm3/lib/cshrc.stub,
pvm3/lib/kshrc.stub or pvm3/lib/bashrc.stub, after your declaration
of PVM_ROOT in your shell startup file.

To build PVM for your system, type "make" in this directory.  Make
will use the "aimk" in the pvm3/lib directory to build the daemon
executable (pvmd3), the C library (libpvm3.a), the Fortran library
(libfpvm3.a) and the console client program (pvm).

The libraries and executables are installed in $PVM_ROOT/lib/$PVM_ARCH,
where $PVM_ARCH is the host architecture name, e.g. "CRAY" or "LINUX".

If you have problems compiling PVM on your system, check the
architecture-specific configuration file:

	$PVM_ROOT/conf/$PVM_ARCH.def

for comments regarding alternative configurations for your system.

The provided scripts $PVM_ROOT/lib/pvm and $PVM_ROOT/lib/pvmd are
used to start the PVM console and the PVM daemon, respectively.
They determine the machine architecture and run the actual programs
in the $PVM_ROOT/lib/$PVM_ARCH directory.  You can either copy these
scripts to your bin directory or add $PVM_ROOT/lib to your shell
search path (using the above *.stub files).

You may wish to add $PVM_ROOT/man to your MANPATH environment variable,
if it's supported on your system.  This will allow you to easily read
the online manual pages.


ALTERNATIVES TO RSH

To use "ssh" instead of "rsh" on your system, you can either:

1. Modify the $PVM_ROOT/conf/$PVM_ARCH.def file to change the absolute
path specified for RSHCOMMAND in the ARCHCFLAGS define.  Replace the
path to rsh with the absolute path to "ssh" on your system and then
recompile PVM and your applications.  Or,

2. Set the "PVM_RSH" environment variable to point to the absolute
path to "ssh" on your system.  This does not require re-compilation
but must be done in every shell from which PVM is executed.  The
"PVM_RSH" environment variable can be set in your $HOME/.cshrc or
equivalent shell startup file to take affect in all new shells.

Once either of these approaches has been applied, the $HOME/.rhosts
file is then no longer necessary for PVM, thereby eliminating a
fundamental security concern.  Now, to add PVM hosts (using ssh),
you can either manually enter your password each time you add a
host, or else you can set up an ssh-agent session.  (You can always
just use a $HOME/.shosts file, but then you're not much more secure
than with standard rsh...)

When manually entering your password, you will not receive a prompt
from PVM or ssh - the text prompts normally returned by ssh are
automatically captured and written to the local /tmp/pvml.<uid> log
file.  However, simply typing your password from inside the "pvm"
console or when running the "pvmd" script will work.

You can bypass this manual password entry for a given session by
executing the following commands (example is for bash, could use
for any shell):

	$ ssh-agent bash
	$ ssh-add
	<now enter your passphrase ONCE>
	$ pvm
	$ pvm> add host2 . . .

For this to work, you must have run "ssh-keygen" on each host to
create a private/public key pair for your login id.  These keys are
typically stored in the $HOME/.ssh directory on each system.  The
$HOME/.ssh/identity.pub public key must be copied to each remote
host and added as a line in the $HOME/.ssh/authorized_keys file
on the remote system.  When this has been set up, any PVM hosts
can be added without password or passphrase entry using the above
ssh-agent session.


STARTING AND STOPPING PVM

To start PVM, run $PVM_ROOT/lib/pvm.  This starts the console task,
which in turn starts a pvmd if one is not already running.  More hosts
can be started and added to your "virtual machine" by using the console
"add" command.

To start the pvmd without starting the console, you can also run the
$PVM_ROOT/lib/pvmd directly.  A number of hosts can all be started at
once by supplying the pvmd with a host file, as in:

	$PVM_ROOT/lib/pvmd my_hosts

where "my_hosts" contains the names of the hosts you wish to add,
one host per line.  See the pvmd man page for other host file
options.

To stop PVM, use the PVM console command "halt".  From within your
user programs, you can use the pvm_halt() function.  You can also
kill the pvmd3 process (always use a catchable signal such as
SIGTERM).  But, if you do kill the pvmd, or if it fails for some
other reason, always be sure to remove any leftover /tmp/pvmd.<uid>
pvmd socket files, where <uid> is your numerical user id.  These
files will make PVM think a pvmd is running, and can cause the
dreaded "Can't Start Pvmd" message.

For more information about the console commands, see the console "help"
function or the console man page.


TROUBLESHOOTING

If you ever have trouble starting PVM or adding a new host to your
virtual machine, verify that there are no leftover /tmp/pvmd.<uid>
daemon socket files on the machines where you are trying to start
PVM (as described above) and then check the local /tmp/pvml.<uid>
log file for any error messages which may help you to determine
the problem.

A common problem is caused by restricted access to a remote machine
via "rsh".  PVM uses "rsh" to start the pvmd on a remote host.  If
you cannot do this:

	% rsh remote_host 'echo $PVM_ROOT'

and successfully get back the correct value of $PVM_ROOT on that
remote host, without typing your password, then that is the problem.
You will need to set up permissions on the remote host to allow rsh
access without a password.  On Unix systems, this is accomplished
by creating a $HOME/.rhosts file on the *remote* machine that
provides access to your *local* machine.  (Note: the $HOME/.rhosts
file is *NOT* a PVM host file...)  The format of a $HOME/.rhosts
file is as follows:

	your_host_1  your_login_on_host_1
	your_host_2  your_login_on_host_2
	your_host_3  your_login_on_host_3
	. . .

If you get really stuck, try checking the online FAQ off of the
PVM Home Page:

 http://www.netlib.org/pvm3/book/node23.html#SECTION00450000000000000000

*** If you see "Master Host IP Address is Loopback!" or get the
PvmIPLoopback error when adding hosts, this means that the networking
on your Master PVM host (the one you initially started PVM on) is
not set up to support multiple, remote hosts in your virtual machine.
By default, especially on many Linux systems, the host name alias
is appended to the 127.0.0.1 loopback / localhost IP address in the
/etc/hosts file.  This is very useful for running in stand-alone mode
without networking, however this alias must be removed for interaction
with remote hosts in PVM.  See the Linux Networking HOWTO for
information on automatically handling this scenario (via ifup-local
and ifdown-local scripts).

In Linux, with the latest version of gcc, if you get the warning:

	"the use of `tmpnam' is dangerous, better use `mkstemp'"

from gcc, you may safely ignore it.  PVM cannot use the mkstemp()
function to create a temporary file for binding to its sockets,
as the mkstemp() function actually opens the given file & returns
a file pointer.  The mktemp() function (without the "s") could be
used in place of tmpnam(), but this function *also* produces the
same gcc warning.  It's all quite annoying really, as tmpnam()
works just fine for our purposes...  Hrmph.


APPLICATION PROGRAMS

C, C++ and Fortran programs must be linked with the main PVM library,
$PVM_ROOT/lib/$PVM_ARCH/libpvm3.a, and Fortran programs must also be
linked with the $PVM_ROOT/lib/$PVM_ARCH/libfpvm3.a library file.
User program source files should include the $PVM_ROOT/include/pvm3.h
header file for C/C++ programs, to define PVM constants and function
prototypes.  The corresponding Fortran header file is
$PVM_ROOT/include/fpvm3.h.

Executables should be installed in $PVM_ROOT/bin/$PVM_ARCH for a
user installation, and in $HOME/pvm3/bin/$PVM_ARCH for a system-wide
PVM installation (where the user may not be able to write to the
$PVM_ROOT/bin/$PVM_ARCH directory).  This default execution path
can be changed; see the "ep=" option on the pvmd man page.

When user programs are spawned by PVM, by default they execute in the
$HOME directory of the user.  Any input or data files should therefore
reside in or be linked to $HOME.  This default working directory
can be changed; see the "wd=" option on the pvmd man page.


CONTACT

The PVM web home page is at

    http://www.csm.ornl.gov/pvm/pvm_home.html .

A newsgroup, comp.parallel.pvm, exists for discussions about PVM
and help with specific problems.

Please direct any e-mail (questions, bugs, bug fixes, etc.) to:

    pvm@msr.CSM.ORNL.GOV.

To report bugs or problems with PVM, please see the file

    $PVM_ROOT/doc/bugreport .

________________________________________________________________________

Sincerely,
The PVM research group

