
Using Globus certificates to authenticate to a remote rootd/proofd server
=========================================================================

To use Globus certificates for remote authentication, the Globus Tool Kit
should  be installed both in client and server machines and ROOT should be
compiled with  the relevant flags set (see below).

Globus Authentication has been tested for the moment only on Linux.

1) Globus Tool Kit (GTK) Version

   The interface has been developed using GTK version 2.2.3 and tested on
   machines with version 2.0, 2.0.21b and 2.4.3; it should work with versions
   2.x ; however, vanilla versions 2.2.3 - 2.4.3 contain a bug which does not 
   allow to overload at the same time the defaults for the user certificate
   and key files. A patch has been provided by the Globus team (see Globus
   bugzilla #892) which is shipped with ROOT.

   If your system does not have GTK installed, you do not need to install
   all the GTK bundles but only the GSI one, i.e.

        globus-gsi-<GTK_version>-src_bundle.tar.gz

   Versions older than 2.1.2 do not have a separate bundle for GSI: in such
   a case you need to install the 

        globus-resource-management-client-<GTK_version>-src_bundle.tar.gz

   If you plan to install binaries, the relevant code should be contained in
   the 

        globus-resource-management-client-<GTK_version>-<arch>-bin.tar.gz

   bundle.
   Please, refer to www.globus.org for any additional information about 
   installation and certificate acquisition. 

2) Compiling ROOT with Globus support

   To compile ROOT with Globus support run the 'configure' script with the flag
   '--enable-globus'; the script expects the variable GLOBUS_LOCATION to be
   defined and pointing to the location with the 'bin', 'lib' and 'include'
   dirs for the  GTK . It is definitely better to have this variable defined
   correctly (especially on the client side); however, for compilation purposes
   it is possible to force a specific directory for the GTK running configure
   with

       --with-globus-dir=<GTK_path>

   The 'configure' script checks if the used GTK needs the patch mentioned 
   above. While running configure you should get the following printed

 ...
Checking for globusdir ... no
Checking for GLOBUS_LOCATION ... <Your_GLOBUS_LOCATION>
Checking for openssl/x509.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for openssl/pem.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for globus_gss_assist.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for globus_gss_common.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for globus_gsi_credential.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for gsi version ... yes
Checking for libglobus_gss_assist_<flavour> ... <Your_GLOBUS_LOCATION>/lib
Checking for libglobus_gssapi_gsi_<flavour> ... <Your_GLOBUS_LOCATION>/lib
Checking for libssl_<flavour> ... <Your_GLOBUS_LOCATION>/lib
Checking for libcrypto_<flavour> ... <Your_GLOBUS_LOCATION>/lib
Checking for libglobus_gsi_credential_<flavour> ... <Your_GLOBUS_LOCATION>/lib
Checking for globus-user-env.sh ... <Your_GLOBUS_LOCATION>/etc
Checking for grid-proxy-init ... <Your_GLOBUS_LOCATION>/bin
Checking pkg_data_<flavour>_rtl.gpt ... <Your_GLOBUS_LOCATION>/etc/globus_packages/globus_gsi_credential
Checking for of globus patch ... yes, version <globus_gsi_credential version>
Checking for patch version consistency ... yes
Checking for globus_gsi_system_config.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Checking for globus_gsi_cert_utils.h ... <Your_GLOBUS_LOCATION>/include/<flavour>
Applying experimental patch to globus_gsi_credential.c
 ...

   where <flavour> is either gcc32 or gcc32dbg, and <Your_GLOBUS_LOCATION>
   is the env $GLOBUS_LOCATION. 
   The last 3 lines appear only if the patch is needed and activated.

   If globus_gsi_credential.c does not compile correctly, it may be that
   the patch version found by configure is not correct. Check in the 
   globusauth/src/patches/globus_gsi_credentials/README if the printed
   <globus_gsi_credential version> matches the one expected for your 
   GTK version (ask the latter to your site administrator if you do not
   it). If the version is wrong you should copy the right patched version
   of the globus_gsi_credentials.c file

cd  globusauth/src
cp -rp patches/globus_gsi_credentials/<right version>/globus_gsi_credentials.c .
cd ../..

   and substitute the right version for the wrong one in the variable 
   GLBPATCHFLAGS defined in config/Makefile.config; re-compile *without* 
   running configure. If the problem persists, mailto: gerardo.ganis@cern.ch.

3) Before running, after compilation/installation

   3.1) On the CLIENT side

        Make sure that valid certificates are in the correct places and that
        valid credentials can be initialized; this can be tested by running

               grid-proxy-init

        and entering the pass phrase; you may have to source the relevant shell
        file to have this command recognized:

           source $GLOBUS_LOCATION/etc/globus-user-env.csh
        or
           source $GLOBUS_LOCATION/etc/globus-user-env.sh .

        If you are sure about your capabilities to get valid credentials, you
        don't need to initialize them outside the root session, for ROOT checks
        the credentials and prompts for the pass phrase if they are missing or
        invalid.

        There are three files/dirs relevant for proxy initialization:

        a) user certificate file; default is $HOME/.globus/usercert.pem ; can
           be  changed by setting the variable "X509_USER_CERT" to the full
           path

        b) user private key file; default is $HOME/.globus/userkey.pem ; can
           be  changed by setting the variable "X509_USER_KEY" to the full path

        c) directory with certificates of the known Certificate Authorities;
           default is /etc/grid-security/certificates ; can be changed by
           setting the variable "X509_CERT_DIR" to the full path.

        All this variables can be changed by the user before running root or
        during a root session. Globus authentication can be requested by:

        1) Setting

                Rootd.Authentication   3
                Proofd.Authentication   3

           in the $HOME/.rootrc file; other parameters can be changed at this
           level:

          - the proxy duration, in hours

                Globus.ProxyDuration :    <hours>

          - the number of bits to be used for the key (should be a power of 2)

                Globus.ProxyKeyBits:      1024

          - a Globus login defining the relevant files mentioned above:

                Globus.Login:             cd:~/.globus cf:usercert.pem  \
                                          kf:userkey.pem  ad:certificates

                with the meaning of the different keys being:

                "cd"    defines the directory with the user certificate and
                        private key; default: $HOME/.globus
                "cf"    user certificate file (in the dir specified by "cd");
                        default: usercert.pem
                "kf"    user key file (in the dir specified by "cd");
                        default: userkey.pem
                "ad"    directory with certificates of the known Certificate
                        Authorities; default: /etc/grid-security/certificates

                Both "cd" and "ad" can be given as absolute paths (starting
                with '/'), relative to  $HOME (starting with ~/) or relative to
                $HOME/.globus (in the other cases).

        2) Specifying an entry in .authrootrc (see README.AUTH for more details)

           This allows to be host and user specific and to initialize proxies
           for two or more different users in the same session.

        3) During a ROOT session by creating (or modifying) a THostAuth
           instantiation (see README.AUTH for more details)

        Upon successful authentication the user is logged on remotely with the
        user name  specified in the gridmap file on the remote server; if there
        is no entry is found in  the gridmap corresponding to the client DN, an
        attempt is made to guess it from the  CN part of the DN.


   3.2) On the SERVER side

        .1) Running as 'root' (or with su privileges)

            The servers rootd/proofd make use of the host certificate ( <CA bla
            bla>/CN=host/<FQDN>) located by default in
	    
                        /etc/grid-security/hostcert.pem
	    
            and the related private key (default in
            /etc/grid-security/hostkey.pem). The environment variable
            X509_USER_CERT should be set to point to this file.
	    
            No proxies are needed for the host certificate.
	    
            Server Configuration file:
            both daemon servers accept as argument
	    
                        -C <server_globus_conf_file>
	    
            By default this file is looked for in /etc/root/hostcert.conf or in
            $ROOTSYS/etc/hostcert.conf and it contains record lines specifying
	    
            <certificates_dir>    <host_cert_file>  <host_cert_key> <grid_mapfile>
	    
            Record lines starting with '#' are considered as comments; an example:
	    
            < bof >
            # This is an example of hostcert.conf ...
            /etc/grid-security/certificates /etc/grid-security/hostcert.pem 
                /etc/grid-security/hostkey.pem /etc/grid-security/grid-mapfile
            < eof >

	    (nb: all the 4 records should be on the same line: here they are split
                 to make it more readable).
	    
            There may be as many line as there are valid certificate settings
            (corresponding to  different Certificates Authorities and potentially
            to different DN-to-UserName mappings ). When a request for globus
            authentication arrives, rootd/proofd look among their  own certificates
            if there is one issued by the same CA which has issued the  client
            certificate; if the search is successful they communicate the related
            subject name to the client and setenv the relevant variables using to
            the chosen  configuration set.

        .2) Running as normal user

            In globus the authentication is governed by a process having superuser
            privileges, usually a gatekeeper. The host certificate is not fully 
            accessible to a normal user, since the key cannot be read. Therefore
            host credentials cannot be acquired by a server running as non-root.

            There is, however, the possibility to grant access to the local user 
            area; this can be done via valid user proxies, to be initialized 
            before the authentication is attempted.
            Is is advised to initialize the user proxies for a very long time, eg. 
             
                       grid-proxy-init -valid 999999:0

            (the actual duration is limited by the lifetime of the user certificate)
            to be sure that for the running time of the daemon authentication is 
            possible. 
            That should be all. The /etc/root/hostcert.conf can still be used to 
            specify non standard certificate directory and mapping file (with the
            same format), the two entries in between being meaningless.
            Make sure that you have read access to hostcert.conf and to the files
            and directories specified inside; it may be wise to use local user-owned
            versions of these files and directories; example

            machine:~> cat $HOME/root/hostcert.conf

            # This is the local configuration
            $HOME/.globus/certificates * * $HOME/.globus/grid-mapfile

            machine:~> rootd -d 3 -p 5151 -C $HOME/root/hostcert.conf -f $ROOTSYS

        .2.1) Using user proxies when running as 'root' (or with su privileges)

            If a valid host certificate is not available, the server administrator 
            can still decide to run as 'root' and use the valid proxies of a user
            registered on the machine to authenticate a client. 
            To activate this, the user certificate and key location should be
            specified in the relevant hostcert.conf file, e.g.

            < bof >
            # This is an example of hostcert.conf ...
            /etc/grid-security/certificates /home/<user>/.globus/usercert.pem 
            /home/<user>/.globus/userkey.pem /etc/grid-security/grid-mapfile
            < eof >
	    
            (nb: all the 4 records should be on the same line: here they are split
                 to make it more readable).

            The program will then look for the proxy file associated with the owner
            of the certificate and key files and use that to acquire the credentials
            needed for authentication; as at point .2), the proxies must be 
            initialized before the relevant software tries to use them, i.e.
            there is no prompt for proxy initialization.

4) Trying out

   Trying to access a remote file should give this on the client side (assuming
   globus credentials have not been initialized before):

root [1] TFile *f1 = TFile::Open("root://<remote_FQDN>/test.root","read")
Your identity: /O=Grid/OU=GlobusTest/OU=simpleCA-arthux.cern.ch/OU=cern.ch/CN=qwerty
Enter GRID pass phrase for this identity:
Creating proxy ....................... Done
Your proxy is valid until Wed May 28 02:46:12 2003
root [2]

   In the /var/log/messages on the server you should get something like ( with -d 3 in 'rootd')

May 27 12:49:46 pcepsft43 rootd[24031]: RootdLoop: kind:2033 -- buf:'11820 -1 2 4 None' (len:17) -- Auth:0
May 27 12:49:46 pcepsft43 rootd[24031]: RpdDefaultAuthAllow: default list of secure methods available:  4 1 2 3
May 27 12:49:46 pcepsft43 rootd[24031]: RpdGlobusAuth: user: /O=Grid/OU=GlobusTest/OU=simpleCA-arthux.cern.ch/OU=cern.ch/CN=qwerty   authenticated
May 27 12:49:46 pcepsft43 rootd[24031]: RpdGlobusAuth: logging as qwerty
May 27 12:49:46 pcepsft43 rootd[24031]: RootdLogin: user qwerty authenticated (OffSet: -1)
May 27 12:49:46 pcepsft43 rootd[24031]: RootdLoop: kind:2004 -- buf:'/test.root read' (len:10) -- Auth:1

   The first authentication to a (remotehost,username) entity may be perceived
   as slow;  this is because a lot of exchanges are need to establish the
   security context; however,  the security context is kept until is valid, so
   subsequent access to the same entity  will be much faster (if this is not
   so, make sure that you do not have set the 'reuse'  flag to '0' or 'no'; see
   README.AUTH).

   In a PROOF cluster, if the master requires Globus credentials for
   authentication  to slaves, these are automatically transmitted using a
   shared memory.

5) If it does not work ...

   Some tips:

   .1) make sure that local and remote times are synchronized within 5 minutes
   .2) if GlobusAuthenticate is not found and you are using /etc/ld.so.conf for
       shared libraries you may need to run /sbin/ldconfig -v after compilation.
   .3) Make sure that remote node accepts Globus authentication from your
       local host
   .4) Check location of certificate and private key files and certificate
       directories; make sure they can be read by the root process
   .5) If you get the error message:

        init.c:497: globus_gss_assist_init_sec_context_async: 
                    Error during context initialization
        init_sec_context.c:187: gss_init_sec_context: 
                                Unable to verify remote side's credentials
        globus_i_gsi_gss_utils.c:898: globus_i_gsi_gss_handshake: 
                    SSLv3 handshake problems: Couldn't do ssl handshake
        OpenSSL Error: s3_clnt.c:836: in library: SSL routines, 
                    function SSL3_GET_SERVER_CERTIFICATE: certificate verify failed
        globus_gsi_callback.c:349: globus_i_gsi_callback_handshake_callback: 
                    Could not verify credential
        globus_gsi_callback.c:435: globus_i_gsi_callback_cred_verify: 
                    Could not verify credential: 
                    self signed certificate in certificate chain (error code 7)

       your certificate issuer is not recognized as trusted.
       Most likely this means that its CA certificate has not been found. 
       Either you don't have it or one of the followings holds:
 
        o your $HOME/.globus/certificates does not exist or is empty
        o your X509_CERT_DIR is either undefined or points to the wrong
          directory

       Solution: you should localize the certificate of the CA issuing your 
       certificate: this is in the form <HASH>.0 and a possible alternative
       location in /etc/grid-securiy/certificates; you can check the content
       using the openssl command:

          openssl x509 -in <HASH>.0 -text

       Do the same with your $HOME/.globus/usercert.pem and make sure the 
       Issuer is the same. 
       Once you have localized the directory with your CA certificate, setenv
       X509_CERT_DIR to point to that directory, for example

          setenv X509_CERT_DIR /etc/grid-securiy/certificates

       (or export X509_CERT_DIR=/etc/grid-securiy/certificates for a bash shell). 
        
   .6) mailto: gerardo.ganis@cern.ch.

--------------------------------------------------------------------------------------
Last update: March 24, 2004

