Monday, July 13, 2015

Globus Toolkit Installation on CentOS

This is a quickstart guide to install Globus Toolkit 6.0 using yum on CentOS-6.6. The steps mentioned in “GT 6 Quickstart Guide” which is the official documentation of Globus Toolkit were followed during the installation process.The GT 6.0 release provides both source and binary RPM packages for CentOS which can be downloaded from here.
We have two two servers out of which one will act as a master and the other as client. The hostnames of the servers are master and client repectively. You can try out this setup on virtual appliances.

PREREQUISITES

This section is common for master and all client nodes.

System Configuration

First and foremost, it is very important to check if the hosts file matches in all nodes and each node is having other two nodes' network address and hostname mentioned properly.
Turn off the iptables service.
Make sure the nodes are connected to internet.

If you are going for a minimal install of CentOS then we need to make sure that all the below mentioned packages are installed using yum.

root@master # yum install epel-release java wget gcc sed make openssl-devel perl pkgconfig httpd elinks openssh-clients rpm-build

root@master # yum groupinstall “Development Tools”

Globus Repository Configuration

Since we are following the “GT 6 Quickstart Guide”, we will be downloading the latest RPM package which is actually contains the yum repository definition for CentOS-6. This will set up our nodes to use the Globus RPM repository for the installation of required Globus software packages and their dependencies.

Download Globus repository package
root@master # wget -c http://toolkit.globus.org/ftppub/gt6/installers/repo/globus-toolkit-repo-latest.noarch.rpm

Install Globus repository package
root@master # rpm -hUv globus-toolkit-repo-latest.noarch.rpm

Verify by listing the repository
root@master # ls /etc/yum.repos.d/

GLOBUS TOOLKIT INSTALLATION ON “MASTER” NODE


Globus Toolkit Installation

Once we have installed the Globus repository package, we can use yum to install the Globus components.
The below mentioned command will install GridFTP, GRAM, MyProxy, GSI C services as well as set up a basic SimpleCA used to issue security credentials for users to run the Globus services.
root@master # yum install globus-gridftp globus-gram5 globus-gsi myproxy myproxy-server myproxy-admin

Setting up Security

As we are not using any other tool to authorize our grid users, SimpleCA will take care of the basic security and manage it's own Certificate Authority. There is a globus trusted certificate directory automatically created after the installation which contains the public certicates, host certicate and host key files. The host certificate and host key files need to be copied so that the myproxy service can use it as well.

root@master # install -o myproxy -m 644 /etc/grid-security/hostcert.pem /etc/grid-security/myproxy/hostcert.pem

root@master # install -o myproxy -m 600 /etc/grid-security/hostkey.pem /etc/grid-security/myproxy/hostkey.pem

root@master # ls -l /etc/grid-security/

Creating MyProxy Server

MyProxy server is used to store the user's certificates. In order to enable MyProxy to use SimpleCA we need to modify the /etc/myproxy-server.config file, by uncommenting every line in the section “Complete Sample Policy #1” such that the section looks like the below truncated ouput.

#
# Complete Sample Policy #1 - Credential Repository
#
# The following lines define a sample policy that enables all
# myproxy-server credential repository features.
# See below for more examples.
accepted_credentials "*"
authorized_retrievers "*"
default_retrievers "*"
authorized_renewers "*"
default_renewers "none"
authorized_key_retrievers "*"
default_key_retrievers "none"
trusted_retrievers "*"
default_trusted_retrievers "none"
cert_dir /etc/grid-security/certificates

Next we will add the “myproxy” user to the “simpleca” group so that MyProxy server can create certificates.
root@master # usermod -a -G simpleca myproxy

Start the MyProxy service.
root@master # service myproxy-server start

Make it permanent across server reboots
root@master # chkconfig myproxy-server on

Verify the service status.
root@master # service myproxy-server status
root@master # netstat -ntulp | grep 7512

User Credentials

We will create a new local user “Globus User” as it's full name and “guser” as it's user name.
root@master # useradd -c "Globus User" guser

Create a password for the user “guser”.
root@master # passwd guser

Now login using “myproxy” user credentials and set the sbin path in .profile file.
root@master # su - -s /bin/sh myproxy

myproxy@master $ vi ~/.profile
export PATH=$PATH:/usr/sbin

myproxy@master $ source ~/.profile
Next run the below command to create the credentials. It will prompt for a passphrase, which I set as “globus”.

myproxy@master $ myproxy-admin-adduser -c "Globus User" -l guser
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
The new signed certificate is at: /var/lib/globus/simple_ca/newcerts/03.pem
using storage directory /var/lib/myproxy
Credential stored successfully
Certificate subject is:
/O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=Globus User

Note: Make a note of the “Certificate subject” mentioned in the last line of from the output generated by the above command.

User Authorization

In order to access globus services, a user must have an entry in grid map file. We need to copy the “Certificate subject” as mentioned in earlier and pass it to the “-dn” switch as a parameter. The below mentioned command will create an entry in already present grid map file for this credential.

root@master # grid-mapfile-add-entry -dn "/O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=Globus User" -ln guser
Modifying /etc/grid-security/grid-mapfile ...
New entry:
"/O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=Globus User" guser
(1) entry added

Verify that the same entry has been generated on /etc/grid-security/grid-mapfile
root@master # cat /etc/grid-security/grid-mapfile

Setting up GridFTP Server

After setting up basic security and authorizing users, we can start globus services. We will start with GridFTP server.

Starting the service related to GridFTP server.
root@master # service globus-gridftp-server start

Making the service persist across server reboots.
root@master # chkconfig globus-gridftp-server on

Verifying the service status.
root@master # service globus-gridftp-server status
root@master # netstat -antupl | grep 2811

Now login as normal user that we initially created “guser” so as to test the GridFTP service.
root@master # su - -s /bin/sh guser

Generate a proxy from myproxy service by using the below mentioned command and provide passphrase.
guser@master $ myproxy-logon -s master

We will test the GridFTP service by copying a file locally.
guser@master $ globus-url-copy gsiftp://master.venus.com/etc/group file:///tmp/guser.test.copy

Verify that both the files are same.
guser@master $ diff /tmp/guser.test.copy /etc/group

After checking that all services are running, a file is transferred successfully and both files are same, we can confirm that the GridFTP server is set up.

Setting up GRAM

Moving on to GRAM, it is a resource manager. GRAM can be configured to use several different Local Resource Managers (LRMs). The default LRM that comes with GRAM is “fork manager”. The GRAM service uses the same host credentials as the GridFTP.

Starting the GRAM service.
root@master # service globus-gatekeeper start

Making the service persist across server reboots.
root@master # chkconfig globus-gatekeeper on

Verifying the service status.
root@master # service globus-gatekeeper status
root@master # netstat -ntupl | grep 2119

Now after verifying that the GRAM service is running, we need to check if the default LRM is enabled.
root@master # globus-gatekeeper-admin -l
jobmanager-fork-poll [DISABLED]

If the LRM is disabled as shown in above output, then we need to enable it using the below mentioned command.
root@master # globus-gatekeeper-admin -e jobmanager-fork-poll

Now login as normal user that we initially created “guser” so as to test the GRAM service.
root@master # su - -s /bin/sh guser

Generate a proxy from myproxy service by using the below mentioned command and provide passphrase.
guser@master $ myproxy-logon -s master

GRAM Authentication test
guser@master # globusrun -a -r master/jobmanager-fork-poll

GRAM Job Submission
Execute the below mentioned commands to run sample jobs locally on a grid compute
guser@master $ globus-job-run master/jobmanager-fork-poll /bin/hostname
master.venus.com

guser@master $ globus-job-run master/jobmanager-fork-poll /usr/bin/whoami
guser

guser@master $ globus-job-run master/jobmanager-fork-poll /bin/date

As a user we can explore more into GRAM commands like globus-job-submit, globus-job-status, globus-job-cancel, and many more.

SimpleCA Configuration

Login as normal user “guser” that we initially created so as to create .globus directory in guser's home directory which consist of necessary authentication certificates.
guser@master $ grid-cert-request

Login back as root and sign the usercert.pem file that we just created and which should be zero kb in size with usercert_request.pem.
root@master # cd /home/guser/.globus/
root@master # grid-ca-sign -in usercert_request.pem -out usercert.pem

Verifying Basic Security

After completing the SimpleCA configuration, the below commands should get valid outputs.

Displaying the Certificate information
guser@master $ grid-cert-info

Displaying the Certificate subject
guser@master $ grid-cert-info -subject

Proxy Verification
guser@master $ grid-proxy-init -verify -debug

GLOBUS TOOLKIT INSTALLATION ON “CLIENT” NODE



Be sure that the prequisites are fixed as a mandatory part before we start with the installation of client machine.

Globus Toolkit Installation

Same as the master node we will start with the installation of globus software using the yum repository that we just set up as suggested in prerequisites section earlier.

root@client # yum install globus-gridftp myproxy globus-gram5

Setting up Security

The below command makes the original SimpleCA that we had set up on master to bootstrap the trust on this client machine.
root@client # myproxy-get-trustroots -b -s master
Bootstrapping MyProxy server root of trust.
New trusted MyProxy server: /O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/CN=master.venus.com
New trusted CA (8209248b.0): /O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/CN=Globus Simple CA

Trust roots have been installed in /etc/grid-security/certificates/.

Next we will create host certificates for the client on master node. Ensure that you are logged into the master node before proceeding further.
root@master # su - -s /bin/sh myproxy
myproxy@master $ myproxy-admin-addservice -c "client.venus.com" -l client
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
The new signed certificate is at: /var/lib/globus/simple_ca/newcerts/04.pem
using storage directory /var/lib/myproxy
Credential stored successfully
Certificate subject is:
 /O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=client.venus.com

Now we will retrieve the generated credentials from the client node. Ensure that you are logged into the client node before proceeding further.

root@client # myproxy-retrieve -s master -k client.venus.com -l client
Enter MyProxy pass phrase:
Credentials for client have been stored in /etc/grid-security/hostcert.pem and /etc/grid-security/hostkey.pem.

After retrieving the credentials, we will destroy the client node's host certificate from the master node as it is no longer required.

root@client # myproxy-destroy -s master -k client.venus.com -l client
MyProxy credential 'client.venus.com' for user client was successfully removed

User Credentials

We will create a new local user “Globus User” as it's full name and “guser” as it's user name same as done in master node mentioned in earlier steps.
root@client # useradd -c "Globus User" guser

Create a password for the user “guser”.
root@client # passwd guser

User Authorization

In order to make the user "guser" access the globus services, we need to add the guser's credentials into the grid-mapfile on client node.
root@client # grid-mapfile-add-entry -dn "/O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=Globus User" -ln guser
master.venus.com/OU=Globus Simple CA/CN=Globus User" -ln guser
Modifying /etc/grid-security/grid-mapfile ...
New entry:
"/O=Grid/OU=GlobusTest/OU=simpleCA-master.venus.com/OU=Globus Simple CA/CN=Globus User" guser
(1) entry added

Setting up GridFTP

After the set up of security, simple authentication, host certificate and user authorization, we will finally turn on the globus services. First we will start the GridFTP service. Follow the below mentioned commands to start the GridFTP service.

Starting the service related to GridFTP server.
root@client # service globus-gridftp-server start

Making the service persist across server reboots.
root@master # chkconfig globus-gridftp-server on

Verifying the service status.
root@master # service globus-gridftp-server status
root@master # netstat -antupl | grep 2811

Login to guser account.
root@client # su - -s /bin/sh guser

We will use the guser to receive the proxy credentials to access the globus services.
guser@client $ myproxy-logon -s master

We will test copying a file between the GridFTP servers running on master and client nodes.
guser@client $ globus-url-copy gsiftp://master.venus.com/etc/group gsiftp://client.venus.com/tmp/from-master

Note: "-nodcau" switch stands for "no data channel authentication" which turns off data channel authentication for GridFTP transfers.

Setting up GRAM

In this final stage we will submit a sample job into the Grid to test it's working. Before we proceed run the below mentioned command for GRAM authentication.
guser@client $ globusrun -a -r master/jobmanager-fork-poll

Job Submission
guser@client $ globus-job-run master/jobmanager-fork-poll /bin/hostname

The above steps have helped us setting up a GRID using Globus Toolkit 6.0 on CentOS-6.6 successfully.

You are most welcome to leave a comment in case if you want to report any errors in the above steps that you have come across or any suggestions and free advices.

2 comments: