UKI-SCOTGRID-GLASGOW enabling VO

From GridPP Wiki
Jump to: navigation, search

cfengine

Firstly, take the steps to enable a VO which are done in cfengine.


Pool Accounts

As pool acounts are controlled by cfengine (with the local_pool_accounts helper script), the distributed group, passwd and shadow files have to be edited on svr031:/var/cfengine/inputs/skel/torque.

groups

Add the groups first to svr031:/var/cfengine/inputs/skel/torque/group - it's wise to create groups for sgm and prd pool accounts at this stage, even if they are not going to be used, because if they are later enabled it is less confusing if all of a VO's groups are contiguous.

poolacct.py

There's a little python script "poolacct.py" which will spit out the appropriate lines for the passwd, shadow, group and users.conf files - these should then be pasted into the correct files. The VO parameters are entered by hand in this script at the moment, but it should probably be rejigged to become the uberscript which can generate all of these files from scratch in the future.

The necessary parameters for a VO which has, e.g., pool production accounts is: 

startuid=18000
vo="supernemo.vo.eu-egee.org"
stub="snemo"
uids = { 'vanilla' : 50,
         'sgm' : 1,
         'prd' : 10 }
gids = {'snemo' : 10051, 'snemosgm' : 10052, 'snemoprd' : 10053}
groupMap = { 'vanilla' : ['snemo'],
          'sgm' : ['snemosgm','snemo'],
          'prd' : ['snemoprd','snemo'] }

If there is only one prd or sgm account an oldstyle single account is created (e.g., snemosgm), but if there are more than one then a new style pool account is created, (e.g. prdsnemo001).

YAIM

site-info.def

Add the new VO's site-info.def parameters to the master site-info.def on svr031. (At the moment this is in svr031:/var/cfengine/inputs/skel/yaim/site-info.def, but will hopefully be version controlled soon.)

Have a look on the CIC Portal VO Users page for an ID card, or try the GridPP approved VOs page.

Note that with new DNS style VOs, this is done as:

  1. Add the full DNS VO name to the VOS variable.
  2. Add a QUEUE_GROUP_ENABLE variable, to bind the VO to an appropriate torque QUEUE

Now add all the other VO configuration parameters to the file vo.d/VONAME: 

SW_DIR=$VO_SW_DIR/supernemo
DEFAULT_SE="$DPM_HOST"
VOMS_SERVERS="vomss://voms.gridpp.ac.uk:8443/voms/supernemo.vo.eu-egee.org"
VOMSES="'supernemo.vo.eu-egee.org voms.gridpp.ac.uk 15012 /C=UK/O=eScience/OU=Manchester/L=HEP/CN=voms.gridpp.ac.uk/Email=hostmaster@hep.man.ac.uk supernemo.vo.eu-egee.org'"
STORAGE_DIR=$DPM_BASE_PATH/supernemo.vo.eu-egee.org

groups.conf

Similarly add the new VO to svr031:/var/cfengine/inputs/skel/yaim/groups.conf, e.g., 

 "/VO=supernemo.vo.eu-egee.org/GROUP=/supernemo.vo.eu-egee.org/ROLE=lcgadmin":::sgm:
 "/VO=supernemo.vo.eu-egee.org/GROUP=/supernemo.vo.eu-egee.org"::::

This makes the VOMS role mappings - there might be other roles required, depending on the VO.

users.conf

Despite the fact that cfengine manages unix groups and users, YAIM needs to know the uids and gids of the different VOs, which is comminucated through users.conf. An appropriate fragment for this file is generated by the poolacct.py script - copy it into users.conf and then it will allow YAIM's "utilities" to run correctly - it's needed by some of the information plugins, etc.

LCG Environment

Add stanzas for the new VOs to svr031:/var/cfengine/inputs/skel/grid/etc/profile.d/lcgenv.{sh,csh} 

 export VO_GRIDPP_DEFAULT_SE=svr018.gla.scotgrid.ac.uk
 export VO_GRIDPP_SW_DIR=/grid/exp_soft/gridpp

 setenv VO_GRIDPP_DEFAULT_SE svr018.gla.scotgrid.ac.uk
 setenv VO_GRIDPP_SW_DIR /grid/exp_soft/gridpp

Although YAIM will generate this file itself, it's more convenient to have cfengine control it, as it allows for fast switching of the top level BDII.

DNS Style VO Names

The current version of YAIM does not define DEFAULT_SE or SW_DIR parameters for DNS style VOs.

Batch System

  1. First, add the new queue to the "torquequeues" cfengine variable on svr031. This will not actually create the queue for you, but will ensure that if the CE goes on fire and has to be rebuilt from scratch then the queue will be readded. See below for the actual adding of the torque queue.
  2. Secondly, for each new queue add the appropriate fairshare in svr031:/var/cfengine/inputs/skel/torque/maui/maui.cfg


Node Specific Tasks

N.B. Tackle the nodes in this order.

WN

Nothing to do - cfengine manages pool accounts.

BDII

Nothing to do. Neither the top level or the site level BDII have any VO specific stuff in them.

DPM Disk Servers

The grid-mapfile and lcgdm-mapfiles need to be remade with the new VOs' VOMS information. Run the YAIM config_mkgridmap function. 

 /opt/glite/yaim/bin/yaim -r -s /opt/glite/yaim/etc/site-info.def -f config_mkgridmap

Cluster Disk Server

As the software directories are on disk037, with root squash on, they have to be made by hand: 

 disk037:/cluster/disk4/exp_soft# mkdir gridpp
 disk037:/cluster/disk4/exp_soft# chown gridppsgm: gridpp

If the VO uses pooled sgm accounts, then make sure that the root directory is group writable as well.

DPM Headnode

Adding DPM Root Directory

Now, this can be done with YAIM, by running the config_sedpm function; however, as this does rather more than just creating the VO root storage areas, then the necessary section can be done by hand, e.g., 

 dpns-entergrpmap --group gridpp
 dpns-mkdir /dpm/gla.scotgrid.ac.uk/home/gridpp
 dpns-chown root:gridpp /dpm/gla.scotgrid.ac.uk/home/gridpp
 dpns-setacl -m d:u::7,d:g::7,d:o:5 /dpm/gla.scotgrid.ac.uk/home/gridpp

Update Information System

Run the config_gip_dpm YAIM function. 

 svr018:~# /opt/glite/yaim/bin/yaim -r -s /opt/glite/yaim/etc/site-info.def -f config_gip_dpm

N.B. This will overwrite the /opt/glite/etc/gip/provider/se-dpm wrapper, so if a new beta plugin is being trialed then restore it as per DPM Information Publishing#Modifying the Plugin Wrapper:

Remake Gridmap Files

The grid-mapfile and lcgdm-mapfiles need to be remade with the new VOs' VOMS information. Run the YAIM config_mkgridmap function. 

 /opt/glite/yaim/bin/yaim -r -s /opt/glite/yaim/etc/site-info.def -f config_mkgridmap

MON

Nothing to do.

UI

WMS Client

If the Vo is supported on one of the UIs, then either run the YAIM config_workload_manager_client function, or just add a new VO specific WMS file to /opt/edg/etc/VO/edg_wl_ui.conf.

VOMS

Run config_vomses to generate the correct VOMS configurations in /opt/glite/etc/vomses.

CE

Create New Queue

  1. Assuming the CE is running, then the cfengine helper script can be run by hand to add the necessary queue: 
 svr016:# /var/cfengine/inputs/scripts/torque_queue_cfg NEW_QUEUE_NAME

This should add the new queue for the VO. You can check it has with "qstat -Q" on the CE.

Information System

Run the config_gip and config_gip_scheduler_plugin: 

 svr016:~# /opt/glite/yaim/bin/yaim -r -s /opt/glite/yaim/etc/site-info.def -f config_gip
 svr016:~# /opt/glite/yaim/bin/yaim -r -s /opt/glite/yaim/etc/site-info.def -f config_gip_scheduler_plugin

Gridmap and LCMAPS

Remake the grid-mapfile and the LCMAPS configuration on the CE: 

 # /opt/glite/yaim/scripts/run_function -r -s /opt/glite/yaim/etc/site-info.def -f config_mkgridmap

This does not need to be done on other nodes, as they will grab their grid-mapfiles from the CE (and don't care about LCMAPS - I think...)

Gatekeeper

Once new VOs are enabled the globus-gatekeeper needs rejigged (I'm unclear why, but submission to the batch system fails until this is done...) 

 # /opt/glite/yaim/scripts/run_function -r -s /opt/glite/yaim/etc/site-info.def -f config_globus

VO Tag Directory

For each VO, make sure there is a VO tag directory in /opt/edg/var/info/VO owned and writable by the VO's sgm account: 

 # mkdir /opt/edg/var/info/gridpp
 # chown gridppsgm: /opt/edg/var/info/gridpp

(It would be very neat if cfengine could do this, but it's not trivial as its colon separated iterator variables cannot then be manipulated to give the correct owner.)

RB

For the moment, re-run YAIM completely on the RB: 

 svr023# /opt/glite/yaim/scripts/configure_node -c -s /opt/glite/yaim/etc/site-info.def -n RB

or 

 svr023# cfagent -qv -D runyaim

Policy Note: As the RB is now advertised in the BDII, it would be desirable to restrict the number os VOs supported. At the moment this is not easy, because of the single site-info.def.

Retrieved from "https://www.gridpp.ac.uk/w/index.php?title=UKI-SCOTGRID-GLASGOW_enabling_VO&oldid=1671"