LFC setup

If one or more of them has different or empty value, please set it (them) in this way:

export LCG_CATALOG_TYPE=lfc                       # define the catalog type
export LFC_HOST=uct2-grid1.uchicago.edu    # define the catalog endpoint
#export LCG_GFAL_INFOSYS=is.grid.iu.edu:2170 # define the BDII endpoint http://is.grid.iu.edu:2170/cgi-bin/data.cgi

The BDII, set in the last variable, is normally used in LCG but is not mandatory for OSG sites. Better not to set the variable at all and use the commands not requiring a BDII.

You can look for help on the syntax using '-h' or '--help'. Not always recognized, but something useful comes out most of the times.

Basic commands

LFC is a virtual file system with LFNs (logical file names) and directories. Basic commands are similar to unix commands.

lfc-ls -l /grid/atlas
export LFC_HOME=/grid/atlas
lfc-ls atlasproddisk

You can define a HOME directory. Since now, all your lfc-command with a relative path (i.e. not starting with / ), will have automatically as prefix $LFC_HOME LFC is a virtual file system with LFNs (logical file names and directories)

lfc-mkdir /grid/atlas/userdirectory
lfc-rename $LFC_HOME/userdirectory/oldnamefile $LFC_HOME/userdirectory/newnamefiel
lfc-ln -s $LFC_HOME/userdirectory/myfile-renamed $LFC_HOME/userdirectory/myfile-renamed-symlink
lfc-rm file
lfc-rm -r empty-dir

Pay attention because lfc-rm removes just entry on catalog : the file on SE will continue to exist. Use lcg-del to remove both and then lfc-rm to remove the directories in LFC.

Permissions

Entries in LFC have owner, group and ACLs. Finally there may be a default, which applies for each new entry created within the directory (if the entry is a directory). Note that ownership is expressed through DN of certificate, in order to grant uniqueness and avoid conflicts, while group is expressed through VO membership. ALCs can have users or groups.

lfc-chmod 750  $LFC_HOME/userdirectory/myfile-renamed
lfc-getacl $LFC_HOME/test-acl
lfc-setacl -m d:u::6,d:g::6,d:o:0 $LFC_HOME/test-acl/

Default permissions are used by commands like lcg-cr and for the ACLs of LFNs or subdirectory created within that path. If you have one default, you need to have default at least for owner, group, other

Commands specifying the group name (using the VO name/group/role):

lfc-setacl -m group:atlas:rwx /grid/atlas/dq2/user09
lfc-setacl -m default:group:atlas:rwx /grid/atlas/dq2/user09
lfc-setacl -m group:atlas/usatlas:rwx /grid/atlas/dq2/user09
lfc-setacl -m default:group:atlas/usatlas:rwx /grid/atlas/dq2/user09
lfc-setacl -d default:group:atlas/usatlas/Role=production:rwx /grid/atlas/dq2/user09
lfc-setacl -m default:other::r-x /grid/atlas/dq2/user09

Other commands

LFC allows to comment its entries

lfc-setcomment $LFC_HOME/userdirectory/myfile-renamed "this is a comment example"
lfc-ls --comment  $LFC_HOME/userdirectory/myfile-renamed
lfc-delcomment $LFC_HOME/userdirectory/myfile-renamed

LFC at MWT2

From the installation host root can modify all the settings. Setup is in /opt/setup.sh.

LFC trees used by user (MWT2_UC_USERSCRATCH) heve to be writable by all atlas users. With lfc-setacl ACLs can be modified one at the time. atlas/usatlas is not necessary (user will have also atlas) but will not hurt (good for redundancy). The final ACL should be something like:

# lfc-getacl /grid/atlas/dq2/user09
# file: /grid/atlas/dq2/user09
# owner: /DC=org/DC=doegrids/OU=People/CN=Nurcan Ozturk 18551
# group: atlas/usatlas/Role=production
user::rwx
group::rwx		#effective:rwx
group:atlas:rwx		#effective:rwx
group:atlas/Role=production:rwx		#effective:rwx
group:atlas/usatlas:rwx		#effective:rwx
group:atlas/usatlas/Role=production:rwx		#effective:rwx
mask::rwx
other::r-x
default:user::rwx
default:group::rwx
default:group:atlas:rwx
default:group:atlas/usatlas:rwx
default:mask::rwx
default:other::r-x

Some LCG commands

This command creates an entry in LFC, with given VO, LFN, real file, guid and using the default permissions of the directory.

lcg-cr -d my-se-name --vo atlas -l lfn:$LFC_HOME/test-acl/uno  file:$HOME/message2.txt guid:7790df6c-a823-4380-a148-01296c2d0ae9

Locating a replica requires the full path of the replica. In LRC LFNs were only the logical name of the file, in LFC they include the whole virtual path in LFC. These commands locate a replica, replicate it, copy it to a local directory and delete it.

lcg-lr --vo atlas lfn:/grid/atlas/userdirectory/text_file.txt 
(In EGEE usually files are stored in dated directories using the GUID as name, e.g.: srm://aliserv6.ct.infn.it/dpm/ct.infn.it/home/gilda/generated/2006-07-19/filef42b726f-d9dc-45f4-b3b5-f4d55a6d9c44)
lcg-rep --vo atlas -d destination-SE lfn:/grid/gilda/tutorials/userdirectory/text_file.txt 
lcg-cp --vo atlas lfn:/grid/atlas/userdirectory/text_file.txt file://$HOME/text_file_copy.txt
lcg-del -a --vo atlas lfn:/grid/atlas/userdirectory/text_file.txt

Not sure what happens:

• when multiple replicas exist of a single LFC entry
• when there are symbolic links in LFC (delete the file, delete the link)

LFC vs LCG

To operate in files, SEs and catalogs in the LGC/EGEE world 2 sets of commands are mainly used:

• "lfc-" commands interact with the LFC catalog server that maps logical filenames to "site URLs".
• "lcg-" commands include ones used to copy files to and from an SE, and to replicate files. These normally require BDII (sometime it is not possible to disable it) and are also responsible for file name mingling (e.g. shortening the name also when the fully qualified name is passed as input). Sometime in OSG is necessary to avoid these commands and provide workarounds.

References

• From GILDA (medical grid in EGEE) https://grid.ct.infn.it/twiki/bin/view/GILDA/LFCMed
  • https://grid.ct.infn.it/twiki/bin/view/GILDA/DataManagement
  • https://grid.ct.infn.it/twiki/bin/view/GILDA/InformationSystems
  • https://grid.ct.infn.it/twiki/bin/view/GILDA/UserTutorials

Old LRC commands

curl http://uct2-grid1.uchicago.edu:8000/dq2/lrc/PoolFileCatalog\?lfns=HIST.022910._50326.root.2+NTUP.022910._50326.root.2+ESD.022910._50326.po
ol.root.2+log.022910._50326.job.log.tgz.2
curl http://uct2-grid1.uchicago.edu:8000/dq2/lrc/PoolFileCatalog\?guids=3fe4d0e5-48ee-45eb-9a87-6e5bc12b5da6
curl http://uct2-grid1.uchicago.edu:8000/dq2/lrc/PoolFileCatalog\?lfns=HIST.022910._51951.root.2
curl http://tier2-01.uchicago.edu:8000/dq2/lrc/storage
curl http://tier2-01.uchicago.edu:8000/dq2/storage
curl http://tier2-01.uchicago.edu:8000/dq2/storages/default

-- MarcoMambelli - 02 Jan 2009