You are here: Foswiki>Main Web>WikiUsers>RobGardner>LinearNotes>DataManageRwg>DeletingDatasetsOld (16 Mar 2009, RobGardner)Edit Attach
  • Login to uct2-grid1, setup grid environment, proxy.
  • Find a dataset which has files for deletion. dq2-client tools not exported to grid1. I go back to uct3-edge5 that has my environment already setup: 
rwg@uct3-edge5:~$ dq2-ls -L MWT2_UC -f user.RobertWGardnerJr.Sox186_shadow

user.RobertWGardnerJr.Sox186_shadow
[X]	AOD.017166._00009.pool.root	E6B71543-6E94-DC11-85AE-001B243E0BA6	md5:9731c5d1d2bfae57b403c60149dd4b66	507871921
[X]	AOD.017166._00007.pool.root	DA0CCA64-6D94-DC11-BDFA-001B243DEAC8	md5:5002751b221d7433570642cf4d8a4db8	507043942
[X]	AOD.017165._00004.pool.root.1	D4E6C456-A294-DC11-A13D-0019B9CC75B0	md5:ca87dedda7338567b510ed979459f897	505609154
[X]	AOD.017165._00002.pool.root.1	EA32AB5D-A394-DC11-8078-0019B9C34395	md5:094684b53892d917d1e599df92b8d31d	495232880
[X]	AOD.017165._00003.pool.root.1	6897F8FF-A394-DC11-9EAC-0019B9C34395	md5:3f71ae1f456154fae2961b98b89a1752	519420794
[X]	AOD.017166._00008.pool.root	D2FFC61E-7194-DC11-AA74-001B243DE654	md5:2b066eab31c3ea0680b6e0dcfb40b934	511945619
[X]	AOD.017166._00006.pool.root	8ECE7B47-6894-DC11-8FC5-001B243E0CE2	md5:98fa0f8674864b50770aca74f878ddc9	493231472
[X]	AOD.017165._00005.pool.root.1	A27104CE-A194-DC11-9632-0019B9C32FA1	md5:45ef08e650b617176404a98992c58667	500733732
[X]	AOD.017165._00001.pool.root.1	405512AD-B594-DC11-874D-0019B9C340D1	md5:08e72bf2381cc6b57023adbd9fcf94ae	522281766
[X]	AOD.017166._00010.pool.root	A09EA275-6D94-DC11-A9A3-001B242DBF66	md5:15114f708f819291593d5cc9cf5a30ea	521252543
total files: 10
local files: 10
total size: 5084623823
date: 2008-04-07 04:15:56

Then back on grid1: 

rwg@uct2-grid1:~$ /usr/local/bin/lrc_delete_dataset_site.py user.RobertWGardnerJr.Sox186_shadow MWT2_UC
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1653    0    59  100  1594   1247  33708 --:--:-- --:--:-- --:--:--  778k
deleted
rwg@uct2-grid1:~$

Was the dataset deleted? 

rwg@uct3-edge5:~$ dq2-ls -L MWT2_UC -f user.RobertWGardnerJr.Sox186_shadow

user.RobertWGardnerJr.Sox186_shadow
[X]	AOD.017166._00009.pool.root	E6B71543-6E94-DC11-85AE-001B243E0BA6	md5:9731c5d1d2bfae57b403c60149dd4b66	507871921
[X]	AOD.017166._00007.pool.root	DA0CCA64-6D94-DC11-BDFA-001B243DEAC8	md5:5002751b221d7433570642cf4d8a4db8	507043942
[X]	AOD.017165._00004.pool.root.1	D4E6C456-A294-DC11-A13D-0019B9CC75B0	md5:ca87dedda7338567b510ed979459f897	505609154
[X]	AOD.017165._00002.pool.root.1	EA32AB5D-A394-DC11-8078-0019B9C34395	md5:094684b53892d917d1e599df92b8d31d	495232880
[X]	AOD.017165._00003.pool.root.1	6897F8FF-A394-DC11-9EAC-0019B9C34395	md5:3f71ae1f456154fae2961b98b89a1752	519420794
[X]	AOD.017166._00008.pool.root	D2FFC61E-7194-DC11-AA74-001B243DE654	md5:2b066eab31c3ea0680b6e0dcfb40b934	511945619
[X]	AOD.017166._00006.pool.root	8ECE7B47-6894-DC11-8FC5-001B243E0CE2	md5:98fa0f8674864b50770aca74f878ddc9	493231472
[X]	AOD.017165._00005.pool.root.1	A27104CE-A194-DC11-9632-0019B9C32FA1	md5:45ef08e650b617176404a98992c58667	500733732
[X]	AOD.017165._00001.pool.root.1	405512AD-B594-DC11-874D-0019B9C340D1	md5:08e72bf2381cc6b57023adbd9fcf94ae	522281766
[X]	AOD.017166._00010.pool.root	A09EA275-6D94-DC11-A9A3-001B242DBF66	md5:15114f708f819291593d5cc9cf5a30ea	521252543
total files: 10
local files: 10
total size: 5084623823
date: 2008-04-07 04:15:56
rwg@uct3-edge5:~$

Delete with verbose - output: 
rwg@uct2-grid1:~$ /usr/local/bin/lrc_delete_dataset_site.py -v user.RobertWGardnerJr.Sox186_shadow MWT2_UC
https://uct2-grid1.uchicago.edu:8443/dq2/lrc/deletePFNs
pfns=srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017166._00010.pool.root+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017165._00002.pool.root.1+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017165._00005.pool.root.1+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017166._00006.pool.root+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017166._00008.pool.root+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017165._00003.pool.root.1+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017165._00001.pool.root.1+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017165._00004.pool.root.1+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017166._00007.pool.root+srm://uct2-dc1.uchicago.edu/pnfs/uchicago.edu/data/ddm1/valid2_valid1/AOD/valid2_valid1.005403.SU3_jimmy_susy.recon.AOD.v13003003/AOD.017166._00009.pool.root
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  1653    0    59  100  1594   1084  29311 --:--:-- --:--:-- --:--:--  518k
Error. Can only delete user datasets. or root daemon files

Note: lrc_delete_dataset_site.py has been renamed to delete_user_dataset.py, please see http://repo.mwt2.org/viewvc/lrc-adler32/

Dataset deletion questions

  • How to verify dataset was actually deleted (if it was)?
  • Tool should give intuitive feedback, like a list of files deleted.
  • How to provide tool as a package to users?

-- RobGardner - 16 Mar 2009
Edit  | Attach | Print version | History: r1 | Backlinks | View wiki text | Edit wiki text | More topic actions
Topic revision: r1 - 16 Mar 2009, RobGardner
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback