DataManagement

Checking unfinished runs

The run is declared finished by the online data mover once all the files have been transfered. it is good to check from time to time that no runs are left behind:

We start from the Reconstruction production (here 67722)

[localhost] ~ $ dirac-bookkeeping-run-information --Prod 67722 --Info Finished --ByValue
Found 347 runs
Successful :
      No : (36 runs) - 201402,201403,201640,201971,201972,202804,202805,202821,[..... more that are active...]

You can also give a run range

[LHCbDirac prod] SRPM $ dirac-bookkeeping-run-information 205957:212325 --Information Finished --ByValue
Successful :
     No : (1 runs) - 212325
    Yes : (2129 runs) - 205957,205958,[...]

A bit of cleaning

From StorageUsagePlot

From the StorageUsage plots, we can see that there are sometimes files left on BUFFER after a production has been finished.

To find them

[lxplus021] ~ $ dirac-dms-find-lfns --Path /lhcb/data/2015/RAW/SMOGPHY/LHCb/LEAD15/ --SE Tier1-Buffer | dirac-dms-replica-stats
Got 3 LFNs
[...]

Very often, it is because the run has been flagged BAD. This can be checked as follows:

dirac-bookkeeping-file-path  --GroupBy dataqualityflag --Summary <LFNS>

We make sure they were not processed before removing them

[lxplus021] ~ $ dirac-bookkeeping-get-file-descendants --Last
Got 3 LFNs
Getting descendants for 3 files (depth 1) : completed in 0.3 seconds
NotProcessed :
    /lhcb/data/2015/RAW/SMOGPHY/LHCb/LEAD15/169028/169028_0000000546.raw
    /lhcb/data/2015/RAW/SMOGPHY/LHCb/LEAD15/169030/169030_0000000354.raw
    /lhcb/data/2015/RAW/SMOGPHY/LHCb/LEAD15/169034/169034_0000000323.raw
[lxplus021] ~ $ dirac-dms-remove-replicas --Last --SE Tier1-Buffer
Got 3 LFNs
Removing replicas : completed in 8.6 seconds
Successfully removed 3 replicas from IN2P3-BUFFER

Files unused in productions

If a run is flagged BAD during the processing, some files may have been added to a production, but then never get used. A very similar process can be done. In case of stripping, we can also go up to the parent files to remove the RA files from buffer:

[localhost] ~ $ dirac-transformation-debug 71500 --Status Unused --Info files | dirac-bookkeeping-get-file-ancestors | dirac-dms-replica-stats
Getting ancestors for 41 files (depth 1) : completed in 12.0 seconds
Got 108 LFNs
Getting replicas for 108 LFNs : completed in 5.8 seconds
108 files found with replicas

Replica statistics:
  0 archive replicas: 108 files
---------------------
  0  other  replicas: 0 files
  1  other  replicas: 41 files
  2  other  replicas: 67 files
---------------------

SE statistics:
        CERN-RAW: 67 files
        CNAF-RAW: 67 files
       CNAF-RDST: 41 files

Sites statistics:
   LCG.CERN.cern: 67 files
     LCG.CNAF.it: 108 files
[localhost] ~ $ dirac-bookkeeping-file-path --Last --GroupBy dataqualityflag --Summary
Got 108 LFNs
Successful :
    DataqualityFlag BAD : 108 files
[localhost] ~ $ dirac-transformation-reset-files --New Removed --Last 71500
Got 108 LFNs
41 files were set Removed in transformation 71500

Files problematic in productions

When a file goes problematic in a production, it can be removed from buffer. If it is for the stripping, also its raw ancestor can be removed. Example for a Stripping (here, all the prods for a given Stripping)

[localhost] ~ $ dirac-transformation-debug 69077,69073,68675,68486,69079,69075,68773,68771 --Status Problematic --Info files | dirac-dms-replica-stats
Got 28 LFNs
Getting replicas for 28 LFNs : completed in 4.7 seconds
28 files found with replicas

Replica statistics:
  0 archive replicas: 28 files
---------------------
  0  other  replicas: 0 files
  1  other  replicas: 0 files
  2  other  replicas: 28 files
---------------------
[...]


[localhost] ~ $ dirac-bookkeeping-get-file-ancestors --Last | dirac-dms-remove-replicas --SE Tier1-Buffer
Getting ancestors for 28 files (depth 1) : completed in 6.5 seconds
Got 56 LFNs
Removing replicas : completed in 228.5 seconds
Successfully removed 12 replicas from CERN-BUFFER
Successfully removed 2 replicas from SARA-BUFFER
Successfully removed 6 replicas from RRCKI-BUFFER
Successfully removed 4 replicas from GRIDKA-BUFFER
Successfully removed 8 replicas from IN2P3-BUFFER
Successfully removed 24 replicas from RAL-BUFFER

We can then set these files as Removed in the removal transformation (setting them Done would not be very clean…)

[localhost] ~ $ dirac-transformation-reset-files --NewStatus Removed --Last 69128,69127,68831,68829
Got 56 LFNs
6 files were set Removed in transformation 69128
7 files were set Removed in transformation 69127
8 files were set Removed in transformation 68831
7 files were set Removed in transformation 68829

From DEBUG

Here we can just clean regularly what is in debug…

dirac-dms-list-directory --Dir /lhcb/debug/ --Rec --Days 7 --NoDirectories | dirac-transformation-add-files Remove-all-replicas