Data Popularity

Whenever production data is accessed by a user job, we keep that information in order to estimate the popularity of the dataset. This is useful for various purposes:

• Freeing disk space
• Making data set consistent in terms of number of replicas
• Produce plots for the RRB

• Components
  • StorageUsageDB
  • StorageUsageAgent
  • StorageHistoryAgent
  • DataUsageHandler
  • PopularityAgent
  • DataPop server
  • PopularityAnalysisAgent
• Popularity scanning
• popularity.csv file
  • Week number
• Popularity Analysis
  • Spreadsheet setup
  • Spreadsheet content