Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-1128

Misleading cache cleanup message?

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • High
    • Resolution: Fixed
    • None
    • CernVM-FS 2.3.3
    • CVMFS
    • None
    • CernVM-FS Test Sprint
    • x86_64-slc6-gcc48-opt

    Description

      The cache we use for data (osgstorage.org) repositories is very small, with CVMFS_QUOTA_LIMIT=1000. We do that because we didn't want to use up much extra space on systems but still wanted it to not interfere with the regular cache so we set CVMFS_CACHE_BASE=$CVMFS_CACHE_BASE/osgstorage. We figure there won't be a lot of reuse of that data, and that the data always comes from high speed servers so it's OK for it to be small. This ticket is because I've noticed on a live system every couple of minutes the following message in /var/log/messages:

      Dec  5 03:56:27 fnpc9074 cvmfs2: cleanup cache until 991216 KB are free

      That seemed odd, since 1000*1024 KB minus 991216KB is only 32MB, so I wondered why it was deleting almost the whole cache. I looked at the code of PosixQuotaManager::DoCleanup() and it looks to me like that parameter is the number of KB to leave, not to delete. Shouldn't that message then be something like "clean up cache until at most XXX KB is used", or else the amount should be subtracted from the total capacity?

      Assuming that's correct, why is it only deleting such a small percentage? Aren't there high and low water marks, so that when it gets over a certain percentage it deletes down to a lower percentage, so it doesn't have to clean up so frequently?

      Ah, this is probably a case of a > 25MB file, which is kBigFile. I see in PosixCacheManager::StartTxn()

          // Opportunistically clean up cache for large files
          if (size > kBigFile) {
            assert(quota_mgr_->GetCapacity() >= size);
            quota_mgr_->Cleanup(quota_mgr_->GetCapacity() - size);
          }

      so that would be happening with large files no matter how big the cache is.

      Attachments

        Issue Links

          Activity

            People

              jblomer Jakob Blomer
              dwd Dave Dykstra
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:
                Actual Start:
                Actual End: