Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-1625

Cache cleanup on big files suboptimal with larger chunk sizes

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Medium
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: CernVM-FS 2.5.2
    • Component/s: CVMFS
    • Labels:
      None
    • Platforms:
      x86_64-slc6-gcc48-opt
    • Development:

      Description

      Whenver a cvmfs client reads a file that is greater than kBigFile, currently set to 25MB, it makes sure that there is at least that much space before reading the file, running a small synchronous cache cleanup if needed. The problem with that is that if the file chunk size is bigger than that, and there are many large files in the repository bigger than that, there will be many of those small synchronous cleanups, and an asynchronous cleanup won't get triggered until the relatively rare times that smaller files get put into the cache. I think there should be some way around that, for example to turn kBigFile into a settable client parameter, or to encode the chunk size in the catalog and automatically increase that limit to be bigger than the chunk size, at least if the chunk size is reasonably sized compared to the size of the cache. Or perhaps there should be a way to trigger the asynchronous cleanup after a synchronous one, in order to clean up all the way down to the low water mark as if the synchronous cleanup hadn't happened.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jblomer Jakob Blomer
                Reporter:
                dwd Dave Dykstra
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:
                  PlannedEnd:
                  PlannedStart: