Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-1826

Make storage location for DUCC configurable



    • Improvement
    • Resolution: Completed
    • High
    • CernVM-FS 2.8
    • CernVM-FS 2.6.3
    • CVMFS
    • None
    • ANY


      By default it appears DUCC puts temporary files in /tmp/ .
      I could not find any option to control the location of temporary files, is there a way?
      The space usage could be around the size of the container, which could be ~> 10 GB for some containers.

      As a result, the /tmp/ (or / ) filesystem can get easily filled up , which can cause system problems. Moreover it can be quite a pain to allocate additional storage space in /tmp/ , since various applications rely on storing files there, so remounting a filesystem on /tmp can be disruptive to other applications or the system. Not only that , on the simple test system I set up , the /tmp filesystem is not getting automatically mounted on boot; I think it might be running into a complication related to systemd, since systemd services may need to write temporary files during bootup, potentially before other filesystems get mounted. Furthermore, usage of /tmp should generally be for only small files; some OSes use RAM for /tmp/ and it is not very suitable to consume multi GB of RAM.

      It is really necessary to be able to configure where DUCC writes these temporary files, whether by a CLI option, environment variable or something else. /tmp/ should not be used by default anyway - to adhere to the Linux FHS, something like /var/cache/cvmfs-ducc should be set up and used by the cvmfs-ducc package.

      In hindsight this may be the cause of the problems with DUCC I was having earlier - a few layers would get published successfully but then /tmp would fill up and the remaining layers failed (with the error message "failed to read data from the tar entry") , requiring multiple invocations to complete publishing of the image. Since it automatically cleaned up afterwards I did not notice the filesystem getting full.




            smosciat Simone Mosciatti (Inactive)
            rptaylor Ryan Taylor
            0 Vote for this issue
            3 Start watching this issue