Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-874

Magic Symlinks Break on Catalog Creation

    Details

    • Type: Bug
    • Status: Closed
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: CernVM-FS 2.1.20
    • Fix Version/s: CernVM-FS 2.2.0
    • Component/s: CVMFS
    • Labels:
      None
    • Platforms:
      x86_64-slc6-gcc48-opt

      Description

      CMS reported that a magic symlink (includes expandable environment variables) in their repository mysteriously became empty. Recreating the symlink's content worked fine and fixed the problem.

      Further investigation shows that the symlink must have been corrupted when it's parent directory was moved into a nested catalog. When a nested catalog is created from an existing directory structure, it needs to retain all this existing directory entry information. Unfortunately the sync routine tries to expand the magic symlink and thus corrupts it due to missing environment variables.

      This needs one or two regression tests (magic symlink handling in general and in connection with nested catalog mangling) and a proper fix.

        Attachments

          Activity

          Hide
          straylen Steve Traylen added a comment - - edited

          The bug was in 2.1.19 as well? i.e was the upgrade the trigger or it was just the re-org of files.

          Show
          straylen Steve Traylen added a comment - - edited The bug was in 2.1.19 as well? i.e was the upgrade the trigger or it was just the re-org of files.
          Hide
          jblomer Jakob Blomer added a comment -

          Most likely the hasty reorganization.

          Show
          jblomer Jakob Blomer added a comment - Most likely the hasty reorganization.
          Hide
          rmeusel Rene Meusel (Inactive) added a comment - - edited

          git blame-ing the responsible code path, I would assume that bug is undiscovered since at least 2013.

          Show
          rmeusel Rene Meusel (Inactive) added a comment - - edited git blame -ing the responsible code path, I would assume that bug is undiscovered since at least 2013.
          Hide
          rmeusel Rene Meusel (Inactive) added a comment -

          Potential related problem: The server mount of CVMFS (/var/spool/.../rdonly) resolves those magic symlink variables if they are set or they become crippled otherwise like so:

          $ cvmfs_server transaction
          $ ln -s '$(foobar)' /cvmfs/test.local/foo
          $ ls -l /cvmfs/test.local
          [...] test -> $(foobar)
          $ cvmfs_server publish
          [...]
          $ ls -l
          [...] test -> 

          Technically this is not a problem, but the displayed symlink doesn't reflect what is stored in the catalogs. Touching this now crippled magic symlink in the next transaction might cause similar problems as described in the ticket initially.

          I suggest to add a client config variable to disable the resolving of those magic symlinks. In this case the client should just report them as-is in the catalogs. This variable could be used for the server's mount both for UX and safety reasons. Thoughts?

          Show
          rmeusel Rene Meusel (Inactive) added a comment - Potential related problem: The server mount of CVMFS ( /var/spool/.../rdonly ) resolves those magic symlink variables if they are set or they become crippled otherwise like so: $ cvmfs_server transaction $ ln -s '$(foobar)' /cvmfs/test.local/foo $ ls -l /cvmfs/test.local [...] test -> $(foobar) $ cvmfs_server publish [...] $ ls -l [...] test -> Technically this is not a problem, but the displayed symlink doesn't reflect what is stored in the catalogs. Touching this now crippled magic symlink in the next transaction might cause similar problems as described in the ticket initially. I suggest to add a client config variable to disable the resolving of those magic symlinks. In this case the client should just report them as-is in the catalogs. This variable could be used for the server's mount both for UX and safety reasons. Thoughts?
          Hide
          jblomer Jakob Blomer added a comment -

          Sounds good to me. We can actually allow for an option that only shows the raw link if the environment variable is undefined. Please make another ticket for this and assign it to me.

          Show
          jblomer Jakob Blomer added a comment - Sounds good to me. We can actually allow for an option that only shows the raw link if the environment variable is undefined. Please make another ticket for this and assign it to me.
          Hide
          rmeusel Rene Meusel (Inactive) added a comment -

          Done in CVM-879.
          However, for the server's CVMFS client I would like to disable the resolving entirely because environment variables might actually be defined and the magic symlinks would potentially become resolved (and therefore lose their 'magic') in a very similar way as stated above.

          Show
          rmeusel Rene Meusel (Inactive) added a comment - Done in CVM-879 . However, for the server's CVMFS client I would like to disable the resolving entirely because environment variables might actually be defined and the magic symlinks would potentially become resolved (and therefore lose their 'magic') in a very similar way as stated above.
          Hide
          rmeusel Rene Meusel (Inactive) added a comment -

          Pull Request containing a fix and regression test for this issue.

          Show
          rmeusel Rene Meusel (Inactive) added a comment - Pull Request containing a fix and regression test for this issue.
          Hide
          rmeusel Rene Meusel (Inactive) added a comment -

          The discussed follow-up issue resulting in bogus reproduction of magic symlinks in the server is tracked in CVM-879.

          Show
          rmeusel Rene Meusel (Inactive) added a comment - The discussed follow-up issue resulting in bogus reproduction of magic symlinks in the server is tracked in CVM-879 .

            People

            • Assignee:
              rmeusel Rene Meusel (Inactive)
              Reporter:
              rmeusel Rene Meusel (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: