Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-1113

Detect empty (corrupted) files on automatic rebuild of the cache db

    XMLWordPrintable

Details

    • Improvement
    • Status: Closed
    • Medium
    • Resolution: Fixed
    • CernVM-FS 2.3.2
    • CernVM-FS 2.3.4
    • CVMFS
    • None
    • x86_64-slc6-gcc48-opt

    Description

      We had a node with a large number of job failures, like this:
      http://bigpanda.cern.ch/job?pandaid=3106264518

      http://aipanda053.cern.ch/media/filebrowser/017142a0-8b48-45b1-9cc3-fee11070992e/panda/tarball_PandaJob_3106264518_ANALY_VICTORIA/athena_stdout.txt

      D3PD2MiniSL: error while loading shared libraries: /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so: file too short

      I found that the file /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so is shown as having size 663740 bytes, but when trying to read it, it is like reading an empty file. No IO errors are reported. It is just empty.

      [root@hermes-kvm007 ~]# ls -l /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so
      -rwxr-xr-x 1 cvmfs cvmfs 663740 Jul 9 2014 /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so
      [root@hermes-kvm007 ~]# md5sum /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so
      d41d8cd98f00b204e9800998ecf8427e /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so
      [root@hermes-kvm007 ~]# cp /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so /tmp/
      [root@hermes-kvm007 ~]# md5sum /tmp/libXMLIO.so
      d41d8cd98f00b204e9800998ecf8427e /tmp/libXMLIO.so
      [root@hermes-kvm007 ~]# ls /tmp/libXMLIO.so -lh
      -rwxr-xr-x 1 root root 0 Dec 2 12:29 /tmp/libXMLIO.so
      [root@hermes-kvm007 ~]# touch /tmp/empty
      [root@hermes-kvm007 ~]# md5sum /tmp/empty
      d41d8cd98f00b204e9800998ecf8427e /tmp/empty
      [root@hermes-kvm007 ~]# cat /cvmfs/atlas.cern.ch/repo/ATLASLocalRootBase/x86_64/root/5.34.19-x86_64-slc6-gcc4.7/lib/libXMLIO.so
      [root@hermes-kvm007 ~]#

      Attachments

        Activity

          People

            jblomer Jakob Blomer
            rptaylor Ryan Taylor
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:
              Actual Start:
              Actual End: