Uploaded image for project: 'CernVM'
  1. CernVM
  2. CVM-1920

Fallback proxy order applied too late when squid not functioning



    • Improvement
    • Status: Closed
    • Medium
    • Resolution: Completed
    • CernVM-FS 2.7.4
    • CernVM-FS 2.8
    • CVMFS
    • None
    • ANY


      I came across a case where CVMFS_HTTP_PROXY was configured to use a non-functioning squid, and even though it was a client in the middle U.S., it used the CERN backup proxies. I was able to reproduce it on my test machine. cvmfs_talk proxy info showed that the backup proxies were properly sorted such that FNAL backup came first, and it was the assigned proxy group, yet /var/log/messages showed it connecting to CERN:

      Load-balance groups:
      [0] (squid.fnal.gov, +5h)
      [1] (cmsextproxy.fnal.gov, +5h), (cmsextproxy.fnal.gov, +5h)
      [2] (cmsextproxy.cern.ch, +53s), (cmsextproxy.cern.ch, +53s)
      Active proxy: [1]
      First fallback group: [1]

      Oct  7 14:41:38 fermicloud327 cvmfs2: (config-osg.opensciencegrid.org) switching proxy from to
      Oct  7 14:41:38 fermicloud327 cvmfs2: (config-osg.opensciencegrid.org) geographic order of servers retrieved from cvmfs-s1bnl.opensciencegrid.org
      Oct  7 14:41:38 fermicloud327 cvmfs2: (config-osg.opensciencegrid.org) CernVM-FS: linking /cvmfs/config-osg.opensciencegrid.org to repository config-osg.opensciencegrid.org

      The problem is that it needs to switch to the fallback proxy to get the geo order, and at that point it couldn't yet sort them so it takes them in the order in which they are defined in the config file. Then after it has retrieved the order, it does not right away reset to use the sorted order.

      I guess that after 5 minutes when it retries the original squid it will at that point switch to the closer backup proxy.

      As a workaround, I plan to change the fallback proxy order in the OSG config repo, but that's not a complete solution. For one, it doesn't affect the config repo, and it takes a long time for everybody to upgrade that config rpm. Also, there are some users of the OSG config outside of North America.




            dwd Dave Dykstra
            dwd Dave Dykstra
            0 Vote for this issue
            2 Start watching this issue