Uploaded image for project: 'CORAL and COOL'
  1. CORAL and COOL
  2. CORALCOOL-2357

Atlas workload tests for COOL data retrieval (end 2005)

    Details

    • Type: Task
    • Status: Closed (View Workflow)
    • Priority: High
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: COOL
    • Labels:
      None

      Description

      Following are COOL read workload use cases, as suggested by Richard Hawkings and summarized by David Front:

      Assumptions:
      - DCS data is updated once every 5-10 minutes, hence 'DCS IOV duration' is 5-10 minutes.
      - The amount of DCS data in each IOV duration is 10-100 MB.

      Use case 1: 'Monitoring':
      Goal: combined reading rate should be as fast as insertion rate.

      Test: A few (10) COOL reading clients, each reading a subset of the folders, read all data of the latest IOV once every IOV duration, for all channelIds.

      Use case 2: 'Reconstruction':
      Goal: (1000) Reconstruction processes, one starting every 5 seconds, all running on 100 machines, may read the whole DCS data of the latest IOV, within a non-growing time period (of a few minutes).
      (Since a process starts every 5 seconds, and IOV duration is 5-10 minutes, 60-120 processes read the same COOL data).

      Test: Start a process every 5 seconds. Read X amount of data, where X starts from 1MB.
      Measure COOL read time. Grow X by 1 MB, until reading time exceeds a predefined limit (for example: 10 minutes).
      Once an optimal X is found, continuously repeat test with this value for a few hours.

      Remarks:
      - Both tests should continuously run for a few hours, to check for memory leaks, slowing down etc.
      - Explore the use of the bulk retrieve over multiple channel IDs when it is available in COOL1-2-3.
      This should make this sort of query significantly more efficient.
      - For both tests, client and server machine and DB statistics should be collected to performance detect bottlenecks
      - Related item: 'overview: Scalability tests for COOL data retrieval performance.' https://savannah.cern.ch/task/?func=detailitem&item_id=2398

        Attachments

          Activity

            People

            • Assignee:
              dfront David Front
              Reporter:
              dfront David Front
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: