[ROOT-6865] 400 MB memory bloat caused by non-locally built allDict.cxx.pch Created: 31/Oct/14  Updated: 27/Aug/17  Resolved: 13/Nov/14

Status: Closed
Project: ROOT
Component/s: Cling
Affects Version/s: 6.02/01
Fix Version/s: None

Type: Bug Priority: High
Reporter: William Tanenbaum Assignee: Axel Naumann
Resolution: Fixed Votes: 0
Labels: None
Environment:

gcc4.8.1 SL6


Attachments: HTML File fl     File igreportbloated.res.gz     File igreportgood.res.gz    
Development:

 Description   

Using an installation of ROOT built on a different system can cause memory bloat. The problem has been traced to alldict.cxx.pch.
CMS sees a 400MB increase in memory usage, which goes away when alldict.cxx.pch is built locally.



 Comments   
Comment by Axel Naumann [ 01/Nov/14 ]

Hi Bill,

This should now be fixed: all combinations of mismatches (location, modification time, size) are now bridged between file system and PCH. Could you verify that this indeed git fixed?

Thanks for all your help yesterday!

Cheers, Axel.

Comment by Axel Naumann [ 01/Nov/14 ]

To be more explicit what you need to test on your side: the patches are in v6-02-00-patches and in the master. They replace the set of patches you received from me yesterday. You should be able to cherry pick to test this on top of the v6-02-00-patches commit the CMS nightly uses.

* 94a0cc3 (HEAD, origin/v6-02-00-patches, v6-02-00-patches) Force the PCH to provide the header even if date/size are off (ROOT-6865).
* b762aac When matching files to PCH, ignore leading "./" which can come from -I. (ROOT-6865)
* 4acd15e Map name as known by PCH to name on disk. (ROOT-6865)

Comment by William Tanenbaum [ 01/Nov/14 ]

I used the new HEAD of the v6-02-00 branch, rather than cherry picking the commits. I made sure LLVM was rebuilt. I still see the 400 MB of bloat when I use the non-locally built PCH.
Of course, the non-locally built PCH is from a different commit of ROOT.
We will advance the IB to the new head of the patches branch, and see what happens.
Bill

Comment by William Tanenbaum [ 01/Nov/14 ]

The latest output of gROOT->ProcessLine(".files") is in attachment "fl".

Comment by Axel Naumann [ 01/Nov/14 ]

Thanks, Bill!

That's good news! Now only 24 ROOT and 27 stdlib headers are parsed, and none of them are in the PCH, so that's expected. This should not account for 400MB anymore. It it does there must be a different source.

Cheers, Axel.

Comment by Axel Naumann [ 03/Nov/14 ]

Hi Bill,

Could you send a memory profile (massif, igprof - whatever you prefer) to allow me to track down where those 400MB still hide?

Axel.

Comment by William Tanenbaum [ 03/Nov/14 ]

igreportbloated.res.gz is the (gzipped) igprof report with the 400MB bloat.
igreportgood.res.gz is the (gzipped) report without the bloat (locally built pch used).
The difference seems to be related to clang::FileManager::getFile(), which is [39] in the bloated report and [531] in the unbloated one.

Comment by Axel Naumann [ 13/Nov/14 ]

Hi,

Should be fixed by 673f438 in v6-02-00-patches. Bill, could you (again! sorry...) verify, please?

Thanks for all your help!

Cheers, Axel.

Comment by William Tanenbaum [ 13/Nov/14 ]

Yes, it's fixed!

Bill

Comment by Axel Naumann [ 13/Nov/14 ]

Thank you for the confirmation, Bill!

Generated at Wed Sep 18 15:53:56 CEST 2019 using Jira 7.13.1#713001-sha1:5e06076c2d215a6f699b7e5c90ab2fae7ba5a1ce.