[ROOT-5698] "import ROOT" would return a success status code even if it fails? Created: 13/Nov/13  Updated: 19/Dec/15  Resolved: 16/Dec/15

Status: Closed
Project: ROOT
Component/s: PyROOT
Affects Version/s: 6.02.00
Fix Version/s: 6.06/00, 6.04/02, 6.08/00

Type: Bug Priority: Low
Reporter: Andrea Valassi Assignee: Axel Naumann
Resolution: Fixed Votes: 0
Labels: None
Environment:

root6 dev2


Development:

 Description   

Hi, this is a followup to
https://sft.its.cern.ch/jira/browse/ROOT-154?focusedCommentId=28658&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-28658

As mentioned there, a fatal_error "warning" was printed out, but the return code from pythos was actually 0 (success). Now the import ROOT succeeds (after fixing issues with relocatability I think), so I cannot tell if the status code returned on failure would be 0 or 1.

I file this just as a very low-priority reminder (and to offload issues from ROOT-154!).

Thanks
Andrea



 Comments   
Comment by Wim Lavrijsen [ 13/Nov/13 ]

Andrea,

not sure what you are asking for here ... what is the "failure"? ROOT can not control the exit status code of python: that is up to the user script. However, if the last statement in python failed with a python error, python will set the status code to something non-zero.

Thus, if "import ROOT" causes a python exception that in turn causes the script to exit, the status code is non-zero. That works, too.

If this 'a fatal_error "warning" was printed out' (could you be more specific?) is coming unexpectedly, it can not be captured or otherwise turned into a python exception. Although presumably most of the time something else in python will fail for the same reason.

Cheers,
Wim

Comment by Andrea Valassi [ 13/Nov/13 ]

Hi Wim, thanks for the followup.
Unfortunately this was long ago and I did not go into too much detail as
this was clearly not the main issue. Anyway, from my link on ROOT-154,
this is what I wrote:

In file included from -:1:
input_line_7:106:10: fatal error: 'core/base/src/TListOfTypes.h' file not 
found
#include "core/base/src/TListOfTypes.h"
          ^
input_line_21:8:10: fatal error: 'core/meta/src/TCling.h' file not found
#include "core/meta/src/TCling.h"
          ^
Warning in <TClass::TClass>: no dictionary for class TCling is available

I guess the problem was that 'import ROOT' (or PyROOT? or cppyy??) seemed
to succeed (did not throw a python exception) even if actually it clearly
failed (the module was not available and the error above were printed).

Anyway, if you think this may have been fixed on the way (those cling
header issues are long gone), please just close this.

Thanks
Andrea

Comment by Wim Lavrijsen [ 13/Nov/13 ]

Andrea, those are a separate issue that is on the TODO list. In fact, there is a FIXCLING roottest for it.

In CINT, there was a message callback handler where I parsed the messages coming from CINT for specific patterns such as 'error'. I expect this to re-appear in Cling at some point but didn't think it important for the beta release.

Leaving it up on 'low' sounds about right.

Cheers, Wim

Comment by Andrea Valassi [ 06/Dec/13 ]

Hi, while doing some cleanup of my notes I realized that this is most likely one of the two issues I reported in ROOT-5759:

PyCintex.gbl.gROOT.ProcessLine('#include "does_not_exist.h"')  

prints an error message but does not throw an exception, hence the control flow continues and no error status is returned at the end.
I suggest we handle the wrong return code here in ROOT-5698, and the segfault in ROOT-5759.
Andrea

Comment by Wim Lavrijsen [ 06/Dec/13 ]

Andrea, yes and it never did throw an exception either.

As said above, in CINT there was an error handler and the error handler would receive the message. The default version would simply print it, but PyROOT replaced the default and that raised the python (not C++) exception.

I'll see whether there are any hooks already available today for use, otherwise I'll request one.

Cheers, Wim

Comment by Andrea Valassi [ 12/Dec/13 ]

Hi Wim, thanks. For the record, Vasil did fix the segfault in ROOT-5759. However ProcessLine still returns a wrong success status code from my simple example above

PyCintex.gbl.gROOT.ProcessLine('#include "does_not_exist.h"') 

So this ROOT-5698 is still pending.
Andrea

Comment by Wim Lavrijsen [ 02/Oct/14 ]

Andrea,

okay, but ProcessLine will return what ProcessLine returns. I'd never make then anything but that on the Python side.

I'll re-assign to Axel.

Cheers,
Wim

Comment by Axel Naumann [ 31/Oct/14 ]

Hi Andrea,

ProcessLine has an error parameter which you need to check. Not its return code, as e.g. ProcessLine("0") returns 0 and happiness: the result and the error state are different dimensions.

Cheers, Axel.

Comment by Andrea Valassi [ 31/Oct/14 ]

Hi Axel and Wim,
thanks for the clarifications.

Axel, I still have two doubts:

1. How do I check the error parameter you are talking about? Now I am actually using gInterpreter.Declare rather than gROOT.ProcessLine, but the pattern seems the same, I get no exception if I try to include a file that does not exist. Indeed it would help to add an explicit check inside PyCool.

2. Just for the record, the original reason why I filed this bug report was that "python -c 'import ROOT'" was succeding (as a linux command line) even if internally import ROOT was failing, most likely because of headers that could not be included. Can you confirm that this is now fixed, i.e. that if even "import ROOT" tried to include missing headers, this would fail?

Thanks
Andrea

Comment by Axel Naumann [ 02/Nov/14 ]

Declare() now returns true for success, false for failure.

I will leave the rest to Wim.

Axel.

Comment by Andrea Valassi [ 28/Nov/14 ]

Hi Axel,
I am doing some cleanup and tried to check the Declare() status. It is weird,

>>> import cppyy
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
input_line_27:1:10: fatal error: 'doesnotexist.h' file not found
#include "doesnotexist.h"
         ^
True
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
True

That is to say:

  • it always seems to return True
  • and it stops to complain (or do anything?) after the first time it was called.

This is on ROOT from the nightlies a few days ago, dev4/20141110Mon/ROOT/v6-02-00-patches/x86_64-slc6-gcc48-dbg.

Is this normal or does it need an additional fix?
Thanks
Andrea

Comment by Axel Naumann [ 03/Dec/14 ]

D'oh. Fixed in master and v6-02-00-patches.

The second issue (no more diagnostics for second inclusion) is now ROOT-6934.

Let me know if I can close this.

Cheers, Axel.

Comment by Andrea Valassi [ 09/Dec/14 ]

Hi Axel, thanks a lot for looking into this!
Indeed it is better, and ROOT-6934 is still open, but it looks a bit more complex. Let's say, the bit that is still open (ROOT-6934, or whatever left in this bug) is NOT only about disagnostics.

What is fixed is that including a file that exists returns true, if this is the first time you call Declare.

However, the moment you include a file that dies not exist, then you stop getting diagnostics (ROOT-6934) BUT you also stop getting correct results from Declare.

Try the following

import cppyy
cppyy.gbl.gInterpreter.Declare('#include <string>')
cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
cppyy.gbl.gInterpreter.Declare('#include <string>')
cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')

This gives

Python 2.7.6 (default, Dec  4 2014, 17:49:04) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cppyy
>>> cppyy.gbl.gInterpreter.Declare('#include <string>')
True
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
input_line_26:1:10: fatal error: 'doesnotexist.h' file not found
#include "doesnotexist.h"
         ^
False
>>> cppyy.gbl.gInterpreter.Declare('#include <string>')
False
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
False

Normally I would expect True from the second inclusion of <string>, right?

It seems that, once you return a False, Declare stops doing anything: it does not print warnings, but it aslo stops returning the right value.

Cheers
Andrea

Comment by Andrea Valassi [ 04/May/15 ]

Hi, for the record I confirm that this is fixed in master from dev3 today, but not in 6.02 patches from dev4.

In dev3 I now get

> python
Python 2.7.9 (default, Mar 27 2015, 16:10:08) 
[GCC 4.8.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cppyy
>>> cppyy.gbl.gInterpreter.Declare('#include <string>')
True
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
input_line_12:1:10: fatal error: 'doesnotexist.h' file not found
#include "doesnotexist.h"
         ^
False
>>> cppyy.gbl.gInterpreter.Declare('#include <string>')
True
>>> cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')
input_line_14:1:10: fatal error: 'doesnotexist.h' file not found
#include "doesnotexist.h"
         ^
False
>>> 

Cheers
Andrea

Comment by Axel Naumann [ 16/Dec/15 ]

Fixed in the relevant branches.

Comment by Andrea Valassi [ 19/Dec/15 ]

Hi Axel, thanks a lot. In the releases I confirm it was fixed between 6.02.12 in LCG78root6 and 6.04.02 in LCG79

[avalassi@aicoral61 bash] ~
> ROOTSYS=/afs/cern.ch/sw/lcg/releases/LCG_78root6/ROOT/6.02.12/x86_64-slc6-gcc48-opt PYTHONPATH=$ROOTSYS/lib LD_LIBRARY_PATH=/afs/cern.ch/sw/lcg/releases/gcc/4.8.4/x86_64-slc6/lib64:$ROOTSYS/lib:$LD_LIBRARY_PATH /afs/cern.ch/sw/lcg/releases/LCG_78root6/Python/2.7.9.p1/x86_64-slc6-gcc48-opt/bin/python -c "import cppyy; print cppyy.gbl.gInterpreter.Declare('#include <string>'); print cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"'); print cppyy.gbl.gInterpreter.Declare('#include <string>'); print cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')"
True
input_line_26:1:10: error: expected "FILENAME" or <FILENAME>
#include doesnotexist.h
         ^
False
False
input_line_30:1:10: error: expected "FILENAME" or <FILENAME>
#include doesnotexist.h
         ^
False
 
[avalassi@aicoral61 bash] ~
> ROOTSYS=/afs/cern.ch/sw/lcg/releases/LCG_79/ROOT/6.04.02/x86_64-slc6-gcc48-opt PYTHONPATH=$ROOTSYS/lib LD_LIBRARY_PATH=/afs/cern.ch/sw/lcg/releases/gcc/4.8.4/x86_64-slc6/lib64:$ROOTSYS/lib:$LD_LIBRARY_PATH /afs/cern.ch/sw/lcg/releases/LCG_79/Python/2.7.9.p1/x86_64-slc6-gcc48-opt/bin/python -c "import cppyy; print cppyy.gbl.gInterpreter.Declare('#include <string>'); print cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"'); print cppyy.gbl.gInterpreter.Declare('#include <string>'); print cppyy.gbl.gInterpreter.Declare('#include "doesnotexist.h"')"
True
input_line_11:1:10: error: expected "FILENAME" or <FILENAME>
#include doesnotexist.h
         ^
False
True
input_line_13:1:10: error: expected "FILENAME" or <FILENAME>
#include doesnotexist.h
         ^
False
 

Generated at Tue Sep 24 11:01:02 CEST 2019 using Jira 7.13.1#713001-sha1:5e06076c2d215a6f699b7e5c90ab2fae7ba5a1ce.