Black list for test cases

[jycamus90]

Now, there is no way that a user can specify the list of test cases that she wants Tacoco to ignore while running. It would be nice if a user can specify the test case black list in Tacoco config file so Tacoco can automatically ignore the specified test cases.

[reetasingh]

Hi J.Y,
Can you specify what the input will be? list of test classes or list of test methods

[jycamus90]

I am thinking of having users specify test case(s) in config file so that Tacoco can ignore them during the execution. For example, adding another option like -Dtacoco.ignore and pass a value that consists of class path + test case name (i.e. org.spideruci.tacoco.TacocoAnalyzer.testCase - this doesn't exist. I'm just giving you an example of a format like test.class.path.testcasemethodname).

[VijayKrishna]

@jycamus90 - when you say test case, do you mean test class or test method?

[jycamus90]

test method. sorry for the confusion.

[VijayKrishna]

@jycamus90 - it might be tricky to actually block the execution of a specific test method, because the Analyzer will run on the entire test class, not on individual test methods. My point is that ignoring test classes will be easier, and totally do-able. But i am not sue exactly how you would block the execution of a single test method. Any ideas?

[jycamus90]

I thought we could maintain a blacklist list like in Blinky and when JUnitRunner.java iterates through test methods in a test class in shouldRun() method, we could search through the list to check that the list contains that test method's name. Please advise me if I'm missing something or wrong.

[VijayKrishna]

We can maintain a list of methods (with their pull class+package name), but how/where exactly would you prevent the execution of those test methods?

[jycamus90]

Oh okay, it seems like that Junit only supports running all test cases in class situation. Hmmm let me re-think about this issue.

[VijayKrishna]

I guess that we could still have an ignore list at a class-level granularity, i.e. ignore an entire test class. And if tomorrow, we are able to figure this out for methods, then we can extend this approach to ignore test methods as well.

[reetasingh]

I was looking through the code and see that PITHandler which has the run method for PITAnalyzer already has some exclusionTest method. any idea what is it for?

[jycamus90]

That is for the mutation testing using PIT :)

[VijayKrishna]

@reetasingh i say that you go ahead with implementing this feature assuming that the input will be an ignore-list of test-classes. We can extend it to specific method later.

[reetasingh]

@VijayKrishna
Yes. I have looked at the code and have rough idea about how to go about it. Lets review that tomorrow.