Skip to content

Adapter only identifies tests in 3 of 5 of my exe files #28

Closed
@loki980

Description

@loki980

We just started using google test a few weeks back. We'd been working with the command line output only. We found this extension and it seems to work correctly, but it only finds tests in 3 of the 5 exes I have implementing tests.

I started going through GoogleTestDiscoverer.cs to see if I can figure out what's wrong, but it takes the entire binary exe file as an argument? Can someone help me figure out how to start debugging this?

Thanks.

Activity

loki980

loki980 commented on Feb 17, 2016

@loki980
Author

Each exe I have correlates to testing one class. We're using IBM Rational Rhapsody for development. The class tests that fail are classes that implement Rhapsody statecharts.

I stripped the files down to nothing and they still failed. The failure occurs when the class I'm testing inherits from OMReactive

I changed:
class shutdownMachine : public OMReactive, public iShutdownAck, public iShutdownRequest {
to:
class shutdownMachine : public iShutdownAck, public iShutdownRequest {

After this, my test is discovered just fine.

Any idea why this is occurring or how I can debug this?

loki980

loki980 commented on Feb 18, 2016

@loki980
Author

OK, who knew I'd go down this rabbit hole. Only been using VS a week. Debugging Visual Studio within Visual Studio is not where I thought I'd end up.

I've been able to attach to the test discovery process and I see the array returned with and without the successfully scraped stdout. I'll update this after I continue debugging tomorrow.

Very nice readme by the way.

csoltenborn

csoltenborn commented on Feb 18, 2016

@csoltenborn
Owner

Thanks for the kudos for the readme! Off the top of my head, I don't have any idea why your tests wouldn't be discovered... Have you seen the "How to debug" section of the readme? Attaching to the discovery process should be rather easy if using that MS extension.

Despite that, I guess we need some minimal example to help you out... If you cannot provide something like that, debugging is probably up to you :-) Here's what I would check: The tests are discovered by means of Dia, which allows to search executables for e.g. method signatures (as you probably have figured out already). We use a simple filter to get only the methods corresponding to actual tests (i.e., *::TestBody), and my first guess is that for some reason, that filter doesn't catch your tests. You could pass a different filter (i.e., *) into the DiaResolver.GetFunctions() method, put a breakpoint and investigate all methods found by Dia - that should help figure out why your tests are not found by the filter we use.

csoltenborn

csoltenborn commented on Feb 18, 2016

@csoltenborn
Owner

Here's one more hint: If you manage to add your code to the SampleTests solution, it should be rather easy to add an according unit test (see GoogleTestDiscovererTests). This allows you to debug the adapter code without leaving your one instance of Visual Studio... And, btw, if this is a bug, and if you are willing to contribute and it's legally possibly to do so, we would happily add your test(s) to our test suite to make sure the bug is gone for good.

loki980

loki980 commented on Feb 18, 2016

@loki980
Author

When executed from visual studio, the .dll my executable relies on (where the OMReactive class is found) is dynamically linked without issues (my environment in project properties, configuration properties, debugging is modified to append this location to PATH). When executed from ProcessLauncher::LaunchProcess, it cannot find this dll and the program fails saying so. I added the location of the dll to my path in windows and the extension finds everything as expected and can execute all my tests.

If you can figure out how to launch the process using the project's environment path, this would resolve this without having to manually register the dll or add it to our system path.

Thanks for your help and thanks for this extension!

csoltenborn

csoltenborn commented on Feb 18, 2016

@csoltenborn
Owner

Thanks for debugging the issue and identifying the root cause! We will certainly look into this...

csoltenborn

csoltenborn commented on Feb 18, 2016

@csoltenborn
Owner

I did some quick research: It probably is rather difficult to figure out the project's path, since discovery as well as execution run in different processes as VS itself, which means that (as far as I know) there is no clean way to access the VS instance.

However, there might be another approach: My guess is that if you enable "Copy local" in the properties of your references (which makes sure they end up in the same dir as the executable), they will be found by the ProcessLauncher. I'm not sure whether this is acceptable for you (although I think that this is a quite common project setup, but I'm far from being an expert in that area) - at least you don't need to change your test code or even the system path. Can you give this a try?

loki980

loki980 commented on Feb 19, 2016

@loki980
Author

The dll isn't added as a reference. It's just added to the project's "configuration properties"->"linker"->"input"->"Additional Dependencies" line, so this won't work for me unless I change a bunch of things around. FYI - I'm on holiday for a week and this is the last thing I'm doing before turning off my PC, so I won't respond to anything for a while. Thanks again.

csoltenborn

csoltenborn commented on Mar 5, 2016

@csoltenborn
Owner

@loki980 We have a solution for your problem (at least I hope so): We have added a new option "Path extension", the content of which will be appended to the PATH of the test executables. This way you should be able to add your additional dependencies to the test project without altering the system's PATH variable (and maybe even share these settings with your fellow developers).

You can download a VSIX containing the patch here - can you please give it a try and let us know if this works for you?

self-assigned this
on Mar 5, 2016
loki980

loki980 commented on Mar 7, 2016

@loki980
Author

Thanks! I tested out the build - it does work. Adding the path allows discovering and running the tests. Removing causes the extension to basically do nothing. Perhaps you should catch the exception and give some feedback when the exe dies. It currently makes it look like your extension is broken, even though it's an issue with the exe.

csoltenborn

csoltenborn commented on Mar 7, 2016

@csoltenborn
Owner

Which exception are you talking about? I wrote a unit test for the feature and tried to first make sure that tests are indeed not found when the DLL path is not configured as described, but was confronted with a windows dialog pointing out that there's a DLL missing (and it seems there is no way to prevent that dialog from showing up).

Is that the dialog you are talking about? If not, please let me know which exception I can catch and how that exception is propagated to the end user...

6 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @csoltenborn@loki980

      Issue actions

        Adapter only identifies tests in 3 of 5 of my exe files · Issue #28 · csoltenborn/GoogleTestAdapter