Setup question - source has changed but `make` is not run

[dabell-cc]

Hi there,
Quite novice to CppUTest and this adapter, please bear with me - it just seems I'm missing something.

I have the Test Executable and Test Executable Path configured in the extension settings like this:

"cpputestTestAdapter.testExecutable": "${workspaceFolder}/t/Example_tests",
"cpputestTestAdapter.testExecutablePath": "${workspaceFolder}/t/"

Now when I change my source I don't see the test result change in the VS Code Testing panel, even though I've intentionally broken it. Example:

int test_func(void)
{
   return 0; //was supposed to return 1
}

I take this to be because this extension is not re-running make, so the production code change isn't being picked up.
What I'm left wondering is how this extension is meant to be used for maximum benefit.

Otherwise my process would look like:

• Save change in source or tests,
• run make (can be linked to the save keybind which makes this one step), - at this point I have shell output for pass/fail
• click 'Run all tests' in Test Explorer to see the test status change.

This sequence of steps does work for picking up changes, I just expected this to be more automatic. What am I missing here?

For info I'm using Ubuntu 14.04, VS Code 1.69.2, CppUTest latest.

[bneumann]

Hi, no this totally fine. There is no automatic rebuild or file watcher at the moment. That is because in C/C++ you have such a big variety of builders, toolchains, tools etc. that can be used to trigger a rebuild.
What I could think if is to have a VSCode task to run before executing the tests. That would be pretty easy to implement i guess and you won't need to support all the specifics inside the test runner tool.

When I was doing C programming I used to have a build task that could be trigger by using Ctrl+Shift+B or something and after the build i clicked run tests. So that was my workflow but I can understand that you aim for more automation.

I quickly browsed through the code and thought I actually added a file watcher on the executables to have at least a retire of tests if the test runner changes but I guess I forgot to implement that ;)

I think this is a good first issue for contributers as I will not be able to do much on the code in the next time. Maybe in winter :)

[dabell-cc]

Thanks for clarifying!
A build task makes sense.

I think in that case I'd modify my Makefile to stop short of running the tests.
If using the MakefileWorker.mk provided with CppUTest, the tests get run immediately after building.
That seems unnecessary if CppUTest-Test-Adapter is going to run the tests anyway.

[malsyned]

I've been working on this issue. I have the basic functionality in place of running a configured pre-launch task from updateTests() and waiting for it to complete before refreshing the test list.

I think if the pre-launch task fails (returns exit code != 0), the right thing to do would be to report that as a failing test somehow. Other kinds of failures result in a call to CreateTestSuiteError() which creates a kind of dummy test suite with a single test that says "ERROR LOADING TESTS". Is this how you'd want to handle pre-launch task errors as well?

[malsyned]

For example:

[bneumann]

Hey, that sounds great. I haven't been programming C for a while and was really busy so I unfortunately neglected this project a bit. Every help is highly appreciated.
I like it that the error is displayed in the test explorer. You have the console but honestly not everyone looks there if something is wrong so this is a good way to inform users.

[malsyned]

OK awesome!

I have it working pretty well, I'm polishing it up.

One issue I'm running into is that when first opening a CMake project, with cmake.configureOnOpen": true, there's a race that leads to an error popup, "Configuration is already in progress". The only way to solve this that I have found is to set cmake.configureOnOpen to false, and to make the CMake Build task in tasks.son depend on the CMake Configure task. It works, but it creates a little bit of latency in various operations.