Getting code coverage for a child process?

In this blog post, I’m going to explain how to collect code coverage for a process that is launched by another process. We’ll be using Coverage Validator to collect the code coverage.

For example, you may have a control process that launches helper programs to do specific jobs and you wish to collect code coverage data for one of the helper programs. I’m first going to show how you do this with the GUI, then I’ll show you how to do this with the command line.

For the purposes of this blog post, I’m going to use a test program called testAppFromOtherProcess.exe as the child program and testAppOtherProcessCpp.exe as the parent process. Once I’ve explained this for C++, I’ll also provide examples for programs launched from Java and for programs launched from Python.

The test program

The test program is simple. It takes two numbers and calculates the sum of all the products. If less than two arguments are supplied they default to 10.

int _tmain(int argc, _TCHAR* argv[])
{
	int	nx, ny;
	int	x, y;
	int	v;

	nx = 10;
	ny = 10;
	v = 0;

	if (argc == 2)
	{
		nx = _tcstol(argv[1], NULL, 10);
	}
	else if (argc >= 3)
	{
		nx = _tcstol(argv[1], NULL, 10);
		ny = _tcstol(argv[2], NULL, 10);
	}

	for(y = 0; y < ny; y++)
	{
		for(x = 0; x < nx; x++)
		{
			v += (x + 1) * (y + 1);
		}
	}

	return v;
}

The parent C++ program

The parent C++ program is a simple MFC dialog that collects two values and launches the test program. The code for launching the child process looks like this:

void CtestAppOtherProcessCppDlg::OnBnClickedOk()
{
	// get data values

	CString	str1, str2;
	DWORD	v = 0;

	GetDlgItemText(IDC_EDIT_COUNT1, str1);
	GetDlgItemText(IDC_EDIT_COUNT2, str2);

	// create command line

	CString	commandline;

	commandline += _T("testAppFromOtherProcess.exe");
	commandline += _T(" ");
	commandline += str1;
	commandline += _T(" ");
	commandline += str2;

	// run child process

	STARTUPINFO         stStartInfo;
	PROCESS_INFORMATION stProcessInfo;

	memset(&stStartInfo, 0, sizeof(STARTUPINFO));
	memset(&stProcessInfo, 0, sizeof(PROCESS_INFORMATION));

	stStartInfo.cb = sizeof(STARTUPINFO);
	stStartInfo.dwFlags = STARTF_USESHOWWINDOW;
	stStartInfo.wShowWindow = SW_HIDE;

	int	bRet;

	bRet = CreateProcess(NULL,
			(TCHAR *)(const TCHAR *)commandline,
			NULL,
			NULL,
			FALSE,
			0,
			NULL,
			NULL,
			&stStartInfo,
			&stProcessInfo);
	if (bRet)
	{
		// wait until complete then get exit code

		WaitForSingleObject(stProcessInfo.hProcess, INFINITE);

		GetExitCodeProcess(stProcessInfo.hProcess, &v);

		// tidy up

		CloseHandle(stProcessInfo.hProcess);
		CloseHandle(stProcessInfo.hThread);
	}

	// display result

	SetDlgItemInt(IDC_STATIC_VALUE, v, FALSE);
}

Configuring the target C++ program

Before we can collect code coverage we need to tell Coverage Validator about the target program and the program that is going to launch it. We do this from the launch dialog (or launch wizard). From the launch dialog, select the program to launch using the Browse… button and selecting the file with the File dialog. Once a file has been chosen a default value will be selected for the Application to Monitor. This is the same program as you just selected with the File dialog.

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit… button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

We need to add our target program to the list of programs to monitor. Click Add…. The Application To Monitor dialog is displayed. Choose our launch program testAppOtherProcessCpp.exe using Browse…. Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don’t want to monitor with the Remove and Remove All buttons. Your dialog should look like the one shown below.

Click OK to close the Application To Monitor dialog.

Click OK to close the Applications To Monitor dialog.

The Application to monitor combo will now have additional entries in it.

Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored.

Click Go! to start the parent process.

You will notice that Coverage Validator is not collecting data. Now click on the Launch Child Process button. The child process is launched, Coverage Validator recognises the parent process is launching a child process that is configured to be monitored and has the correct launch count (this is the first time it is being launched and the launch count is set to “1”) – the child process is instrumented for code coverage. You can see the instrumentation progress in the title bar and pretty soon code coverage statistics are being displayed by Coverage Validator.

Command Line, example for C++

OK, that’s wonderful, we can collect code coverage using the GUI to launch one program and collect data from a child process. All without any coding. Super. So how do we do that from the command line? Glad you asked!

"c:\Coverage Validator\coverageValidator.exe" -program "e:\test\release\testAppOtherProcessCpp.exe"
                                              -directory "e:\test\release" 
                                              -programToMonitor "e:\test\release\testAppFromOtherProcess.exe" 

How does this work?

• -directory. Specify the startup directory.
• -program. Specify the program to launch.
• -programToMonitor. Specify the program to that will be monitored for code coverage.

Very straightforward and simple. Paths must have quotes if they contain spaces. If in doubt always use quotes. Note also that where you’ve installed  Coverage Validator will be different, most likely in C:\Program Files (x86)\Software Verify. We shortened it for the example to make it fit the page.

Java

The parent program in Java is very simple. It takes any arguments passed to it and passes them to the target program.

import java.io.IOException;
import java.lang.ProcessBuilder;
import java.util.ArrayList;

public class testAppFromOtherProcessJava 
{
    public static void main(String[] args) throws IOException, InterruptedException
    {
        String            target = "e:\\om\\c\\testApps\\testAppFromOtherProcess\\Release\\testAppFromOtherProcess.exe";
        ProcessBuilder    p = new ProcessBuilder();

        // add the args to be passed to the target program, unlike C/C++, args[0] is not the program name

        ArrayList    targetArgs;

        targetArgs = new ArrayList();
        targetArgs.add(target);
        for(int i = 0; i < args.length; i++)
        {
            targetArgs.add(args[i]);
        }

        p.command(targetArgs);

        // run the process, wait for it to complete and report the value calculated

        Process    proc;

        proc = p.start();
        proc.waitFor();

        System.out.println("Result: " + proc.exitValue()); 
    }
}

You can compile this program with this simple command line. This assumes you have a Java Development Kit installed and javac.exe on the command line.

javac testAppFromOtherProcessJava.java

Configuring the target Java program

As with the C++ target program we need to tell Coverage Validator about the target program and the program that is going to launch it. We’re running a Java program so the executable to launch is the Java runtime. Click the Browse… button and select the Java runtime you are using.

The launch directory is automatically configured to be the same as the launch program. In the case of a Java program, that is almost certainly incorrect. We’re going to choose the directory where our Java class is located. Click the Dir… button and choose that directory.

We also need to tell the Java runtime what class to execute. This is provided as an argument to the program being run (the Java rutnime). In the arguments field, type the name of the class. In this case testAppFromOtherProcessJava (without the .class extension).

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit… button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

We need to add our target program to the list of programs to monitor. Click Add…. The Application To Monitor dialog is displayed. Choose the Java runtime java.exe using Browse…. Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don’t want to monitor with the Remove and Remove All buttons. We now need to add the target program to the list of programs we want to monitor. Click Add… and select testAppFromOtherProcess.exe. Your dialog should look like the one shown below.

Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored. Click Go! to start the parent process.

The Java process launches testAppFromOtherProcess.exe immediately. As such you will notice that Coverage Validator starts collecting code coverage almost instantly because it has recognised the Java process is launching a child process that is configured to be monitored and has the correct launch count.

Command Line, example for Java

As you can see, it’s slightly more complicated for Java than for C++, but only because the Java runtime is located in a different folder than the test executable and because we also have to specify a Java class to execute. We still managed to collect code coverage for a child process of a just-in-time compiled language without any coding.

Of course, you now want to know how to do this for the command line. Is this any more complicated than the C++ example? No! Just as easy. Here’s how you do it:

"c:\Coverage Validator\coverageValidator.exe" -program "c:\program files\java\jdk1.8.0_121\bin\java.exe"
                                              -directory "e:\test\release" 
                                              -arg testAppFromOtherProcessJava
                                              -programToMonitor "e:\test\release\testAppFromOtherProcess.exe"

How does this work?

• -arg. Specify an argument for the program to launch. In this example, this specifies the Java class to execute.
• -directory. Specify the startup directory.
• -program. Specify the program to launch. In this example, this specifies the Java runtime.
• -programToMonitor. Specify the program that will be monitored for code coverage.

Use as many -arg options as you need. We only used one because that’s all we need for the example.

Python

The parent program in Python is very simple.

import sys
import subprocess

cmdLine = r"E:\om\c\testApps\testAppFromOtherProcess\Release\testAppFromOtherProcess.exe"
for arg in range(1, len(sys.argv)):
  cmdLine += " "
  cmdLine += sys.argv[arg]
  
subprocess.call(cmdLine, stdin=None, stdout=None, stderr=None, shell=False)

Configuring the target Python program

As with the C++ target program we need to tell Coverage Validator about the target program and the program that is going to launch it. We’re running a Python program so the executable to launch is the Python interpreter. Click the Browse… button and select the Python interpreter you are using.

The launch directory is automatically configured to be the same as the launch program. In the case of a Python program, that is almost certainly incorrect. We’re going to choose the directory where our Python script is located. Click the Dir… button and choose that directory.

We also need to tell Python what script to launch. This is provided as an argument to the program being run (the Python interpreter). In the arguments field, type the name of the script. In this case testAppFromOtherProcess.py.

To allow us to monitor other programs we need to edit the list of applications we can monitor. Click the Edit… button to the right of the Application to monitor combo box. The Applications To Monitor dialog is displayed.

We need to add our target program to the list of programs to monitor. Click Add…. The Application To Monitor dialog is displayed. Choose the Python interpreter python.exe using Browse…. Coverage Validator will identify any other executables in the same folder and add these to the list of target programs you may want to monitor. You can remove any programs you don’t want to monitor with the Remove and Remove All buttons. We now need to add the target program to the list of programs we want to monitor. Click Add… and select testAppFromOtherProcess.exe. Your dialog should look like the one shown below.

Select testAppFromOtherProcess.exe in the Application to monitor combo. Leave the launch count set to 1. The first time testAppFromOtherProcess.exe is launched it will be monitored. Click Go! to start the parent process.

The Python process launches testAppFromOtherProcess.exe immediately. As such you will notice that Coverage Validator starts collecting code coverage almost instantly because it has recognised the Python process is launching a child process that is configured to be monitored and has the correct launch count.

Command Line, example for Python

As you can see, it’s slightly more complicated for Python than for C++, but only because the Python interpreter is located in a different folder than the test executable and because we also have to specify a Python script. We still managed to collect code coverage for a child process of a scripted language without any coding.

Of course, you now want to know how to do this for the command line. Is this any more complicated than the C++ example? No! Just as easy. Here’s how you do it:

"c:\Coverage Validator\coverageValidator.exe" -program "c:\python36-32\python.exe"
                                              -directory "e:\test\release" 
                                              -arg testAppFromOtherProcess.py
                                              -programToMonitor "e:\test\release\testAppFromOtherProcess.exe"

How does this work?

• -arg. Specify an argument to the program to launch. In this example this specifies the Python script to run.
• -directory. Specify the startup directory.
• -program. Specify the program to launch. In this example, this specifies the Python interpreter.
• -programToMonitor. Specify the program that will be monitored for code coverage.

Use as many -arg options as you need. We only used one because that’s all we need for the example.

Conclusion

We’ve demonstrated how to monitor code coverage in a target program launched from C++, Java and Python, using both the GUI and the command line. Each example is slightly different, showing you the changes required for each situation. If you have any questions please email support@softwareverify.com

You can download the C++, Java and Python code used in these examples here.