Detecting memory leaks in an IIS ISAPI DLL

This tutorial describes how to detect memory leaks in an IIS ISAPI DLL.

This tutorial covers the following:

1. 1. Modifying an ISAPI DLL to use the NT Service API.
   2. How to use the Memory Validator user interface to detect memory leaks in the IIS ISAPI DLL.

Related tutorials:

Detecting memory leaks in a service.
Detecting memory leaks in an application that is a child process of a service.
Detecting memory leaks in ASP.Net with IIS.
Detecting memory leaks in ASP.Net with Web Development Server.
Detecting detect memory leaks for a child process.

Native ISAPI and mixed-mode ISAPI

This tutorial applies to all native ISAPI DLLs and to mixed-mode ISAPI DLLs that uses the native Win32 services API.

Example ISAPI DLL

Memory Validator ships with an example ISAPI DLL in the examples\isapiExample folder in the Memory Validator installation directory.

The ISAPI DLL has already been modified to use the NT Service API. In this tutorial we’ll describe the modification you would make to the ISAPI DLL to make it work correctly with Memory Validator.

What is the NT Service API?

The NT Service API is a simple API that allows you to load the Memory Validator profiling DLL and start the process of detecting memory leaks.

The API also includes some debugging functions to help provide debugging information via log files (the only way to get data out of an ISAPI DLL without a connection to the Memory Validator user interface).

Modifying your ISAPI DLL to use the NT Service API

1. Identify your service’s GetExtensionVersion() function, and just before that function, add a new function definition called attachToMemoryValidator(). 

   The purpose of attachToMemoryValidator() is to use the NT Service API to instrument the ISAPI DLL with Memory Validator.

   // code to load Memory Validator into the IIS process for this ISAPI
   // this assumes the ISAPI is in C:\testISAPIWebsite\
   
   #include "..\..\svlMVStubService\svlMVStubService.h"
   #include "..\..\..\svlCommon\svlServiceError.h"
   
   static void attachToMemoryValidator()
   {
       svlMVStub_setLogFileName(L"C:\\testISAPIWebsite\\svl_MV_log.txt");
       svlMVStub_deleteLogFile();
   
       SVL_SERVICE_ERROR   errCode;
   #ifdef IS6432
       // x86 with x64 GUI
       errCode = svlMVStub_LoadMemoryValidator6432();
   #else   //#ifdef IS6432
       // x86 with x86 GUI
       // x64 with x64 GUI
       errCode = svlMVStub_LoadMemoryValidator();
   #endif   //#ifdef IS6432
   
       if (errCode != SVL_OK)
       {
           DWORD   lastError;
   
           lastError = GetLastError();
           svlMVStub_writeToLogFileW(L"Memory Validator load failed. \r\n");
           svlMVStub_writeToLogFileLastError(lastError);
           svlMVStub_writeToLogFile(errCode);
   
           svlMVStub_dumpPathToLogFile();
       }
       else
       {
           svlMVStub_writeToLogFileW(L"Memory Validator load success. \r\n");
   
           errCode = svlMVStub_StartMemoryValidatorForIIS();
           if (errCode != SVL_OK)
           {
               DWORD   lastError;
   
               lastError = GetLastError();
               svlMVStub_writeToLogFileW(L"Starting Memory Validator failed. \r\n");
               svlMVStub_writeToLogFileLastError(lastError);
               svlMVStub_writeToLogFile(errCode);
           }
   
           svlMVStub_writeToLogFileW(L"Finished starting Memory Validator\r\n");
       }
   }
   

2. At the start of GetExtensionVersion() just after setting the ISAPI description pVer->lpszExtensionDesc add a call attachToMemoryValidator().
3. At the end of TerminateExtension() add a call to svlMVStub_UnloadMemoryValidator().

Detecting memory leaks in the service

Now that the NT Service API has been implemented in your service, we can start collecting memory allocation data from the service.

1. Choose the Launch > Services > Monitor IIS and ISAPI… option.

   

2. The Monitor IIS and ISAPI dialog is displayed.

   

   1. Select the ISAPI DLL you are going to monitor. For this example, the application is C:\testISAPIWebsite\isapiExample_x64.dll. This assumes that IIS has been configured for this ISAPI DLL.
   2. Choose the appropriate native/mixed-mode/.Net option to specify which types of code you want to detect memory leaks for. Mixed-mode is the default, as this collects memory allocation information for all types of code.
   3. The web root will be automatically set to the same directory as that containing the ISAPI DLL. Directory permissions for the current user and for user group IIS_ISURS will be automatically set if the Automatically set directory permissions check box is selected. If this is not selected, permissions won’t be set, and a Show me how… link will be displayed beneath the web root. This link will open a tutorial explaining how to set the correct directory permissions to work with IIS_ISURS.
   4. Select a web browser of your choice. For this example we’ll choose firefox.exe.
   5. Specify the URL that you wish to test with the ISAPI DLL. For this example we’re going to test http://localhost:81\isapiExample_x64.dll?10
3. When you click OK, Memory Validator will set up everything needed to interact with the IIS and the NT Service API and if all is OK, the web browser will be started to open the specified URL.

   If errors are found you will be presented with a dialog box specific to the error so that you can correct the error before starting the web browser.

4. Memory Validator will instrument your ISAPI DLL and start collecting memory allocation data.

Finishing detecting memory leaks

To finish detecting memory leaks, you need to stop IIS.

If you had selected Stop IIS when web browser is closed on the Monitor IIS and ISAPI dialog then you have nothing to do.

Otherwise, choose Launch > Services > Stop IIS

Memory Validator will monitor memory allocations and deallocations the service shutdown procedure and then present you with the memory leak report.

I’m not getting any memory allocation data. What can I do?

There are a few things to check.

1. Have you correctly added the NT Service API to the ISAPI DLL? 
2. Check the log file for any errors. You specified the log file in attachToMemoryValidator() with the call

   svlMVStub_setLogFileName(SZLOGFILENAME);

3. Check the diagnostics tab. If the NT Service API is working correctly, Memory Validator will have some data. Information on instrumentation failures will be on the diagnostic tab.
4. Check the debug information dialog. You can access this from the Tools > DLL Debug Information… menu. This dialog will tell you which DLLs have debug information and which do not. Any DLLs that don’t have debug information you’ll need to ensure that debug information is built for these DLLs and is findable.

Conclusion

You have learned how to add the NT Service API to an ISAPI DLL, how to use Memory Validator to monitor ISS and ISAPI, and what to look at to diagnose errors if things don’t work first time.