Code coverage analysis tools for PIN.

Developed by:

Oleksiuk Dmitry, eSage Lab
mailto:dmitry@esagelab.com
http://www.esagelab.com/

My article about PIN and this tools:
http://d-olex.blogspot.com/2011/03/blog-post.html
(in Russian, use Google Translate)

==============================================================
  ARCHIVE CONTENTS
==============================================================

./Coverager.dll - PIN instrumentation module for code coverage analysis.
./coverage_test.exe - Test application to buid code coverage map for Internet Explorer process.
./coverage_parse.py - Program for parsing the logs, that has been generated by instrumentation module.
./coverage_to_callgraph.py - Program to generates log files in Calltree Profile Format.
./symlib.pyd - PDB symbols library for Python 2.6 (see symlib_test.py for usage details).
./symlib25.pyd - PDB symbols library for Python 2.5
./EXAMPLES/ - Samples of output logs.


==============================================================
  BUILDING CODE COVERAGE MAP BY FUNCTIONS AND BASIC BLOCKS
==============================================================

1) Download and install PIN toolkit (http://www.pintool.org).

2) Copy Coverager.dll into the PIN toolkit root directory.

3) Edit execute_pin.bat scenario and put PIN toolkit root directory path into the PINPATH variable.

4) Use execute_pin.bat from command line to run some aaplication and generate code coverage map for it. 
   Example:

    > execute_pin.bat "C:\Program Files\Internet Explorer\iexplore.exe"
    
5) After the target applicaion termination 4 log files will be created (CoverageData.log, CoverageData.log.modules, CoverageData.log.routines and CoverageData.log.blocks).

6) Use coverage_parse.py program to extract information from the generated logs. 
   Example:

   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    C:\> python coverage_parse.py Coverager.log --dump-routines --modules "iexplore, ieframe" --outfile routines.txt
    
    SYMLIB: DLL_PROCESS_ATTACH
    SYMLIB: Symbols path is "C:\Symbols;SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols"

    Code Coverage Analysis Tool for PIN
    by Oleksiuk Dmitry, eSage Lab (dmitry@esagelab.com)

    Filtering by module name "iexplore"
    Filtering by module name "ieframe"
    [+] Output file: "routines.txt"
    [+] Parsing routines list, please wait...

    SYMLIB: Module loaded from "c:\Program Files\Internet Explorer\iexplore.exe"
    SYMLIB: 395 symbols loaded for "c:\Program Files\Internet Explorer\iexplore.exe"
    SYMLIB: Module loaded from "C:\Windows\system32\IEFRAME.dll"
    SYMLIB: 33516 symbols loaded for "C:\Windows\system32\IEFRAME.dll"

    [+] Processed modules list:

    #
    # Routines count -- Module Name
    #
               3576 -- flash10n.ocx
                 47 -- jp2ssv.dll
                195 -- wdmaud.drv
                 15 -- rasadhlp.dll
                208 -- msls31.dll

                ... skipped ...

    [+] DONE

    SYMLIB: DLL_PROCESS_DETACH
    
   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    
Sample log file from the coverage_parse.py can be found in ./EXAMPLES/IEXPLORE_Routines.txt
For detailed information about coverage_parse.py usage see comments in the Python source.


==============================================================
  BUILDING AND EXPLORING CALL TREE MAP
==============================================================

1) To enable call tree logging execute your target applicaion with execute_pin_calls.bat scenario:

   > execute_pin_calls.bat "C:\Program Files\Internet Explorer\iexplore.exe"
   
2) After the target applicaion termination in addidition to CoverageData.log, CoverageData.log.modules, CoverageData.log.routines and CoverageData.log.blocks also will be created a few files with the names like CoverageData.log.<N>, where <N> - thread number.

3) Use coverage_to_callgraph.py scenario to converting CoverageData.log.<N> files into the Calltree Profile Format (that uses in Valgrind):

   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

    C:\> python coverage_to_callgraph.py CoverageData.log *
    SYMLIB: DLL_PROCESS_ATTACH
    SYMLIB: Symbols path is "C:\Symbols;SRV*C:\Symbols*http://msdl.microsoft.com/download/symbols"

    Code Coverage Analysis Tool for PIN
    by Oleksiuk Dmitry, eSage Lab (dmitry@esagelab.com)

    [!] Psyco is not available
    [+] Input file(s): CoverageData.log.0, CoverageData.log.1, CoverageData.log.10, CoverageData.lo
    g.11, CoverageData.log.12, CoverageData.log.13, CoverageData.log.14, CoverageData.log.15, Cover
    ageData.log.16, CoverageData.log.17, CoverageData.log.18, CoverageData.log.19, CoverageData.log
    .2, CoverageData.log.20, CoverageData.log.21, CoverageData.log.22, CoverageData.log.3, Coverage
    Data.log.4, CoverageData.log.5, CoverageData.log.6, CoverageData.log.7, CoverageData.log.8, Cov
    erageData.log.9
    [+] Output file: Callgrind.out
    [+] 80 modules readed
    [+] Parsing routines list, please wait...

    [+] 27806 routines readed
    [+] Parsing call tree, please wait...

    SYMLIB: Module loaded from "C:\Windows\SYSTEM32\ntdll.dll"
    SYMLIB: 4239 symbols loaded for "C:\Windows\SYSTEM32\ntdll.dll"
    SYMLIB: Module loaded from "C:\Windows\system32\IEFRAME.dll"
    SYMLIB: 33516 symbols loaded for "C:\Windows\system32\IEFRAME.dll"
    SYMLIB: Module loaded from "C:\Windows\System32\mshtml.dll"
    SYMLIB: 35150 symbols loaded for "C:\Windows\System32\mshtml.dll"
    SYMLIB: Module loaded from "C:\Windows\system32\OLEAUT32.dll"
    SYMLIB: 3940 symbols loaded for "C:\Windows\system32\OLEAUT32.dll"

    ... skipped ...

    [+] DONE (15 mins., 33 secs.)

    SYMLIB: DLL_PROCESS_DETACH

   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

coverage_to_callgraph.py creates Callgrind.out file, that can be explored with Kcachegrind program.
Sample Callgrind.out for Internet Explorer process execution can be found in ./EXAMPLES/ directory.
For detailed information about coverage_to_callgraph.py usage see comments in the Python source.

Useful liks:

 - Official Kcachegrind page:
 http://kcachegrind.sourceforge.net/html/Home.html

 - Windows port of Kcachegrind (by Lailin Chen):
 http://sourceforge.net/projects/precompiledbin/

 - Calltree Profile Format specification:
 http://valgrind.org/docs/manual/cl-format.html