Sitemap | Contact

Company Evaluations Purchase Support What´s new

Testwell CTC++

Test Coverage Analyzer for C/C++

Information in this document corresponds to the CTC++ version 7.3.1

1. INTRODUCTION

Testwell CTC++ is a powerful instrumentation-based code coverage and dynamic analysis tool for C and C++ code. With certain add-on components CTC++ can be used also on C# and Java code. Further, again with certain add-on components, CTC++ can be used to analyse code basically at any embedded target machines, also in very small ones (limited memory, no operating system,...).
 
CTC++ provides Statement Coverage, Function Coverage, Decision Coverage, Multicondition Coverage, Modified Condition/Decision Coverage (MC/DC), Condition Coverage.
 
As a dynamic analysis tool, CTC++ shows the execution counters (how many times executed) in the code, i.e. more than a plain executed/not executed information. You can also use CTC++ to measure function execution costs (normally time) and to enable function entry/exit tracing at test time.

CTC++ is easy to use. When used in command-line mode (by makefiles or other build scripts), the instrumentation is just a front end phase at the compile/link command. No changes to the source files or build scripts are needed. Test runs are done with the instrumented program version, in the same way as with the original program. Coverage and other execution profiling reports can be obtained easily in straight text, HTML, XML and Excel input form. On some environments, e.g. Microsoft Visua Studio, CTC++ use  is possible directly from the compiler IDE.

CTC++'s overhead on the size and execution speed of the instrumented code is very reasonable. CTC++'s reporting is informative and well-organised. The reports give both a top-level view, which show the coverage percentages at various summary levels, and a detailed view, where the executed/not executed information is mapped to the actual source code locations.

Commercially CTC++ is available as follows:
  • "CTC++ host-only": This is the basic package of the tool, so called "host-only". It is available Windows, Linux, Solaris and HP-UX, see  CTC++ availability. The tool utilities are run at the selected host environment. The used C/C++ compiler is one of the supported compilers that generate code for the selected host environment. The tests with the instrumented code are run at the same host environment.
  • "CTC++ with Host-Target add-on": This contains the basic CTC++ host-only on the selected host environment and Host-Target add-on package (HOTA). HOTA gives certain additional components, which facilitate cross-compiling the instrumented code for a target, running tests at the target, getting the coverage data back to the host, and viewing the coverage reports at the host. The cross-compiler can effectively be "whatever" (except it needs to run at the host where the basic CTC++ runs). The target machine architecture and its operating system (if any) can be effectively "whatever".
  • "CTC++ with Host-Target add-on and Bitcov add-on": This is like "CTC++ with Host-Target add-on" but contains additionally the Bitcov add-on package (Bitcov). Bitcov is a variant of HOTA, and facilitates CTC++ use in targets, which have very limited memory. The Bitcov add-on package is developed by Verifysoft Technology.
Additionally there are separate add-on packages supporting C# and Java code. From CTC++ point of view C# and Java are seen as special dialects of C++, and with certain arrangements CTC++ instrumentation is connected to the C#/Java compilation phase, similarly as when instrumenting and compiling C/C++ code. The C#/Java run-time context is modeled as a special type of target for which the CTC++ support library has been implemented of the HOTA components, i.e. rewritten in C#/Java. The net result is that CTC++ gives of C#/Java code similar coverage and dynamic analysis information as here described for C/C++ code. The C# and Java add-on packages are developed by Verifysoft Technology.

CTC++ is an industry-strength tool, which has been used over 25 years in the IT industry.

Here are some quick links on some of the CTC++ capabilities.
CTC++ facilitates
  • Measuring code coverage = > ensure thorough testing, you know when to stop testing, etc.
    • Function coverage (instrumentation mode: functions called)
    • Decision coverage (instrumentation mode: additionally to function coverage, conditional expressions true and false in program branches, case branches in switch statements, catch exception handlers in C++, control transfers)
    • Statement coverage (analyzed from program flow: percent of statements executed in functions/files/overall)
    • Multicondition coverage (instrumentation mode: additionally to decision coverage, in program branches have all the possible ways to evaluate a conditional expressions having && and || operators been exercised. Direct assignment statements like "variable = ... && ... || ... ;" are also measured for multicondition coverage.)
    • MC/DC coverage (like multicondition coverage, but it suffices that each elementary condition is shown to independently affect the conditional expression outcome--not so demanding than full multicondition coverage, used in DO-178B/DO-178C level A projects)
    • Condition coverage (like multicondition coverage, but it suffices that each elementary condition is shown to have been evaluated to true and false--not so demanding than full multicondition coverage)
  • Searching execution bottlenecks => no more guessing in algorithm tunings
    • Function execution timing (instrumentation mode: total, average, maximum time--if needed, the user may introduce his own time-taking function for measuring whatever is interesting in regard of function resource consumption)
    • Execution counters (how many times the CTC++ instrumentation probes have been executed)
  • Displaying function call trace => helps in analyzing program behavior
    • You can provide the function, which makes the call tracing (perhaps displaying on the screen). At instrumentation phase CTC++ is made to call your trace function at the entry/exit of each function under test.
  • Conveniently presented test results
    • Hierarchical, color-coded, HTML-browsable coverage reports
    • Pure textual reports
    • Coverage data can be converted to Excel input file
    • XML report
  • Ease of use
    • Instrumentation phase as a "front end" to the compilation command => very simple to use
    • No changes are needed to the C/C++ source files to be measured
    • "Ctc-builds" can be done with your existing build makefiles, which normally can be used as is
    • automated script-based use from command line
  • Mature product
    • Has been in demanding use in the IT industry for over 25 years
    • Long use experience with the most commonly used C/C++ compilers (VC++, gcc/g++,...) and with their versions. Many of them have their "extreme corner specialities", but which the tool can handle.
    • Also a proven record of working with numerous (~30+) cross-compilers
    • Full C and C++ support, including the new C++11 additions (lambda functions, trailing return type, range-based loop, etc.)
  • Usable "in the large"
    • Instrumentation overhead very reasonable
    • You can select what source files to instrument and with what instrumentation options
    • Besides full executables, static and dynamically loaded libraries (.lib/.a/.dll/.so) can also be measured
    • Capturing coverage data of never-ending processes is conveniently supported
    • Combined coverage report can be obtained of test runs of different programs and there are powerful means to select the source files that will be reported and in what coverage view they will be reported.
    • Combined coverage report can be obtained of test runs that are run at different machines.
    • Combined coverage report can be obtained of a code base, which has been built and tested for different "configurations" (in the code files there has been conditional compilations and other macro trickery, which have made the actually executed program variants different).
  • On some environments usable via IDE
  • Usable at embedded targets (CTC++ Host-Target add-on is needed)
    • The target can be effectively "whatever"
  • Good management and visibility of testing
    • Easy to read listings (textual and HTML)
    • In terms of the original source code
    • Untested code highlighted
    • Various summary level reports (in HTML)
    • TER-% (test effectiveness ratio) calculated per function, source file, directory, and overall

 2. HOW CTC++ WORKS

There are basically three steps in the use of CTC++ (command-line mode of use):
  • You use the CTC++ Preprocessor (ctc) utility for instrumenting and compiling the C or C++ source files of interest and for linking the instrumented program with the CTC++ run-time library. At this phase ctc maintains a symbol file, MON.sym by default, where it remembers the names of the instrumented files and what they contained. If you build your program by a makefile, you just prepend the make command by ctcwrap ctc-options , and all the emitted compile and link commands that the make emits will be done "in ctc control".
  • You execute the test runs with the instrumented program as you see necessary, in the same way as you would do test runs on the original not-instrumented program. When the instrumented code portions are executed, CTC++ collects the coverage and function timing history in memory. Normally at the end of the program, and automatically by CTC++, the collected counters are written  to a data file, MON.dat by default. If there was previous counters in the data file, they are summed up.
  • You use the CTC++ Postprocessor (ctcpost) utility for putting one or more symbol files and data files together and produce the human readable textual reports. One of them, the Execution Profile Listing, can be further processed with ctc2html utility for getting an easy-to-view hierarchical and color-coded HTML representation of the coverage information. You can obtain the coverage information also in XML and Excel form.

2.1. Building the Instrumented Program

Use of ctc is connected to the command by which you compile and link your programs. Adding 'ctc' and possible 'ctc-options'  in front of the original compilation command is all that is needed for instrumenting and compiling the source file. If the command also links, ctc automatically adds the CTC++ run-time library to the linkage. The ctc on-line help is:
Usage:
ctc [ctc-options] comp/link-command comp/link-options-and-files

[ctc-options]:
   [-i {f|d|m|te|ti}...] [-n symbolfile]
   [-v] [-V] [-k] [-h] [@optionsfile]...
   [-c conf-file[;conf-file]...]... [-C conf-param{=value|+value|-value}]...
   [-2comp] [-no-comp] [-no-templates] [-no-warnings]
On Windows with these actual source files with Visual C++ compiler in command-line mode we could give command:
ctc -i mte cl -Feprime.exe calc.c io.c prime.c
which instruments these three C files with multicondition and (exclusive) timing instrumentation modes, compiles the instrumented code with the 'cl' compiler, and links the instrumented target 'prime.exe' with 'cl' (the CTC++ run-time library is automatically added to the linkage). The same could have been obtained also with the following sequence of commands:
ctc -i mte cl -c calc.c
ctc -i mte cl -c io.c
ctc -i mte cl -c prime.c
ctc link -out:prime.exe calc.obj io.obj prime.obj
The first three commands instrument their argument source file. ctc also invokes the compiler on the instrumented version of the source file resulting an object file to the same place as the original compilation would have generated it. No changes to the source file is needed by the user and it remains intact.

In the last command ctc just repeats the linking command and adds CTC++ run-time library to it. The result is an instrumented executable in the same place as the original linking would have generated it.

Normally real programs are not built by explicitly issuing compile and link commands. Instead they are built by a makefile, for example as follows
nmake -f Makefile
which uses its own ruling to emit the elementary compile and link commands.  In this case the Makefile can be modified to emit the compile and link commands prefixed with ctc, for example
nmake -f Makefile "CC=ctc -i mte cl" "LNK=ctc link"
Or a simpler way is to use ctcwrap command to make the "build" to be a "ctc-build", as follows:
ctcwrap -i mte nmake -f Makefile
The ctcwrap command executes its argument command (here 'nmake') in a special context. In it all compile and link commands are changed to behave "ctc-wise" with the given instrumentation options (here with '-i mte'), i.e. they are run as if they would have 'ctc -i mte' in front of them. The net effect is that the building is "with CTC++".  The makefiles do not need modifications for the sake of using with CTC++. There are means to specify to CTC++ which files only will be instrumented of all the files that the makefile compiles.

2.2. Running the Tests with the Instrumented Program

You run the tests with the instrumented program. Due to instrumentation it is somewhat bigger and slower than originally. How much? It depends on what kind program control structures you have in your code, what instrumentation mode you have selected, what compiler optimization has been used, and have you instrumented all code files of your program or only some of them.  Increase to program size is normally a concern only in some embedded target cases, which have limited memory. When the largest multicondition instrumentation mode is used, a size increase of 50-70% could result on the instrumented code portions. But note that normally the programs have also non-instrumented portions like system libraries that are linked to it. Lower instrumentation gives lower overhead. In the Windows example Cube.exe case, which contains many system libraries (not instrumented) and where the CTC++ run-time library is in a separate DLL (see HTML form coverage report ) the actual program size grew only with 6%.
 
CTC++'s impact to the execution speed has been found to be very modest.

When the program logic executes the code in the instrumented files, the inserted instrumentation probes collect execution history in main memory. When the program ends (or at some explicit user-determined places), the execution counters are automatically written to a data file on disk. If the instrumented program is a never-ending process, there are simple means to add to the instrumented executable an auxiliary thread, which periodically writes the coverage data to the disk. Multiple executions accumulate the counters in the file as long as the instrumented file is the same as before.

To continue the example, the instrumented executable could be run as follows:

prime
Enter a number (0 for stop program): 2
2 IS a prime.

Enter a number (0 for stop program): 5
5 IS a prime.

Enter a number (0 for stop program): 20
20 IS NOT a prime.

Enter a number (0 for stop program): 0

The program was used and it behaved just like the original program. At the program end the CTC++ run-time system, which  has been linked to program, wrote the collected execution counters data to a data file, here to MON.dat.

2.3. Getting the Results of the Test Runs

Finally you use the ctcpost and ctc2html  utilities to get the results for the analysis, i.e. the various types of listings showing the information you initially asked for. ctcpost  is used first. Its on-line help is:
Usage:
ctcpost [general-options] [symbolfile]... [datafile]...
        [-ff|-fd|-fc|-fmcdc] [-w report-width] {{-p|-u|-t|-x} reportfile}...
ctcpost [general-options] datafile... -a target-datafile
ctcpost [general-options] symbolfile... -a target-symbolfile
ctcpost [general-options] {-l|-L} {symbolfile|datafile}...


[general-options]:
   [-h] [-V] [@optionsfile]...
   [-c conf-file[;conf-file]...]... [-C conf-param{=value|+value}]...
   [-f source-file[;source-file]...]... [-nf source-file[;source-file]...]...
and Execution Profile Listing could be obtained as follows:
ctcpost MON.sym MON.dat -p profile.txt
Here is a more complex example where coverage report is generated of two independently instrumented progams, and some files, which however have been instrumented, are excluded from the report:
ctcpost MON.sym MON.dat ..\prj2\MON.sym ..\prj2\MON.dat \
-nf "*\harnessdir\*" -nf "*\moc_*.cpp" -p profile.txt
With ctcpost you can get the following textual reports:
  • Execution Profile Listing shows the missing coverage as well as how many times each code location has been visited. This is the primary CTC++ report, normally worked onwards to HTML form. The report can be also generated with somewhat reduced coverage information compared how the code was instrumented. See examples
  • Untested Code Listing is similar to execution profile listing but shows only the places where test coverage is inadequate. The HTML form report shows also the untested information, at summary levels (TER%) and individual code locations.
  • Execution Time Listing shows the total (times of all calls summed up), average and maximum (longest) execution times of functions. In timing instrumentation there are two submodes of which you can select: inclusive timing (the time spent in the called instrumented function is included in the time of the calling function) and exclusive timing (the time spent in the called instrumented functions is excluded from the time of the calling function).
  • XML report contains the information that is in Execution Profile Listing and in Execution Time Listing, but is in XML form. This report is meant for postprocessing of the coverage data by your own XML-utility. This report is also used in getting a combined coverage report of a code base, which has been built and tested for different configurations.
Normally the  Execution Profile Listing is straight away worked onwards to HTML form using ctc2html utility. Its on-line help is:
Usage:
  ctc2html [-i inputfile] [-o outputdir] [-s sourcedir][;sourcedir]...]...
           [-t threshold] [-p prefix] [-nsb] [-no-sources]
           [-no-javascript] [-li number]
  ctc2html [-h] [--enable-help]


Command-line options:
  -i inputfile   Input Execution Profile Listing file. Default stdin.
  -o outputdir   Output HTML directory. Default CTCHTML.
  -s sourcedir   Source files are searched also from this directory.
  -t threshold   Set the structural threshold percent (0-100). Default 100.

  -nsb           Do not start HTML browser automatically (only Windows).
  -no-sources    Do not convert sources to HTML, use only ctcpost listing.
  -no-javascript Do not generate javascript on HTML pages.
  -li number     For load speed reasons reduce Index frame at Execution
                 Profile page to contain max 'number' lines. Default 2000.
  -h             Display this command line help.
  --enable-help  Display help of advanced --enable-XXX options.
Start browsing from file 'outputdir/index.html'.
An example:
ctc2html -i profile.txt -t 85 -nsb
start CTCHTML\index.html
ctc2html   converts the Execution Profile Listing information to hierarchical, easily navigable, color-coded HTML representation. Also the actual source files are incorporated to the HTML. The generated HTML files can be viewed with normal web browsers.

The HTML representation is called "CTC++ Coverage Report". It is hierarchical and has six levels:
  • Overall Summary: General header information in full (from what data generated, when generated, what options used, etc. -- sometimes, in hard use, these can make up quite many lines) and overall summary of the code base with TER percentages.
  • Directory Summary: General header information (cutted to few first lines, if would be longer), directory TERs shown in histograms and numerically, coverage-% not meeting the suggested threshold percent (-t option) are shown in red color. TER over all directories.
  • Files Summary: Zoom-in to the files in the directories. Similar TERs and color-coding are shown but at file levels.
  • Functions Summary: Zoom-in to the methods and functions in the files. Similar TERs and color-coding are shown but at function levels.
  • Untested Code: Compact listing of untested code locations with links to the actual source file to the pertinent line.
  • Execution Profile: Zoom-in to the detailed view where the execution counters are shown with the source code. Lines that are not fully exercised with respect to the selected coverage criteria are highlighted in red.
See the example  HTML report (started in a new window).

2.4. Getting combined coverage reports

When same instrumented program is run multiple times, CTC++ automatically accumulates the execution coverage data to the results of the previous test runs.

When you have many independently instrumented programs and wish to get a combined coverage report of them, you can do it for example as follows:
ctcpost MON.sym ..\test2\MON.sym MON.dat ..\test2\MON.dat -p profile.txt
If your independently instrumented separate programs contain partially same source files, the above kind of ctcpost command will give you combined coverage of them, too. I.e. in the profile.txt file there is one entry of the file and it contains aggregated coverage of the file execution in different programs. The requirement on this to succeed is that the common files really represent the same level as CTC++ sees them, e.g. compiled with same flags, and the files have been instrumented in the same way.
 
In getting the coverage report you can specify that some selected files only are shown in the report and some others not. Further, you can specify that the coverage information is shown in a lower coverage measure (e.g. in a compact function coverage view only) than the file was actually instrumented with.
 
Another usage scenario, which can occur in bigger development organisations, is the following: There is a code base, which is compiled and tested in many configurations. The original C/C++ files are unchanged as such, but in compiling the different configirations conditional compilation and macro resolvings make the actual code (C translation unit after C prepocessing) different. The "basic CTC++" refuses to generate combined coverage report of files, which are in the above way different. The management may however want to see one report, especially its "bottom line TER%", which sums up the coverage of the whole code base over its all configurations. For this need there is ctcxmlmerge utility. Its on-line help is:
Usage:
ctcxmlmerge input.xml... [-p profilefile] [-x xmlfile] [@optionsfile]
            [-f file[;file]...]... [-nf file[;file]...]... [-ndl]
ctcxmlmerge [-h]

Options:
   input.xml       Input XML file. One or more CTC++ XML coverage reports.
   -p profilefile  Output merged textual execution profile listing.
   -x xmlfile      Output merged summary XML report.
   @optionsfile    Input and options can be given also at optionsfile.
   -f file         Only these instrumented files are reported.
   -nf file        These instrumented files are not reported.
   -ndl            No drive letter (removed from reported filenames).
   -h              Display command line help.
The idea is  that each configuration is first instrumented and tested independently, and XML form report of its coverage is taken. Then the XML form coverage reports are merged with this utility to get one combined text form execution profile listing (-p). An XML summary report can also be obtained (-x), if wanting to further process the TER%s from XML form.
 
If the instrumented code files have header code (can be determined at instrumentation time what headers, if any, are instrumented), that code is "pulled out" from their code files and reported (and execution hits summed up and TER% calculated) as separate header file entities. Execution hits on same code files, from different program configurations, even if due to macros and conditional compilation they were a bit different as CTC++ sees the code, are summed up and TER% is recalculated in the merged report.
 
The merged execution profile listing can be further converted to HTML for by cct2html utility.

2.5. Connection to Excel

The coverage data from an Execution Profile Listing can be converted to converted to a file, which is suitable input to Excel (or to another spreadsheet application). The ctc2excel utility is used here. Its on-line help is:
Usage:
  ctc2excel [-i infile] [-o outfile] [-u] [-efs 'c'] [-full] [-nopath]
  ctc2excel -h

Command-line options:
  -i infile  Input Execution Profile Listing file . Default 'stdin'.
  -o outfile Output Excel TSV file. Contains coverage data of the functions,
             one line of each. Default 'stdout'.
  -u         Only untested (under 100% TER) functions are reported in outfile.
  -efs 'c'   Use c as Excel field separator. Default tab. Example -efs ';'.
  -full      Write to outfile also the header information and the summary
             information of the instrumented files and of overall.
  -nopath    Of the reported instrumented files drop off the path portion.
  -h         Displays this help text.
And an example of the use:
ctc2excel -i profile.txt -o excelinput.txt
start excel excelinput.txt

2.6. Function call trace

With certain simple arrangements you can instrument your code to produce a function call trace. It means that you provide your own trace function (and link to the executable), which CTC++ calls at the begin and return of each instrumented function. As a parameter the trace function gets the name of the function that was just called. Your trace function, then, can do whatever you see necessary, for example display the called function name on the screen.

As an example, the trace function might be the following:

#include <stdio.h>
void mytrace(char* fname, char* start_or_end) {
printf("mytrace: function %s %s\n", fname, start_or_end);
}

You may find this useful when wanting to analyze the dynamic behavior of your program. Another situation might be when your program crashes and you do not know where it happens. You just instrument the code for producing this "brute force call trace" and you see how far your program managed to get.

3. IDE INTEGRATIONS

CTC++ has been integrated to a couple of compilation system IDEs, all at Windows platform. The integration means that in the IDE Tools menu there has been added new commands, like
  • CTC++ Set... : for setting "instrumentation mode on" and specifying the ctc-options under which the coming builds in the IDE will be done. Later this command is used for setting "instrumentation mode off", i.e.  the builds to normal/ctc-free again.
  • CTC++ Report...: for getting the various coverage reports and starting some appropriate viewer (like notepad, html browser, Excel) on them.
See CTC++/Visual Studio Integration on how the integration looks at Visual Studio IDE.

Some IDEs support that the build can be commanded from command line. With many of them, in the next example with Visual Studio 2003, the"ctc-build" can be made with ctcwrap  as follows:
ctcwrap -i d devenv mySolution.sln /rebuild debug

4. SUPPORT FOR TARGET TESTING

HOTA:
Information in this chapter corresponds to the CTC++ Host-Target add-on (HOTA) version 5.1

With "target testing" it is here meant that you have a host environment, where you do builds for some target machine, typically an embedded system. The used C/C++ cross-compiler and the target machine can be something with which CTC++ may not have been used before. You anyway want to instrument the code, compile it with the cross-compiler, run it at the target machine, and get the coverage data back to the host for reporting. The "CTC++ Host-Target add-on" (HOTA) package  provides this capability.

Technically the arrangement goes roughly as follows:
  • At the host you need to have normal CTC++ (host-only) copy running. Additionally you need this HOTA package.
  • You "teach" to CTC++ the cross-compiler/linker command names and options. This is a one-time job. If the cross-compiler is a decent one and follows the common conventions, there is no bigger problem in this. We also have working configuration files for a number of cross-compilers.
  • The instrumented code needs a little CTC++ run-time support layer at the target. The HOTA package contains it in C source code form. It is "vanilla C", about 1000 lines of well-commented code, and it compiles with any C compiler, also with your cross-compiler. You need not touch that code.
  • However, into the run-time layer's use you need to implement the low-level data transfer layer, with which the coverage data is transferred to the host. The coverage data is in CTC++-internal encoded form as a sequence of printable ascii characters. No CTC++ internal knowledge is needed in its handling, just writing it one char at a time to somewhere.  Ultimately the char stream needs to be transferred to the host machine to a text file. If you can write at the target machine the data to a local text file (and separately move it to the host later), the work is simple. For this alternative the delivery package has a compile-ready implementation, which uses normal C text file I/O. In some cases the coverage data needs to be written to some communication channel, which  your program in the host listens and writes to a file. Developing this low-level data transfer layer is a one-time job. At test time the coverage data writing is normally a one-time act at the end of the instrumented program execution. The data volumes that are transferred are rather modest.
  • The instrumentation phase at the host for the target is similar as instrumentation for the host. Only the cross-compiler is used (not the one that compiles for the host) and the slightly modified target-specific run-time layer is linked to the instrumented code (not the one that is linked in the ctc-builds to the host).
  • You run the tests at the target. Depending how you have arranged to data transfer, you get the coverage data to host side where you feed it CTC++'s ctc2dat  utility and you get MON.dat file. That after the reports can be taken normally at the host by ctcpost and ctc2html utilities.

CTC++ Host-Target is designed to cope with situations where it is not known

  • what hardware architecture the target has
  • what operating system, if any, the target runs
  • what brand is the C/C++ cross compiler for the target
  • do the host and target machines use the same endianness in binary data
  • do the target machine (cross-compiler) use same amount of bits for basic C integral types for storing the execution counters
The Host-Target add-on package is also usable when the code under test is operating system kernel code. Read more from  Kernelcoverage.

BITCOV:
Bitcov is a derivative work based on HOTA. It is meant for small embedded micro-controller kind of targets, which have very little free data memory for CTC++'s use, or where the HOTA style to transfer the coverage data as an encoded ASCII stream to the host is difficult.

In Bitcov there is one global bit array in the target main memory where the execution hits are recorded, one bit per probe. For example with a 1000 byte array 8000 probes could be recorded. It might be well enough for a reasonable size instrumented program, around 30000 lines of instrumented code. In normal instrumentation CTC++ run-time data area consumption would be in a similar case 8000 * 4 bytes (one counter is normally 4 bytes) = 32000 bytes + something more for CTC++'s internal control data needs. At the target there essentially is no CTC++ run-time layer at all.

Of Bitcov there is also a variant, where the not_hit/hit (0/1) information is stored in a byte array, one byte per probe. In typical machine instruction sets setting a byte in memory takes less instructions than setting an individual bit in memory ("code-bloat"), and it saves more than what we lose when a byte array (not bit array) is used for storing the coverage information ("data-bloat").

After the test run the bit/byte array is captured to the host to a file, either so that the initiative is at the host who pulls the coverage data (e.g. by debugger means) from the target, or so that the initiative is at the target side who pushes the coverage data to the host (as it is possible at the target, e.g. embedded in the instrumented code). At the host the coverage data is converted to a form, which is suitable to the HOTA tool chain for further processing (ctc2dat, ctcpost, ctc2html).

In Bitcov timing instrumentation is not supported. In coverage reports the counters are reduced to 0 (not executed) and 1 (executed) while in normal coverage reports the counter value tells also how many times the code at the probe location was executed.

5. SUMMARY OF BENEFITS

CTC++ is a versatile tool to be used in testing and tuning of all kinds of applications written in C or C++.
Testing becomes more efficient
The execution profile listing reveals the parts of code which have not yet been executed. The coverage information helps to design the missing test cases. On the other hand, CTC++ helps to determine when to stop testing (from code coverage point of view), thus preventing the waste of the costly human resources.
Testing becomes a measurable, well-managed activity
The summary listing with TER-% histograms characterizing the reached test coverage gives valuable information for project management purposes at a glance.
Usable for program dynamic analysis and performance tuning
When CTC++ is used for improving the performance of a program, it can easily show where the bottlenecks of the program are (functions that are executed most often and functions that consume most time). Also production level applications can be monitored when they are used in their real environment. The function call "brute force" tracing capability can be useful in analyzing the program behavior when no better tools are available.
Use of CTC++ is easy and fast
CTC++ is easy to use. No modifications of the user's code is needed. Instrumentation takes place by just adding 'ctc' in front of the compilation/link command. Enforcing the existing makefiles to build instrumented targets instead of the original non-instrumented ones is very straightforward. This does not need any changes to the makefiles itself.  Browsing the coverage results in HTML is very easy. The overall picture is shown in color-coded histograms of coverage percentages. Zooming to the detailed level can be be done with only a few mouse clicks and the untested code locations are clearly shown mapped to the original source code.
 Independent instrumentation of source file
The source files of the executable program are instrumented independently of each other, or left as non-instrumented. The source files can be instrumented with such instrumentation modes that are appropriate in the situation at hand. Instrumented source files can be linked to different executables and yet the merged coverage of their executions in different executables can be obtained.
Configurability
CTC++ is easy to configure for specific needs by simply editing the textual configuration file. At the time instrumentation and getting the reports there are powerful means to fine-tune the process to obtain the desired result.
Usable for many purposes
CTC++ can be used for many purposes: measuring code coverage at various testing phases (module testing, integration testing, system testing), performance testing, optimization, comparing efficiency of algorithms, locating dead code, ...
Support for host-target and kernel code testing
CTC++ has powerful support for measuring code coverage at embedded targets. The instrumentation overhead is very moderate. The used C/C++ cross-compiler for the target, the target operating system and the target hardware type can be virtually of any brand and type. Applying on kernel code is something unique among coverage tools.
Supports both C and C++
With one CTC++ tool you can work both with C and C++ code.
Work motivation
The programmers/testers are likely to design more and better test cases to achieve higher test coverage when they have an easy-to-use tool for measuring it.
Usable with CTA++
CTC++ can be used together with Testwell's test harnessing tools CTA++ (C++ Test Aider), see  CTA++ description. Such usage combines the "black box" (behavioral, functional) and "white box" (structural) testing strategies for purposes of systematic module testing and reducing of testing costs.
Usable with other vendors' test execution frameworks
CTC++ can be seamlessly used with other vendors' unit test and system (GUI) test execution frameworks.
CTC++ has been compared to US Army Jeep...
Simple to use. Works. Can be driven on almost whatever terrain (especially when Host-Target add-on package is used).


6. OPERATING ENVIRONMENTS

CTC++ is available on several machine / operating system / C/C++ compiler environments, see the detailed list: CTC++ availability

The resource requirements of your normal C or C++ development environment are sufficient for using the CTC++.

Since July 2013 the "Testwell CTC++" tool is owned by Verifysoft Technology GmbH. See what's new .


To frontpage