6eab34f3dc
gdb/testsuite/ChangeLog: * gdb.perf/README: New file. * lib/perftest.exp (tcl_string_list_to_python_list): New function. * lib/gen-perf-test.exp: New file.
211 lines
6.9 KiB
Text
211 lines
6.9 KiB
Text
The GDB Performance Testsuite
|
|
=============================
|
|
|
|
This README contains notes on hacking on GDB's performance testsuite.
|
|
For notes on GDB's regular testsuite or how to run the performance testsuite,
|
|
see ../README.
|
|
|
|
Generated tests
|
|
***************
|
|
|
|
The testcase generator lets us easily test GDB on large programs.
|
|
The "monster" tests are mocks of real programs where GDB's
|
|
performance has been a problem. Often it is difficult to build
|
|
these monster programs, but when measuring performance one doesn't
|
|
need the "real" program, all one needs is something that looks like
|
|
the real program along the axis one is measuring; for example, the
|
|
number of CUs (compilation units).
|
|
|
|
Structure of generated tests
|
|
****************************
|
|
|
|
Generated tests consist of a binary and potentially any number of
|
|
shared libraries. One of these shared libraries, called "tail", is
|
|
special. It is used to provide mocks of system provided code, and
|
|
contains no generated code. Typically system-provided libraries
|
|
are searched last which can have significant performance consequences,
|
|
so we provide a means to exercise that.
|
|
|
|
The binary and the generated shared libraries can have a mix of
|
|
manually written and generated code. Manually written code is
|
|
specified with the {binary,gen_shlib}_extra_sources config parameters,
|
|
which are lists of source files in testsuite/gdb.perf. Generated
|
|
files are controlled with various configuration knobs.
|
|
|
|
Once a large test program is built, it makes sense to use it as much
|
|
as possible (i.e., with multiple tests). Therefore perf data collection
|
|
for generated tests is split into two passes: the first pass builds
|
|
all the generated tests, and the second pass runs all the performance
|
|
tests. The first pass is called "build-perf" and the second pass is
|
|
called "check-perf". See ../README for instructions on running the tests.
|
|
|
|
Generated test directory layout
|
|
*******************************
|
|
|
|
All output lives under testsuite/gdb.perf in the build directory.
|
|
|
|
Because some of the tests can get really large (and take potentially
|
|
minutes to compile), parallelism is built into their compilation.
|
|
Note however that we don't run the tests in parallel as it can skew
|
|
the results.
|
|
|
|
To keep things simple and stay consistent, we use the same
|
|
mechanism used by "make check-parallel". There is one catch: we need
|
|
one .exp for each "worker" but the .exp file must come from the source
|
|
tree. To avoid generating .exp files for each worker we invoke
|
|
lib/build-piece.exp for each worker with different arguments.
|
|
The file build.piece.exp lives in "lib" to prevent dejagnu from finding
|
|
it when it goes to look for .exp scripts to run.
|
|
|
|
Another catch is that each parallel build worker needs its own directory
|
|
so that their gdb.{log,sum} files don't collide. On the other hand
|
|
its easier if their output (all the object files and shared libraries)
|
|
are in the same directory.
|
|
|
|
The above considerations yield the resulting layout:
|
|
|
|
$objdir/testsuite/gdb.perf/
|
|
|
|
gdb.log, gdb.sum: result of doing final link and running tests
|
|
|
|
workers/
|
|
|
|
gdb.log, gdb.sum: result of gen-workers step
|
|
|
|
$program_name/
|
|
|
|
${program_name}-0.worker
|
|
...
|
|
${program_name}-N.worker: input to build-pieces step
|
|
|
|
outputs/
|
|
|
|
${program_name}/
|
|
|
|
${program_name}-0/
|
|
...
|
|
${program_name}-N/
|
|
|
|
gdb.log, gdb.sum: for each build-piece worker
|
|
|
|
pieces/
|
|
|
|
generated sources, object files, shlibs
|
|
|
|
${run_name_1}: binary for test config #1
|
|
...
|
|
${run_name_N}: binary for test config #N
|
|
|
|
Generated test configuration knobs
|
|
**********************************
|
|
|
|
The monster program generator provides various knobs for building various
|
|
kinds of monster programs. For a list of the knobs see function
|
|
GenPerfTest::init_testcase in testsuite/lib/perftest.exp.
|
|
Most knobs are self-explanatory.
|
|
Here is a description of the less obvious ones.
|
|
|
|
binary_extra_sources
|
|
|
|
This is the list of non-machine generated sources that go
|
|
into the test binary. There must be at least one: the one
|
|
with main.
|
|
|
|
class_specs
|
|
|
|
List of pairs of keys and values.
|
|
Supported keys are:
|
|
count: number of classes
|
|
Default: 1
|
|
name: list of namespaces and class name prefix
|
|
E.g., { ns0 ns1 foo } -> ns0::ns1::foo_<cu#>_{0,1,...}
|
|
There is no default, this value must be specified.
|
|
nr_members: number of members
|
|
Default: 0
|
|
nr_static_members: number of static members
|
|
Default: 0
|
|
nr_methods: number of methods
|
|
Default: 0
|
|
nr_inline_methods: number of inline methods
|
|
Default: 0
|
|
nr_static_methods: number of static methods
|
|
Default: 0
|
|
nr_static_inline_methods: number of static inline methods
|
|
Default: 0
|
|
|
|
E.g.,
|
|
class foo {};
|
|
namespace ns1 { class bar {}; }
|
|
would be represented as:
|
|
{
|
|
{ count 1 name { foo } }
|
|
{ count 1 name { ns1 bar } }
|
|
}
|
|
|
|
The naming of each class is "class_<cu_nr>_<class_nr>",
|
|
where <cu_nr> is the number of the compilation unit the
|
|
class is defined in.
|
|
|
|
There's currently no support for nesting classes in classes,
|
|
or for specifying baseclasses or templates.
|
|
|
|
Misc. configuration knobs
|
|
*************************
|
|
|
|
These knobs control building or running of the test and are specified
|
|
like any global Tcl variable.
|
|
|
|
CAT_PROGRAM
|
|
|
|
Default is /bin/cat, you shouldn't need to change this.
|
|
|
|
SHA1SUM_PROGRAM
|
|
|
|
Default is /usr/bin/sha1sum.
|
|
|
|
PERF_TEST_COMPILE_PARALLELISM
|
|
|
|
An integer, specifies the amount of parallelism in the builds.
|
|
Akin to make's -j flag. The default is 10.
|
|
|
|
Writing a generated test program
|
|
********************************
|
|
|
|
The best way to write a generated test program is to take an existing
|
|
one as boilerplate. Two good examples are gmonster1.exp and gmonster2.exp.
|
|
gmonster1.exp builds a big binary with various custom manually written
|
|
code, and gmonster2 is (essentially) the equivalent binary split up over
|
|
several shared libraries.
|
|
|
|
Writing a performance test that uses a generated program
|
|
********************************************************
|
|
|
|
The best way to write a test is to take an existing one as boilerplate.
|
|
Good examples are gmonster1-*.exp and gmonster2-*.exp.
|
|
|
|
The naming used thus far is that "foo.exp" builds the test program
|
|
and there is one "foo-bar.exp" file for each performance test
|
|
that uses test program "foo".
|
|
|
|
In addition to writing the test driver .exp script, one must also
|
|
write a python script that is used to run the test.
|
|
This contents of this script is defined by the performance testsuite
|
|
harness. It defines a class, which is a subclass of one of the
|
|
classes in gdb.perf/lib/perftest/perftest.py.
|
|
See gmonster-null-lookup.py for an example.
|
|
|
|
Note: Since gmonster1 and gmonster2 are treated as being variations of
|
|
the same program, each test shares the same python script.
|
|
E.g., gmonster1-null-lookup.exp and gmonster2-null-lookup.exp
|
|
both use gmonster-null-lookup.py.
|
|
|
|
Running performance tests for generated programs
|
|
************************************************
|
|
|
|
There are two steps: build and run.
|
|
|
|
Example:
|
|
|
|
bash$ make -j10 build-perf RUNTESTFLAGS="gmonster1.exp"
|
|
bash$ make -j10 check-perf RUNTESTFLAGS="gmonster1-null-lookup.exp" \
|
|
GDB_PERFTEST_MODE=run
|