mirror of
https://github.com/SSCLI/sscli_20021101
synced 2026-06-08 12:28:57 +00:00
yallie
162 lines
6.7 KiB
HTML
162 lines
6.7 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
|
|
<html>
|
|
<head>
|
|
<title>The Shared Source CLI Testing Environment</title>
|
|
<link rel="stylesheet" type="text/css" href="rotor.css">
|
|
</head>
|
|
|
|
<body>
|
|
|
|
<h1>The Shared Source CLI Testing Environment</h1>
|
|
<h3>Contents</h3>
|
|
<ul>
|
|
<li><a href="#Overview">Overview</a></li>
|
|
<li><a href="#PALTestSuite">PAL Test Suite</a></li>
|
|
<li><a href="#SSCLIQualitySuites">SSCLI Quality Suites</a><ul>
|
|
<li><a href="#QualitySuitesTestDriver">Quality Suites Test Driver</a></li>
|
|
<li><a href="#QualitySuitesandBVTs">Quality Suites and BVTs</a></li>
|
|
</ul>
|
|
</li>
|
|
<li><a href="#BuildandLogging">Build and Logging</a></li>
|
|
<li><a href="#TestStrategy">Test Strategy</a></li>
|
|
</ul>
|
|
<h2><a name="Overview"></a>Overview</h2>
|
|
<p>
|
|
The tests in the Shared Source CLI (SSCLI) are broken into two large sections, with each
|
|
section
|
|
broken into several smaller subsections. The large sections are:
|
|
<ul>
|
|
<li><a href="#PALTestSuite">PAL Test Suite</a> - This test suite and test driver
|
|
were developed to act as a quality suite for the
|
|
Platform Adaptation Layer (PAL) which is used to implement the SSCLI on
|
|
multiple platforms.</li>
|
|
<li><a href="#SSCLIQualitySuites">SSCLI Quality Suites</a> - These suites and
|
|
the associated driver test everything else in the system. </li>
|
|
</ul>
|
|
|
|
<h2><a name="PALTestSuite"></a>PAL Test Suite</h2>
|
|
<p>The Platform Adaptation Layer (PAL) Test Suite is designed to validate the
|
|
quality of the PAL. You can run this suites of tests before anything else in the SSCLI is running; therefore, it has no
|
|
dependencies on anything running other than the PAL itself.</p>
|
|
|
|
<p>The driver script for the PAL Test Suite is called <a
|
|
href="../tests/pvtrun.pl">pvtrun.pl</a>. After you build the PAL,
|
|
you can execute the tests by running this script; it will execute around 1200
|
|
tests from the <a
|
|
href="../tests/palsuite">palsuite</a> directory.
|
|
</p>
|
|
|
|
<p>The PAL test harness is located in the <a href="../tests/harness">test_harness directory</a>. The
|
|
test harness uses several environment variables that
|
|
indicate where to find its test list, where to store results
|
|
and where to find its helper driver <a
|
|
href="../tests/harness/xrun">xrun</a>. These environment variables are
|
|
usually set in
|
|
a wrapper script, such as <a href="../tests/pvtrun.pl">pvtrun.pl</a>,
|
|
but they can also be set individually in the environment for manual execution.</p>
|
|
|
|
<p>Additional tests are not run by default from the driver
|
|
script. See the PAL test <a
|
|
href="../tests/palsuite/readme.txt">readme.txt</a> for more details on running the remaining
|
|
suites.</p>
|
|
|
|
<h2><a name="SSCLIQualitySuites"></a>SSCLI Quality Suites</h2>
|
|
|
|
<p> The SSCLI Quality Suites are the tests for everything other
|
|
than the PAL. The test inventory is not a complete breadth test bed but is
|
|
focused on key areas of functionality. </p>
|
|
|
|
<h3><a name="QualitySuitesTestDriver"></a>Quality Suites Test Driver</h3>
|
|
|
|
<p>The main test driver is implemented in <code><a href="../tests/rrun.pl">
|
|
rrun.pl</a></code>. The rrun driver assumes that tests are distributed in a directory tree,
|
|
with their configuration information located in the directory with the
|
|
tests. Tests are required to be self-verifying (no baselines are
|
|
used). The rrun driver traverses the directory tree starting with the working
|
|
directory of the process that invoked the driver and working down the
|
|
directory structure collecting
|
|
the test information. All tests that match a simple set of
|
|
criteria are executed and the results reported.</p>
|
|
|
|
<p>The source comments at the top of the <code><a href="../tests/rrun.pl">
|
|
rrun.pl</a> </code>file should be consulted for more
|
|
complete details of driver operation.</p>
|
|
|
|
<p>Running the rrun driver from the <a href="../tests">tests</a> directory causes the entire test
|
|
inventory to be run using the default criteria. To run a subset
|
|
of tests simply change the working directory to
|
|
the one listed, and then run the rrun driver.</p>
|
|
|
|
<h3><a name="QualitySuitesandBVTs"></a>Quality Suite Tests</h3>
|
|
|
|
<h4>Base Verification Tests</h4>
|
|
|
|
<p>The Base Verification Tests (BVTs) are a set of quick tests that can be run after
|
|
a source modification is made to ensure that basic functionality continues to work
|
|
correctly. These are
|
|
sometimes called "smoke tests." These tests are
|
|
located in <a href="../tests/bvt">tests/bvt</a>.</p>
|
|
|
|
<h4>CIL Tests</h4>
|
|
|
|
<p>The base common intermediate language (CIL) tests are located in <a
|
|
href="../tests/il_bvt">tests/il_bvt</a>. These tests provide core
|
|
coverage of the CIL instructions in the system.</p>
|
|
|
|
<h4>Verifier Tests</h4>
|
|
|
|
<p>Verifier tests check the verifier in the JIT-compiler.
|
|
These tests are located in <a
|
|
href="../tests/security/verifier">tests/security/verifier</a>. In
|
|
addition to these tests, many tests throughout the tree require verification to
|
|
be either off or on; these are marked with the attributes <code><VERIFIERMUSTBEOFF></code> and
|
|
<code><VERIFIERMUSTBEOON></code> in the appropriate
|
|
<code>rsources</code> file.</p>
|
|
|
|
<h4>Base Class Library Tests</h4>
|
|
|
|
<p>You can find tests for the base class libraries in <a
|
|
href="../tests/bcl">tests/bcl</a>.</p>
|
|
|
|
<h4>Miscellaneous Tests and Support Infrastructure</h4>
|
|
|
|
<p>The <a href="../tests/utilities">tests/utilities</a> directory
|
|
contains support functions for other tests. </p>
|
|
|
|
<p>The <a
|
|
href="../tests/dev">tests/dev</a> directory contains test that cover specific
|
|
cases not covered by other tests but discovered during the development process.</p>
|
|
|
|
<h3><a name="BuildandLogging"></a>Build and Logging</h3>
|
|
|
|
<p>The test driver rrun respects build type.
|
|
Executables are built into the obj, objd, objdf subdirectory depending on
|
|
build type (see <a href="buildtools/building_sscli.html">
|
|
buildtools\building_sscli.html</a>).</p>
|
|
|
|
<p>Build log files are named rrun.log and rrun.err on the free build, rrundf.log and
|
|
rrundf.err on the fastchecked build, and rrund.log and rrund.err on the checked
|
|
build.</p>
|
|
|
|
<h3><a name="TestStrategy"></a>Test Strategy</h3>
|
|
|
|
<p>The following are some quick guidelines to help you plan your test usage:</p>
|
|
|
|
<ul>
|
|
<li>To quickly verify that a build or change is working and didn't
|
|
break anything obvious, run the tests in the <a
|
|
href="../tests/bvt">tests/bvt</a> directory. </li>
|
|
<li>To check extensive changes or to verify that you have built without
|
|
problems, use full test run from the <a
|
|
href="../tests">tests</a> directory.</li>
|
|
<li>To run all the tests in the inventory, use the -l option when
|
|
running the rrun driver. Note that this can
|
|
take a long time to complete.</li>
|
|
</ul>
|
|
|
|
|
|
<hr>
|
|
|
|
<p><i>Copyright (c) 2002 Microsoft Corporation. All rights reserved.</i></p>
|
|
</body>
|
|
</html> |