mirror of
https://github.com/rn10950/RetroZilla.git
synced 2024-11-16 12:30:13 +01:00
270 lines
12 KiB
HTML
270 lines
12 KiB
HTML
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
|
||
|
|
<html>
|
||
|
|
<head>
|
||
|
|
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
||
|
|
<meta name="GENERATOR" content="Mozilla/4.7 [en] (X11; U; SunOS 5.8 sun4u) [Netscape]">
|
||
|
|
</head>
|
||
|
|
<body text="#000000" bgcolor="#FFFFFF" link="#0000EE" vlink="#551A8B" alink="#FF0000">
|
||
|
|
|
||
|
|
<h3>
|
||
|
|
<b><font face="Times New Roman,Times">Author Sonja Mirtitsch</font></b></h3>
|
||
|
|
|
||
|
|
<h3>
|
||
|
|
<b><font face="Times New Roman,Times">Last updated: 4/4/2001</font></b></h3>
|
||
|
|
|
||
|
|
<h1>
|
||
|
|
<b><font face="Times New Roman,Times">NSS 3.2.QA Wrapper</font></b></h1>
|
||
|
|
|
||
|
|
<p><br>The QA wrapper tests the nightly builds of NSS. The actual
|
||
|
|
tests are being run are called from the QA script all.sh. I will add documentation
|
||
|
|
for the actual QA soon. The main purpose of the wrapper is: find out which
|
||
|
|
build (NSS version, date, Build Platform) to test on which machine (OS,
|
||
|
|
OS version) and construct a summary report, which is then mailed to the
|
||
|
|
nss developers (aka mailing list nss-qa-report@netscape.com). Please see
|
||
|
|
also the <a href="#advertisement">feature</a> section.
|
||
|
|
<p><a href="#nssqa">nssqa</a> - the script that calls the actual
|
||
|
|
qa script all.sh
|
||
|
|
<br><a href="#qa_stat">qa_stat</a> - sends out status reports
|
||
|
|
<br><a href="#qaclean">qaclean</a> - if everything else fails
|
||
|
|
<p>Sample <a href="/u/sonmi/doc/publish/glob_result.html">global result</a>,
|
||
|
|
<a href="/u/sonmi/doc/publish/results.html">individual result </a>and <a href="/u/sonmi/doc/publish/output.log">log
|
||
|
|
files</a>
|
||
|
|
<p>The QA wrapper consistst mainly of scripts, most located in security/nss/tests
|
||
|
|
and subdirectories, but run from /u/sonmi/bin
|
||
|
|
<p>nssqa and qa_stat, the main scripts both include a common header (<a href="../header">header</a>)
|
||
|
|
and a common environment (<a href="../set_environment">set_environment</a>).
|
||
|
|
<br>Also used is <a href="../mksymlinks">mksymlinks</a> and <a href="../path_uniq">path_uniq</a>
|
||
|
|
and <a href="#qaclean">qaclean</a>.
|
||
|
|
<p>The scripts that are used on a daily basis are located in /u/sonmi/bin
|
||
|
|
and checked into security/nss/tests
|
||
|
|
<p>Parameters and Options are the same for most scripts.
|
||
|
|
<p><a NAME="Parameters"></a><b><u><font size=+1>Parameters</font></u></b>
|
||
|
|
<br> nssversion (supported: 30b, 31, tip, default tip)
|
||
|
|
<br> builddate (default - today, format mmdd)
|
||
|
|
<p><a NAME="Options"></a><b><u><font size=+1>Options</font></u></b>
|
||
|
|
<br> -y answer all questions with y - use at your own
|
||
|
|
risk... ignores warnings
|
||
|
|
<br> -s silent (only usefull with -y)
|
||
|
|
<br> -h, -? -help you guessed right - displays the usage
|
||
|
|
<br> -d debug
|
||
|
|
<br> -f <filename> - write the (error)output to filename
|
||
|
|
<br> -fcron writes resultfile in the same location as
|
||
|
|
would the -cron
|
||
|
|
<br> -m <mailinglist> - send filename to mailinglist
|
||
|
|
(csl) only useful
|
||
|
|
<br> with -f on nssqa
|
||
|
|
<br> -l <mozroot> run on a local build - does not
|
||
|
|
work at this time
|
||
|
|
<br> -cron equivalient to -y -s -d -f $RESULTDIR/$HOST.<scriptname>
|
||
|
|
<br>
|
||
|
|
<p>Please be aware that some iPlanet specific environments and features
|
||
|
|
are being used.
|
||
|
|
<p>-d Debug option might be removed from cron in a few weeks - or maybe
|
||
|
|
not
|
||
|
|
<br>-l QA on local build is not fully implemented yet - will not be implemented,
|
||
|
|
all.sh can be called directly instead
|
||
|
|
<p>Please do not use on Windows 95 and 98, ME platforms yet.
|
||
|
|
<p>use -d if script behaves strange or exits unexpectedly
|
||
|
|
<p><b><font size=+1>How to use the QA-wrapper</font></b>
|
||
|
|
<br>To test a build, first run nssqa on the required QA platforms (some
|
||
|
|
buildplatforms require QA to be run on additional platforms - for example
|
||
|
|
Solaris 2.6 has to be tested on 2.8 32 and 64bit) If QA has been run on
|
||
|
|
multiple or all required platforms it makes sense to run qa_stat on the
|
||
|
|
output of nssqa as well.
|
||
|
|
<br>Before used on a new system (even if the same platform has been tested
|
||
|
|
before) please use completely interactive, to see what the variables are
|
||
|
|
being initialized to, and read the warnings. Same is true if being run
|
||
|
|
from a different user account than svbld.
|
||
|
|
<p>In any case, if you are using it, please let me know the results.
|
||
|
|
<p><a NAME="nssqa"></a><b><u><font size=+1>nssqa:</font></u></b>
|
||
|
|
<p>the script that calls the actual qa script all.sh
|
||
|
|
<p>nssqa <a href="#Parameters">parameters</a> and <a href="#Options">options</a>
|
||
|
|
<p><a href="../nssqa">view the script</a>
|
||
|
|
<p><b><u><font size=+1>Pseudocode Description of nssqa</font></u></b>
|
||
|
|
<br>not quite up to date
|
||
|
|
<p> header:init (global)
|
||
|
|
<br> set flags and variables
|
||
|
|
to default values
|
||
|
|
<br> signal trap (for interupts
|
||
|
|
and kills)
|
||
|
|
<br> set HOST and DOMSUF variables
|
||
|
|
if running from cron
|
||
|
|
<br> parse parameters and options
|
||
|
|
<br> determine os and set up
|
||
|
|
the environment (espec. PATH)
|
||
|
|
<br> set the directories to run
|
||
|
|
in (influenced by parameters and -l option)<br>
|
||
|
|
set the directories for backward
|
||
|
|
compatibility testing
|
||
|
|
<br> set and initialize the tmp
|
||
|
|
/ debugging / output files
|
||
|
|
<p> nssqa:init (local)
|
||
|
|
<br> locking: if nssqa is already
|
||
|
|
running on this systems (yes-exit,
|
||
|
|
<br>
|
||
|
|
no-lockfile)
|
||
|
|
<br> set HOST and DOMSUF variables
|
||
|
|
if running interavtively
|
||
|
|
<br> set flag to kill remaining
|
||
|
|
selfserv processes during cleanup
|
||
|
|
<br> if QA platform different
|
||
|
|
from build platform create neccessary
|
||
|
|
<br>
|
||
|
|
symbolic links
|
||
|
|
<br> wait for the build to finish
|
||
|
|
(max of 5h)
|
||
|
|
<p> main:
|
||
|
|
<br> repeated per test (optimized,
|
||
|
|
debug, 32, 64 bit)
|
||
|
|
<br>
|
||
|
|
set flags for this run of all.sh (optimized, debug, 32, 64 bit)
|
||
|
|
<br>
|
||
|
|
set the DIST directory (where the binaries reside)
|
||
|
|
<br>
|
||
|
|
kill running selfservers (sorry - just don't use the svbld
|
||
|
|
<br>
|
||
|
|
account if you need to do your own testing... I will fix
|
||
|
|
<br>
|
||
|
|
selfserv as soon as I can - but it hangs too often and
|
||
|
|
<br>
|
||
|
|
disturbs all following QA)
|
||
|
|
<br>
|
||
|
|
run all.sh
|
||
|
|
<p> header:exit (global)
|
||
|
|
<br> remove temporary files
|
||
|
|
<p> kill remaining selfservers
|
||
|
|
<br> send email to the list
|
||
|
|
<br>
|
||
|
|
<p> errorhandling
|
||
|
|
<br> Option / Parameter errors:
|
||
|
|
Exit with usage information
|
||
|
|
<p> Severe errors: Exit wit errormessage
|
||
|
|
<br>
|
||
|
|
example: directory in which all.sh resides does not exist
|
||
|
|
<br>
|
||
|
|
can't create files or directories
|
||
|
|
<br>
|
||
|
|
build not done after 5 hours
|
||
|
|
<br>
|
||
|
|
is already running
|
||
|
|
<p> Other errors: User is prompted
|
||
|
|
with the "errormessage - continue (y/n)?"
|
||
|
|
<br>
|
||
|
|
example: local DIST dir does not exist (continues with next all.sh)
|
||
|
|
<br>
|
||
|
|
outputdirectory does not exist (user can specify other)
|
||
|
|
<p> Signals 2, 3, 15 are treated
|
||
|
|
as severe errors
|
||
|
|
<br>
|
||
|
|
<br>
|
||
|
|
<br>
|
||
|
|
<p><img SRC="clean.gif" height=129 width=92 align=LEFT><a NAME="qaclean"></a><b><u><font size=+2>qaclean:</font></u></b>/u/sonmi/bin/qaclean
|
||
|
|
<br>
|
||
|
|
<p>Use qaclean as user "svbld" to get the propper permissions. It is supposed
|
||
|
|
to clean up after a "hanging" QA and will also brutally kill, interupt
|
||
|
|
and disturb any other nss related test or performance meassurement on the
|
||
|
|
named machine. NT and 2000 might require an additional reboot, since the
|
||
|
|
ps is not so good about telling us the actual programmname - so we can't
|
||
|
|
kill them... Please note that this is a brute force script, it should not
|
||
|
|
be used on a regular basis, file a bug whenever you have to use it, since
|
||
|
|
hanging QA is nothing that should occur frequently
|
||
|
|
<p> <a href="../qaclean">view the script</a>
|
||
|
|
<p>What it does:
|
||
|
|
<ol>
|
||
|
|
<li>
|
||
|
|
see if there is a lockfile (/tmp/nssqa.$$ or $TMP/nssqa.$$)</li>
|
||
|
|
|
||
|
|
<br>if yes:
|
||
|
|
<ol>kill the process of the lockfile <font color="#666666">(future expansion
|
||
|
|
and if possible it's children )</font>
|
||
|
|
<br>rm the lockfile</ol>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
kill selfservers</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
kill whatever other qa related processes might be hanging</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
clean up tmp files</li>
|
||
|
|
</ol>
|
||
|
|
<b>QAClean Parameters:</b>
|
||
|
|
<br> machinename.
|
||
|
|
<br> for example
|
||
|
|
<br> qaclean kentuckyderby
|
||
|
|
<br> started on any machine, will clean up on kentuckyderby
|
||
|
|
<p><a NAME="qa_stat"></a><b><u><font size=+2>qa_stat</font></u></b>
|
||
|
|
<p>qa_stat is the script that is being started from the svbld cron on kentuckyderby
|
||
|
|
every morning at 10:00 and runs some (very primitive) analysis on the qa
|
||
|
|
results.
|
||
|
|
<br>I'd like to rewrite the whole thing in perl, and in a few weeks I might
|
||
|
|
just do this...
|
||
|
|
<p> <a href="../qa_stat">view the script</a>
|
||
|
|
<p>qa_stat <a href="#Parameters">parameters</a> and <a href="#Options">options</a>
|
||
|
|
<p><a NAME="advertisement"></a><b><u><font size=+1>Why we need the QA wrapper</font></u></b>
|
||
|
|
<p>We need the new QA wrapper, because we have to test on so many platforms,
|
||
|
|
that running the tests and evaluating the results for the nightly builds
|
||
|
|
took about an average workday.
|
||
|
|
<p><b><font size=+1>New Features:</font></b>
|
||
|
|
<ul>
|
||
|
|
<li>
|
||
|
|
runs from <b>cron</b> / rsh or <b>interactive</b> if desired</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
generates <b>summary</b> (no need to look through 60-90 directories)</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
sends <b>email</b> about results</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
automatically <b>recognizes common errors</b> and problems and conflicts
|
||
|
|
and corrects them</li>
|
||
|
|
|
||
|
|
<br>(or attempts to correct them :-)
|
||
|
|
<li>
|
||
|
|
automatically determines <b>which build </b>to test (waits if build in
|
||
|
|
progress, exits if no build)</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
runs on <b>all required platforms</b> (Windows 98 and before not functional
|
||
|
|
yet)</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
Windows version runs on <b>free Cygnus</b> as well as on MKS</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
debug mode, normal mode and silent mode</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
<b>locking</b> mechanism so it won't run twice</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
<b>cleanup</b> after being killed and most errors (no remaining selfservers,
|
||
|
|
tmpfiles, lock files)</li>
|
||
|
|
</ul>
|
||
|
|
The 1st script is started via cron between 5:00 and 8:00 am on different
|
||
|
|
systems, and starts QA on the nightly build. At 10:00 the next script is
|
||
|
|
started, and sends a QA summary to the nss developers.
|
||
|
|
<p><b>Cygnus Advantages</b>:
|
||
|
|
<ul>
|
||
|
|
<li>
|
||
|
|
<b>free</b></li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
better handling of <b>processes</b> (background, processIDs, Signals)</li>
|
||
|
|
|
||
|
|
<li>
|
||
|
|
Unix / Linux <b>compatible</b> sh / bash</li>
|
||
|
|
</ul>
|
||
|
|
<b>Disadvantages</b>
|
||
|
|
<ul>
|
||
|
|
<li>
|
||
|
|
MKS functionality needs to be preserved (makes <b>8 Windows platforms</b>
|
||
|
|
instead of 4 for the QA suites - makes 32 testruns on Windows alone)</li>
|
||
|
|
|
||
|
|
<br>In certain functionality's <b>slow</b>
|
||
|
|
<br><b></b> </ul>
|
||
|
|
<b>Porting the windows QA to Uwin as well is also being considered</b>
|
||
|
|
</body>
|
||
|
|
</html>
|