Document:         Classics.txt 
File Group:       PCHW Forum Benchmarks & Standards
Creation Date:    19 November 1996
Revision Date:    19 June 2002

Title:            Classic Numeric Benchmarks
Keywords:         BENCHMARK PERFORMANCE LIVERMORE WHETSTONE DHRYSTONE
                  LINPACK MIPS MFLOPS

Abstract:         Pre-compiled codes to run on PCs via Windows 9X,
                  NT, 2K and XP for Livermore Kernels, Whetstone, 
                  Linpack and Dhrystone benchmarks. Performance
                  ratings are saved in log files with numeric
                  results being checked by the programs.

Contributor:      Roy Longbottom 101323.2241@compuserve.com
                       or     Roy_Longbottom@compuserve.com


CLASSIC NUMERIC BENCHMARKS - THE PC COLLECTION


1.  INTRODUCTION

There are many standard benchmarks available to run on PCs but not a lot of 
the "load and go" variety for demonstrating potential hardware performance 
on scientific number crunching applications. Pre-compiled codes have been 
produced for the benchmarks that set performance standards and influenced 
the design of computers from the early days. The benchmarks and main areas 
of influence are:

             Livermore Kernels    -    Supercomputers

             Whetstone Benchmark  -    Minicomputers

             Dhrystone Benchmark  -    Unix systems

             Linpack Benchmarks   -    Workstations 


The benchmarks were compiled using the now unavailable Watcom 10.5 C/C++
compiler and optimised results checked to verify that measured speeds 
were of the same order of magnitude as the best from other compilers. Non-
optimised codes are also provided to show worst case performance and 
identify possible over-optimisation.

This package is for recompiled NT versions for Win9X/NT/2K/XP. The numeric
results are not now saved in the log files but checked by the programs with
any errors being logged. The Windows high resolution timer is used in order
to reduce the rather excessive run times necessary to obtain accurate speed
measurements. Watcom 10.5 (rather than 11.0) was used to produce the same
speeds as earlier versions.

The Livermore Loops benchmark has been shown to increase CPU temperatures
on some PCs and occasionally produce the wrong numeric results on over-
clocked systems. The benchmark runs each of the 24 kernels multiple times
to produce the results. My version repeats these an extra number of times
(see Passes E in table below) for timing purposes. The standard benchmark
only check results from the last E pass. Default running time for each
kernel is now 1 second rather than 5 seconds but the program can now be
run as a reliability test using a command line (e.g. LinpCONT Runsecs 10)
to increase running times. When the command is used, the program checks
each E pass for correct results.

Except for the Whetstone benchmark, the source codes were obtained from the 
accepted repositories via Internet. Although the main execution code was 
unchanged, timing arrangements and the reporting procedures were modified 
to meet a common standard:

a.  automatic run time calibration rather than fixed parameters, taking
    into account the poor PC timer resolution

b.  initial calibration with display to show linearity
 
c.  results displayed at reasonable rate for viewing and at a fairly
    constant rate on PCs of different speeds

d.  details of CPU and OS used are detected 

e.  date run included in code
 
f.  constant floating point results to confirm that all PCs produce
    "correct" results.
 
g.  results, speeds, run details appended to a text file and, where
    appropriate, in a format suitable to read into a spreadsheet.


Send the text results files with one sample of each run and system details 
fully completed to Roy Longbottom  101323.2241@compuserve.com


2.  THE BENCHMARK COLLECTION

2.1  ZIP FILES

Four versions of the benchmark collection are available all of which 
produce 32 bit executables:

BENCHDOS.ZIP  600K bytes for programs to run via DOS or an NT DOS Window.
              With Windows 3.1 and Windows 95, the best results are likely
              to be obtained in direct DOS mode. This collection also
              includes the DOS/4GW protected run-time program - Copyright
              (c) Rational Systems Inc.

BENCHWIN.ZIP  600K bytes for programs which run in a window via Windows 3.1
              Windows 95 or Windows NT. The window allows copy and paste.

BENCHNT.ZIP   280K bytes for programs that run via Windows 95 or NT. The
              programs are run in a DOS window due to limitations of
              display width of the compiler default window. This
              is for original version containing only EXE files.

BENCHOS2.ZIP  240K bytes for programs that run via OS/2. These were
              compiled via Windows 95 and are released untested via OS/2.


2.2  EXE FILES

BENCHNT.ZIP contains the following benchmarks. BENCHDOS files have a "D"
suffix, instead of "W", BENCHWIN have "W" with "O2" for BENCHOS2:

  DHRY1NNT.EXE    Dhrystone benchmark 1.1                   non-optimised
  DHRY1ONT.EXE    Dhrystone benchmark 1.1                   optimised
  DHRY2NNT.EXE    Dhrystone benchmark 2.1                   non-optimised
  DHRY2ONT.EXE    Dhrystone benchmark 2.1                   optimised
  LINPCNNT.EXE    Linpack "C" double precision benchmark    non-optimised
  LINPCONT.EXE    Linpack "C" double precision benchmark    optimised
  LIVECNNT.EXE    Livermore Kernels "C" double precision    non-optimised
  LIVECONT.EXE    Livermore Kernels "C" double precision    optimised
  WHETCNNT.EXE    Whetstone "C" benchmark single precision  non-optimised
  WHETCONT.EXE    Whetstone "C" benchmark single precision  optimised
  WHETDCNT.EXE    Whetstone "C" benchmark double precision  optimised


3.  STANDARD OUTPUT

All programs include the following in the output text file (example).

 Livermore Loops Benchmark Optimised via C/C++ Wed Jun 19 15:24:05 2002

 Performance numbers

 Numeric results were as expected

 Windows NT Version 5.0, build 2195, Service Pack 2
 CPU AuthenticAMD, Features Code 0183F9FF, Model Code 00000630, 950 MHz
 Memory 523760 KB, Free 435540 KB

The examples below are for the computer identified in this output and show 
the results that are included in the output text file.
 

4.  LIVERMORE KERNELS (LIVERMORE LOOPS)

This supercomputer benchmark was first introduced in 1970, initially 
comprising 14 kernels of numerical application, written in Fortran. The 
numbers of kernels were increased to 24 in the 1980's. Performance 
measurements are in terms of Millions of Floating Point Operations Per 
Second or MFLOPS. The program also checks the results for computational 
accuracy. One main aim was to avoid producing single number performance 
comparisons, the 24 kernels being executed three times at different Do-loop 
spans to produce short, medium and long vector performance measurements. If 
overall averages are quoted, the benchmark reference below indicates that 
the geometric mean may be interpreted as a characteristic rate of 
computation but it would be more realistic to retain the range of 
statistics in terms of geometric, harmonic and arithmetic means, minimum 
and maximum.

For the uninitiated, performance of supercomputers depends upon the data 
being in arrays and the provision of functional units that can complete 
executions at the rate of one per clock cycle plus a compiler which can 
generate the appropriate vector instructions. The supercomputers usually 
have a multiply unit and an add unit, where the output from one can be 
directly connected to the input of the other with a different data stream. 
Thus a loop of the form a(i) = a(i) + b(i) * c(i) can produce two results 
per clock cycle, for example a peak of 160 MFLOPS on the 80 MHz Cray 1. 
Faster peak performance can be obtained by having multiple pipelines or 
multiple processors. In practice, few real programs are capable of 
producing this peak performance on a continuous basis. The best achieved on 
the Cray 1 with the CFT compiler was 82 MFLOPS on one kernel, worst 1.57 
MFLOPS and geometric mean of 11.9 MFLOPS. The latter mean is not much 
better than the results shown below for a downmarket 100 MHz Pentium based 
PC.

The "C" version was produced later but this requires Fortran procedures for 
data generation, statistical analysis and report generation.


4.1  REFERENCE  

F.H. McMahon,   The Livermore Fortran Kernels: A Computer Test Of The 
Numerical Performance Range, Lawrence Livermore National Laboratory, 
Livermore, California, UCRL-53745, December 1986.


ORIGINAL SOURCE CODE   Fortran and "C" netlib.att.com:/netlib/benchmark


FORTRAN RESULTS        In Fortran source code and Reference. Geometric mean
                       results for Alpha based systems via www.digital.com


4.2  CHANGES FOR PC VERSION

The Fortran version of the benchmark has 8000 lines of code and 
documentation or about 4000 lines excluding documentation. The "C" version 
has some 1000 lines of code and relies upon using Fortran routines for data 
generation and result analysis. The package probably needs over 1M bytes of 
memory, mainly for double precision floating point data arrays.

This version, with around 2000 lines of code, is essentially the "C" 
program with data generation, result production and accuracy checking 
converted from the Fortran code. Comprehensive facilities for statistical 
and sensitivity analysis have been omitted. The program generates 60K bytes 
of machine code (plus up to 260 Kb for the particular run time option) and 
600 Kb of data storage, still double precision but lower than the original.

As for all of the benchmarks in the collection, the main computational 
sections are repeated many times. The original has one global value for the 
number of repeats that tends to make the overall running time too long when 
adjusted to provide accurate timing for the fastest kernel. This version 
adjusts the number of repeat passes for each kernel independently with the 
aim that each should run for 5 seconds.

The overhead and calibration times are displayed as are results on 
completion of each kernel (see below). The OK column represents the number 
of correct significant digits out of 16 as calculated in the Fortran 
listing. On completion, running details can be typed in and these are 
appended to file llloops.txt along with the results.


4.3  SCREEN OUTPUT

The output and calibration are displayed as follows, the example being for 
the 100 MHz Pentium system shown above. The main calibration is carried out 
until each kernel has recorded at least 0.07 seconds (the clock is updated 
at intervals of 0.05 or 0.06 seconds). The extra loops required for a first 
estimate of 5 seconds are shown under E in the results table. The times for 
calibration include an overhead 24 occurrences of data generation that is 
excluded from the main timing, in this case about 0.6 seconds.


L.L.N.L. 'C' KERNELS: MFLOPS   P.C.  VERSION

Calculating outer loop overhead
      1000 times   0.00 seconds
     10000 times   0.00 seconds
    100000 times   0.06 seconds
   1000000 times   0.38 seconds
   2000000 times   0.82 seconds
   4000000 times   1.70 seconds
   8000000 times   3.30 seconds
  16000000 times   6.48 seconds
Overhead for each loop  4.0500e-007 seconds

Calibrating part 1 of 3

Loop count          4  0.93 seconds
Loop count         16  2.09 seconds
Loop count         32  3.57 seconds
Loop count         64  6.64 seconds
Loop count        128 12.58 seconds

Loops  200 x  1 x Passes

Kernel       Floating Pt ops
No  Passes E No    Total      Secs.  MFLOPS Span     Checksums          OK
------------ -- ------------- ----- ------- ---- ---------------------- --
 1   7 x  15  5 1.051050e+008  5.20   20.21 1001 5.114652693224671e+004 16
 2  67 x  20  4 1.039840e+008  4.83   21.52  101 1.539721811668384e+003 15
 3   9 x  19  2 6.846840e+007  5.43   12.62 1001 1.000742883066364e+001 15
 4  14 x  30  2 1.008000e+008  5.96   16.92 1001 5.999250595473891e-001 16
 5  10 x  10  2 4.000000e+007  4.61    8.67 1001 4.548871642387267e+003 16
 6   3 x  19  2 4.523520e+007  4.61    9.82   64 4.375116344729986e+003 16
 7   4 x  10 16 1.273600e+008  4.50   28.32  995 6.104251075174761e+004 16
 8  10 x   7 36 9.979200e+007  5.16   19.32  100 1.501268005625795e+005 15
 9  36 x   6 17 7.417440e+007  5.03   14.74  101 1.189443609974981e+005 16
10  34 x   4  9 2.472480e+007  4.99    4.96  101 7.310369784325296e+004 16
11  11 x  12  1 2.640000e+007  4.55    5.80 1001 3.342910972650109e+007 16
12  12 x  21  1 5.040000e+007  4.64   10.86 1000 2.907141294167248e-005 16
13  36 x   3  7 9.676800e+006  5.10    1.90   64 1.202533961842804e+011 15
14   2 x   4 11 1.761760e+007  5.71    3.09 1001 3.165553044000335e+009 15
15   1 x  12 33 3.960000e+007  4.50    8.80  101 3.943816690352044e+004 15
16  25 x  21 10 5.565000e+007  4.90   11.36   75 5.650760000000000e+005 16
17  35 x   7  9 4.454100e+007  5.03    8.85  101 1.114641772902486e+003 16
18   2 x   9 44 7.840800e+007  4.95   15.84  100 1.015727037502299e+005 15
19  39 x  20  6 9.453600e+007  5.82   16.25  101 5.421816960147207e+002 16
20   1 x  12 26 6.240000e+007  5.38   11.60 1000 3.040644339351238e+007 16
21   1 x   1  2 2.525000e+007  6.20    4.07  101 1.597308280710200e+008 15
22  11 x  12 17 4.532880e+007  5.43    8.35  101 2.938604376566698e+002 15
23   8 x   9 11 7.840800e+007  5.37   14.59  100 3.549900501563624e+004 15
24   5 x  30  1 3.000000e+007  4.93    6.09 1001 5.000000000000000e+002 16

                     Maximum   Rate   28.32 
                     Average   Rate   11.86 
                     Geometric Mean    9.97 
                     Harmonic  Mean    7.88 
                     Minimum   Rate    1.90 

                     Do Span    471

Calibrating part 2 of 3

Loop count          8  0.93 seconds
Loop count         32  1.98 seconds
Loop count         64  3.40 seconds
Loop count        128  6.21 seconds
Loop count        256 11.86 seconds

Loops  200 x  2 x Passes

Kernel       Floating Pt ops
No  Passes E No    Total      Secs.  MFLOPS Span     Checksums          OK
------------ -- ------------- ----- ------- ---- ---------------------- --
 1  40 x  15  5 1.212000e+008  4.73   25.61  101 5.253344778937972e+002 16
 2  40 x  21  4 1.303680e+008  6.07   21.46  101 1.539721811668384e+003 15
 3  53 x  21  2 8.993040e+007  5.32   16.91  101 1.009741436578952e+000 16
 4  70 x  32  2 1.075200e+008  5.40   19.92  101 5.999250595473891e-001 16
 5  55 x  13  2 5.720000e+007  4.72   12.11  101 4.589031939600982e+001 16
 6   7 x  19  2 5.107200e+007  5.31    9.62   32 8.631675645333210e+001 16
 7  22 x  12 16 1.706496e+008  5.51   30.99  101 6.345586315784055e+002 16
 8   6 x   6 36 1.026432e+008  5.27   19.46  100 1.501268005625795e+005 15
 9  21 x   5 17 7.211400e+007  4.92   14.65  101 1.189443609974981e+005 16
10  19 x   4  9 2.763360e+007  5.54    4.99  101 7.310369784325296e+004 16
11  64 x  21  1 5.376000e+007  4.72   11.38  101 3.433560407475758e+004 16
12  68 x  20  1 5.440000e+007  4.39   12.39  100 7.127569130821465e-006 16
13  41 x   3  7 1.102080e+007  5.75    1.92   32 9.816387810944356e+010 15
14  10 x   4 11 1.777600e+007  5.43    3.27  101 3.039983465145392e+007 15
15   1 x   8 33 5.280000e+007  5.99    8.82  101 3.943816690352044e+004 15
16  27 x  21 10 6.350400e+007  5.68   11.18   40 6.480410000000000e+005 16
17  20 x   6  9 4.363200e+007  4.87    8.96  101 1.114641772902486e+003 16
18   1 x   9 44 7.840800e+007  4.94   15.88  100 1.015727037502299e+005 15
19  23 x  16  6 8.920320e+007  5.49   16.25  101 5.421816960147207e+002 16
20   8 x   9 26 7.488000e+007  6.03   12.42  100 3.126205178815432e+004 16
21   1 x   2  2 5.000000e+007  7.03    7.11   50 7.824524877232093e+007 16
22   7 x   9 17 4.326840e+007  5.15    8.40  101 2.938604376566698e+002 15
23   5 x   8 11 8.712000e+007  5.92   14.71  100 3.549900501563624e+004 15
24  31 x  30  1 3.720000e+007  5.95    6.25  101 5.000000000000000e+001 16

                     Maximum   Rate   30.99 
                     Average   Rate   13.11 
                     Geometric Mean   11.13 
                     Harmonic  Mean    8.77 
                     Minimum   Rate    1.92 

                     Do Span     90

Calibrating part 3 of 3

Loop count         32  0.83 seconds
Loop count        128  1.54 seconds
Loop count        256  2.53 seconds
Loop count        512  4.56 seconds
Loop count       1024  8.67 seconds

Loops  200 x  8 x Passes

Kernel       Floating Pt ops
No  Passes E No    Total      Secs.  MFLOPS Span     Checksums          OK
------------ -- ------------- ----- ------- ---- ---------------------- --
 1  28 x  22  5 1.330560e+008  5.10   26.09   27 3.855104502494961e+001 16
 2  46 x  22  4 7.124480e+007  4.18   17.03   15 3.953296986903060e+001 16
 3  37 x  23  2 7.352640e+007  4.12   17.85   27 2.699309089320672e-001 16
 4  38 x  34  2 6.201600e+007  4.49   13.80   27 5.999250595473891e-001 16
 5  40 x  35  2 1.164800e+008  6.73   17.30   27 3.182615248447483e+000 16
 6  21 x  32  2 5.160960e+007  4.78   10.79    8 1.120309393467088e+000 15
 7  20 x  16 16 1.720320e+008  5.56   30.93   21 2.845720217644024e+001 16
 8   9 x   8 36 1.078272e+008  5.50   19.59   14 2.960543667875005e+003 15
 9  26 x  16 17 1.697280e+008  5.72   29.67   15 2.623968460874250e+003 16
10  25 x   7  9 3.780000e+007  5.05    7.49   15 1.651291227698265e+003 16
11  46 x  36  1 6.888960e+007  6.34   10.87   27 6.551161335845770e+002 16
12  48 x  36  1 7.188480e+007  5.86   12.27   26 1.943435981130448e-006 16
13  31 x   4  7 1.111040e+007  5.80    1.92    8 3.847124199949431e+010 15
14   8 x   6 11 2.280960e+007  5.07    4.50   27 2.923540598672009e+006 15
15   1 x  12 33 4.435200e+007  4.82    9.20   15 1.108997288134785e+003 16
16  14 x  21 10 5.174400e+007  4.42   11.71   15 5.152160000000000e+005 16
17  26 x   9  9 5.054400e+007  5.28    9.58   15 2.947368618589360e+001 16
18   2 x  12 44 1.098240e+008  5.31   20.67   14 9.700646212337040e+002 16
19  28 x  16  6 6.451200e+007  3.99   16.17   15 1.268230698051004e+001 15
20   7 x   9 26 6.814080e+007  5.18   13.16   26 5.987713249475302e+002 16
21   1 x   2  2 8.000000e+007  6.15   13.01   20 5.009945671204667e+007 16
22   8 x  12 17 3.916800e+007  4.61    8.50   15 6.109968728263973e+000 16
23   7 x  16 11 1.281280e+008  4.93   26.00   14 4.850340602749970e+002 16
24  23 x  20  1 1.913600e+007  3.05    6.27   27 1.300000000000000e+001 16

                     Maximum   Rate   30.93 
                     Average   Rate   14.76 
                     Geometric Mean   12.60 
                     Harmonic  Mean    9.91 
                     Minimum   Rate    1.92 

                     Do Span     19

                Overall

                Part 1 weight 1
                Part 2 weight 2
                Part 3 weight 1

                     Maximum   Rate   30.99 
                     Average   Rate   13.21 
                     Geometric Mean   11.17 
                     Harmonic  Mean    8.77 
                     Minimum   Rate    1.90 

                     Do Span    167


4.4  REPORTED RESULTS

With such a large number of performance measures it is advisable to produce 
a reduced set of figures for comparison purposes. Suppliers have often just 
quoted the final weighted geometric mean. The Livermore Fortran listing 
quotes the overall weighted averages plus MFLOPS for each kernel at Do 
Span 471. These are the ones collected for results files here.

Rather than repeating all the above, the revised version just shows the
MFLOPS ratings and some comparisons for use in file CPUSpeed.txt. Example:

 MFLOPS for 24 loops
 841.5 436.6 156.4 360.4  98.8 144.3 1053.3 944.7 889.7  51.3  67.4 234.9
  22.3  56.9 164.4 147.2  91.5 614.1  129.9 128.0 260.5 171.3 323.7 101.2

 Overall Ratings
 Maximum Average Geomean Harmean Minimum
  1053.3   298.5   187.1   119.0    22.1

 Classic Benchmark Float Loops Ratings for where 100 MHz Pentium = 100
 Maximum Geomean Minimum
    3392    1628    1125

When used as a reliability test the following is also reported:

 Reliability test   2 seconds each loop x 24 x 3

 Part 1 of 3 start at Thu Jun 20 15:16:25 2002

 Part 2 of 3 start at Thu Jun 20 15:17:14 2002

 Part 3 of 3 start at Thu Jun 20 15:18:03 2002

The above might identify the time of any crash as overheating can cause
the CPU to generate errors other than incorrect computations.


5.  WHETSTONE BENCHMARK

The Whetstone benchmark was written by Harold Curnow of CCTA, the British 
government computer procurement agency, based on work by Brian Wichmann of 
the National Physical Laboratory. An Algol version of the benchmark was 
released in November 1972 and Fortran single and double precision varieties 
in April 1973. The Fortran codes became the first general purpose 
benchmarks that set industry standards of performance. 

The benchmark produced speed ratings in terms of Thousands of Whetstone 
Instructions Per Second (KWIPS). In 1978, self timing versions (by Roy 
Longbottom also of CCTA) produced speed ratings in MOPS and MFLOPS and 
overall rating in MWIPS (see below).

Although long overtaken by superior performance standards, comparative 
results are little different from current popular benchmarks. Whetstone 
programs also addressed the question of the efficiency of different 
languages, an important issue not covered to the same extent by other 
contemporary standard benchmarks (see Whets.tbl). 


5.1  REFERENCES

H J Curnow and B A Wichmann, "A Synthetic Benchmark", Computer Journal Vol
19, No 1 1976

H J Curnow, "Whither Whetstone?", Seminar on Software for Parallel
Computers, June 1989

Vector processor version - R Longbottom, "Performance of Multi-user
Supercomputing Facilities", 4th International Conference on Supercomputing,
April 1989


SOURCE CODE       Whets.c can be obtained from ftp.nosc.mil/pub/aburto
                  This is a C or C++, single or double precision version
                  and includes various timers for running on systems other
                  than PCs. Versions in the same format are also available
                  in Fortran, Qbasic and Visual Basic. The benchmarks are
                  approved by Harold Curnow 100421.1615@compuserve.com.

RESULTS           Whets.tbl can be obtained from ftp.nosc.mil/pub/aburto
                  This includes results of systems from the 1960's to
                  modern computers. System costs are also given to enable
                  historic comparisons. Results for Alpha based systems can
                  be obtained via www.digital.com.


5.2  CHANGES FOR PC VERSION

All official versions produce speed ratings for the different loops. The 
pre-compiled PC versions append the results to whets.txt. Whets.c puts the 
results in whets.res. Also, the numeric results produced are constant, 
irrespective of running time.


5.3  SCREEN OUTPUT

The output and calibration are displayed as follows, the example being for 
the 100 MHz Pentium system shown above.

   Single Precision C/C++ Whetstone Benchmark
 
   Calibrate
         0.17 Seconds          1   Passes (x 100)
         0.77 Seconds          5   Passes (x 100)
         3.70 Seconds         25   Passes (x 100)
 
   Use 676  passes (x 100)
 
   676 passes are used for an approximate duration of 100
   seconds, providing an initial estimate of a speed rating
   of 67.6 MWIPS.


          Single Precision C/C++ Whetstone Benchmark

Loop content                 Result            MFLOPS     MOPS    Seconds

N1 floating point    -1.12475025653839100      26.751               0.478
N2 floating point    -1.12274754047393800      17.148               5.220
N3 if then else       1.00000000000000000               19.922      3.460
N4 fixed point       12.00000000000000000               15.978     13.130
N5 sin,cos etc.       0.49904659390449520                2.663     20.810
N6 floating point     0.99999988079071040      10.077              35.650
N7 assignments        3.00000000000000000               22.877      5.380
N8 exp,sqrt etc.      0.75110864639282230                1.513     16.370
 
MWIPS                                          66.270             100.498


          Double Precision C/C++ Whetstone Benchmark
 
Loop content                 Result           MFLOPS      MOPS   Seconds

N1 floating point    -1.12398255667391900     26.548               0.486
N2 floating point    -1.12187079889284400     16.542               5.460
N3 if then else       1.00000000000000000               19.647     3.540
N4 fixed point       12.00000000000000000               15.680    13.500
N5 sin,cos etc.       0.49902937281515140                3.019    18.520
N6 floating point     0.99999987890802820      9.977              36.330
N7 assignments        3.00000000000000000               22.620     5.490
N8 exp,sqrt etc.      0.75100163018457870                1.493    16.740

MWIPS                                         67.156             100.066

Note different numeric results to single precision. Slight variations are 
normal with different compilers and sometimes optimisation levels.


5.4  REPORTED RESULTS

An example of results reported in the log file is as follows. These also
show geometric mean MFLOPS and VAX MIPS (based on last 3 tests shown)

 Whetstone Single Precision Benchmark Optimised via C/C++ 

 MFLOPS    Vax  MWIPS MFLOPS MFLOPS MFLOPS    Cos    Exp  Fixpt     If  Equal
  Gmean   MIPS            1      2      3    MOPS   MOPS   MOPS   MOPS   MOPS
    158   1420    731    189    176    120   20.8   15.8    501    194    236

 Classic Benchmark Ratings for CPUSpeed.txt where 100 MHz Pentium = 100
 Integer Whets 1463, Float Whets 948

 Whetstone Single Precision Benchmark Non-optimised via C/C++ 

 MFLOPS    Vax  MWIPS MFLOPS MFLOPS MFLOPS    Cos    Exp  Fixpt     If  Equal
  Gmean   MIPS            1      2      3    MOPS   MOPS   MOPS   MOPS   MOPS
    124    736    496    172    151   73.5   17.7   15.1    284    183   61.5

Note that non-optimised IF MOPS can be faster than optimised as, for the
latter, the compiler translates If statements into more complex code 
which appears to defeat branch prediction.


6.  DHRYSTONE BENCHMARKS

The Dhrystone "C" benchmark, a sort of Whetstone without floating point,
became the key standard benchmark, from 1984, with the growth of Unix 
systems. The first version was produced by Reinhold P. Weicker in ADA and 
translated to "C" by Rick Richardson.

Two versions are available Dhrystone versions 1.1 and 2.1. The second 
version was produced to avoid over-optimisation problems encountered with 
version 1. Although it is recommended that advanced optimisation levels 
should be avoided with the latter, it is clear from published results that 
the recommendation is usually ignored. The default option in the Watcom 
compiler produces high levels of optimisation and omits some constant 
calculations from the timing loop. Version 2 is compiled from three source 
files.

Original versions of the benchmark gave performance ratings in terms of 
Dhrystones per second. This was later changed to VAX MIPS by dividing 
Dhrystones per second by 1757, the DEC VAX 11/780 result.

 

6.1  REFERENCE

Reinhold P. Weicker, CACM Vol 27, No 10, 10/84,pg.1013


ORIGINAL SOURCE CODE   ftp.nosc.mil/pub/aburto (.sha format needs
                       unshar.exe via unshar.zip from same site)

RESULTS TABLE          ftp.nosc.mil/pub/aburto (dhry.tbl text format)

OTHER RESULTS          DEC Alpha based systems via www.digital.com
                       In Dhrystones per second (divide by 1757)


6.2  CHANGES FOR NEW VERSIONS

No changes were made to the benchmark code. Automatic calibration was 
included, the original (from nosc.mil) requiring manual input for run time 
parameters. There is no checking for correct output in version 1. One line 
of output was added to show the number of passes (+10) and to check that 
the loop index register does not overflow.


6.3  SCREEN OUTPUT

Dhrystone Benchmark, Version 1.1 (Language: C)

Compiler        Watcom C/ C++ 10.5 Win386 
Optimisation      -otexan -zp8 -5r

       10000 runs   0.00 seconds 
      100000 runs   0.38 seconds 
      200000 runs   0.66 seconds 
      400000 runs   1.43 seconds 
      800000 runs   2.80 seconds 
     1600000 runs   5.55 seconds 

Array2Glob8/7: O.K.     1600010

Microseconds for one run through Dhrystone:         3.47 
Dhrystones per Second:                          288288 
VAX  MIPS rating =                                164.08 

Dhrystone Benchmark, Version 2.1 (Language: C)

Compiler        Watcom C/ C++ 10.5 Win386 
Optimisation    -otexan -zp8 -fp5 -5r
Register option not selected

       10000 runs   0.05 seconds 
      100000 runs   0.44 seconds 
      200000 runs   0.94 seconds 
      400000 runs   1.86 seconds 
      800000 runs   3.68 seconds 
     1600000 runs   7.36 seconds 

Final values (* implementation-dependent):

Int_Glob:      O.K.  5  Bool_Glob:     O.K.  1
Ch_1_Glob:     O.K.  A  Ch_2_Glob:     O.K.  B
Arr_1_Glob[8]: O.K.  7  Arr_2_Glob8/7: O.K.     1600010
Ptr_Glob->              Ptr_Comp:       *    -5959980
  Discr:       O.K.  0  Enum_Comp:     O.K.  2
  Int_Comp:    O.K.  17 Str_Comp:      O.K.  DHRYSTONE PROGRAM, SOME STRING
Next_Ptr_Glob->         Ptr_Comp:       *    -5959980 same as above
  Discr:       O.K.  0  Enum_Comp:     O.K.  1
  Int_Comp:    O.K.  18 Str_Comp:      O.K.  DHRYSTONE PROGRAM, SOME STRING
Int_1_Loc:     O.K.  5  Int_2_Loc:     O.K.  13
Int_3_Loc:     O.K.  7  Enum_Loc:      O.K.  1
Str_1_Loc:                             O.K.  DHRYSTONE PROGRAM, 1'ST STRING
Str_2_Loc:                             O.K.  DHRYSTONE PROGRAM, 2'ND STRING

Microseconds for one run through Dhrystone:         4.60 
Dhrystones per Second:                          217391 
VAX  MIPS rating =                                123.73 

The above results (less calibration details) are appended to file Dhry.txt, 
the version 2 results format being slightly different.


6.4 REPORTED RESULTS EXAMPLE

 Dhrystone Benchmark  Version 1.1 Non-optimised via C/C++ 

 VAX MIPS rating:            566.92

 Dhrystone Benchmark  Version 1.1 Optimised via C/C++ 

 VAX MIPS rating:           2205.12

 Dhrystone Benchmark  Version 2.1 Non-optimised via C/C++ 

 VAX MIPS rating:            588.49

 Classic Benchmark Ratings for CPUSpeed.txt where 100 MHz Pentium = 100
 Integer Dhry2 NoOpt 1839

 Dhrystone Benchmark  Version 2.1 Optimised via C/C++ 

 VAX MIPS rating:           1683.19

 Classic Benchmark Ratings for CPUSpeed.txt where 100 MHz Pentium = 100
 Integer Dhry2 Opt 1294


7.  LINPACK BENCHMARK

This benchmark was produced by Jack Dongarra from the "LINPACK" package of 
linear algebra routines. It became the primary benchmark for scientific 
applications from the mid 1980's with a slant towards supercomputer 
performance.

The original version was produced in Fortran but a "C" version appeared 
later. The standard "C" version operates on 100x100 matrices in double 
precision with rolled/unrolled and single/double precision options. The 
pre-compiled versions are double precision, rolled. Other versions are 
available with different sizes of matrices.

Performance rating is in terms of MFLOPS.

7.1  REFERENCE

Jack Dongarra, Performance of Various Computers Using Standard Linear 
Algebra Software in a Fortran Environment from netlib.att.com


ORIGINAL SOURCE CODE   ftp.nosc.mil/pub/aburto (.sha format), with timers
                       netlib.att.com:/netlib/benchmark, "C" and Fortran

RESULTS TABLES         ftp.nosc.mil/pub/aburto (clinpack.dpr text format,
                       other versions .dpu, .spr .spu)
                           netlib.att.com Fortran results

OTHER RESULTS          DEC Alpha based systems via www.digital.com
                       IBM RS/6000 systems from www.ibm.com
                       HP workstations from www.hp.com

The source code used to produce the pre-complied versions is available on 
netlib.att.com as linpack-pc.c (.z format, needs gunzip.exe). The code with 
additional timers for other types of system will shortly be available from 
ftp.nosc.mil.


7.2  CHANGES FOR NEW VERSIONS

The original version executes one pass of the functions a number of times 
then one run with 100 passes of the functions. Timing is based on summing 
the time taken by the main functions. With the speed of current PCs and 
their poor clock resolution, this method of timing is inadequate.

The new version executes the functions once with one pass to produce the 
standard numeric values, calculates data generation overheads separately 
and executes the main timing loops five times for approximately five 
seconds each, determined by calibration.


7.3  SCREEN OUTPUT 

Rolled Double Precision Linpack Benchmark - PC Version in 'C/C++'

Compiler     Watcom C/C++ 10.5 Win386    C++
Optimisation -zp8 -otexan -fp5 -5r -dDP  -dROLL

norm resid      resid           machep         x[0]-1          x[n-1]-1
  0.4   7.41628980e-014  1.00000000e-015 -1.49880108e-014 -1.89848137e-014

Times are reported for matrices of order          100
1 pass times for array with leading dimension of  201

      dgefa      dgesl      total     Mflops       unit      ratio
    0.05000    0.00000    0.05000      13.73     0.1456     0.8929

Calculating matgen overhead
        10 times   0.11 seconds
        20 times   0.22 seconds
        40 times   0.44 seconds
        80 times   0.87 seconds
       160 times   1.76 seconds
       320 times   3.46 seconds
       640 times   6.92 seconds
Overhead for 1 matgen      0.01081 seconds

Calculating matgen/dgefa passes for 5 seconds
        10 times   0.72 seconds
        20 times   1.43 seconds
        40 times   2.80 seconds
        80 times   5.65 seconds
Passes used         70 

Times for array with leading dimension of 201

      dgefa      dgesl      total     Mflops       unit      ratio
    0.05990    0.00157    0.06147      11.17     0.1790     1.0977
    0.05990    0.00143    0.06133      11.20     0.1786     1.0952
    0.05976    0.00157    0.06133      11.20     0.1786     1.0952
    0.05976    0.00243    0.06219      11.04     0.1811     1.1105
    0.06047    0.00157    0.06204      11.07     0.1807     1.1079
Average                                11.13

Calculating matgen2 overhead
Overhead for 1 matgen      0.01089 seconds

Times for array with leading dimension of 200

      dgefa      dgesl      total     Mflops       unit      ratio
    0.06211    0.00157    0.06368      10.78     0.1855     1.1372
    0.06125    0.00243    0.06368      10.78     0.1855     1.1372
    0.06197    0.00157    0.06354      10.81     0.1851     1.1346
    0.06211    0.00157    0.06368      10.78     0.1855     1.1372
    0.06197    0.00157    0.06354      10.81     0.1851     1.1346
Average                                10.79

Rolled Double  Precision       10.79 Mflops 


7.4  REPORTED RESULTS EXAMPLE

 Linpack Double Precision Unrolled Benchmark n @ 100
 Non-optimised via C/C++ Wed Jun 19 15:07:53 2002

 Speed      61.85 MFLOPS

 Linpack Double Precision Unrolled Benchmark n @ 100
 Optimised via C/C++ Wed Jun 19 15:26:03 2002

 Speed     349.28 MFLOPS

 Classic Benchmark Ratings for CPUSpeed.txt where 100 MHz Pentium = 100
 Float Linp 2886


8.  INTERNET ADDRESSES

Compuserve PCHW Forum Library 10 Benchmarks & Standards (Main home)

ftp://ftp.nosc.mil/pub/aburto for benchmark source codes and results tables

http://performance.netlib.org/performance for Performance Database

http://netlib.att.com:/netlib/benchmark  for benchmark codes

http://www.digital.com/www-cgi-
bin/textit?/info/alphaserver/products/perform3.html for AlphaServer results

http://www.digital.com

http://www.rs6000.ibm.com/hardware/  for IBM RS/6000 results

http://www.ibm.com

http://hpcc997.external.hp.com/wsg/products/work/workhome.html HP results

http://www.hp.com

ftp://ftp.cdf.toronto.edu/pub/spectable for SPEC benchmark results


