John L. Larson Sun Microsystems, Inc.
The presentation describes two projects in the area of performance evaluation. The first project, Parallelism Evaluation Methodology, demonstrates the relationship of parallelism to performance, and shows where various agents change the parallelism on the path to performance. A simulation tool is described which allows the study of executed parallelism within a single CPU of a CRAY Y-MP. A methodology is presented and applied to the measurement and comparison of executed parallelism within several of the Perfect Benchmarks.
In the second project, Benchmark Set Construction Methodology, results are presented on the profile of usage, by populations of academic users in different application areas, on the CRAY Y-MP/464 at the National Center for Supercomputing Applications. Hardware Performance Monitor data collected over a period of one year is analyzed. The collected data contains HPM group 0 performance information for approximately 300,000 jobs representing about 18,000 CPU-hours. This data serves as a specification to be approximated by a benchmark set which represents the workload at this site.
If time allows, a short presentation will be given on the practice of BenchMARKETing: The Art of Selling Inferior Goods.