These massively parallel systems have really long set up times and rarely scale well when the applications are written as they are formulated. The SPEC HPC benchmarks are much more indicative of the application performance. Application developers complained behind closed doors that these massively parallel machines were impossible to scale past a few CPUs.

Peak numbers and the RMAX/RPEAK Linpack numbers have become the macho-FLOPS of old. These figures are mostly numbers to impress funding agencies and the public. The game became how many boxes can you get into a warehouse, and how big a warehouse can you build.

What you have now are throughput clusters, whereby my associates have found that they can farm out scalable applications to Amazon and the cloud at a tiny fraction of the cost and hassle of requesting time on these systems.

Are taxpayers are getting value from these efforts? It is time to consider whether these projects are worthwhile.

Buddy is right, and procurement data can be protected from FOIA requests in some cases. However the government doesn’t have to agree to these terms; the costs for some very large military weapons programs (all of which dwarf HPC investments in the United States) are known. With enough citizen or media pressure, the current practice could (and probably should) change.

This is exactly the kind of information I wish was publicly available. it would be interesting to do a real TCO analysis on these massive machines.

Math error not in my favor. In flipping between my teras and petas I left a factor of 1,000 in the equation. Deepest apologies to all. Fixed now. I hope.

On the other hand, the CMOS microprocessor-based massively-parallel systems that came later were very hard to get high performance rates out of, although they made up for it by having much higher peaks.

On the log-scale of history, this isn’t really that important, but the difference is more significant that the article implies. On real programs, the Crays could be 5-10 times better when measured by percent of peak, compared to the MPPs. It’s a shame that the ECL circuits and static RAM of the vector Crays were too expensive and power hungry to scale to the performance levels of the microprocessor MPPs, since the ease-of-programming of the Crays would have saved a lot of time and effort on the part of their users.

I wonder how your chart would change when you add the cost of
cooling, power, software, heroic datacenters. These have risen exponentially over the last ten years.