Comparison of the von Neumann Bottleneck and the Efficiency of the Systolic Array Architecture
Jeonghoon Bang, Seojun Kim, Yoongi Hong, Yesung Choi
This research directly implemented the sequential memory access approach of the Von Neumann architecture and the parallel processing approach of the systolic array architecture using five Arduino Nano-compatible boards, comparing the processing time and the number of data movements for the identical matrix multiplication operation. The control group simulated the Von Neumann access pattern through sequential access routed via global bus variables, while the experimental group implemented an output-stationary (OS) data flow by configuring four processing elements (PEs) in a $2 \times 2$ grid. The internal dimension $K$ of the matrices was varied to 2, 4, 6, and 8, with measurements repeated at least 5 times under each condition. The measurement results showed that the logical number of data element movements was 25% lower for the systolic array across all intervals ($8K$ versus $6K$). However, the processing time was 295 to 326 times slower. The slope ratio of the two regression equations was 313.2, and the intercept was also greater for the grid structure, meaning no break-even point appeared even as $K$ was increased. The cause lay in the difference in cost per data movement: while an in-chip variable access took 3.23 µs, inter-board GPIO communication took 1,346.9 µs—making it approximately 417 times more expensive. Reducing the number of movements by 25% was insufficient to offset this unit cost difference. These results demonstrate that the data reuse principle of the systolic array operated effectively even in this experimental environment. At the same time, the experiment confirmed the prerequisite that for this principle to translate into a performance advantage, communication costs between PEs must be sufficiently lower than memory access costs. The performance of a systolic array is ultimately determined by the physical integration density between PEs. This inversely illustrates the rationale behind the design choice of actual TPUs to integrate hundreds of PEs within a single chip.