Transistors have been continuously scaled to be smaller, faster and more energy-efficient. Yet, wires connecting the transistors, blocks and chips have been unable to be scaled as much as the transistors. As a result, the performance and energy consumption of chips and systems began to be dominated by those of data movements over the wires. Tackling this challenge, many researchers proposed various architectures and circuits that compute near data. These proposals, nonetheless, often fell short to consider various practical constraints imposed by complex ecosystems and practices of the computing industry.

Aiming to impact the industry with innovative yet practical proposals, I have worked closely with the industry to consider the practical constraints and developed hardware and (system) software co-design proposals for computing near data. Among these, for this talk, I chose two proposals that just began to be evaluated by two major hyperscalers for production system deployments. More specifically, in this talk, I first present a computing near memory architecture with a software driver that allows any existing application to run on a server without any change in server hardware and application software. Second, I present a computing in network architecture with an enhanced distributed computing framework software for accelerating distributed deep neural network (DNN) training, where a large fraction of training time is spent for communicating weights and gradients over the network. Demonstrating the feasibility and effectiveness of both the proposals with commercial servers, I implemented the proposed architectures with FPGA cards, one of which is a custom design to be connected to a main memory channel, and necessary supporting software.