Description:
Reference #: 1690
The University of South Carolina is offering licensing opportunities for Integration of In-Memory Analog Computing Architectures with Tensor Processing Units.
Background:
Tensor processing units (TPUs), specialized hardware accelerators for machine learning tasks, have shown significant performance improvements when executing general matrix-matrix multiplication in machine learning models. However, they struggle to maintain the same efficiency for matrix-vector multiplication (MVM) operations, leading to suboptimal hardware utilization. In-memory analog computing (IMAC) architectures, on the other hand, have demonstrated notable speedup in MVM operations.
Invention Description:
This invention is a hybrid TPU-IMAC architecture that combines the strengths of TPU and IMAC to efficiently execute both matrix-matrix and matrix-vector multiplications. It leverages resistive memory technologies along with conventional transistors that enable faster processing of artificial intelligence operations at low running cost (e.g. power consumption) and with compact size (cheaper devices). This integration also enables the deployment of artificial intelligence models on small IOT devices.
Potential Applications:
The integration of IMAC and TPU architectures has the potential to impact the broad machine-learning landscape, including convolutional neural networks (CNNs) and large language models (LLMs). The TPU-IMAC architecture shows potential for various applications where energy efficiency and high performance are essential, such as edge computing and real-time processing in mobile devices.
Advantages and Benefits:
The simulations demonstrate that the TPU-IMAC configuration achieves up to 2.59× performance improvements and 88% memory reductions compared to conventional TPU architectures for various CNN models while maintaining comparable accuracy. In addition, the invention’s configuration and architecture require lower energy consumption and lower resource utilization in order to achieve comparable accuracy with that of the conventional TPU architectures.