Enhanced Distributed Computation for Machine Learning: Coded Strategies and Multidisciplinary Impact

University of California, San Diego, La Jolla 92093, United States

^* Corresponding author: xit058@ucsd.edu

Abstract

This article explores methods to accelerate distributed computation, focusing on its application in machine learning. It discusses two primary concepts: coded multiplication and data shuffling, along with a non-linear core to Random Access Memory (RAM) approach, presenting new avenues for future research. The challenges and future developments of coded computation in machine learning are examined, along with a general discussion on its applications across various scientific fields. The need for systems capable of handling massive data volumes has led to the adoption of large-scale distributed systems. These systems, however, introduce complexities such as straggler nodes and system failures, which can disrupt computational efficiency. Coded computation offers a solution by introducing redundancy that guards against system failures and enhances data noise mitigation. This paper highlights the strategic importance of coded matrix computation and data shuffling in improving fault tolerance and reducing communication costs within machine learning operations, thereby enhancing the overall efficiency and reliability of large-scale data analytics.