Literature by the same author
plus at Google Scholar

Bibliografische Daten exportieren
 

The tensorflow partitioning and scheduling problem: it's the critical path!

Title data

Mayer, Ruben ; Mayer, Christian ; Laich, Larissa:
The tensorflow partitioning and scheduling problem: it's the critical path!
In: Proceedings of the 1st Workshop on Distributed Infrastructures for Deep Learning. - New York : Association for Computing Machinery , 2017 . - pp. 1-6
ISBN 978-1-4503-5169-0
DOI: https://doi.org/10.1145/3154842.3154843

Abstract in another language

State-of-the-art data flow systems such as TensorFlow impose iterative calculations on large graphs that need to be partitioned on heterogeneous devices such as CPUs, GPUs, and TPUs. However, partitioning can not be viewed in isolation. Each device has to select the next graph vertex to be executed, i.e., perform local scheduling decisions. Both problems, partitioning and scheduling, are NP-complete by themselves but have to be solved in combination in order to minimize overall execution time of an iteration. In this paper, we propose several heuristic strategies to solve the partitioning and scheduling problem in TensorFlow. We simulate the performance of the proposed strategies in heterogeneous environments with communication-intensive workloads that are common to TensorFlow. Our findings indicate that the best partitioning and scheduling heuristics are those that focus on minimizing the execution time of the critical path in the graph. Those strategies provide a speed-up of up to 4 times in comparison to strategies that are agnostic to the critical path, such as hash-based partitioning and FIFO scheduling.

Further data

Item Type: Article in a book
Refereed: Yes
Institutions of the University: Faculties
Faculties > Faculty of Mathematics, Physics und Computer Science
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Computer Science > Chair Data Systems
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Computer Science > Chair Data Systems > Chair Data Systems - Univ.-Prof. Dr. Ruben Mayer
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Computer Science
Result of work at the UBT: No
DDC Subjects: 000 Computer Science, information, general works > 004 Computer science
Date Deposited: 26 Apr 2023 08:56
Last Modified: 05 Feb 2024 07:44
URI: https://eref.uni-bayreuth.de/id/eprint/76027