GLite: A Fast and Efficient Automatic Graph-Level Optimizer for Large-Scale DNNs
TimeTuesday, July 12th2:15pm - 2:37pm PDT
Location3000, Level 3
ML Algorithms and Applications
DescriptionWe propose Kroak, an automatic and efficient graph-level optimizer aiming to optimize large computation graph. Kroak discovers the potential to reduce the searching space of the graph rewriting in search-based method. Hence, we employ an graph splitting strategy to divide an large-scale graph into several subgraphs. Then, owing to the observation that the parallelism of DNNs would impact the optimizing effect after graph splitting, we separately design two graph splitting algorithm on low-parallelism DNNs and high-parallelism ones. To avoid redundant subgraph matching, Kroak applies dynamic programming method to store the explored matching pattern and reuse them in subsequence searching iterations.