在21世紀做自動微分?你需要Zygote.jl!
IB501 13:10 ~ 14:00 漢語在現今得深度學習研究中,基於微分的優化技術是非常重要的,如何快速有效的計算微分值也成為了同要重要的議題,這個演算法叫做「自動微分」,透過這樣的自動微分系統,我們可以輕鬆的做許多不同的模型而不需要手動去推導微分公式。同時可微分編程概念的出現,讓計算微分這件事情不再只是優化需要去考慮的問題,然而要將自動微分系統與現有的軟體開發結合卻不是那麼容易的事情。在這場演講中,我會先簡單介紹過去所使用的自動微分技術接著進入我們的重點 —— Zygote.jl 這個 Julia 的套件以及他所提出的21世紀的自動微分技術。 Gradient-based optimization is a very important technique in modern deep learning research. The way of efficiently computing gradient of a model becomes an essential issue. Automatic Differentiation (AD) allow one not paying much effort on derivation of a model. Thus, we can focus on the architecture of model, regardless of the gradient. Meanwhile, the idea of Differentiable Programming shows up. Therefore, gradient calculation is not limit to optimization field. However, integration of AD system and the software development stack is not that trivial. I will introduce the basic knowledge about different ways to conduct differentiation in the past and also the AMAZING package Zygote.jl --- the 21st century package for gradient calculation.