Literature by the same author
plus at Google Scholar

Bibliografische Daten exportieren
 

Frobenius-Type Norms and Inner Products of Matrices and Linear Maps with Applications to Neural Network Training

Title data

Herzog, Roland ; Köhne, Frederik ; Kreis, Leonie ; Schiela, Anton:
Frobenius-Type Norms and Inner Products of Matrices and Linear Maps with Applications to Neural Network Training.
Bayreuth ; Heidelberg , 2023 . - 14 p.
DOI: https://doi.org/10.48550/arXiv.2311.15419

Official URL: Volltext

Project information

Project title:
Project's official title
Project's id
Multilevel Architectures and Algorithms in Deep Learning
464103607

Project financing: Deutsche Forschungsgemeinschaft

Abstract in another language

The Frobenius norm is a frequent choice of norm for matrices. In particular, the underlying Frobenius inner product is typically used to evaluate the gradient of an objective with respect to matrix variable, such as those occuring in the training of neural networks. We provide a broader view on the Frobenius norm and inner product for linear maps or matrices, and establish their dependence on inner products in the domain and co-domain spaces. This shows that the classical Frobenius norm is merely one special element of a family of more general Frobenius-type norms. The significant extra freedom furnished by this realization can be used, among other things, to precondition neural network training.

Further data

Item Type: Preprint, postprint
Refereed: Yes
Keywords: Frobenius norm; inner product; trace estimation; neural network training; backpropagation; K-FAC preconditioner
Institutions of the University: Faculties
Faculties > Faculty of Mathematics, Physics und Computer Science
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Mathematics
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Mathematics > Chair Mathematics V (Applied Mathematics)
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Mathematics > Chair Applied Mathematics
Faculties > Faculty of Mathematics, Physics und Computer Science > Department of Mathematics > Chair Applied Mathematics > Chair Applied Mathematics - Univ.-Prof. Dr. Anton Schiela
Profile Fields
Profile Fields > Advanced Fields
Profile Fields > Advanced Fields > Nonlinear Dynamics
Result of work at the UBT: Yes
DDC Subjects: 500 Science > 510 Mathematics
Date Deposited: 30 Nov 2023 06:00
Last Modified: 30 Nov 2023 06:00
URI: https://eref.uni-bayreuth.de/id/eprint/87951