>"Here is the crux of it: Multiplying matrices of numbers is hard, even for computers with matrix multiplier units. But adding banks of numbers is a lot easier. (Even your own neurons like addition more than multiplication.) And the key insight from Tensordyne’s founders is that
if you convert data to logarithms, you can add them and avoid the multiplication overhead entirely.
And, the Napier chip, named after Scottish mathematician, astronomer, and physicist John Napier of the late Renaissance, who invented logarithms and also was an early user of the decimal point and was therefore one of the first geeks to get us on the path to a floating point processing unit, does all of this conversion from matrix to logs all under the covers, invisibly.
Here’s the upshot:
This shift to log math provides more than an order of magnitude better performance
, lower price, and lower power consumption than hybrid architectures from Nvidia and Amazon Web Services."
if you convert data to logarithms, you can add them and avoid the multiplication overhead entirely.
And, the Napier chip, named after Scottish mathematician, astronomer, and physicist John Napier of the late Renaissance, who invented logarithms and also was an early user of the decimal point and was therefore one of the first geeks to get us on the path to a floating point processing unit, does all of this conversion from matrix to logs all under the covers, invisibly.
Here’s the upshot:
This shift to log math provides more than an order of magnitude better performance
, lower price, and lower power consumption than hybrid architectures from Nvidia and Amazon Web Services."