Deriving convolution from first principles

By Medium - 2020-10-21

Description

During my undergraduate studies, which I did in Electrical Engineering at the Technion in Israel, I was always appalled that such an important concept as convolution [1] just landed out of nowhere…

Summary

  • Have you ever wondered what is so special about convolution?
  • 2] Technically speaking, what I define here is circular convolution.
  • [ 3] Note that the rows of C(w) have the vector w transposed, resulting in the reflection that appears in the convolution formula and distinguished it from a related notion of correlation.
  • 6] Some often confuse invariance (meaning “unchanged” in Latin) and equivariance (“changing in the same way”), with many signal processing books referring to the property I discuss here as “shift invariance”.

 

Topics

  1. Machine_Learning (0.21)
  2. NLP (0.17)
  3. Backend (0.1)

Similar Articles

Self-Organising Textures

By Distill - 2021-02-27

Neural Cellular Automata learn to generate textures, exhibiting surprising properties.

Python enumerate(): Simplify Looping With Counters

By realpython - 2020-12-15

Once you learn about for loops in Python, you know that using an index to access items in a sequence isn't very Pythonic. So what do you do when you need that index value? In this tutorial, you'll lea ...

The Rules of Margin Collapse

By joshwcomeau - 2020-12-11

“Margin collapse” has a dastardly reputation, one of the trickier parts of CSS. Fortunately, it gets a lot easier once you learn a few rules! In this tutorial, we take a deep dive into the governing p ...