Functors, Comonads, and Digital Image ProcessingJUSTIN LE, CHAPMAN UNIVERSITY SCHMID COLLEGE OF SCIENCE AND TECHNOLOGY

Let’s talk about Filters

getglasses.com.au

The problem with filters

Categories

Objects Morphisms

Composition

ℤ ℝ

ℙ(people)

(integers)

(real numbers)

𝑟𝑜𝑢𝑛𝑑 :ℝ→ℤ

𝑎𝑔𝑒 :ℙ→ℝ

𝑟𝑜𝑢𝑛𝑑∘𝑎𝑔𝑒 :ℙ→ℤ(𝑟𝑜𝑢𝑛𝑑∘𝑎𝑔𝑒) (𝑝 )=𝑟𝑜𝑢𝑛𝑑 (𝑎𝑔𝑒 (𝑝 ))

𝒮Set

Functors•Objects to Objects

•Morphisms to Morphisms

Ex: Infinite List Functor

𝐿 ( 𝑋 )=𝑋ℕ

X to infinite lists of things in X

92 4, 9, 8, 75, -3, ...∈ℤ ∈𝐿(ℤ)

𝑓 : 𝑋→𝑌𝐿 ( 𝑓 ) :𝐿(𝑋 )→𝐿(𝑌 )

Comonads1. Extract 2. Duplicate 3. Laws , etc.

𝜖 ([ 4 ,9 ,8 ,75 ,−3. .])=4 𝛿 ([4 ,9 ,8 ,75 ,−3. .])=?? ?

Infinite List Functor

Cokleisli Arrows

𝑓 :𝑊 ( 𝑋 )→𝑌 𝑔 :𝑊 (𝑌 )→𝑍

𝑔∘ 𝑓 :𝑊 (𝑋 )→𝑍Comonads only!

Extension

𝑓 :𝑊 ( 𝑋 )→𝑌

𝑓 ∗ :𝑊 (𝑋 )→𝑊 (𝑌 )

Functor 1: “Image with Focus”

𝐼 ( 𝑋 )=ℤ2× 𝑋ℤ2

5 ((5,2 ) ,[⋱ ⋮ ⋮ ⋮ ⋰⋯ 7 3 19 ⋯⋯ 22 4 120 ⋯⋯ 8 79 1 ⋯⋰ ⋮ ⋮ ⋮ ⋱

])∈ℕ ∈ 𝐼 (ℕ)

∈ 𝐼 ( [ 0 ,𝑁 ]⊂ℕ)

Position-Aware Transformation

[0 −1 𝑤1 0 00 0 1 ]

Affine Transformation Matrix

𝐼 (ℕ )→ℕ

𝑓 ,𝑔

𝛼 𝑓 ,𝛼𝑔

𝑓 ∘𝑔

𝛼 𝑓 ∘𝛼𝑔=𝛼 𝑓 ∘𝑔

Compose Affine

Compose Cokleisli

EncodeEncode

Functor 2: “Local Neighborhood”

G

5 [⋱ ⋮ ⋮ ⋮ ⋰⋯ 7 3 19 ⋯⋯ 22 4 120 ⋯⋯ 8 79 1 ⋯⋰ ⋮ ⋮ ⋮ ⋱

]∈ℕ ∈𝐺(ℕ)

Local/Relative Transformations

[ 0 −1 0−1 5 −10 −1 0 ]

Kernel/Convolution Matrix

[⋱ ⋮ ⋮ ⋮ ⋰⋯ 7 3 19 ⋯⋯ 2 5 6 ⋯⋯ 8 1 9 ⋯⋰ ⋮ ⋮ ⋮ ⋱

] 13ℤ𝐺 (ℤ)

𝑓 ,𝑔

𝜅 𝑓 ,𝜅𝑔

𝑓 ∗𝑔

𝜅 𝑓 ∘ 𝜅𝑔=𝜅 𝑓 ∗𝑔

Convolve Kernels

Compose Cokleisli

EncodeEncode

Extensions of I are Decoded Filters

𝑓 : 𝐼 ( 𝑋 )→𝑌 𝑓 ∗ : 𝐼 (𝑋 )→ 𝐼 (𝑌 )

𝑓 ∗ : 𝑋ℕ2

→𝑌ℕ 2

Classical image filter

focus stays same

Commutation Abounds

𝑓 ,𝑔

𝑓 ∗ ,𝑔∗

𝑓 ∘𝑔

𝑓 ∗∘𝑔∗= ( 𝑓 ∘𝑔)∗

Compose Cokleisli

Compose Normally

Extend Extend

Decoded Neighborhoods

𝑓 :𝐺 ( 𝑋 )→𝑌

Γ ( 𝑓 ) : 𝐼 ( 𝑋 )→𝑌

“Globalization”,

Γ ( 𝑓 )∗ : 𝑋ℕ2

→𝑌ℕ2

Classical image filter

𝑓 ,𝑔

Γ ( 𝑓 )∗∘ Γ (𝑔 )∗

Γ ( 𝑓 ∘𝑔 )∗

(Γ ( 𝑓 )∘Γ (g ) )∗Globalize, extend, compose

Cokleisli compose, globalize, extend

Globalize, cokleisli compose, extend

Extension, Globalization are Cheap•Written once: less bugs

•Optimize once, unlimited return on performance

•Trivially parallelizable

•Globalization can handle boundary conditions, low-level

Generalization

𝐼𝑛 (𝑋 )=ℤ𝑛×𝑋ℤ𝑛

𝐺𝑛 ( 𝑋 )=𝑋ℤ𝑛

n Application1 Audio, Time signals

2 Images

3 Video

1000+ Difference Equations

In Conclusion•Better math•Better engineering•Better development process•Better world