Fourier Analysis on the Hypercube, the CoefficientProblem, and Applications
by
Ganesh AjjanagaddeS.B., Massachusetts Institute of Technology (2015)
M.Eng., Massachusetts Institute of Technology (2016)
Submitted to the Department of Electrical Engineering and ComputerScience
in partial fulfillment of the requirements for the degree of
Doctor of Philosophy in Computer Science
at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
May 2020
c Massachusetts Institute of Technology 2020. All rights reserved.
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Department of Electrical Engineering and Computer Science
April 28, 2020Certified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Gregory WornellProfessor of Electrical Engineering and Computer Science
Thesis SupervisorCertified by. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Henry CohnSenior Principal Researcher, Microsoft Research New England
Thesis Supervisor
Accepted by . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Leslie A. Kolodziejski
Professor of Electrical Engineering and Computer ScienceChair, Department Committee on Graduate Students
2
Fourier Analysis on the Hypercube, the Coefficient Problem,
and Applications
by
Ganesh Ajjanagadde
Submitted to the Department of Electrical Engineering and Computer Scienceon April 28, 2020, in partial fulfillment of the
requirements for the degree ofDoctor of Philosophy in Computer Science
Abstract
In this dissertation, we primarily study some problems that revolve around Fourieranalysis. More specifically we focus on the magnitudes of the frequency components.Firstly, we perform a study on the hypercube. It is well known that the Delsarte lin-ear programming bounds provide rich information on the magnitudes of the Fouriercoefficients, grouped by Hamming weight. Classically, such information is primarilyused to attack coding problems, where the objective is to maximize cardinality of asubset of a metric space subject to a minimum distance constraint. Here, we use it tostudy anticoding problems, where the objective is to maximize cardinality of a subsetof a metric space subject to a maximum distance (diameter) constraint. One moti-vation for such study is the problem of finding memories that are cheap to update,where the cost of an update is a function of the distance in the metric space. Such aview naturally supports the study of different cost functions going beyond hard diam-eter constraints. We work accordingly with different cost functions, with a particularemphasis on completely monotonic functions. Our emphasis is on the phenomenon of“universal optimality”, where the same subset (anticode) simultaneously optimizes awide range of natural cost functions. Among other things, our work here gives someanswers to a question in computer science, namely finding Boolean functions withmaximal noise stability subjected to an expected value constraint.
Secondly, we work with Fourier analysis on the integers modulo a number by draw-ing upon Nazarov’s general solution to the “coefficient problem”. Roughly speaking,the coefficient problem asks one to construct time domain signals with prescribedmagnitudes of frequency components, subject to certain natural constraints on thesignal. In particular, Nazarov’s solution works with 𝑙𝑝 constraints in time. Thissolution to the coefficient problem allows us to give an essentially complete resolu-tion to the mathematical problem of designing optimal coded apertures that arises incomputational imaging. However, the resolution we provide is for an 𝑙∞ constrainton the aperture, corresponding to partial occlusion. We believe it is important toalso examine a binary valued (0, 1) constraint on the aperture as one does notneed to synthesize partial occluders for such apertures. We therefore provide some
3
preliminary results as well as directions for future research.Finally, inspired by the recent breakthroughs in understanding the 𝑑 = 8, 24
cases of sphere packing and universal optimality in R𝑑, we attempt to show that theassociated lattices (𝐸8 and the Leech lattice for 𝑑 = 8, 24 respectively) are also optimalfor the problem of vector quantization in the sense of minimizing mean squared error.Accordingly, we develop a dispersion and anticoding based approach to lower boundson the mean squared error. We also generalize Tóth’s method, which shows optimalityof the hexagonal lattice quantizer for 𝑑 = 2, to arbitrary 𝑑. To the best of ourknowledge, these methods give the first rigorous improved lower bounds for the meansquared error for all large enough 𝑑 since the work of Zador over 50 years ago.
Thesis Supervisor: Gregory WornellTitle: Professor of Electrical Engineering and Computer Science
Thesis Supervisor: Henry CohnTitle: Senior Principal Researcher, Microsoft Research New England
4
Acknowledgments
It is a Herculean task to express my gratitude to all who have contributed directly
and indirectly to my adventures in graduate school and in this dissertation. I can only
offer a few succinct remarks here. For those of you not explicitly mentioned here: first
off, I am confident that you are already aware of my deep respect for you, and that you
would not derive much meaning or pleasure from an explicit acknowledgement here.
Secondly, I am grateful for the very fact that you are examining this dissertation.
I hope that you find it at best illuminating, and at worst somewhat entertaining.
Keeping this in my mind, I now turn to some of the most important people, hoping
that you understand how much you have meant to me.
First, I thank my parents, Venkataramana and Vijaya Ajjanagadde. To put it
simply: I am who I am because of you; any merits I might have are due to you, and
any demerits are mine alone.
Second, I thank my thesis advisers, Profs. Gregory Wornell and Henry Cohn. Both
of you have not only been extremely patient, supportive, infectiously optimistic, and
insightful with respect to research, but have also extended it to my development as a
person. It was an honor to be your student.
Third, I thank Prof. Yury Polyanskiy, my thesis committee member. It is pretty
safe to say that I was forged as a researcher in his smithy as an undergraduate. I am
also grateful to him for exposing me to the wonders of the Russian intelligentsia.
Fourth, I acknowledge several professors with whom I have had stimulating inter-
actions over the past eight years here at MIT: Profs. Guy Bresler, Polina Golland,
Alexandre Megretski, Elchanan Mossel, John Tsitsiklis, George Verghese, Alan Will-
sky, and Lizhong Zheng.
Fifth, I acknowledge the friends who have served as a bedrock during my time
here at MIT: Mohamed AlHajri, Nirav Bhan, Kishor Bhat, Austin Collins, Matthew
de Courcy-Ireland, Igor Kadota, Pranav Kaundinya, Joshua Ka-Wing Lee, Eren
Kizildag, Suhas Kowshik, Fabián Kozynski, Pavitra Krishnaswamy, Ashwin Kumar,
Tarek Lahlou, Bhavesh Lalwani, Anuran Makur, Dheeraj Nagaraj, Deepak Narayanan,
5
James Noraky, Or Ordentlich, Rishi Patel, David Qiu, Govind Ramnarayan, Ankit
Rawat, Arman Rezaee, Hajir Roozbehani, Tuhin Sarkar, Anjan Soumyanarayanan,
Harihar Subramanyam, James Thomas, Christos Thrampoulidis, Aditya Venkatra-
mani, and Adam Yedidia.
Sixth, I acknowledge the outstanding labmates (past and present) who have made
the Signals, Information, and Algorithms Lab a special place: Toros Arikan, Yuheng
Bu, Qing He, Tejas Jayashankar, Gauri Joshi, Gary Lee, Emin Martinian, Safa Medin,
Gal Shulkind, Atulya Yellepeddi, and Xuhong Zhang. I also acknowledge the always
friendly and helpful administrative assistant Tricia O’Donnell.
Finally, I thank my extended family. They have been a very deep foundation
during the times when I could not see the light at the end of the tunnel. In particular,
I dedicate this thesis to my grandmothers Parvathi Ajjanagadde and Shankari Bhat,
who instilled a deep love for education and knowledge in their descendants in spite of
very difficult circumstances.
The material in this dissertation is based upon work supported by the National
Science Foundation Graduate Research Fellowship Program under Grant No. CCF-
1816209. Any opinions, findings, and conclusions or recommendations expressed in
this material are those of the author and do not necessarily reflect the views of the
National Science Foundation.
Much of the work presented in Chapter 4 was performed while I was at Microsoft
Research New England, whose support I gratefully acknowledge.
6
Contents
1 Introduction 13
1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.1.1 Hamming space . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.1.2 Discrete Fourier transforms . . . . . . . . . . . . . . . . . . . 15
1.1.3 Euclidean space . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.1.4 General remarks . . . . . . . . . . . . . . . . . . . . . . . . . 17
2 Linear Programming Bounds and Anticodes in Hamming Space 19
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2.1 Linear programming and isodiametry in Hamming space . . . 24
2.2.2 Universal optimality for special subcubes . . . . . . . . . . . . 24
2.2.3 A mean value theorem for noise stability . . . . . . . . . . . . 27
2.2.4 Large 𝑛 and balls versus subcubes . . . . . . . . . . . . . . . . 28
2.3 Linear programming and isodiametry in Hamming space . . . . . . . 28
2.3.1 LP bounds and Fourier analysis on Hamming space . . . . . . 28
2.3.2 Proof of Theorem 1 . . . . . . . . . . . . . . . . . . . . . . . . 39
2.4 Universal optimality of some subcubes . . . . . . . . . . . . . . . . . 47
2.5 Connection to maximum noise stability . . . . . . . . . . . . . . . . . 56
2.6 A mean value theorem for noise stability . . . . . . . . . . . . . . . . 60
2.7 Large 𝑛 and balls versus subcubes . . . . . . . . . . . . . . . . . . . . 62
2.8 Open problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
7
3 Near-Optimal Coded Apertures for Imaging via Nazarov’s Theorem 79
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.2.1 Background on linear estimation . . . . . . . . . . . . . . . . . 88
3.2.2 Model clarifications and intuition . . . . . . . . . . . . . . . . 91
3.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.3.1 Lower bound . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
3.3.2 Upper bound . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.3.3 Greedy algorithm . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.3.4 Simulations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
3.4 Discussion and Future Work . . . . . . . . . . . . . . . . . . . . . . . 111
4 New lower bounds for the mean squared error of vector quantizers119
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.2 Main results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.3 Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
4.3.1 Rederivation of Zador’s lower bound . . . . . . . . . . . . . . 126
4.3.2 The general case . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.3.3 Tóth/Newman’s hexagon theorem . . . . . . . . . . . . . . . . 132
4.3.4 Conjectured bound for lattices . . . . . . . . . . . . . . . . . . 136
4.3.5 Towards a proof for lattices . . . . . . . . . . . . . . . . . . . 140
4.4 Discussion and Future Work . . . . . . . . . . . . . . . . . . . . . . . 144
5 Conclusion and Future Directions 147
Appendices 150
A Proofs for Chapter 3 151
8
List of Figures
3-1 A pinhole camera. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
3-2 Dicke’s coded aperture proposal [37]. . . . . . . . . . . . . . . . . . . 81
3-3 Maximum spectral magnitude of random on-off sequences. . . . . . . 82
3-4 Minimum spectral magnitude of random on-off sequences. . . . . . . . 83
3-5 Synthetic performance comparison of various coded aperture proposals. 117
9
10
List of Tables
4.1 Numerics of various bounds on the normalized second moment of vector
quantizers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
11
12
Chapter 1
Introduction
Regarding the researches of d’Alembert and Euler could one not add
that if they knew this expansion, they made but a very imperfect use of
it. They were both persuaded that an arbitrary and discontinuous
function could never be resolved in series of this kind, and it does not
even seem that anyone had developed a constant in cosines of multiple
arcs, the first problem which I had to solve in the theory of heat.
Joseph Fourier,1808-9
1.1 Motivation
In science and engineering, the modern problems we face often have a rich history
lying underneath their surface. Understanding this history is often crucial in the
resolution of these problems. This dissertation attempts to defend this point of view
through a set of case studies, all revolving around the topic of Fourier analysis.
Fourier analysis may be attributed to the work of Joseph Fourier on the theory of
heat transfer [48]. Roughly speaking, Fourier analysis allows one to study the way a
general function can be represented as a linear combination of simpler trigonometric
functions (sinusoids). The viewpoint of a function as a combination of sinusoids is
certainly a very useful one, and seems to be how Fourier himself envisioned it.
Over time, however, as Fourier’s ideas were cultivated and their impact realized,
13
more sophisticated points of view emerged. For example, the trigonometric functions
can be replaced by complex exponentials, and the fundamental role of the complex
exponentials comes from the fact that they are simultaneous eigenfunctions for the
translation group generated from the basic translation: (𝑇𝑓)(𝑥) , 𝑓(𝑥+ 1).
Such a viewpoint may be developed further into a theory of the Fourier transform
on groups. In the abelian case (including Hamming space (Z/𝑞Z)𝑑), the theory is
simpler as the Fourier transform remains scalar valued, and the fundamental dual
objects are characters. Finiteness of the group also helps keep things simple. Details
at a level suitable for both engineers and mathematicians may be found in e.g. the
wonderful book of Terras [112], or the article of Forney [47]. We shall review this
material in the context of finite abelian groups, such as Hamming space (Z/𝑞Z)𝑑,
with minimal prerequisites. The elementary approach for finite abelian groups has
the advantage of being accessible to more people, though to appreciate the generality
and richness of these ideas and plumb deeper one can study the representation theory
of finite groups (e.g. [103, 72]) and compact/locally compact groups (e.g. [97]).
We now briefly describe the contents of this dissertation and how they relate to
Fourier analysis on three spaces, namely (Z/𝑞Z)𝑑,Z/𝑛Z,R𝑑.
1.1.1 Hamming space
The primary motivation for our study on Hamming space in Chapter 2 is understand-
ing “anticodes” better. Here, the classical question is to maximize the size of a subset
of 0, 1𝑛 subject to an upper bound on distances between pairs of points. We call this
problem a isodiametric problem. The reason we call such subsets “anticodes” is that
one can replace the upper bound with lower bound to yield the classical central ques-
tion of coding theory. It is somewhat remarkable that in spite of the coding theory
question remaining unresolved, Ahlswede and Khachatrian [7, Diametric Theorem]
obtained a complete resolution to the anticoding question in the above isodiametric
sense.
Here, we generalize the definition of optimality from the isodiametric sense to
that of optimizing a two point potential function subject to a cardinality constraint.
14
We employ the classical Delsarte linear programming (LP) bounds, which are in-
timately connected to Fourier analysis. The LP bounds turn out to yield sharp
answers for some special values of the cardinality. We give an application to a
problem in theoretical computer science, namely that of finding a Boolean function
𝑓 : −1, 1𝑛 → −1, 1 which maximizes noise stability subject to an expected value
constraint. Previously, sharp answers were known only for E(𝑓) = 0, where the answer
is a dictator function 𝑓(𝑥𝑛) = 𝑥1 irrespective of the value of noise. Here, we resolve
E(𝑓) = ±1/2, where an answer is given by 𝑓(𝑥𝑛) = 𝑥1 ∧ 𝑥2 and 𝑓(𝑥𝑛) = 𝑥1 ∧ 𝑥2,
irrespective of the value of noise. We generalize such results to a non-binary setting,
namely 𝑓 : 0, 1, . . . , 𝑞 − 1𝑛 → −1, 1. We also exhibit a stacking construction of
anticodes, and utilize it to prove that the set of universal optima for noise stability
across the noise level is a sparse set.
1.1.2 Discrete Fourier transforms
The primary motivation for our study of the discrete Fourier transform in Chapter 3
is computational imaging, specifically the problem of designing good coded aperture
systems. Roughly speaking, coded aperture imaging systems consist of a perforated
plate placed before the imaging plane, and an associated computational inversion
procedure to recover the scene of interest from the image formed by a superposition
of shifted copies of the scene (a convolution) through the various perforations. The
basic design problem is to come up with a good perforation pattern.
In [8], we characterize the fundamental limits of coded aperture imaging systems
up to universal constants by drawing upon a theorem of Nazarov regarding Fourier
transforms. The theorem itself is more general, and we will elaborate on this. Our
work is performed under a simple propagation and sensor model that accounts for
thermal and shot noise, scene correlation, and exposure time. Focusing on mean
square error as a measure of linear reconstruction quality, we show that appropriate
application of a theorem of Nazarov leads to essentially optimal coded apertures,
up to a constant multiplicative factor in exposure time. Additionally, we develop a
heuristically efficient algorithm to generate such patterns that explicitly takes into
15
account scene correlations. This algorithm finds apertures that correspond to local
optima of a certain potential on the hypercube, yet are guaranteed to be tight. Finally,
for i.i.d. scenes, we show improvements upon prior work by using spectrally flat
sequences with bias. The development primarily focuses on one dimensional apertures
for conceptual clarity; the natural generalizations to 2D are also discussed.
1.1.3 Euclidean space
Fourier analysis played a crucial role in the resolution of the sphere packing problem
(the coding problem for Euclidean space) and associated universal optimality phe-
nomena for R𝑑, 𝑑 = 8, 24 [120], [27], [28]. We agree with the experts and believe that
the associated universally optimal structures, namely the lattices 𝐸8 for 𝑑 = 8 and
the Leech lattice for 𝑑 = 24 are also optimal for the quantization problem in the sense
of minimizing mean squared error in the so-called “high-resolution limit”, and refer
the impatient reader to Conjecture 6 for a precise statement.
We still believe that Fourier analysis will play a role in resolving Conjecture 6, but
are currently unable to shed any light on such an idea. Instead, we develop alternative
methods that are still capable of yielding improved lower bounds on the mean squared
error for lattice as well as non-lattice quantizers. These bounds represent the first rig-
orous improvement over Zador’s sphere bound [128], [129], though Conway and Sloane
have a conjectured bound [31]. We obtain distinct lower bounds for lattice quantizers
versus general quantizers, with stronger results for lattice quantizers. The results for
lattices are numerically verifiable albeit conjectural. In either setting, our bounds are
not as strong as the one conjectured by Conway and Sloane. At a high level, our
approach may be viewed as a generalization of the work of Tóth/Newman [116], [88]
from dimension 𝑑 = 2 to larger 𝑑. We achieve this by utilizing upper bounds on
face counts of Voronoï cells from Minkowski/Voronoï [84], [121] in the lattice case,
and considerations of dispersion and upper bounds on sphere packing density in the
general case. One route to the sphere packing density is through LP bounds. This
second “dispersion method” thus indirectly relies on Fourier analysis, through the
links between the LP bounds and Fourier analysis that we first alluded to above with
16
Hamming space and that we will describe in greater detail in Chapter 2. However,
there are other approaches to nontrivial upper bounds on packing density, such as
Rankin’s method [94], which extended earlier work of Blichfeldt [20]. As such, we do
not recommend reading too much into this link with Fourier analysis.
1.1.4 General remarks
We shall develop these ideas in a self-contained manner in as elementary a fashion as
possible. In particular, we provide proofs of facts that may be found in the original
sources or other expositions either explicitly or implicitly, unless they are well known
to a general audience of scientists/engineers, or take us too far away from our main
thread. As a concrete example, we assume Cauchy-Schwarz is a well-known inequality,
but that the discrete Fourier transform of quadratic residue sequences [51], the Euler-
Maclaurin formula, or Voronoï’s upper bound on the face counts of Voronoï cells of
a lattice [121] are not well-known.
For the reader who wishes to refresh their familiarity with certain concepts, we
provide some book references. For Chapter 2, we assume some rudimentary famil-
iarity with Hamming space. The freely available book by O’Donnell [89] covers that
and much more. For Chapter 3, we assume some familiarity with stastical signal
processing, estimation, and elementary number theory. For the signal processing and
estimation aspects, the book by Luenberger [80] covers all that we need and more at
a level suitable for both mathematicians and engineers. There are countless sources
for elementary number theory freely available online, such as the book by Stein [108].
For Chapter 4, we assume some familiarity with the basic notions of quantization.
The excellent survey by Gray and Neuhoff [57] covers all that we need and more. It
also has the virtue of tracing the history of the field accurately.
We apologize in advance to the mathematicians who desire a higher level of so-
phistication, and to the engineers who just want to build things and move on. Good
examples of the balance we are striving towards are anything written by Donald
Knuth, in particular the outstanding discrete mathematics book with Graham and
Patashnik [56]. Naturally, we do not reach that level either, so we must apologize yet
17
again.
A few words on notation. We assume knowledge of standard asymptotic notation
𝑜(), 𝑂(),Θ(),Ω(), 𝜔() with their usual meanings. We simply use the word constant
to refer to what many authors call a universal constant. We shall call scenarios with
exactly matching upper and lower bounds sharp, and the analagous situations with
upper and lower bounds that remain within a constant multiplicative factor of each
other tight. We assume that the reader is familiar with the “indicator/characteristic
function” notation 1(𝑥 = 0),1(𝑥 ∈ 𝒜). We sometimes find it convenient to follow the
analytic number theorists and write
𝑒(𝑧) , 𝑒2𝜋𝑖𝑧.
Conclusions and directions for future research are provided on a chapter by chap-
ter basis, once again in line with the self-contained philosophy. As such, our final
Chapter 5 contains only certain general remarks about this dissertation and where
one can cultivate ideas further.
18
Chapter 2
Linear Programming Bounds and
Anticodes in Hamming Space
That combinatorics and information sciences often come together is no
surprise - they were born as twins (Leibniz “Ars Combinatoria” gives
credit to Raimundus Lullus from Catalania, who wanted to create a
formal language).
Rudolf Ahlswede, 2006 Shannon Lecture
2.1 Introduction
In coding theory, the basic question is to maximize the size of a set subject to a
minimum distance requirement. The analogous dual question is to maximize the size
of a set subject to a maximum distance constraint. This maximization problem may
be termed as an anticoding or more precisely an isodiametric problem. We note that
this duality can be made precise in terms of the anticoding bound of Delsarte [35,
Thm 3.9]. This anticoding bound is sharper than the sphere-packing bound when a
optimal solution to the isodiametric problem is not given by a ball.
In Hamming space, a complete resolution of the isodiametric problem was given
by [7, Diametric Theorem], building upon techniques as well as the complete reso-
lution of the analogous question for Johnson space given in [6]. In the non-binary
19
setting of Hamming space, the optimal anticode is not in general a ball, but rather
a Cartesian product of a ball and a subcube. In particular, this implies that the an-
ticoding bound is sharper than the classical sphere-packing bound in the non-binary
setting, as noted in [4].
Our primary goal in this chapter is to address anticoding problems in a com-
plementary manner to the diametric perspective of [6, 7]. Specifically, note that in
isodiametric problems the goal is to maximize the cardinality subject to a constraint
that may be viewed as an upper bound on a potential energy characterized by a hard
pair potential function. Our aim here is to “flip” this perspective, and ask energy
minimization questions subject to a cardinality constraint. Although the two ques-
tions are obviously related (see e.g. (2.1)), we find the phenomena sufficiently rich
(e.g. Theorems 2, 3, 4) to warrant investigation in their own right. We note that the
complementary investigation of anticodes via potential energy may be traced in [5, pg
vii,239]. There, the authors describe the question of the average cost of a uniformly
randomly chosen update within a subset of a metric space. The authors then special-
ize to a cost function that decomposes on a product space as a sum of cost functions
on the individual coordinates.1 In general, taking a hard cost constraint with cost
0 for distances below a threshold, and ∞ otherwise leads one naturally to diametric
problems. Our work may be viewed in that framework as considering other cost func-
tions, with particular emphasis on completely monotone ones which we define in 4.
A more direct motivation is viewing our work as addressing the anticoding analog of
the ground state version of the coding problem as described in [29]. Along the way,
we draw a connection with the problem of maximum noise stability in theoretical
computer science, and thereby answer a folklore question of Mossel that we heard
from Razenshteyn and Ramnarayan in Corollary 1. We use this perspective to guide
our work in Section 2.5 onwards.
We first define what we mean by energy below.
Definition 1. Let (𝒳 , 𝑑) be a finite metric space, and 𝑓 : R → R∪±∞ a potential
1This perspective and a motivation for such study in terms of updating memories with costconstraints was also emphasized in Ahlswede’s 2006 Shannon lecture [3].
20
function. Let 𝒞 ⊆ 𝒳 . We then define the potential energy of 𝒞 with respect to the
potential function 𝑓 to be
𝐸𝑓 (𝒞) ,1
|𝒞|∑𝑥,𝑦∈𝐶𝑥 =𝑦
𝑓(𝑑(𝑥, 𝑦)).
Then we may define two fundamental limits associated with the problem of finding
energy minimizing (ground) states.
Definition 2.
𝑒*(𝑓, 𝑐) , min𝒞⊆𝒳 :|𝒞|=𝑐
𝐸𝑓 (𝒞).
𝑐*(𝑓, 𝑒) , max𝒞⊆𝒳 :𝐸𝑓 (𝒞)≤𝑒
|𝒞|.
In general, sharp information about one of these functions does not necessarily
translate into sharp information about the other function, even when one confines
oneself to “interesting” potential functions, such as exponential decays. What is triv-
ially clear however is that 𝑐*, 𝑒* are related by
𝑐*(𝑓, 𝑒*(𝑓, 𝑐)) ≥ 𝑐 (2.1a)
𝑒*(𝑓, 𝑐*(𝑓, 𝑒)) ≤ 𝑒. (2.1b)
A natural question then is what constitutes an interesting potential function. One
class of examples is readily furnished by the isodiametric problem, that is finding
max𝒞⊆𝒳 :𝑑(𝒞)≤𝑑 |𝒞|, where:
Definition 3. The diameter of a set 𝒞 in a finite metric space (𝒳 , 𝑑) is given by:
𝑑(𝒞) , max𝑥,𝑦∈𝒞 𝑑(𝑥, 𝑦).
It is then clear that the isodiametric problem is nothing but the question of de-
termining 𝑐*(𝑓, |𝒳 |) for
𝑓(𝑥) =
⎧⎪⎨⎪⎩1 𝑥 ≤ 𝑑
∞ 𝑥 > 𝑑.
21
Another class of functions, namely that of completely monotonic functions has
proved to be very fruitful from a theoretical perspective in these investigations. More-
over, in the Euclidean setting, special cases of completely monotonic functions such
as power laws have a natural physical interpretation. In coding theory, optimizing
these potential functions gives information on the probability of error via the union
bound.
In the discrete setting, completely monotonic functions are defined as follows:
Definition 4. Let ∆ denote the finite difference operator, defined by ∆𝑓(𝑛) , 𝑓(𝑛+
1) − 𝑓(𝑛). Then, a function 𝑓 : 𝑎, 𝑎 + 1, . . . , 𝑏 is said to be completely monotonic
if its iterated differences alternate in sign, that is (−1)𝑘∆𝑘𝑓(𝑖) ≥ 0 whenever 𝑘 ≥ 0
and 𝑎 ≤ 𝑖 ≤ 𝑏− 𝑘.
Of crucial importance for us is the fact that 𝑓(𝑟) = 𝛾𝑟 for 0 ≤ 𝛾 ≤ 1 is a
completely monotonic function. Minimizing potential energy with respect to such 𝑓
favors repulsion between points, and is a ground state analog of the coding problem. A
natural, perhaps naive view of anticoding is simply to flip the sign of 𝑓 , or equivalently
maximize the potential energy associated to such 𝑓 .
What we find surprising is that this approach still retains an “operational sig-
nificance” in the anticoding setting via a connection with the problem of maximal
noise stability in theoretical computer science. Noise stability was first studied ex-
plicitly in [18]; see for instance [89, 2.4] for an introduction to the topic. Typically,
noise stability is studied for Boolean functions 𝑓 : 0, 1𝑛 → 0, 1. However, as the
methods we employ apply more generally, we shall define an analogous notion for
𝑓 : F𝑛𝑞 → 0, 1. Although this notation suggests that F𝑞 is a finite field, we shall not
use the field structure.
As the name may suggest, noise stability is measured by Pr(𝑓(𝑥) = 𝑓(𝑦)) where
𝑦 is a noisy version of 𝑥. Typically, one is interested in the behavior of functions
on product spaces, and thus it is natural to consider product transition probability
kernels, though it can be defined in greater generality.
We define it rigorously as follows in the context of Hamming space, and also define
22
the problem of maximal noise stability subject to an expected value constraint.
Definition 5. Let 𝑓 : F𝑛𝑞 → 0, 1 be a Boolean valued function. Let 𝑟(·|·) be a row-
stochastic 𝑞× 𝑞 transition probability matrix. We define a kernel 𝑠(·|·) on the product
space F𝑛𝑞 . Typically, 𝑠 is a product kernel given by 𝑠(𝑏𝑛|𝑎𝑛) =
∏𝑛𝑖=1 𝑟(𝑏𝑖|𝑎𝑖) ∀𝑎𝑛 ∈ F𝑛
𝑞 .
Let x ∼ 𝑈(F𝑛𝑞 ) be a uniformly distributed random variable. Let y be coupled with x
by sending x through the kernel 𝑠. Then the noise stability of 𝑓 is given by
Stab𝑠(𝑛, 𝑓) , Pr(𝑓(x) = 𝑓(y)).
We also define the maximal noise stability function as
Stab*𝑠(𝑛, 𝜇) , max
𝑓 :E[𝑓(x)]=𝜇Stab𝑠(𝑛, 𝑓).
Since it is usually clear that we are referring to a product kernel, we will often simply
write Stab𝑟(𝑛, 𝜇). Furthermore, when 𝑟 is parametrized in a natural way, such as a
binary symmetric channel (BSC) with parameter 𝜖, we may write Stab𝜖(𝑛, 𝜇). Similar
remarks apply to Stab*𝑟,Stab
*𝜖 .
We also find it convenient to define the notation
Stab𝑠(𝑛, 𝒞) , Stab𝑠(𝑛,1(𝑥 ∈ 𝒞)).
Note that we do not strictly follow the conventions of [89, pg 53], which defines
Stab𝑠(𝑛, 𝑓) , E[𝑓(x)𝑓(y)]. The reason for this is that the above definition is better
suited for generality. Note also that it is common in the study of Boolean functions
to work with 𝑓 : −1, 1𝑛 → −1, 1; this simply corresponds to a relabeling 0 ↔
−1, 1 ↔ 1. With the −1, 1 output convention, it is clear that the definition of noise
stability adopted in [89, pg 53] is simply an affine transformation of Definition 5. For
the sake of notational clarity, in all rigorous statements we shall make it clear which
representation we are working with.
23
2.2 Main results
We now give precise statements of the main results that we establish in this chapter.
Lemmas, proofs, and establishment of statements of possible independent interest will
occupy subsequent sections.
2.2.1 Linear programming and isodiametry in Hamming space
We first revisit in Section 2.3 the diametric theorem of [7] from a linear program-
ming (LP) bound perspective, and rederive a sharp bound for the subcube cases.
Our demonstration of the sharp cases of the LP bound for isodiametry in Hamming
space may be viewed as an analog of the work of Wilson [124], who used LP to
establish special cases of the complete diametric theorem obtained in [6]. We note
that Shinkar [104] has also established the subcube cases using spectral techniques.
Indeed, the spectral techniques used in [104] are implicitly contained in the language
of association schemes and LP bounds of [35].
Definition 6. A subcube of cardinality 𝑞𝑘 in Hamming space F𝑛𝑞 is defined by 𝒞𝑞,𝑘 =
𝑥 : 𝑥𝑖 = 0 ∀1 ≤ 𝑖 ≤ 𝑛− 𝑘.
Theorem 1. Let 𝑁𝑞(𝑛, 𝑑) , max𝒞⊆F𝑛𝑞 :𝑑(𝒞)≤𝑑 |𝒞|. Then if 𝑑 ≤ 1 or 𝑑 ≥ 𝑛 − 𝑞 + 1,
𝑁𝑞(𝑛, 𝑑) = 𝑞𝑑, and this may be deduced from the LP bounds. Moreover, equality is
attained by subcubes: 𝒞 = 𝒞𝑞,𝑑.
2.2.2 Universal optimality for special subcubes
We next establish in Section 2.4 the fact that some special subcubes are simultaneous
ground states in the anticoding sense for classes of potential functions. This phe-
nomenon is called universal optimality as defined in [26]. However, it turns out that
for some subcubes we get even stronger information than being universally optimal
with respect to all completely monotonic functions, and in fact we can deduce uni-
versal optimality with respect to all monotonic functions. Most of Theorem 2 follows
24
in a natural manner from the LP bounds, except for (2.6) that relies on a certain
inequality for Krawtchouk polynomials established in Lemma 13.
Theorem 2. Consider the class ℱ of all nonnegative monotonically nondecreasing
potential functions 𝑓 : 0, 1, . . . , 𝑛 → R, that is with 𝑓(𝑖) ≤ 𝑓(𝑖+ 1) ∀0 ≤ 𝑖 ≤ 𝑛− 1,
and 𝑓(0) ≥ 0. Then ∀𝑓 ∈ ℱ , 𝑞 ≥ 2 ∈ N, 𝑛 ≥ 2
𝑒*(𝑓, 𝑞) = 𝐸𝑓 (𝒞𝑞,1) (2.2)
𝑒*(𝑓, 𝑞2) = 𝐸𝑓 (𝒞𝑞,2) (2.3)
𝑒*(𝑓, 𝑞𝑛−1) = 𝐸𝑓 (𝒞𝑞,𝑛−1) (2.4)
𝑒*(𝑓, 𝑞𝑛) = 𝐸𝑓 (𝒞𝑞,𝑛). (2.5)
Furthermore, if 𝑞 > 2, we have for all 𝑓 ∈ ℱ and 𝑛 ≥ 2
𝑒*(𝑓, 𝑞𝑛−2) = 𝐸𝑓 (𝒞𝑞,𝑛−2). (2.6)
Now consider the class 𝒢 of all negations of completely monotonic potential func-
tions 𝑓 : 0, 1, . . . , 𝑛 → R. In other words, 𝑓 ∈ 𝒢 iff (−1)𝑘+1∆𝑘𝑓(𝑖) ≥ 0 whenever
𝑘 ≥ 0 and 0 ≤ 𝑖 ≤ 𝑛− 𝑘. Then for 𝑞 = 2, we have for all 𝑓 ∈ 𝒢 and 𝑛 ≥ 2
𝑒*(𝑓, 2𝑛−2) = 𝐸𝑓 (𝒞2,𝑛−2). (2.7)
As rather simple corollaries of Theorem 2, we obtain the following implications for
the problem of maximal noise stability subject to an expected value constraint. Our
deduction of these statements for maximal noise stability is based on a connection
between anticoding and maximum noise stability that we develop in Section 2.5.
In binary Hamming space, we have the following.
Corollary 1. Let 𝑞 = 2, and let us work with functions 𝑓 : −1, 1𝑛 → −1, 1. Let
the transition probability kernel 𝑤 be given by the family of BSC(𝜖). In other words,
25
𝑤(1|1) = 𝑤(−1| − 1) = 1 − 𝜖 and 𝑤(−1|1) = 𝑤(1| − 1) = 𝜖. Let
𝑔(𝑥) =𝑥1𝑥2 + 𝑥1 + 𝑥2 − 1
2= 𝑥1 ∧ 𝑥2
where ∧ denotes logical "and". Then ∀0 ≤ 𝜖 ≤ 12
and ∀𝑛 we have
Stab*𝜖
(𝑛,
−1
2
)= Stab𝜖(𝑛, 𝑔).
Similarly, we have
Stab*𝜖
(𝑛,
1
2
)= Stab𝜖(𝑛,−𝑔).
We also have the (well known)
Stab*𝜖(𝑛, 0) = Stab𝜖(𝑛, ℎ)
where
ℎ(𝑥) = 𝑥1.
Corollary 1 answers a “folklore” question of Mossel that we first heard of from
Razenshteyn and Ramnarayan.
In non-binary Hamming space, we have the following.
Corollary 2. Let 𝑞 > 2, and let us work with functions 𝑓 : F𝑛𝑞 → 0, 1. Let the
transition probability kernel 𝑤 be given by the family of 𝑞-SC(𝜖). In other words,
𝑤(𝑦|𝑥) = 1 − 𝜖 if 𝑦 = 𝑥, and 𝑤(𝑦|𝑥) = 𝜖𝑞−1
otherwise. Let 𝑔1(𝑥) = 𝑥1 and 𝑔2(𝑥) =
1(𝑥1 = 0)1(𝑥2 = 0). Then ∀0 ≤ 𝜖 ≤ 1 − 1𝑞
and ∀𝑛 we have
Stab*𝜖
(𝑛,
1
𝑞
)= Stab𝜖(𝑛, 𝑔1),
Stab*𝜖
(𝑛,𝑞 − 1
𝑞
)= Stab𝜖(𝑛, 𝑔1),
26
We also have
Stab*𝜖
(𝑛,
1
𝑞2
)= Stab𝜖(𝑛, 𝑔2),
Stab*𝜖
(𝑛,𝑞2 − 1
𝑞2
)= Stab𝜖(𝑛, 𝑔2),
Here, 𝑓 denotes the logical complement of 𝑓 .
We note that the results for measures 14, 12
are in some sense anticipated by the
work of [50], who show (in our language) the optimality of subcubes of measure 14, 12
for the cost function 𝑓(𝑥) = 𝑥.
2.2.3 A mean value theorem for noise stability
We note that Theorem 2 and the corresponding noise stability corollaries 1, 2 refer
to a 𝑞-SC channel. The LP bounds (or their SDP generalizations) do not apply to
general channels, and we are therefore unable to give sharp answers for such channels,
even ones coming from a product noise. However, in Section 2.6, we prove a channel
comparison, and show that the maximum noise stability for a general channel can be
compared with the corresponding quantity for a 𝑞-SC with appropriate noise level 𝜖.
All of the statements in Section 2.6 follow from a statement that we call a mean value
theorem for noise stability :
Theorem 3. Let Aut denote the group of distance preserving automorphisms of Ham-
ming space F𝑛𝑞 . Define a group action of Aut on Boolean valued functions 𝑓 : F𝑛
𝑞 →
0, 1 by (𝜎𝑓)(𝑥) = 𝑓(𝜎𝑥) where 𝜎 ∈ Aut. Let 𝑠(·|·) denote a 𝑞𝑛 × 𝑞𝑛 probability
kernel. Let 𝑡(·|·) be a “symmetrized version” of 𝑠, given by
𝑡(𝑦|𝑥) =1
|Aut|∑𝜎∈Aut
𝑠(𝜎𝑦|𝜎𝑥). (2.8)
Then we have1
|Aut|∑𝜎∈Aut
Stab𝑠(𝑛, 𝜎𝑓) = Stab𝑡(𝑛, 𝑓). (2.9)
27
2.2.4 Large 𝑛 and balls versus subcubes
From the above discussion, it is clear that both Hamming balls and subcubes have
a role to play in Hamming space for anticoding problems. In Section 2.7 we com-
bine Hamming balls and subcubes by a stacking construction to prove that universal
optima (in the sense of noise stability across the 𝑞-SC(𝜖) family) form a sparse set:
Theorem 4. Let 𝒮 denote the set of cardinalities 0 ≤ 𝑐 ≤ 𝑞𝑛 where there exists a
universally optimal anticode 𝒞 with |𝒞| = 𝑐, where the universal optimality is in the
sense of noise stability across the 𝑞-SC(𝜖) with 𝜖 ∈ [0, 1 − 1𝑞]. Then, |𝒮|
𝑞𝑛= 𝑜(1) as
𝑛→ ∞.
Along the way, we establish a rigorous definition of Stab*𝑟(∞, 𝜇) for any product
noise generated by the kernel 𝑟(·|·) in Proposition 5. The rigorous definition is meant
to capture a large 𝑛 limit. The Lemmas 15, 17 that we use to establish 5 also play a
role in our proof of 4.
2.3 Linear programming and isodiametry in Ham-
ming space
We now prove Theorem 1. As noted in Section 2.2, the theorem itself is completely
subsumed by the complete diametric theorem of [7], and the subcube cases have been
derived independently of us by spectral techniques in [104]. As such, the purpose
of this section is to introduce the LP bounds in Hamming space which cover the
techniques used in [104] and more importantly play a key role in the remainder of
this chapter.
2.3.1 LP bounds and Fourier analysis on Hamming space
First, we formulate the LP bounds. Suppose
𝒞 ⊆ F𝑛𝑞 .
28
The Delsarte bounds are linear constraints on the distance distribution
𝐴𝑖 ,1
|𝒞||(𝑥, 𝑦) ∈ 𝒞 × 𝒞 : 𝑑(𝑥, 𝑦) = 𝑖|. (2.10)
As we are working in Hamming space, here 𝑑(𝑥, 𝑦) is the Hamming distance between
𝑥 and 𝑦.
We now define the Krawtchouk polynomials.
Definition 7.
𝐾𝑘(𝑥) = 𝐾𝑘(𝑥;𝑛) = 𝐾𝑘(𝑥;𝑛, 𝑞)
=𝑘∑
𝑗=0
(−1)𝑗(𝑞 − 1)𝑘−𝑗
(𝑥
𝑗
)(𝑛− 𝑥
𝑘 − 𝑗
). (2.11)
Then the Delsarte inequalities [35, Thm 3.3,4.2] are
𝑛∑𝑖=0
𝐴𝑖𝐾𝑗(𝑖) ≥ 0 ∀0 ≤ 𝑗 ≤ 𝑛. (2.12)
These inequalities are central in coding theory, and so we believe it is worth having
a look at a proof of these inequalities and how they are connected to Fourier analysis.
All of this exposition on the Delsarte inequalities is in some sense “classical” and is
either explicitly or implicitly contained in his seminal work [35]. For readers who
want a quick derivation of the inequalities themselves, we recommend [118, Sec. 5.3].
Perhaps it is useful to first understand where the Krawtchouk polynomials come
from, as their definition 7 is relatively unilluminating. We have the following Lemma
(see e.g. [118, Lemma 5.3.1]):
Lemma 1. Let ⟨𝑥, 𝑦⟩ denote the usual inner product in (Z/𝑞Z)𝑛. Let 𝜔 = 𝑒(
1𝑞
)be
a primitive 𝑞th root of unity. Let 𝑥 ∈ F𝑛𝑞 be a fixed word of weight 𝑖, in other words
|𝑥| = 𝑖. Then, ∑𝑦∈F𝑛
𝑞 ,|𝑦|=𝑘
𝜔⟨𝑥,𝑦⟩ = 𝐾𝑘(𝑖).
Proof of Lemma 1. By the underlying symmetries of Hamming space, we may assume
29
without loss that 𝑥 = (𝑥1, 𝑥2, . . . , 𝑥𝑖, 0, 0, . . . , 0), where 𝑥𝑖 = 0. Choose 𝑘 positions
ℎ1, ℎ2, . . . , ℎ𝑘 with 0 < ℎ1 < ℎ2 < · · · < ℎ𝑗 ≤ 𝑖 < ℎ𝑗+1 < · · · < ℎ𝑘 ≤ 𝑛. Let 𝒟 be
the set of all words of weight 𝑘 that have their nonzero coordinates in precisely these
positions. Then, we have
∑𝑦∈𝒟
𝜔⟨𝑥,𝑦⟩ =∑
𝑦ℎ1∈F𝑞∖0· · ·
∑𝑦ℎ𝑘∈F𝑞∖0
𝑘∏𝑖=1
𝜔𝑥ℎ𝑖𝑦ℎ𝑖
=
⎡⎣ 𝑗∏𝑖=1
∑𝑦∈𝐹𝑞∖0
𝜔𝑥ℎ𝑖𝑦
⎤⎦ (𝑞 − 1)𝑘−𝑗
=
[𝑗∏
𝑖=1
(𝜔 + 𝜔2 + . . . 𝜔𝑞−1
)](𝑞 − 1)𝑘−𝑗
= (−1)𝑗(𝑞 − 1)𝑘−𝑗.
Since there are(𝑖𝑗
)(𝑛−𝑖𝑘−𝑗
)choices for 𝒟, we get the result once we recall Definition 7.
The key role played by the roots of unity should already suggest the Fourier ana-
lytic nature of the linear programming bounds. Let us now develop Fourier analysis
on the abelian group (Z/𝑞Z)𝑛. Once again, all this material is classical. For a reader
who does not want to delve deeper into algebraic aspects such as representation theory
and is more comfortable with analysis, we recommend [107, Ch. 7].
As remarked in Chapter 1, in the abelian case, it suffices to study certain scalar
functions called characters :
Definition 8. Let (𝐺, ·) be a finite abelian group, and let 𝑆1 = 𝑧 ∈ C : |𝑧| = 1 be
the unit circle in the complex plane. A character 𝜒 on 𝐺 is a complex valued function
𝜒 : 𝐺→ 𝑆1 which satisfies
𝜒(𝑎 · 𝑏) = 𝜒(𝑎)𝜒(𝑏).
In other words, it is a group homomorphism from 𝐺 to 𝑆1. The trivial character
(denoted by 𝑒) is given by ∀𝑔 ∈ 𝐺, 𝑒(𝑔) = 1. Our notation here does collide with our
notation 𝑒(𝑧) = 𝑒2𝜋𝑖𝑧, but in practice the usage is unambiguous.
30
The astute reader familiar with the DFT but not with characters should already
realize that the DFT basis consisting of complex exponentials is a set of characters
on Z/𝑛Z.
A very important property of characters is that distinct characters are orthogonal.
In order to show this, first we show
Lemma 2. If 𝜒 is a nontrivial character for (𝐺, ·),
∑𝑔∈𝐺
𝜒(𝑔) = 0.
Proof of Lemma 2. As 𝜒 is nontrivial, there exists a 𝑏 with 𝜒(𝑏) = 1. Then,
𝜒(𝑏)∑𝑔∈𝐺
𝜒(𝑔) =∑𝑔∈𝐺
𝜒(𝑏)𝜒(𝑔)
=∑𝑔∈𝐺
𝜒(𝑏 · 𝑔)
=∑𝑔∈𝐺
𝜒(𝑔),
where the last step follows as 𝑏 · 𝑔 sweeps over all elements of 𝐺 exactly once. Now
as 𝜒(𝑏) = 1, we must have the required∑
𝑔∈𝐺 𝜒(𝑔) = 0.
We may now prove the important orthonormality of characters:
Proposition 1. Let 𝜒, 𝜒′ be two characters. Then ⟨𝜒, 𝜒′⟩ = 1(𝜒 = 𝜒′). Here
⟨𝑎, 𝑏⟩ , 1
|𝐺|∑𝑔∈𝐺
𝑎(𝑔)𝑏(𝑔).
Proof of Proposition 1. First, if 𝜒 = 𝜒′, then we have
⟨𝜒, 𝜒′⟩ =1
|𝐺|∑𝑔∈𝐺
1 = 1,
as required.
31
If 𝜒 = 𝜒′, we have
⟨𝜒, 𝜒′⟩ =1
|𝐺|∑𝑔∈𝐺
𝜒(𝑔)𝜒′(𝑔)
=1
|𝐺|∑𝑔∈𝐺
(𝜒𝜒′−1)(𝑔)
= 0.
Here, we used the fact that the characters themselves form a group (called the dual
group ( 𝐺, ·)) as is easily verified, and that 𝜒𝜒′−1 is a nontrivial character to which we
can apply Lemma 2.
Already we have some neat consequences, such as | 𝐺| ≤ |𝐺| due to orthogonality
implying independence and the fact that the dimension of the space of functions over
𝐺 is |𝐺|. In fact, we shall now show:
Theorem 5. For a finite abelian group (𝐺, ·), |𝐺| = | 𝐺|. In other words, the char-
acters of a finite abelian group form a basis for the vector space of functions over
𝐺.
Before turning to the proof of this nontrivial theorem, we do note that we do not
strictly speaking need this to develop Fourier analysis for something as “concrete” as
(Z/𝑞Z)𝑛. Indeed, the astute and impatient reader may already think: characters on
product groups are just products of characters, and we know characters for Z/𝑞Z from
knowledge of the DFT. More generally, one can invoke the structure theorem for finite
abelian groups which allows one to decompose any such group into a direct product
of cyclic groups, and in fact prove the above Theorem 5 from such a consideration.
However, we believe that with a little more patience more light may be shed.
For example, going back to Chapter 1, we made some remarks about “simultaneous
eigenfunctions for the translation group”. The approach we present now elucidates
this, assuming a basic grasp of linear algebra.
We first prove that commuting unitary operators on a finite dimensional inner
product space are simultaneously diagonalizable:
32
Lemma 3. Suppose 𝑇1, . . . , 𝑇𝑘 is a commuting family of unitary operators on the
finite dimensional inner product space 𝑉 ; that is for all 𝑖, 𝑗
𝑇𝑖𝑇𝑗 = 𝑇𝑗𝑇𝑖.
Then 𝑇𝑖 are simultaneously diagonalizable. In other words, there is a basis for 𝑉
consisting of eigenvectors for every 𝑇𝑖, 1 ≤ 𝑖 ≤ 𝑘.
Proof. We shall induct on 𝑘. The case 𝑘 = 1 is simply the spectral theorem. Suppose
Lemma 3 is true for 𝑘 − 1 commuting unitary operators. Applying the spectral
theorem to 𝑇𝑘, we see that
𝑉 = 𝑉𝜆1 ⊕ . . . 𝑉𝜆𝑠 ,
where 𝑉𝜆𝑖denotes the subspace of all eigenvectors with eigenvalue 𝜆𝑖. We now claim
that each one of 𝑇1, . . . , 𝑇𝑘−1 maps 𝑉𝜆𝑖to itself. For if 𝑣 ∈ 𝑉𝜆𝑖
, and 1 ≤ 𝑗 ≤ 𝑘 − 1,
then we have
𝑇𝑘𝑇𝑗(𝑣) = 𝑇𝑗𝑇𝑘(𝑣) = 𝑇𝑗(𝜆𝑖𝑣) = 𝜆𝑖𝑇𝑗(𝑣),
and so 𝑇𝑗(𝑣) ∈ 𝑉𝜆𝑖as needed.
Now, the restrictions of 𝑇1, 𝑇2, . . . , 𝑇𝑘−1 to 𝑉𝜆𝑖are clearly well defined operators,
and inherit pairwise commuting and the unitary nature. Furthermore, by the in-
duction hypothesis, these are simultaneously diagonalizable. Thus, we get a suitable
basis for each 𝑉𝜆𝑖that works for 𝑇1, 𝑇2, . . . , 𝑇𝑘. As 𝑉 is a direct sum of the 𝑉𝜆𝑖
, we
are done.
With the above Lemma 3, we may now turn to the proof of Theorem 5.
Proof of Theorem 5. Define “translation” operators on the vector space of complex
valued functions on 𝐺
(𝑇𝑎𝑓)(𝑥) , 𝑓(𝑎 · 𝑥).
𝐺’s abelian nature implies that 𝑇𝑎, 𝑇𝑏 commute. Furthermore, 𝑇𝑎 is clearly unitary,
since 𝑎 · 𝑥 sweeps across 𝐺 when 𝑥 does. Then by Lemma 3 𝑇𝑎 is simultaneously
diagonalizable. Thus, we have a basis of eigenvectors 𝑣𝑏(·), where 𝑏 varies over 𝐺.
33
Let 𝑣 be one of these 𝑣𝑏, and we claim that 𝑤(𝑔) , 𝑣(𝑔)𝑣(1)
is well-defined and also a
character.
First, we need to show that 𝑣(1) = 0. Suppose not. Then
𝑣(𝑎) = 𝑣(𝑎 · 1) = (𝑇𝑎𝑣)(1) = 𝜆𝑎𝑣(1) = 0,
forcing 𝑣(𝑎) = 0 for an arbitrary 𝑎, which is impossible.
Thus 𝑤 is indeed well-defined. We now check that it is indeed a character by
𝑤(𝑎 · 𝑏) =𝑣(𝑎 · 𝑏)𝑣(1)
=𝜆𝑎𝑣(𝑏)
𝑣(1)= 𝜆𝑎𝜆𝑏 = 𝑤(𝑎)𝑤(𝑏).
This completes the proof of Theorem 5.
Note that the above proof is quite hands-on and “effective”, since it gives an
actual procedure (compute eigenvectors of the translation operators) to obtaining the
characters. As such, the following consequence should be transparent either by the
above discussion, or by the earlier remarks regarding characters on product groups
being products of characters:
Corollary 3. Let 𝜔 = 𝑒(
1𝑞
). The characters of ((Z/𝑞Z)𝑛,+) are 𝜒𝑥, where 𝜒𝑥(𝑧) =
𝜔⟨𝑥,𝑧⟩.
Proof of Corollary 3. It is obvious that 𝜒𝑥(𝑧) are characters, since their range is on
the unit circle, and
𝜒𝑥(𝑧1 + 𝑧2) = 𝜔⟨𝑥,𝑧1+𝑧2⟩ = 𝜔⟨𝑥,𝑧1⟩𝜔⟨𝑥,𝑧2⟩ = 𝜒𝑥(𝑧1)𝜒𝑥(𝑧2).
Furthermore, we have one character for each of the elements of (Z/𝑞Z)𝑛, so we are
done by Theorem 5.
We now introduce the notion of a positive definite function.
34
Definition 9. Let (𝐺,+) be a finite abelian group, and 𝑓 : 𝐺 → C. 𝑓 is called
positive definite iff for all 𝑁 and 𝑥1, . . . , 𝑥𝑁 ∈ 𝐺, 𝑐1, . . . , 𝑐𝑁 ∈ C,
𝑁∑𝑗,𝑘=1
𝑓(𝑥𝑗 − 𝑥𝑘)𝑐𝑗𝑐𝑘 ≥ 0.
In other words, for all 𝑁 and all “codes” 𝑥1, . . . , 𝑥𝑁 , the matrix 𝐴 given by 𝐴𝑗,𝑘 =
𝑓(𝑥𝑗 − 𝑥𝑘) is a positive definite matrix.
Before developing the theory further, let us look at some simple illustrations and
consequences of Definition 9. For example, the function 𝑓(𝑥) = 1(𝑥 = 0) is a positive
definite function; this simply follows from the fact that the identity is positive definite.
We also have the following simple
Lemma 4. Let 𝑓 be a positive definite function. Then
1. 𝑓(0) ≥ 0,
2. ∀𝑥 ∈ 𝐺, 𝑓(0) ≥ |𝑓(𝑥)|,
3. ∀𝑥 ∈ 𝐺, 𝑓(−𝑥) = 𝑓(𝑥).
Proof of Lemma 4. First take 𝑁 = 1, 𝑥1 = 0, 𝑐1 = 1 to get 𝑓(0) ≥ 0. Next take
𝑁 = 2, 𝑥1 = 0, 𝑥2 = 𝑥, 𝑐1 = 1, 𝑐2 = 𝑐 where 𝑐 is an arbitrary complex number on the
unit circle, to get
𝑐𝑓(−𝑥) + 𝑐𝑓(𝑥) + 2𝑓(0) ≥ 0.
Thus 𝑓(−𝑥) = 𝑓(𝑥) (the third item), and also 2ℜ(𝑐𝑓(𝑥)) ≤ 2𝑓(0). Take 𝑐 proportional
to 𝑓(−𝑥) to get the second item.
It is also easy to show that characters are positive definite, and that positive
definite functions form a cone:
Lemma 5. Let 𝑔 ∈ 𝐺 be a character. Then 𝑔 is positive definite. Also, a nonnegative
linear combination of positive definite functions is positive definite. In other words,
positive definite functions form a cone.
35
Proof of Lemma 5. We have
𝑁∑𝑗,𝑘=1
𝑔(𝑥𝑗 − 𝑥𝑘)𝑐𝑗𝑐𝑘 =𝑁∑
𝑗,𝑘=1
𝑔(𝑥𝑗)𝑔(𝑥𝑘)𝑐𝑗𝑐𝑘 =
𝑁∑𝑗=1
𝑔(𝑥𝑗)𝑐𝑗
2
≥ 0,
completing the proof of the first statement by the definition of positive definiteness 9.
The second statement is obvious, and follows directly from linearity.
We may now prove a (weak) form of Bochner’s theorem (see e.g. [22] for original,
or an exposition [70, VI.2.7] for the version on the locally compact R/Z.)2 The
“weakness” here refers to the fact that we are working in the technically simple setting
of finite abelian groups.
Theorem 6 (Bochner). Let (𝐺,+) be a finite abelian group, and let 𝑓 : 𝐺→ C have
Fourier expansion
𝑓(𝑥) =∑𝑔∈ 𝐺
𝑎𝑔𝑔(𝑥).
Then, 𝑓 is positive definite iff all its Fourier coefficients 𝑎𝑔 are nonnegative.
Proof of Theorem 6. Let ℎ ∈ 𝐺 be a character. Now take 𝑥1, 𝑥2, . . . , 𝑥𝑁 = 𝐺 (each
element of 𝐺 occuring precisely once), and 𝑐𝑗 = ℎ(𝑥𝑗). Applying the definition of
positive definiteness 9, we have
∑𝑔∈ 𝐺
𝑎𝑔∑𝑗,𝑘
𝑔(𝑥𝑗 − 𝑥𝑘)ℎ(𝑥𝑗)ℎ(𝑥𝑘) ≥ 0
⇒∑𝑔∈ 𝐺
𝑎𝑔∑𝑗,𝑘
𝑔(𝑥𝑗)𝑔(𝑥𝑘)ℎ(𝑥𝑗)ℎ(𝑥𝑘) ≥ 0
⇒∑𝑔∈ 𝐺
𝑎𝑔∑
𝑗
𝑔(𝑥𝑗)ℎ(𝑥𝑗)
2
≥ 0
⇒ | 𝐺|𝑎ℎ ≥ 0
⇒ 𝑎ℎ ≥ 0,
2It can be argued that these ideas for R/Z should be traced further back to the work of Her-glotz [62] and independently Riesz [96].
36
where we used the orthonormality of characters (Proposition 1). ℎ was arbitrary, so
we proved one direction.
We now need to show that if all the Fourier coefficients are nonnegative, 𝑓 is
positive definite. But this is precisely the content of Lemma 5.
Most of the above machinery is not strictly speaking needed for the proof of the
central Delsarte inequalities (2.12). However, it does serve to place them in a broader
Fourier analytic context, and helps reduce the “mystery” of where the inequalities
come from. Let us now prove the Delsarte inequalities.
Proposition 2. Let the Krawtchouk polynomials be defined via (2.11), and let the dis-
tance distribution of a code 𝒞 be given by 𝐴𝑖, defined by (2.10). Then, we have (2.12)
∀0 ≤ 𝑗 ≤ 𝑛,𝑛∑
𝑖=0
𝐴𝑖𝐾𝑗(𝑖) ≥ 0.
Proof of Proposition 2. Let 𝑥1, . . . , 𝑥𝑁 = 𝒞, where each 𝑥𝑖 occurs precisely once for
each element of the code 𝒞. Let 𝑐𝑖 = 1 for all 𝑖. 𝜒𝑥 is positive definite, so∑
𝑥:|𝑥|=𝑗 𝜒𝑥 is
positive definite as well by Lemma 5. The definition of positive definiteness 9, together
with the expression of Krawtchouk polynomials in terms of characters (Lemma 1)
completes the proof. More explicitly, we have
|𝒞|𝑛∑
𝑖=0
𝐴𝑖𝐾𝑗(𝑖) =𝑛∑
𝑖=0
∑(𝑥,𝑦)∈𝒞2,|𝑥−𝑦|=𝑖
⎡⎣ ∑𝑧∈F𝑛
𝑞 ,|𝑧|=𝑘
𝜔⟨𝑥−𝑦,𝑧⟩
⎤⎦=
∑𝑧∈F𝑛
𝑞 ,|𝑧|=𝑘
∑𝑥∈𝒞
𝜔⟨𝑥,𝑧⟩
2
≥ 0.
Remark 1. The above proof also reveals an interpretation of the Delsarte inequalities.
Consider 𝑓(𝑥) = 1(𝑥 ∈ 𝒞), that is the “characteristic function” of the code. Then,
the Delsarte inequalities state that the sum of the squares of the magnitudes of the
Fourier coefficients of 𝑓 , grouped by Hamming weight 𝑗, are nonnegative. For the
37
reader familiar with Boolean Fourier analysis in the spirit of [89], this provides a link
to the Delsarte inequalities.
Remark 2. The astute reader may have noticed that nothing in our above discussion
really sheds light as to why the Krawtchouk polynomials are actually polynomials,
though we did verify this by direct computation. As our direct computation in the proof
of Lemma 1 suggests, one may suspect that the polynomial behavior arises somehow
from the combinatorics (in particular the regularity) of the underlying Hamming space.
This guess is indeed correct, and there is a rich theory of association schemes that
studies this further. Delsarte’s original work [35] is written in that framework. We
also refer the interested reader to the expository paper [36]. Although we could provide
an exposition here as well, the theory of association schemes is very tangential to our
focus here, and we feel that it would not shed much light on the subject matter at
hand.
We shall need a few properties of Krawtchouk polynomials for the subsequent
development in Hamming space. The above remark illustrates how one may develop
the theory systematically. Here, we shall content ourselves with a more “ad-hoc”
development.
First, we obtain a closed form for the generating function of Krawtchouk polyno-
mials (see e.g [118, 1.2.3]).
Lemma 6. Recall (7)
𝐾𝑘(𝑥) =𝑘∑
𝑗=0
(−1)𝑗(𝑞 − 1)𝑘−𝑗
(𝑥
𝑗
)(𝑛− 𝑥
𝑘 − 𝑗
).
Then we have∞∑𝑖=0
𝐾𝑖(𝑥)𝑧𝑖 = (1 + (𝑞 − 1)𝑧)𝑛−𝑥(1 − 𝑧)𝑥. (2.13)
Proof of Lemma 6. Compare the coefficient of 𝑧𝑘 on both sides of (2.13). The left
hand side has 𝐾𝑘(𝑥), while on the right hand side we may use the binomial theorem
to get a coefficient of
(−1)𝑗(𝑞 − 1)𝑘−𝑗
(𝑥
𝑗
)(𝑛− 𝑥
𝑘 − 𝑗
),
38
as required.
Next, we obtain an orthogonality relation for Krawtchouk polynomials (see e.g. [36,
Thm 1]).
Lemma 7. We have𝑛∑
𝑖=1
𝐾𝑖(𝑗) = 𝑞𝑛 1(𝑗 = 0) − 1. (2.14)
Proof of Lemma 7. It is obvious by the definition of Krawtchouk polynomials 7 that
𝐾0(𝑗) = 1, so we may rewrite the desired (2.14) as
𝑛∑𝑖=0
𝐾𝑖(𝑗) = 𝑞𝑛 1(𝑗 = 0).
Let 𝑥 be a fixed vector of Hamming weight 𝑗. We may use the expression of
Krawtchouk polynomials in terms of characters (Lemma 1) to obtain
𝑛∑𝑖=0
𝐾𝑖(𝑗) =𝑛∑
𝑖=0
∑𝑦∈F𝑛
𝑞 ,|𝑦|=𝑖
𝜔⟨𝑥,𝑦⟩
=∑𝑦∈F𝑛
𝑞
𝜔⟨𝑥,𝑦⟩
= 𝑞𝑛 1(𝑗 = 0),
where for the last line we used a basic property of characters (Lemma 2).
2.3.2 Proof of Theorem 1
With these inequalities in hand, we may turn to the proof of Theorem 1.
Proof of Theorem 1. We may first dispose of trivialities 𝑑 = 𝑛 (the full space), and
𝑑 = 0 (a single point). By (2.12), we see that 𝑁𝑞(𝑛, 𝑑)− 1 is less than or equal to the
39
solution of the following LP
max𝐴𝑖
𝑛∑𝑖=1
𝐴𝑖 (2.15)
s.t. 𝐴𝑖 = 0 ∀𝑑 < 𝑖 ≤ 𝑛
𝐴𝑖 ≥ 0 ∀1 ≤ 𝑖 ≤ 𝑛
𝑛∑𝑖=1
𝐴𝑖𝐾𝑗(𝑖) ≥ −𝐾𝑗(0) ∀1 ≤ 𝑗 ≤ 𝑛.
The subtraction of 1 is simply due to 𝐴0 = 1, and hence can be removed from the
LP.
By weak duality, the primal LP given by (2.15) is upper bounded by the solution
of the dual LP
min𝑝𝑖
𝑛∑𝑖=1
𝐾𝑖(0)𝑝𝑖 (2.16)
s.t. 𝑝𝑖 ≥ 0 ∀1 ≤ 𝑖 ≤ 𝑛
𝑛∑𝑖=1
𝑝𝑖𝐾𝑖(𝑗) ≤ −1 ∀1 ≤ 𝑗 ≤ 𝑑.
The goal now is to construct dual variables 𝑝𝑖 that yield the optimum cost 𝑞𝑑 − 1
under the assumed constraints on 𝑑, and check their satisfiability. The way we will
achieve this is by making the 𝑝𝑖 satisfy a linear recurrence of order 𝑛 − 𝑑 + 1 with
initial conditions
𝑝𝑖 = 0 ∀1 ≤ 𝑖 ≤ 𝑛− 𝑑
𝑝𝑛−𝑑+1 =1
(𝑞 − 1)𝑛−𝑑.
Note that for 𝑑 = 1, this completely specifies the dual variables; with no need to
define the recurrence relation. For the 𝑛 − 𝑞 + 1 ≤ 𝑑 ≤ 𝑛 − 1 case, one considers a
recurrence relation with characteristic polynomial (𝑧+ 1𝑞−1
)𝑛−𝑑(𝑧−1). Explicitly, this
40
yields the recurrence
𝑝𝑘 =𝑛−𝑑+1∑𝑖=1
𝑎𝑖𝑝𝑘−𝑖
where the weights 𝑎𝑖 are
𝑎𝑖 =
(𝑛−𝑑𝑖−1
)(𝑞 − 1)𝑖−1
−(𝑛−𝑑𝑖
)(𝑞 − 1)𝑖
(2.17)
and thus satisfy∑𝑛−𝑑+1
𝑖=1 𝑎𝑖 = 1 by telescoping.
More importantly for our proof is the fact that 𝑎𝑖 ≥ 0 ∀𝑖, which follows from (2.17),
crucially using the assumption that 𝑛−𝑞+1 ≤ 𝑑. This yields immediately by induction
that 𝑝𝑖 ≥ 0 ∀𝑖.
By the general theory of linear recurrences, we know that
𝑝𝑖 = 𝑎+
(𝑛−𝑑−1∑𝑗=0
𝑏𝑗𝑖𝑗
)(1
1 − 𝑞
)𝑖
for some constants 𝑎, 𝑏𝑗 (depending on 𝑛, 𝑑, 𝑞 but independent of 𝑖). It will be of con-
venience to switch to the falling factorial basis (in order to use Newton interpolation),
and reindex by defining 𝑞𝑖 = 𝑝𝑖+1, yielding
𝑞𝑖 = 𝑐+
(𝑛−𝑑−1∑𝑗=0
𝑑𝑗
(𝑖
𝑗
))(1
1 − 𝑞
)𝑖
for some constants 𝑐, 𝑑𝑗. The constants 𝑐, 𝑑𝑗 are determined by have the specified
boundary conditions
𝑞0 = 𝑞1 = · · · = 𝑞𝑛−𝑑−1 = 0, 𝑞𝑛−𝑑 =1
(𝑞 − 1)𝑛−𝑑.
Transposing and using the fact that ∆𝑘[(1 − 𝑞)𝑛](0) = (−𝑞)𝑘 along with ∆𝑘[(𝑛𝑗
)] =
41
(𝑛
𝑗−𝑘
), one gets
𝑑𝑖 = −𝑐(−𝑞)𝑖 ∀1 ≤ 𝑖 ≤ 𝑛− 𝑑− 1.
Using 𝑞0 = 0, we get 𝑑0 = −𝑐, and hence
𝑞𝑖 = 𝑐
[1 −
(𝑛−𝑑−1∑𝑗=0
(−𝑞)𝑗(𝑖
𝑗
))1
(1 − 𝑞)𝑖
].
From 𝑞𝑛−𝑑 = 1(𝑞−1)𝑛−𝑑 and the binomial theorem, we can finally determine 𝑐 = 1
𝑞𝑛−𝑑 ,
yielding the explicit formula for the dual variables
𝑝𝑖 =1
𝑞𝑛−𝑑
[1 −
(𝑛−𝑑−1∑𝑗=0
(𝑖− 1
𝑗
)(−𝑞)𝑗
)1
(1 − 𝑞)𝑖−1
](2.18)
Nonnegativity of the 𝑝𝑖 has been verified above, so it remains to compute the dual
objective value and verify feasibility. For this purpose, we prove the following
Lemma 8.
𝑛∑𝑖=1
(𝑖− 1
𝑠
)𝐾𝑖(𝑗)
1
(1 − 𝑞)𝑖= (−1)𝑠+1 ∀0 ≤ 𝑠 ≤ 𝑛− 𝑗 − 1. (2.19)
Proof of Lemma 8. The proof is by induction on 𝑠.
For 𝑠 = 0, we may write (2.19) as
𝑛∑𝑖=1
𝐾𝑖(𝑗)1
(1 − 𝑞)𝑖= −1
or equivalently∞∑𝑖=0
𝐾𝑖(𝑗)1
(1 − 𝑞)𝑖= 0.
But this follows immediately from the generating function governing Krawtchouk
polynomials (2.13). Now suppose (2.19) holds for 𝑠 ≤ 𝑘, and we wish to prove it for
𝑠 = 𝑘 + 1 where 𝑘 + 1 ≤ 𝑛 − 𝑗. Using(𝑖−1𝑘+1
)=(
𝑖𝑘+1
)−(𝑖−1𝑘
), we reduce to showing
42
that𝑛∑
𝑖=1
(𝑖
𝑘 + 1
)𝐾𝑖(𝑗)
1
(1 − 𝑞)𝑖= 0
or equivalently∞∑𝑖=0
(𝑖
𝑘 + 1
)𝐾𝑖(𝑗)
1
(1 − 𝑞)𝑖= 0 (2.20)
Observe that as 𝑘 + 1 ≤ 𝑛− 𝑗,(
𝑖𝑘+1
)is a polynomial in 𝑖 of degree at most 𝑛− 𝑗.
Thus, taking 𝑥 = 𝑗, differentiating (2.13) at most 𝑛− 𝑗 times, and using the fact that1
1−𝑞is a root of order 𝑛− 𝑗, we get (2.20) as desired.
We now turn to checking the dual LP’s objective value for 𝑝𝑖 governed by (2.18).
First, note that
𝑛∑𝑖=1
𝐾𝑖(𝑗) =𝑛∑
𝑖=0
𝐾𝑖(𝑗)𝐾0(𝑖) − 1 (2.21)
= 𝑞𝑛 1(𝑗 = 0) − 1 (2.22)
by an orthogonality relation of Krawtchouk polynomials (Lemma 7). Using (2.21)
along with Lemma 2.19, we get
𝑛∑𝑖=1
𝑝𝑖𝐾𝑖(0) =1
𝑞𝑛−𝑑
[𝑞𝑛 − 1 + (1 − 𝑞)
(𝑛−𝑑−1∑𝑗=0
𝑞𝑗
)]
= 𝑞𝑑 − 1,
as desired.
Turning to the verification of dual feasibility, we have
𝑛∑𝑖=1
𝑝𝑖𝐾𝑖(𝑗) =1
𝑞𝑛−𝑑
[−1 + (1 − 𝑞)
(𝑛−𝑑−1∑𝑗=0
𝑞𝑗
)]
= −1 ∀1 ≤ 𝑗 ≤ 𝑑,
as required.
It is obvious that subcubes achieve this objective value, thus completing the proof.
43
Note that this is an example of a sharp bound on the value of 𝑐*(𝑓, 𝑞𝑛) for specific
choices of 𝑓 by the discussion in Section 2.1. In general, it seems like the LP bounds
tend to yield richer information for the problem of 𝑒*(𝑓, 𝑐); at the very least we know
of many more examples where the LP bounds are sharp for 𝑒*(𝑓, 𝑐). Apart from
Theorem 1 and the work of [124, 49], where sharp answers are given for specific
cases of the Johnson graph and for all cases of their 𝑞-analog, namely the Grassman
graph respectively, we do not know of any other interesting families of finite distance-
regular graphs where LP bounds yield sharp information for the isodiametric problem,
although we do think this is highly plausible.
The remainder of this chapter will focus on some 𝑒*(𝑓, 𝑐) questions. Before turn-
ing to specific choices of 𝑓, 𝑐, we note that one can without loss restrict study to
anticodes that are down-set (for any 𝑞) and right-compressed (when 𝑞 = 2) as long as
𝑓 is a monotone potential. The arguments here are classical, with right compression
operations going back to Erdős-Ko-Rado [42] and pushing down arguments that are
essentially due to Kleitman [71].
It is convenient to define the notions of slice and projection of a set for this
purpose, as in [6]. We also define a natural lexicographic order on 𝒳 𝑛𝑞 .
Definition 10. Let 𝒞 ⊆ F𝑛𝑞 , and let 𝒥 ⊆ [𝑛]. For 𝑥 ∈ F𝑛
𝑞 , denote by 𝑥𝐽 the sub-
sequence of 𝑥 obtained by deleting components 𝑥𝑡 for 𝑡 /∈ 𝒥 . We then denote the
slice
𝒞𝒥 (𝑥[𝑛]∖𝒥 ) , 𝑥𝒥 ∈ F𝒥𝑞 : 𝑥 ∈ 𝒞 for 𝑥[𝑛]∖𝒥 ∈ F[𝑛]∖𝒥
𝑞 .
We then denote the projection
𝒞𝒥 ,⋃
𝑥[𝑛]∖𝒥∈F[𝑛]∖𝒥𝑞
𝒞𝒥 (𝑥[𝑛]∖𝒥 ).
We also denote by L a natural lexicographic order generated by 0 ≤ 1 · · · ≤ 𝑞 − 1 on
F𝑛𝑞 , with the least significant positions towards the right end. We denote the set of the
lexicographically first 𝑚 elements in F𝒥𝑞 by L(F𝒥
𝑞 ,𝑚), and call this a lex-set.
44
With these notations in hand, we may readily define the pushing down operations
as follows, again following [6].
Definition 11. The pushing down operation with respect to the order L, a subset 𝒥
of indices, and an anticode 𝒞 is given by
𝐷𝒥 (L, 𝒞) ,⋃
𝑥[𝑛]∖𝒥∈𝒞[𝑛]∖𝒥
𝑦 : 𝑦[𝑛]∖𝒥 = 𝑥[𝑛]∖𝒥 and 𝑦𝒥 ∈ L(F𝒥𝑞 , |𝒞(𝑥[𝑛]∖𝒥 )|)
Then 𝒞 is said to be down-set if 𝐷𝑖(L, 𝒞) = 𝒞 for all 1 ≤ 𝑖 ≤ 𝑛.
In the special case of 𝑞 = 2, we also define the right compression operations as
follows.
Definition 12. For any 𝒞 ∈ F𝑛2 , any 𝑐 = (𝑐1, . . . , 𝑐𝑛) ∈ 𝒞, and 1 ≤ 𝑖 < 𝑗 ≤ 𝑛 we
define the right compressing operators 𝑆𝑖,𝑗 by
𝑆𝑖,𝑗(𝑐) ,⎧⎪⎨⎪⎩(𝑐1, . . . , 𝑐𝑖−1, 0, 𝑐𝑖+1, . . . , 𝑐𝑗−1, 1, 𝑐𝑗+1, . . . , 𝑐𝑛), if not in 𝒞 and 𝑐𝑗 = 0, 𝑐𝑖 = 1
𝑐 otherwise.
Also, let
𝑆𝑖,𝑗(𝒞) , 𝑆𝑖,𝑗(𝑐) : 𝑐 ∈ 𝒞.
Then 𝒞 is said to be right-compressed if 𝑆𝑖,𝑗(𝒞) = 𝒞 for all 1 ≤ 𝑖 < 𝑗 ≤ 𝑛.
With these definitions 11, 12, we may prove the following
Proposition 3. Let 𝑓 be a monotonically nondecreasing potential function, that is
𝑓 ∈ ℱ or in other words 𝑓(𝑖) ≤ 𝑓(𝑖 + 1) ∀0 ≤ 𝑖 ≤ 𝑛 − 1. Let a cardinality 𝑐 with
0 ≤ 𝑐 ≤ 𝑞𝑛 be given. Then there exists a down-set 𝒞 ⊆ F𝑛𝑞 such that for any other set
𝒞 ′ of the same cardinality, we have
𝐸𝑓 (𝒞) ≤ 𝐸𝑓 (𝒞 ′).
45
Furthermore, if 𝑞 = 2, we may take 𝒞 to be simultaneously down-set and right-
compressed.
Proof of Proposition 3. Suppose 𝒞0 minimizes 𝐸𝑓 (𝒜) over all anticodes 𝒜 with |𝒜| =
𝑐. Consider the sets 𝒞𝑖 = 𝐷𝑖 (mod 𝑛)+1(L, 𝒞𝑖−1) defined inductively for 𝑖 ≥ 1. It is
clear that this process stabilizes eventually, and moreover that it does not change the
cardinality. For example, consider a bounded potential
𝐹 (𝒞) ,∑𝑐∈𝒞
𝑐𝑞, (2.23)
where 𝑐𝑞 =∑𝑛−1
𝑖=0 𝑐𝑛−𝑖𝑞𝑖. This potential clearly can not increase with 𝑖, so it eventually
stabilizes. Once the potential stabilizes, it is clear that any further iterations do not
change the set. Let us denote this stabilized down-set by 𝒟. We argue now that
𝐸𝑓 (𝒞0) ≥ 𝐸𝑓 (𝒟). Clearly it suffices by symmetry to show that 𝐸𝑓 (𝐷𝑛(L, 𝒞)) ≤
𝐸𝑓 (𝒞) for any 𝒞. But this is easy to see. Consider two arbitrary slices of the anticode
𝒜 = 𝑥[𝑛]∖𝑛 × 𝒞𝑛(𝑥[𝑛]∖𝑛),
ℬ = 𝑦[𝑛]∖𝑛 × 𝒞𝑛(𝑦[𝑛]∖𝑛),
and note that the action of 𝐷 on 𝒜,ℬ can not increase the number of pairs of code-
words of 𝒜,ℬ of Hamming distance ≥ 𝑖 for any 0 ≤ 𝑖 ≤ 𝑛. 𝐸𝑓 (𝒞0) ≥ 𝐸𝑓 (𝒟) then
follows by the trivial spanning set characterization of ℱ established in Lemma 10.
Taking 𝒞 = 𝒟 completes the proof for 𝑞 = 2.
Now suppose 𝑞 = 2. Let 𝒟0 = 𝒟, 𝒟𝑖 = 𝑆𝑘𝑙(𝑖)(𝒟𝑖−1) defined inductively for
𝑖 ≥ 1. Here, 𝑘𝑙(𝑖) denotes some periodically repeating indexing of all (𝑘, 𝑙) pairs from
1 ≤ 𝑘 < 𝑙 ≤ 𝑛. Once again, it is clear that this process eventually stabilizes and
that it does not change the cardinality; indeed the same potential (2.23) works as a
certificate. Let 𝒞 denote the stabilized right-compressed set. It remains to check that
𝒞 is also down-set and that the number of pairs of codewords with Hamming distance
≥ 𝑖 does not increase for any 0 ≤ 𝑖 ≤ 𝑛. Again by symmetry it suffices to study
46
𝑆𝑛−1,𝑛. Consider two arbitrary slices of the anticode
𝒜 = 𝑥[𝑛]∖𝑛,𝑛−1 × 𝒞𝑛,𝑛−1(𝑥[𝑛]∖𝑛,𝑛−1),
ℬ = 𝑦[𝑛]∖𝑛,𝑛−1 × 𝒞𝑛,𝑛−1(𝑦[𝑛]∖𝑛,𝑛−1),
and note by a case by case analysis of the bits in positions 𝑛−1, 𝑛 of 𝒜 that 𝑆𝑛−1,𝑛
preserves the down-set property. Moreover, by looking at both 𝒜,ℬ, it is clear that
the number of pairs of codewords with Hamming distance past a threshold does not
increase. This completes the proof for 𝑞 = 2.
We remark that the powerful pushing-pulling operations that serve a crucial role
in establishing the complete diametric theorems of [6, 7] in general modify the cardi-
nality of the anticode, unlike the classical compression and pushing down operations.
Pushing-pulling thus seem better suited to 𝑐*(𝑓, 𝑒) questions or generalizations with
a non-uniform ground measure such as a Bernoulli weighted case (see e.g. [46] and
references therein). As our focus here is on 𝑒*(𝑓, 𝑐) questions, we do not examine this
further.
2.4 Universal optimality of some subcubes
Before we turn to the proof of Theorem 2, it is natural to wonder why we focus
on subcubes. For example, we know by the diametric theorem [7] that Hamming
balls or nontrivial Cartesian products of Hamming balls and subcubes are optimal
anticodes in the diametric sense, with pure subcubes addressing only a small range
of the diametric problem.
The reason for this is because of our focus on the phenomenon of universal opti-
mality. Consider a potential function 𝑓(𝑥) = −(𝑛−𝑥𝑛−1
). −𝑓 is completely monotonic,
and the problem of 𝑒*(𝑓, 𝑐) is nothing but the edge-isoperimetric problem in Hamming
space. In the binary case this was first solved by Harper [60] and in the non-binary
case by Lindsey [76]. In either case, the answer is given by taking the first 𝑐 vertices
in lexicographic order in Hamming space. Furthermore, this answer is unique up to
47
symmetry, specifically the group of (Hamming) distance preserving automorphisms.
Thus, for the problem of universal optimality, where the class of functions includes
one that minimizes the edge boundary (such as completely monotonic functions or
monotonic functions), we can focus on such lex-sets, which we can denote formally
using our notation as L(F𝑛𝑞 ,𝑚).
Of course, this does not address why we focus on the further specialization to
subcubes. Our restricted focus on subcubes comes from a limitation of the LP bounds
observed in [29, Prop. 29]. Suppose our set of potential functions contains 𝑛 +
1 linearly independent potential functions; this is true of the cones of completely
monotonic and monotonic functions. Then by [29, Prop. 29] we know that 𝒞 ∖ 𝑐
must have the same distance distribution for all 𝑐 ∈ 𝒞. If 𝒞 is a lex-set, this can
only happen when 𝒞 is a subcube. To see this, consider 𝑐1 = 0, 𝑐2 = 𝑚 where 𝑚 is
the maximum element in 𝒞 in lexicographic order, and look at how many neighbors
each of them has at Hamming distance 1. The only way these equate is if 𝑐2 is the
maximum element of a subcube in lexicographic order. It would be interesting if one
could get around this limitation (for example, using semidefinite programming (SDP)
bounds of some constant order [102, 53]) and prove an analog of (2.7) for cardinality
(3/8)2𝑛, or other candidate cardinalities outlined later in this chapter, such as the
presumably easier Conjecture 1 which asks the question for (1/8)2𝑛, (1/16)2𝑛.
We now turn to the proof of Theorem 2. With a good exact LP solver that sup-
ports rational arithmetic (we used [54], which incorporates exact rational LP support
from [55]) and a reasonable ability to identify sequences, for instance by looking at
finite differences, it is possible to “eye-ball” families of dual certificates of optimality
and prove Theorem 2. However, by stepping back and taking a more conceptual view,
one can get a better feel for what is happening. Accordingly, the proof we give here
uses slightly more machinery about the LP bound, most of which can be found in [29].
Following [29], we use the term quasicode for a feasible point in the Delsarte
LP [29, Defn 6].
Definition 13. A quasicode a of length 𝑛 and size 𝑁 over F𝑞 is a real column vector
48
(𝐴0, 𝐴1, . . . , 𝐴𝑛) satisfying the Delsarte inequalities (2.12). Explicitly, we have
a ≥ 0, 𝐾a ≥ 0,𝑛∑
𝑖=0
𝐴𝑖 = 𝑁, and 𝐴0 = 1.
Here 𝐾 is an (𝑛+ 1) × (𝑛+ 1) matrix with (𝑖, 𝑗) entry 𝐾𝑖(𝑗) for 0 ≤ 𝑖, 𝑗 ≤ 𝑛. Also,
a ≥ 0 means coordinate wise inequality.
We let |a| = 𝑁 denote the size of the quasicode, and define the dual of a quasicode
to be the quasicode
a⊥ ,1
|a|𝐾a.
If 𝒞 ∈ 𝐹 𝑛𝑞 is a code, its distance distribution a is a quasicode with |a| = |𝒞|. Also,
if 𝒞 is a linear code, its dual 𝒞⊥ has distance distribution a⊥.
As we are studying subcubes, we first prove that the dual of a subcube is a
subcube.
Lemma 9. Let a = (𝐴0, . . . , 𝐴𝑛) where 𝐴𝑖 =(𝑑𝑖
)(𝑞 − 1)𝑖. Then
a⊥ = (𝐴⊥0 , . . . , 𝐴
⊥𝑛 ), 𝐴⊥
𝑖 =
(𝑛− 𝑑
𝑖
)(𝑞 − 1)𝑖.
Proof of Lemma 9. Suppose F𝑞 is a field. There the dual of 𝒞𝑞,𝑑 is 𝒞𝑞,𝑛−𝑑; this follows
from the orthogonal direct sum decomposition F𝑛𝑞 = F𝑑
𝑞 ⊕ F𝑛−𝑑𝑞 with F𝑑
𝑞 ⊥ F𝑛−𝑑𝑞 .
Moreover, the distance distribution a of 𝒞𝑞,𝑑 is (𝐴0, . . . , 𝐴𝑛) with 𝐴𝑖 =(𝑑𝑖
)(𝑞 − 1)𝑖.
Now if 𝑞 is not a prime power, we are still fine since the dual distance distribution
of a subcube is a rational function of finite degree in 𝑞 given by some combination of
Krawtchouk polynomials, and we have an infinite number of 𝑞 where the expressions
agree. Alternatively, one may do a direct computation.
We now turn to examining the cone of monotonically increasing functions ℱ , and
provide a spanning set for it.
Lemma 10. ℱ is the nonnegative span of 𝑓0, 𝑓1, . . . , 𝑓𝑛 where 𝑓𝑗(𝑥) = 1(𝑥 ≥ 𝑗).
Proof of Lemma 10. 𝑓𝑗 are obviously monotonically increasing. Moreover, any mono-
49
tonically increasing 𝑓 may be written as
𝑓(𝑥) = 𝑓(0)𝑓0(𝑥) +𝑛∑
𝑗=1
(𝑓(𝑗) − 𝑓(𝑗 − 1))𝑓𝑗(𝑥).
We note that the analogous characterization for the cone of completely monotonic
functions 𝒢 is already provided by [29, Lemma 4].
A slightly more delicate issue is what happens to ℱ under duality. 𝒢 behaves very
well, and turns out to be invariant under duality [29, Lemma 10]. ℱ is unfortunately
not invariant under duality. In order to get useful information about the dual of ℱ ,
it is helpful to develop some recurrence relations for Krawtchouk polynomials:
Lemma 11.
𝐾𝑗(𝑖;𝑛, 𝑞) = 𝐾𝑗(𝑖− 1;𝑛− 1, 𝑞) −𝐾𝑗−1(𝑖− 1;𝑛− 1, 𝑞). (2.24a)
𝐾𝑗(𝑖;𝑛, 𝑞) = 𝐾𝑗(𝑖;𝑛− 1, 𝑞) + (𝑞 − 1)𝐾𝑗−1(𝑖;𝑛− 1, 𝑞). (2.24b)
Proof of Lemma 11. Basically (2.24) follows from Definition 7 by applying Pascal’s
rule in two different ways. We note that (2.24a) is widely known in coding theory
(see e.g [118, 1.2.15]).
Explicitly, we have
𝐾𝑗(𝑖;𝑛, 𝑞) =
𝑗∑𝑘=0
(−1)𝑘(𝑞 − 1)𝑗−𝑘
(𝑖
𝑘
)(𝑛− 𝑖
𝑗 − 𝑘
)
=
𝑗∑𝑘=0
(−1)𝑘(𝑞 − 1)𝑗−𝑘
((𝑖− 1
𝑘
)+
(𝑖− 1
𝑘 − 1
))(𝑛− 𝑖
𝑗 − 𝑘
)
=
𝑗∑𝑘=0
(−1)𝑘(𝑞 − 1)𝑗−𝑘
(𝑖− 1
𝑘
)((𝑛− 1) − (𝑖− 1)
𝑗 − 𝑘
)
−𝑗∑
𝑘=0
(−1)𝑘−1(𝑞 − 1)(𝑗−1)−(𝑘−1)
(𝑖− 1
𝑘 − 1
)((𝑛− 1) − (𝑖− 1)
(𝑗 − 1) − (𝑘 − 1)
)= 𝐾𝑗(𝑖− 1;𝑛− 1, 𝑞) −𝐾𝑗−1(𝑖− 1;𝑛− 1, 𝑞).
50
This proves (2.24a).
Using Pascal’s rule on the other binomial coefficient in the expansion, we have
𝐾𝑗(𝑖;𝑛, 𝑞) =
𝑗∑𝑘=0
(−1)𝑘(𝑞 − 1)𝑗−𝑘
(𝑖
𝑘
)(𝑛− 𝑖
𝑗 − 𝑘
)
=
𝑗∑𝑘=0
(−1)𝑘(𝑞 − 1)𝑗−𝑘
(𝑖
𝑘
)(((𝑛− 1) − 𝑖
𝑗 − 𝑘
)+
((𝑛− 1) − 𝑖
(𝑗 − 1) − 𝑘
))= 𝐾𝑗(𝑖;𝑛− 1, 𝑞) + (𝑞 − 1)𝐾𝑗−1(𝑖;𝑛− 1, 𝑞).
This proves (2.24b).
With the recurrence relation (2.24a) in hand, we may characterize the dual of ℱ
as follows.
Lemma 12. Let f𝑗 denote the column vector (of length 𝑛+1) corresponding to 𝑓𝑗(𝑥) =
1(𝑥 ≥ 𝑗), where 0 ≤ 𝑗 ≤ 𝑛. Suppose also that 𝑛 > 1. Then,
𝐾𝑡f𝑗 =(𝑛∑
𝑘=𝑗
(𝑛
𝑘
)(𝑞 − 1)𝑘,−𝐾𝑗−1(0;𝑛− 1),−𝐾𝑗−1(1;𝑛− 1), . . . ,−𝐾𝑗−1(𝑛− 1;𝑛− 1)
).
We denote the set of functions lying in the nonnegative span of 𝐾𝑡f𝑗 the dual cone of
the cone of monotonic functions ℱ . Symbolically, we denote this cone by ℱ⊥.
Proof of Lemma 12. If 𝑖 = 0, we get
𝑛∑𝑘=𝑗
𝐾𝑘(0) =𝑛∑
𝑘=𝑗
(𝑛
𝑘
)(𝑞 − 1)𝑘.
If 𝑖 > 0, we have
𝑛∑𝑘=𝑗
𝐾𝑘(𝑖) = −𝑗−1∑𝑘=0
𝐾𝑘(𝑖)
= −𝐾𝑗−1(𝑖− 1;𝑛− 1, 𝑞),
51
where we used orthogonality followed by telescoping (2.24a).
There are a number of essentially equivalent ways of proceeding with the proof
of Theorem 2, which differ chiefly in how one views dual certificates of optimality in
the LP. For example, in the proof of Theorem 1 we worked explicitly with the dual
variables 𝑝𝑖. A more conceptually satisfying approach that we shall adopt now is to
use the idea of positive definite functions. Specifically we shall use [29, Prop 5].
The construction of dual certificates shall rest on two key inequalities involving
Krawtchouk polynomials. The first of these holds for general reasons. Our proof of
the second uses (2.24b) to fuel an induction argument.
Lemma 13. Suppose 0 ≤ 𝑖, 𝑗 ≤ 𝑛. Then we have the following inequalities involving
Krawtchouk polynomials
∀𝑞 ≥ 2 𝐾𝑗(0) ≥ 𝐾𝑗(𝑖). (2.25)
If 𝑞 > 2, we have
𝑞 − 1
𝑞𝐾𝑗(1) +
1
𝑞𝐾𝑗(0) +
(−1)𝑖(𝐾𝑗(0) −𝐾𝑗(1))
𝑞(𝑞 − 1)𝑖−1≥ 𝐾𝑗(𝑖). (2.26)
Proof of Lemma 13. By positive-definiteness of Krawtchouk polynomials, we imme-
diately conclude (2.25).
For (2.26), we shall induct on 𝑛 and use (2.24b) to do the induction step. The
base case 𝑛 = 1 is trivial since we have equality in (2.26) for 𝑖 = 0, 1 regardless of 𝑛.
We also note that for any 𝑛 the case 𝑗 = 0 is trivial as it simply asserts 1 ≥ 1.
Using
𝐾𝑗(0) =
(𝑛
𝑗
)(𝑞 − 1)𝑗
and
𝐾𝑗(1) =
(𝑛
𝑗
)(𝑞 − 1)𝑗 − 𝑞(𝑞 − 1)𝑗−1
(𝑛− 1
𝑗 − 1
)we may rewrite (2.26) as
𝐾𝑗(𝑖;𝑛) ≤(𝑛− 1
𝑗
)(𝑞 − 1)𝑗 + (−1)𝑖(𝑞 − 1)𝑗−𝑖
(𝑛− 1
𝑗 − 1
). (2.27)
52
Now suppose (2.27) is true for 𝑛 − 1 with 𝑛 − 1 ≥ 1. We wish to establish it for
0 ≤ 𝑖, 𝑗 ≤ 𝑛. If 𝑗 = 𝑛 we have equality in (2.27) since 𝐾𝑛(𝑖;𝑛) = (−1)𝑖(𝑞 − 1)𝑛−𝑖. If
0 < 𝑖, 𝑗 ≤ 𝑛− 1, we may use the induction hypothesis to obtain
𝐾𝑗(𝑖;𝑛− 1) ≤(𝑛− 2
𝑗
)(𝑞 − 1)𝑗 + (−1)𝑖(𝑞 − 1)𝑗−𝑖
(𝑛− 2
𝑗 − 1
), (2.28)
(𝑞 − 1)𝐾𝑗−1(𝑖;𝑛− 1) ≤(𝑛− 2
𝑗 − 1
)(𝑞 − 1)𝑗 + (−1)𝑖(𝑞 − 1)𝑗−𝑖
(𝑛− 2
𝑗 − 2
). (2.29)
Adding (2.28) and (2.29) to each other and using (2.24b) we get
𝐾𝑗(𝑖;𝑛)
≤((
𝑛− 2
𝑗 − 1
)+
(𝑛− 2
𝑗
))(𝑞 − 1)𝑗 + (−1)𝑖(𝑞 − 1)𝑗−𝑖
((𝑛− 2
𝑗 − 1
)+
(𝑛− 2
𝑗 − 2
))=
(𝑛− 1
𝑗
)(𝑞 − 1)𝑗 + (−1)𝑖(𝑞 − 1)𝑗−𝑖
(𝑛− 1
𝑗 − 1
),
thereby proving the induction step as long as 𝑖 ≤ 𝑛− 1.
All that remains is establishing (2.27) for 𝑖 = 𝑛 and 0 < 𝑗 ≤ 𝑛− 1. Using
𝐾𝑗(𝑛;𝑛) = (−1)𝑗(𝑛
𝑗
)
and cancelling common factors, we reduce our task to establishing
(−1)𝑗𝑛 ≤ (𝑛− 𝑗)(𝑞 − 1)𝑗 + (−1)𝑛𝑗(𝑞 − 1)𝑗−𝑛. (2.30)
If 𝑗 is odd, it suffices to prove that
(𝑞 − 1)𝑛 ≥ 𝑗
𝑛− 𝑗.
Since we have the well known 2𝑛 ≥ 𝑛− 1, we get
(𝑞 − 1)𝑛 ≥ 2𝑛 ≥ 𝑛− 1 ≥ 𝑗
𝑛− 𝑗
as 𝑗 ≤ 𝑛− 1, thus proving the 𝑗 odd case.
53
Now suppose 𝑗 is even. If 𝑛 is even, it suffices to prove
𝑛
𝑛− 𝑗≤ (𝑞 − 1)𝑗.
But we have𝑛
𝑛− 𝑗≤ 𝑗 + 1 ≤ 2𝑗 ≤ (𝑞 − 1)𝑗
using the well known 2𝑗 ≥ 𝑗 + 1. Now suppose 𝑛 is odd. We note that it suffices to
prove (2.30) for 𝑛 = 𝑗 + 1, since for a fixed 𝑗 incrementing 𝑛 by 1 increases the left
hand side by 1, while the right hand side increases by at least 4 since 𝑞 ≥ 3. Now for
𝑛 ≥ 3 we have by an obvious induction
3𝑛
2− 1
2≤ 2𝑛−1.
Thus, we have
𝑛+𝑛− 1
𝑞 − 1≤ 𝑛+
𝑛− 1
2≤ 2𝑛−1 ≤ 𝑞𝑛−1,
completing the proof of (2.30).
Lemma 13 will allow us to prove Theorem 2 quite easily. We use the framework
of [29, Prop 5].
Proof of Theorem 2. We begin by noting that the statement for the full space (2.5)
is trivial. Slightly less trivial, but also clear is (2.2).
Nevertheless, we shall examine (2.2) from the LP perspective, in order to then use
duality and obtain (2.4).
As we are working with the subcube of cardinality 𝑞, we already have quite a lot
of information on what to use for ℎ. Specifically, we know ℎ(𝑖) = 𝑐0 + 𝑐𝑛𝐾𝑛(𝑖) for
some 𝑐𝑛 ≥ 0. We need ℎ(1) = 𝑓(1), this gives us
𝑐0 − 𝑐𝑛(𝑞 − 1)𝑛−1 = 𝑓(1).
54
A simple choice is to use 𝑐0 = 𝑓(1) and 𝑐𝑛 = 0. This choice works as long as
𝑓(1) ≤ 𝑓(𝑘) ∀𝑘 > 1, (2.31)
yielding the desired objective value of (𝑞 − 1)𝑓(1). Constraint (2.31) is met by all
𝑓 ∈ ℱ , thus proving (2.2) from the LP perspective, although it is trivial to see directly
as well. More importantly we note that by (2.25) and Lemma 12 all functions in ℱ⊥
also satisfy (2.31). By a simple modification of [29, Prop 9] to use ℱ and ℱ⊥ instead
of 𝒢, we have proved the 𝒞𝑞,𝑛−1 case (2.4).
We now turn to the cube of size 𝑞2. Duality will then allow us to obtain the
corresponding statements for 𝑞𝑛−2. Once again, we know that
ℎ(𝑖) = 𝑐0 + 𝑐𝑛𝐾𝑛(𝑖) + 𝑐𝑛−1𝐾𝑛−1(𝑖)
for some 𝑐𝑛, 𝑐𝑛−1 ≥ 0 and that we also need ℎ(1) = 𝑓(1) and ℎ(2) = 𝑓(2). A simple
choice is to use 𝑐𝑛−1 = 0. Thus, we have ℎ(𝑖) = 𝑐0 + 𝑐𝑛(−1)𝑖(𝑞 − 1)𝑛−𝑖. Solving the
simultaneous linear equations ℎ(1) = 𝑓(1) and ℎ(2) = 𝑓(2) for 𝑐0, 𝑐𝑛, we get
ℎ(𝑖) = 𝑓(1)1
𝑞+ 𝑓(2)
𝑞 − 1
𝑞+
(−1)𝑖(𝑓(2) − 𝑓(1))
𝑞(𝑞 − 1)𝑖−2. (2.32)
We note that such an ℎ, if it satisfies ℎ(𝑖) ≤ 𝑓(𝑖) ∀1 ≤ 𝑖 ≤ 𝑛, yields the desired
objective value since
𝑞2𝑐0 − ℎ(0) = 𝑞𝑓(1) + 𝑞(𝑞 − 1)𝑓(2) − 𝑐0 − 𝑐𝑛(𝑞 − 1)𝑛
= 𝑞𝑓(1) + 𝑞(𝑞 − 1)𝑓(2) − 𝑐0 −(𝑞 − 1)2(𝑓(2) − 𝑓(1))
𝑞
=
(𝑞 − 1
𝑞+
(𝑞 − 1)2
𝑞
)𝑓(1) +
(𝑞(𝑞 − 1) − 𝑞 − 1
𝑞− (𝑞 − 1)2
𝑞
)𝑓(2)
= 2(𝑞 − 1)𝑓(1) + (𝑞 − 1)2𝑓(2).
It is clear from the form of ℎ (2.32) that ℎ(𝑖) lies in between 𝑓(1) and 𝑓(2) for
all 1 ≤ 𝑖 ≤ 𝑛, regardless of the value of 𝑞. Now, if 𝑓 is a monotonically increasing
55
function, such an ℎ is a valid dual certificate. In particular, any 𝑓 ∈ ℱ satisfies this
constraint, so we have proved (2.3). As 𝒢 ⊆ ℱ and 𝒢 is invariant under duality,
by [29, Prop 9] we have proved (2.7). In fact, we can conclude the analog of (2.7) for
general 𝑞. Nonetheless, we wish to prove something stronger for 𝑞 > 2, namely (2.6).
Here (2.26) plays an important role. First, note that by the analog of [29, Prop 9]
for ℱ ,ℱ⊥ it suffices to show universal optimality over ℱ⊥ for 𝒞𝑞,2. Equivalently we
may show it over a spanning set of ℱ⊥, without loss that given by 12.
The validity of ℎ given by (2.32) would follow from the inequality
− 1
𝑞𝐾𝑗−1(0) − 𝑞 − 1
𝑞𝐾𝑗−1(1) +
(−1)𝑖(−𝐾𝑗−1(1) +𝐾𝑗−1(0))
𝑞(𝑞 − 1)𝑖−2≤ −𝐾𝑗−1(𝑖− 1). (2.33)
Negating and relabelling the indices 𝑗 − 1 → 𝑗, 𝑖 − 1 → 𝑖, and the implicit index
𝑛 − 1 → 𝑛, (2.33) is nothing but (2.26). This completes the proof of (2.6), and in
turn Theorem 2.
2.5 Connection to maximum noise stability
The main goal of this section is to elaborate upon the connection between the “ground
state” problem for anticodes and “maximal noise stability”. The key idea is that an
anticode may be viewed as the inverse image of 0 (or by complementarity 1) of a
Boolean valued function. Furthermore, it is not unreasonable to expect a connection,
since we know that by the edge isoperimetric inequality, subcubes are optimal sets
in terms of their edge boundary. Thus the edge isoperimetric inequality implies
that subcubes are optimal anticodes with respect to a potential function that is zero
everywhere except at distance 1, where it is negative. The corresponding functions,
namely the and𝑘 functions have very good noise stability when 𝑘 is small, especially
in the low noise limit. In fact, the derivative of noise stability with respect to the noise
parameter at noise level 0 is an affine function of the size of the edge boundary [89,
Prop 2.51].
We first prove a lemma that makes this connection precise. As this lemma does not
56
necessarily require a product noise structure, we derive it in slightly greater generality.
Lemma 14. Let 𝑓 : F𝑛𝑞 → 0, 1 be a Boolean valued function, and let 𝒞 = 𝑓−1(0).
Let 𝑠(·|·) be a row-stochastic 𝑞𝑛 × 𝑞𝑛 transition probability matrix. We assume that 𝑠
has an additive noise structure that is equidistributed over Hamming shells. In other
words, 𝑠 is determined by the vector (𝑠0, . . . , 𝑠𝑛) with 𝑠(𝑦|𝑥) = 𝑠|𝑥−𝑦|, 𝑠𝑖 ≥ 0, and∑𝑛𝑖=0
(𝑛𝑖
)(𝑞 − 1)𝑖𝑠𝑖 = 1. Let ℎ(𝑖) = 𝑠𝑖 be a potential function. Then, we have:
Stab𝑠(𝑛, 𝑓) = 1 + 2|𝒞|𝑞−𝑛(𝐸ℎ(𝒞) + 𝑠0 − 1). (2.34)
Proof of Lemma 14.
Stab𝑠(𝑛, 𝑓) = Pr(𝑓(x) = 𝑓(y))
=1
𝑞𝑛
(∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥) +∑𝑥∈𝒞𝑐
Pr(y ∈ 𝒞𝑐|x = 𝑥)
)
=1
𝑞𝑛
(∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥) + 𝑞𝑛 − |𝒞| −∑𝑥∈𝒞𝑐
Pr(y ∈ 𝒞|x = 𝑥)
)
=1
𝑞𝑛
(∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥) + 𝑞𝑛 − |𝒞| −∑𝑥∈𝒞
Pr(y ∈ 𝒞𝑐|x = 𝑥)
)
=1
𝑞𝑛
(∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥) + 𝑞𝑛 − |𝒞| − |𝒞| +∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥)
)
=1
𝑞𝑛
(2∑𝑥∈𝒞
Pr(y ∈ 𝒞|x = 𝑥) + 𝑞𝑛 − 2|𝒞|
)(2.35)
= 1 − 2|𝒞|𝑞−𝑛 + 2𝑞−𝑛
( ∑𝑥∈𝒞,𝑦∈𝒞
𝑠|𝑥−𝑦|
)
= 1 + 2|𝒞|𝑞−𝑛(𝐸ℎ(𝒞) + 𝑠0 − 1).
We have chosen to highlight (2.35) as we will need this intermediate step later in
this chapter. This step depends upon the symmetry 𝑠(𝑦|𝑥) = 𝑠(𝑥|𝑦), but does not
need additive noise structure. We remark that Stab𝑠(𝑛, 𝑓) = Stab𝑠(𝑛, 𝑓) together
57
with the above proof imply the “particle-antiparticle” relation involving the potential
energy [29, Sec VII].
In an application to noise stability, one naturally specializes the above Lemma 14
to the BSC and 𝑞-SC families. This gives us Corollary 1 and Corollary 2.
Proof of Corollaries 1 and 2. For general 𝑞, we have for the 𝑞-SC
𝑠𝑖 = (1 − 𝜖)𝑛−𝑖
(𝜖
𝑞 − 1
)𝑖
= (1 − 𝜖)𝑛(
𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
Observe that if
0 ≤ 𝜖 ≤ 1 − 1
𝑞,
we have
0 ≤ 𝜖
(𝑞 − 1)(1 − 𝜖)≤ 1.
Thus 𝑠 gives a completely monotonic potential ℎ in Lemma 14. Lemma 14 shows that
maximizing noise stability subject to a cardinality constraint is equivalent to maxi-
mizing 𝐸ℎ(𝒞). We may then use Theorem 2 (specifically eqs. (2.4), (2.6) and (2.7))
together with complementarity (Stab𝑠(𝑛, 𝑓) = Stab𝑠(𝑛, 𝑓)) to obtain Corollary 1
and Corollary 2.
We remark that [29, Prop 29] allows us to remove an arbitrary point of a subcube
of measure 1/(𝑞2) and get universal optimality for a set of measure 1/(𝑞2) − 1/(𝑞𝑛).
By complementarity, one can add an arbitrary point to the complement of a subcube
to get universal optimality for a set of measure (𝑞2−1)/(𝑞2)+1/(𝑞𝑛). Similar remarks
apply to the other subcubes. We did not explicitly record these facts in Corollary 1
and Corollary 2 as such operations result in an asymptotically vanishing perturbation
of measure. Similarly, we did not record explicitly the noise stability analog for 𝒞𝑞,2or 𝒞𝑞,1 since these subcubes have an asymptotically vanishing measure.
We also find it illustrative to express the noise stability across the 𝑞-SC of an anti-
code in terms of its dual distance distribution. One application of this is in providing
58
a quick way to deduce the intuitive fact that the noise stability is monotonically non-
increasing from 𝜖 = 0 to 𝜖 = 1 − 1/𝑞 (Corollary 4), something which is unclear from
the expression (2.34). We note that we are essentially rephrasing the discussion of [89,
Sec. 2.4] in slightly different language and for general 𝑞; see also [89, Ex. 5.28]. The
link to Fourier analysis on the hypercube should not be too mysterious to the reader,
especially in view of our preliminary remarks 1.
Proposition 4. Let 𝜖 ∈ [0, 1]. Define the correlation factor
𝜌 , 1 − 𝑞𝜖
𝑞 − 1.
Then we have
Stab𝜖(𝑛, 𝒞) = 1 + 2𝜇
(𝜇
𝑛∑𝑘=0
𝜌𝑘𝐴⊥𝑘 − 1
). (2.36)
Here, 𝐴⊥𝑘 denotes the dual distance distribution of 𝒞.
Proof. By the generating function for Krawtchouk polynomials (2.13), we have
ℎ(𝑖) = (1 − 𝜖)𝑛(
𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
= 𝑞−𝑛(1 + (𝑞 − 1)𝜌)𝑛−𝑖(1 − 𝜌)𝑖
= 𝑞−𝑛
𝑛∑𝑘=0
𝐾𝑘(𝑖;𝑛, 𝑞)𝜌𝑘.
Then we have by (2.34)
Stab𝜖(𝑛, 𝑓) = 1 + 2𝜇
(𝑛∑
𝑖=0
𝐴𝑖ℎ(𝑖) − 1
)
= 1 + 2𝜇
(𝑞−𝑛
𝑛∑𝑘=0
𝜌𝑘𝑛∑
𝑖=0
𝐴𝑖𝐾𝑘(𝑖;𝑛, 𝑞) − 1
)
= 1 + 2𝜇
(𝜇
𝑛∑𝑘=0
𝜌𝑘𝐴⊥𝑘 − 1
).
As an immediate corollary of 4, we have the desired monotonicity in 𝜖 of the noise
59
stability.
Corollary 4. Stab𝜖(𝑛, 𝒞) is monotonically decreasing in 𝜖 for 0 ≤ 𝜖 ≤ 1 − 1𝑞. Fur-
thermore, with 𝜇 = |𝒞|𝑞𝑛
, we have
Stab𝜖(𝑛, 𝒞) ≥ 𝜇2 + (1 − 𝜇)2.
Proof of Corollary 4. The interval for 𝜖 corresponds precisely to 0 ≤ 𝜌 ≤ 1. The
lower bound follows since 𝜌 = 0 corresponds to x ⊥ y.
2.6 A mean value theorem for noise stability
Although the BSC and 𝑞-SC are entirely reasonable channel models for noise stability
that are perhaps most useful for current applications, one may wonder what can be
said about maximal noise stability with respect to other channels. In general, this is
not an easy question. For example, the LP bounds (or the SDP generalizations) will
work only for a noise that respects the underlying symmetries of Hamming space as
seen in Lemma 14. In particular, it is easy to see that the only product noise that
respects such symmetries is the one given by the 𝑞-SC.
Intuitively, it seems clear that for a general channel maximal noise stability should
be at least as high as that for a 𝑞-SC with noise level being chosen appropriately to
match the “average” chance of a bit flip. This is because one should be able to “tailor” a
function better for a channel that does not treat coordinates and letters symmetrically.
One route to proving such things is by formulating a random coding/mean value
theorem for noise stability. Making these intuitive ideas precise is the subject of
Theorem 3.
Proof of Theorem 3. First, note that 𝑡 is nonnegative and row-stochastic, and thus a
valid probability kernel, by its definition (2.8). This ensures that (2.9) is well defined.
We have the following
60
1
|Aut|∑𝜎∈Aut
Stab𝑠(𝑛, 𝜎𝑓)
=1
|Aut|∑𝜎∈Aut
Pr((𝜎𝑓)(x) = (𝜎𝑓)(y))
=1
|Aut|∑𝜎∈Aut
1
𝑞𝑛
∑𝑥,𝑦∈F𝑛
𝑞
1(𝑓(𝜎𝑥) = 𝑓(𝜎𝑦))𝑠(𝑦|𝑥)
=1
𝑞𝑛
∑𝑥,𝑦∈F𝑛
𝑞
∑𝜎∈Aut
1
|Aut|1(𝑓(𝜎𝑥) = 𝑓(𝜎𝑦))𝑠(𝑦|𝑥)
=1
𝑞𝑛
∑𝑥,𝑦∈F𝑛
𝑞
∑𝜎∈Aut
1
|Aut|1(𝑓(𝑥) = 𝑓(𝑦))𝑠(𝜎−1𝑦|𝜎−1𝑥)
=1
𝑞𝑛
∑𝑥,𝑦∈F𝑛
𝑞
1(𝑓(𝑥) = 𝑓(𝑦))
(1
|Aut|∑𝜎∈Aut
𝑠(𝜎−1𝑦|𝜎−1𝑥)
)
=1
𝑞𝑛
∑𝑥,𝑦∈F𝑛
𝑞
1(𝑓(𝑥) = 𝑓(𝑦))𝑡(𝑦|𝑥)
= Stab𝑡(𝑛, 𝑓).
This completes the proof.
Using Theorem 3 we can immediately derive the following corollary for maximal
noise stability.
Corollary 5. Using the notation of Theorem 3, we have
Stab*𝑠(𝑛, 𝜇) ≥ Stab*
𝑡 (𝑛, 𝜇). (2.37)
Proof of Corollary 5. An important feature of the proof of Theorem 3 is that the
expected value of a Boolean function is invariant under our chosen group action, that
is E[𝑓(x)] = E[(𝜎𝑓)(x)]. Let a function 𝑓 be chosen such that subject to E[𝑓(x)] = 𝜇,
𝑓 attains maximal noise stability under 𝑡(·|·). By the above Theorem 3, we see that
there exists a 𝜎 ∈ Aut such that Stab𝑠(𝑛, 𝜎𝑓) ≥ Stab*𝑡 (𝑛, 𝜇). Thus, Stab*
𝑠(𝑛, 𝜇) ≥
Stab𝑠(𝑛, 𝜎𝑓) ≥ Stab*𝑡 (𝑛, 𝜇). This completes the proof.
61
We may specialize 5 to the case of an i.i.d product kernel 𝑠(·|·) to get a “sym-
metrized” 𝑡(·|·) that corresponds to a 𝑞-BSC.
Corollary 6. Let 𝑠(𝑏𝑛|𝑎𝑛) =∏𝑛
𝑖=1 𝑟(𝑏𝑖|𝑎𝑖) ∀𝑎𝑛 ∈ F𝑛𝑞 be a product kernel, where 𝑟(·|·)
is a 𝑞×𝑞 probability kernel. Let 𝜖 = 𝑞−tr(𝑟)𝑞
, where tr(𝑟) denotes the trace of the kernel
𝑟 viewed as a 𝑞 × 𝑞 row-stochastic matrix. Then, we have
Stab*𝑟(𝑛, 𝜇) ≥ Stab*
𝜖(𝑛, 𝜇). (2.38)
Proof of Corollary 6. Let Π𝑞 denote the permutation group on 𝑞 letters. We know
that the distance preserving automorphisms of F𝑛𝑞 consist of permutations of the
𝑛 coordinates composed with arbitrary permutations of the individual coordinates.
Using this decomposition, by (2.8), we have
𝑡(𝑦|𝑥) =1
|Aut|∑𝜎∈Aut
𝑠(𝜎𝑦|𝜎𝑥)
=1
|Aut|∑𝜎∈Aut
𝑛∏𝑖=1
𝑟((𝜎𝑦)𝑖|(𝜎𝑥)𝑖)
=𝑛!
|Aut|
𝑛∏𝑖=1
∑𝜎∈Π𝑞
𝑟(𝜎𝑦𝑖|𝜎𝑥𝑖)
=𝑛∏
𝑖=1
⎡⎣ 1
𝑞!
∑𝜎∈Π𝑞
𝑟(𝜎𝑦𝑖|𝜎𝑥𝑖)
⎤⎦=
(tr(𝑟)𝑞
)𝑛−|𝑥−𝑦|(𝑞 − tr(𝑟)𝑞(𝑞 − 1)
)|𝑥−𝑦|.
This proves (2.38).
2.7 Large 𝑛 and balls versus subcubes
One aspect of the anticoding problem that we find intriguing is the role of balls versus
subcubes in Hamming space. For example, the optimal anticodes in the isodiamet-
ric sense given by [7, Diametric Theorem] involve Cartesian products of balls and
subcubes. In the context of binary noise stability, at small expected value 𝜇 → 0
62
and high noise 𝜖 → 1/2, it is well known that Hamming balls of appropriate radius
maximize noise stability in a sharp sense as 𝑛 → ∞ [110, Prop. 2.2]. On the other
hand, we know thanks to Corollary 1 that for 𝜇 ∈ 1/4, 1/2, 3/4 we maximize noise
stability regardless of 𝜖 ≤ 1/2 by using Hamming subcubes and their complements.
The goal of this section is to further explore the question of which sets maximize noise
stability subject to an expected value constraint.
It is perhaps useful to define a limiting notion of noise stability for a product noise
that captures 𝑛 → ∞. The formulation of a limiting value for noise stability avoids
issues such as how many points one needs to pick from the last shell of the Hamming
ball in order to meet a cardinality constraint, at the cost of possibly missing out
on finite 𝑛 phenomena. Although we do not make much further use of this limiting
notion once we establish it, we find it conceptually satisfying. Moreover, we do make
further use in our proof of Theorem 4 of some auxiliary Lemmas 15, 18 that are
established en route to this definition.
Once defined rigorously, we shall denote the maximum noise stability for a sym-
metric kernel 𝑟 on 𝒳 ×𝒳 by Stab*𝑟(∞, 𝜇). In an analogous manner to our other nota-
tion, we may specialize to the 𝑞-SC with parameter 𝜖 and denote it by Stab*𝜖(∞, 𝜇).
Our approach shall be to define the limit on 𝑞-adic 𝜇 first, and then extend by
uniform continuity to all 𝜇 ∈ [0, 1]. The first task is simple, and so we do it first.
In order to do so, it is helpful to understand the general behavior of Cartesian
products of anticodes under a product noise.
Lemma 15. Let 𝒞𝑚 ⊆ F𝑚𝑞 have measure 𝜇𝑚 = |𝒞𝑚|/𝑞𝑚. Similarly, let 𝒞𝑛 ⊆ F𝑛
𝑞 have
measure 𝜇𝑛 = |𝒞𝑛|/𝑞𝑛. Let 𝑟(·|·) be a 𝑞 × 𝑞 symmetric transition probability matrix.
Let 𝒞 = 𝒞𝑚 × 𝒞𝑛. Then its measure 𝜇 = |𝒞|/𝑞𝑚+𝑛 = 𝜇𝑚𝜇𝑛, and its noise stability is
given by
Stab𝑟(𝑚+ 𝑛, 𝒞) + 2𝜇− 1 =1
2(Stab𝑟(𝑚, 𝒞𝑚) + 2𝜇𝑚 − 1)(Stab𝑟(𝑛, 𝒞𝑛) + 2𝜇𝑛 − 1).
Proof of Lemma 15. The measure statement is trivial. The stability statement follows
63
readily from (2.35), as follows.
Stab𝑟(𝑚+ 𝑛, 𝒞) + 2𝜇− 1
=2
𝑞𝑚+𝑛
∑𝑥𝑚∈𝒞𝑚
∑𝑥𝑛∈𝒞𝑛
Pr(y ∈ 𝒞|x = 𝑥)
=2
𝑞𝑚+𝑛
∑𝑥𝑚∈𝒞𝑚
∑𝑥𝑛∈𝒞𝑛
Pr(y𝑚 ∈ 𝒞𝑚|x𝑚 = 𝑥𝑚) Pr(y𝑛 ∈ 𝒞𝑛|x𝑛 = 𝑥𝑛)
=1
2
[2
𝑞𝑚
∑𝑥𝑚∈𝒞𝑚
Pr(y𝑚 ∈ 𝒞𝑚|x𝑚 = 𝑥𝑚)
][2
𝑞𝑛
∑𝑥𝑛∈𝒞𝑛
Pr(y𝑛 ∈ 𝒞𝑛|x𝑛 = 𝑥𝑛)
]
=1
2(Stab𝑟(𝑚, 𝒞𝑚) + 2𝜇𝑚 − 1)(Stab𝑟(𝑛, 𝒞𝑛) + 2𝜇𝑛 − 1).
Lemma 16. Let 𝜇 = 𝑎𝑞𝑘
be a 𝑞-adic fraction. Let 𝑟(·|·) be a row-stochastic 𝑞 × 𝑞
transition probability matrix. Then the following limit exists and may be used to
define
Stab*𝑟(∞, 𝜇) , lim
𝑛→∞Stab*
𝑟(𝑛, 𝜇). (2.39)
Proof of Lemma 16. The sequence at hand is uniformly bounded by 1. Furthermore,
we claim that it is non-decreasing. Let 𝒜*𝑚 denote an optimal set for noise stability
at 𝜇 for 𝑛 = 𝑚. Consider 𝒜𝑚+1 = 𝒜*𝑚 × F𝑞. Then by Lemma 15,
Stab*𝑟(𝑚+ 1, 𝜇) ≥ Stab𝑟(𝑚+ 1,𝒜𝑚+1) = Stab𝑟(𝑚,𝒜*
𝑚) = Stab*𝑟(𝑚,𝜇).
The slightly trickier task is to prove uniform continuity. Our approach to this is
to use a randomly chosen subset of the appropriate cardinality to get a reasonably
good subanticode given an optimal anticode. Our approach in fact yields Lipschitz
continuity.
The “averaging” step is contained in the following
Lemma 17. Let 𝑎𝑖𝑗, 1 ≤ 𝑖, 𝑗 ≤ 𝑛 denote a collection of reals. Let 𝑚 ≤ 𝑛, and let 𝒜
64
denote the collection of 𝑚-subsets of [𝑛]. Then, we have
1
|𝒜|∑ℬ∈𝒜
∑(𝑖,𝑗)∈ℬ×ℬ
𝑎𝑖𝑗 =𝑚
𝑛
𝑛∑𝑖=1
𝑎𝑖𝑖 +𝑚(𝑚− 1)
𝑛(𝑛− 1)
∑𝑖 =𝑗
𝑎𝑖𝑗. (2.40)
Furthermore, if 𝑎𝑖𝑗 are nonnegative, we have the immediate estimate
1
|𝒜|∑ℬ∈𝒜
∑(𝑖,𝑗)∈ℬ×ℬ
𝑎𝑖𝑗 ≥𝑚(𝑚− 1)
𝑛(𝑛− 1)
∑𝑖,𝑗
𝑎𝑖𝑗. (2.41)
Proof of Lemma 17. The fraction of the number of times a given diagonal element
appears is (𝑚1 )
(𝑛1)
. Similarly, for an off-diagonal element, it is (𝑚2 )
(𝑛2)
. This proves (2.40).
The estimate (2.41) follows immediately from 𝑚 ≤ 𝑛 and 𝑎𝑖𝑗 ≥ 0.
Lemma 17 together with the noise stability expression (2.35) allow one to readily
understand the noise stability of random subanticodes.
Lemma 18. Let 𝑚′ ≤ 𝑚 ≤ 𝑞𝑛 denote two cardinalities, and let 𝜇′ = 𝑚′𝑞𝑛, 𝜇 = 𝑚
𝑞𝑛
denote their respective measures. Let 𝒞 denote an anticode of size 𝑚. Let 𝒜 denote
the collection of anticodes 𝒞 ′ of size 𝑚′ obtained as 𝑚′-subsets of 𝒞. Then
1
|𝒜|∑𝒞′∈𝒜
Stab𝑟(𝑛, 𝒞 ′) ≥ (1 − 2𝜇′) +𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)(Stab𝑟(𝑛, 𝒞) + 2𝜇− 1) . (2.42)
Proof of Lemma 18.
1
|𝒜|∑𝒞′∈𝒜
Stab𝑟(𝑛, 𝒞 ′) = (1 − 2𝜇′) +2
𝑞𝑛
⎛⎝ 1
|𝒜|∑𝒞′∈𝒜
∑(𝑥,𝑦)∈𝒞′×𝒞′
Pr(y = 𝑦|x = 𝑥)
⎞⎠≥ (1 − 2𝜇′) +
𝑚′(𝑚′ − 1)
𝑚(𝑚− 1)
⎛⎝ 2
𝑞𝑛
∑(𝑥,𝑦)∈𝒞×𝒞
Pr(y = 𝑦|x = 𝑥)
⎞⎠= (1 − 2𝜇′) +
𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)(Stab𝑟(𝑛, 𝒞) + 2𝜇− 1) .
The above Lemmas 16, 18 allow us to uniquely define Stab*𝑟(∞, 𝜇) via
65
Proposition 5. There exists a unique Lipschitz continuous (in 𝜇) extension of Stab*𝑟(∞, 𝜇)
defined on a dense subset via (2.39) to all 𝜇 ∈ [0, 1].
Proof of Proposition 5. Let 𝜇′ = 𝑚′/𝑞𝑘, 𝜇 = 𝑚/𝑞𝑘 be two 𝑞-adic measures in (0, 1).
Without loss of generality suppose 𝜇′ ≤ 𝜇. Let 𝜖 > 0, and choose 𝑛 ≥ 𝑘 large enough
such that we have simultaneously
Stab*𝑟(∞, 𝜇′) − Stab*
𝑟(𝑛, 𝜇′) ≤ 𝜖, (2.43)
Stab*𝑟(∞, 𝜇) − Stab*
𝑟(𝑛, 𝜇) ≤ 𝜖, (2.44)
Stab*𝑟(∞, 1 − 𝜇′) − Stab*
𝑟(𝑛, 1 − 𝜇′) ≤ 𝜖, (2.45)
Stab*𝑟(∞, 1 − 𝜇) − Stab*
𝑟(𝑛, 1 − 𝜇) ≤ 𝜖, (2.46)
𝜇′2
𝜇2− 𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)≤ 𝜖,
(1 − 𝜇)2
(1 − 𝜇′)2− (1 − 𝜇)(𝑞𝑛 −𝑚− 1)
(1 − 𝜇′)(𝑞𝑛 −𝑚′ − 1)≤ 𝜖.
We note that estimates (2.43), (2.44) are ineffective, as we do not know how fast
we converge to the large 𝑛 limit, though Lemma 16 guarantees that we get there
eventually. The following two (2.45), (2.46) follow from (2.43), (2.44) by taking
complements, and the remainder are effective.
Then by Lemma 18, we have a lower bound on Stab*𝑟(∞, 𝜇′) − Stab*
𝑟(∞, 𝜇)
Stab*𝑟(∞, 𝜇′) − Stab*
𝑟(∞, 𝜇)
≥ −2𝜖+ (1 − 2𝜇′) +
(𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)− 1
)Stab*
𝑟(𝑛, 𝜇) +𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)(2𝜇− 1)
≥ −2𝜖+ (1 − 2𝜇′) +
(𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)− 1
)+𝜇′(𝑚′ − 1)
𝜇(𝑚− 1)(2𝜇− 1)
≥ −2𝜖+ (1 − 2𝜇′) +
(𝜇′2
𝜇2− 1 − 𝜖
)+𝜇′2
𝜇2(2𝜇− 1) − 𝜖|2𝜇− 1|
≥ −4𝜖+ (1 − 2𝜇′) +
(𝜇′2
𝜇2− 1
)+𝜇′2
𝜇2(2𝜇− 1)
= −4𝜖− 2𝜇′(𝜇− 𝜇′)
𝜇
≥ −4𝜖− 2(𝜇− 𝜇′). (2.47)
66
Using the complementarity relations Stab*𝑟(𝑛, 𝜇) = Stab*
𝑟(𝑛, 1−𝜇) and their limiting
equivalents Stab*𝑟(∞, 𝜇) = Stab*
𝑟(∞, 1−𝜇) (valid at this stage for 𝑞-adic 𝜇), we may
get an upper bound in similar manner to (2.47) as
Stab*𝑟(∞, 𝜇′) − Stab*
𝑟(∞, 𝜇) = −(Stab*𝑟(∞, 1 − 𝜇) − Stab*
𝑟(∞, 1 − 𝜇′))
≤ 4𝜖+ 2(𝜇− 𝜇′). (2.48)
As 𝜖 > 0 was arbitrary, combining eqs. (2.47) and (2.48) gives the estimate
|Stab*𝑟(∞, 𝜇′) − Stab*
𝑟(∞, 𝜇)| ≤ 2|𝜇− 𝜇′|. (2.49)
The uniform continuity on the dense subset given by the 𝑞-adic fractions then extends
uniquely to a uniformly continuous function of all 𝜇 ∈ [0, 1] (see e.g. [100, Prob. 4.13]),
and in fact a Lipschitz continuous function with Lipschitz constant 2.
Remark 3. The constant 2 is the best possible in the estimate (2.49). For example,
consider 𝜖 = (𝑞 − 1)/𝑞 and 𝑟 the 𝑞-SC(𝜖). Then, y ⊥ x, and so Stab*𝑟(∞, 𝜇) =
𝜇2 + (1 − 𝜇)2. Differentiating at 𝜇 = 0 demonstrates the tightness of (2.49).
We also note that Stab*𝑟(∞, 𝜇) is not necessarily differentiable everywhere. Take
for instance 𝑟(𝑦|𝑥) = 1(𝑦 = 0). Then,
Stab*𝑟(∞, 𝜇) = Stab*
𝑟(𝑛, 𝜇) = max(𝜇, 1 − 𝜇).
Our goal now is to explore the role of balls versus subcubes in the context of noise
stability for the 𝑞-SC in more detail. First, we give an example where balls do better
than subcubes for high values of noise. We have the following
Proposition 6. For 𝑞 ≥ 3 and 𝑛 ≥ 𝑞2+𝑞+1, 𝒞𝑞,3 is not universally optimal for noise
stability across the family of 𝑞-SC(𝜖), 0 ≤ 𝜖 ≤ 1 − 1/𝑞. More specifically, for such
𝑛, 𝒞𝑞,3 is not optimal for noise stability in the interval ((2𝑞 + 1)/(1 + 𝑞)2, 1 − 1/𝑞).
For 𝑞 = 2, for all 𝑛 ≥ 3, 𝒞2,3 is universally optimal. For 𝑞 = 2, 𝑛 ≥ 12, 𝒞2,4 is
not universally optimal. More specifically, for such 𝑛, 𝒞2,4 is not optimal for noise
67
stability in the interval((19 −
√137)/16, 1/2
)⊃ (0.456, 0.5).
Proof of Proposition 6. As in Lemma 9, we know that the distance distribution of
𝒞𝑞,𝑘 is 𝐴 where 𝐴𝑖 =(𝑘𝑖
)(𝑞 − 1)𝑖. Thus its energy with respect to
ℎ(𝑖) = (1 − 𝜖)𝑛(
𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
is
𝐸ℎ(𝒞𝑞,𝑘) = (1 − 𝜖)𝑛𝑘∑
𝑖=1
(𝑘
𝑖
)(𝑞 − 1)𝑖
(𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
= (1 − 𝜖)𝑛((1 − 𝜖)−𝑘 − 1). (2.50)
Now suppose 𝑞 ≥ 3, and consider the Hamming ball of radius 1, denoted by
ℬ𝑞2+𝑞+1,𝑞,1. The cardinality of this ball is 1 + 𝑛(𝑞 − 1) = 𝑞3. We remark that this is
the smallest cube we could hope for by Theorem 2. The distance distribution of this
Hamming ball is 𝐵 where 𝐵0 = 1, 𝐵1 = (𝑞3 − 1)/𝑞2, 𝐵2 = (𝑞3 − 1)(𝑞2 − 1)/𝑞2 and
𝐵𝑖 = 0 for 𝑖 > 2. Thus
𝐸ℎ(ℬ𝑞2+𝑞+1,𝑞,1) = (1 − 𝜖)𝑛[
𝜖(𝑞3 − 1)
(1 − 𝜖)(𝑞 − 1)𝑞2+𝜖2(𝑞3 − 1)(𝑞2 − 1)
(1 − 𝜖)2(𝑞 − 1)2𝑞2
]= (1 − 𝜖)𝑛
[𝜖(𝑞2 + 𝑞 + 1)(𝜖𝑞 + 1)
(1 − 𝜖)2𝑞2
]. (2.51)
Thus for 𝑘 = 3, we see by eqs. (2.50) and (2.51) that 𝐸ℎ(ℬ𝑞2+𝑞+1,𝑞,1) ≥ 𝐸ℎ(𝒞𝑞,3)
precisely when(𝑞2 + 𝑞 + 1)(𝜖𝑞 + 1)(1 − 𝜖)
𝑞2≥ 𝜖2 − 3𝜖+ 3. (2.52)
Treating (2.52) as a quadratic in 𝜖, we see that the roots of this quadratic are
𝑟1 = 1 − 1
𝑞, 𝑟2 =
2𝑞 + 1
(1 + 𝑞)2.
Note that 𝑟1 is expected as it corresponds to y ⊥ x, in which case the noise stability
does not depend on the actual anticode.
68
We claim that for 𝑞 ≥ 3, 𝑟2 < 𝑟1. Cross multiplying, this reduces to showing that
for 𝑞 ≥ 3,
𝑓(𝑞) = 𝑞3 − 𝑞2 − 2𝑞 − 1 > 0.
This claim may be proved readily. For example, at 𝑞 = 3 the inequality is true as
11 > 0. Differentiating, we get 𝑓 ′(𝑞) = 3𝑞2 − 2𝑞 − 2, and the roots of 𝑓 ′(𝑞) = 0 are
(1 ±√
7)/3, which are both less than 3. This root check completes the proof of the
𝑞 ≥ 3 case, since for 𝑛 > 𝑞2 + 𝑞 + 1, we may simply take a Cartesian product with
0 on 𝑛− (𝑞2 + 𝑞 + 1) coordinates.
We now turn to 𝑞 = 2. Here we can not rely on using 𝒞2,3, since in fact it may
easily be checked by hand (using Proposition 3 to simplify the case analysis) that 𝒞2,3is universally optimal for 𝑛 ≥ 3. Hence we move to 𝒞2,4, and use an “almost-Hamming
ball” for 𝑛 = 12 and of cardinality 16. This time, we take care of the last shell by
filling it in lexicographic order. The distance distribution 𝐵 of this anticode ℬ is
given by 𝐵0 = 1, 𝐵1 = 9/4, 𝐵2 = 9, 𝐵3 = 15/4 and 𝐵𝑖 = 0 for 𝑖 > 3. Then
𝐸ℎ(ℬ) = (1 − 𝜖)𝑛[
9𝜖
4(1 − 𝜖)+
9𝜖2
(1 − 𝜖)2+
15𝜖3
4(1 − 𝜖)3
]. (2.53)
Thus by eqs. (2.50) and (2.53), we see that 𝐸ℎ(ℬ) ≥ 𝐸ℎ(𝒞2,4) precisely when
4𝜖− 6𝜖2 + 4𝜖3 − 𝜖4
(1 − 𝜖)4≤ 3𝜖(3 + 6𝜖− 4𝜖2)
4(1 − 𝜖)3.
Cross multiplying, this boils down to studying when
𝑔(𝑥) , 16𝑥3 − 46𝑥2 + 33𝑥− 7 ≥ 0.
As we already know one root 𝑠1 = 1/2, we may easily find the other roots
𝑠2, 𝑠3 =19 ±
√137
16.
The root below 1/2 is what matters for us. Once again, the derivative checks out, so
we have our desired interval. This completes the proof.
69
We remark that by no means is Proposition 6 tight in the sense of the obtained
measures. The parameters were chosen above to reflect the smallest cube cardinalities
where universal optimality does not exist. As can be seen in the proof above, it also
has the advantage of producing a low degree polynomial that can be analyzed by
hand readily. For example, the above proof yields for 𝑞 = 2 an anticode example of
measure 1/256. It turns out one can do far better:
Remark 4. Let 𝑛 = 19 and 𝑞 = 2. Then, 𝒞2,14 is not universally optimal across the
BSC-𝜖 family. More specifically, 𝒞2,14 is not optimal for 𝜖 ∈ (0.484, 0.5). Note that
the measure here is 1/32. The example is simply an “almost-Hamming ball” of the
appropriate cardinality, with the last shell filled in lexicographic order.
We note that the lack of universal optimality at measure 1/32, or more broadly
Proposition 6, is not mysterious, and may be understood more conceptually as fol-
lows. The discussion here follows closely [89, Sec 5.4], and we give a summary. The
derivative of noise stability with respect to 𝜖 at high noise 𝜖 = 1/2 is proportional to
the degree-1 Fourier weight. For a Hamming ball, as 𝑛 → ∞, the degree-1 Fourier
weight as a function of the measure is given by the square of the Gaussian isoperimet-
ric function due to the central limit theorem ( [89, Propn. 5.25]). The corresponding
quantity for subcubes is also given in [89, pg 125]. One may then numerically com-
pute the values for measure 1/32 and compare. This consideration tells us that for
some large enough 𝑛, one should be able to construct an “almost-Hamming ball” of
measure 1/32 that does better than the corresponding cube for high noise. The above
Remark 4 is simply a numerical quantification; 𝑛 = 19 was the smallest 𝑛 for which
the “almost-Hamming ball” happened to work.
We also note that 1/32 represents the largest measure where this phenonmenon
occurs; at 1/16, 1/8 one can check that subcubes do better than balls at high noise,
and for 1/4, 1/2 we have universal optimality of subcubes by Theorem 2. We suspect
that there is universal optimality for measures 1/16, 1/8, and raise the following
Conjecture 1. Let 𝑞 = 2. Then for all 𝑛 ≥ 4, 𝒞2,𝑛−3, 𝒞2,𝑛−4 are universally optimal
with respect to the cone of all negations of completely monotonic functions 𝒢 (defined
70
in Theorem 2).
We have the following numerical evidence in favor of the 1/8 case of Conjecture 1.
One may use the “order-1” SDP bounds of [102] to study this problem (we used
SDPA-GMP for this purpose, see e.g. [85]); they represent a natural generalization of
the LP bounds. These bounds do manage to certify universal optimality for the 1/8
case for 𝑛 ≤ 8, but unfortunately do not do so for 𝑛 = 9 onwards. For the 1/16 case,
there seems to be no nontrivial certificates: even 𝑛 = 7 is not certified, even though
we know that 𝒞2,3 is universally optimal (see e.g. Proposition 6). It is possible that a
higher constant order SDP bound will certify universal optimality for all 𝑛, at least
for 1/8.
We shall now use Proposition 6 to show that “universal optima are sparse for
anticoding”.
Our approach to showing this is by understanding how anticodes behave in Ham-
ming space when they are stacked. We accordingly have
Lemma 19. Let 𝑐 = 𝑘𝑞𝑙, 𝑐′ = 𝑟 be given, where 𝑐 + 𝑐′ ≤ 𝑞𝑛, 0 < 𝑐′ < 𝑞𝑙. Let
𝜇 = (𝑐+ 𝑐′)/𝑞𝑛, 𝜇′ = 𝑐′/𝑞𝑙. Let 𝒞1, 𝒞2 ⊂ F𝑙𝑞 be two anticodes of cardinality 𝑐′. Let
𝒟1 =(𝑘𝑞 × 𝒞1
),𝒟2 =
(𝑘𝑞 × 𝒞2
),
live in F𝑛𝑞 . Let ℬ = L(F𝑛
𝑞 , 𝑐). Consider the anticodes 𝒜1,𝒜2 obtained by disjoint
union
𝒜1 = ℬ ∪𝒟1,
𝒜2 = ℬ ∪𝒟2.
We say that 𝒜1,𝒜2 are obtained by stacking. Then we have
Stab𝜖(𝑛,𝒜1) − Stab𝜖(𝑛,𝒜2) =𝜇(1 − 𝜖)𝑛−𝑙
𝜇′ (Stab𝜖(𝑙, 𝒞1) − Stab𝜖(𝑙, 𝒞2)). (2.54)
Proof of Lemma 19. We observe that the only differences between the distance dis-
71
tributions of 𝒜1 and 𝒜2 come from the distances internal to 𝒟1,𝒟2. This is because
ℬ is common to both, and the interactions between 𝒟1,𝒟2 and ℬ are identical due
to the symmetric nature of the projection of ℬ onto the lower 𝑙 coordinates. As such,
letting
ℎ(𝑖) = (1 − 𝜖)𝑛(
𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
, ℎ′(𝑖) = (1 − 𝜖)𝑙(
𝜖
(𝑞 − 1)(1 − 𝜖)
)𝑖
,
we have by Lemma 14
Stab𝜖(𝑛,𝒜1) − Stab𝜖(𝑛,𝒜2) = 2𝜇(𝐸ℎ(𝒜1) − 𝐸ℎ(𝒜2))
= 2𝜇(𝐸ℎ(𝒟1) − 𝐸ℎ(𝒟2))
= 2𝜇(𝐸ℎ(𝒟1) − 𝐸ℎ(𝒟2))
= 2𝜇(1 − 𝜖)𝑛−𝑙(𝐸 ′ℎ(𝒟1) − 𝐸 ′
ℎ(𝒟2))
=𝜇(1 − 𝜖)𝑛−𝑙
𝜇′ (Stab𝜖(𝑙, 𝒞1) − Stab𝜖(𝑙, 𝒞2)).
With Lemma 19, we may deduce the lack of universal optimality at cardinality
𝑐 + 𝑐′ for F𝑛𝑞 from the lack of universal optimality at 𝑐′ for F𝑙
𝑞. Thus combining
with Proposition 6 already gives us many more cardinalities where we lack universal
optimality than the examples furnished by Proposition 6 itself. Nevertheless, we
may do much better by combining further with Lemmas 15, 18. In particular, this
combination suffices to achieve our aim here, namely that “universal optima are sparse
for anticoding”.
We have all the ingredients in place to prove Theorem 4, except for a technicality
that involves estimating the difference in noise stability between lex-sets of different
sizes. This technicality may be viewed as analogous to Lemma 18, except that this
time the anticodes at hand are nicely structured. In fact, this Lemma 20 only depends
on the nesting of two anticodes 𝒞 ⊆ 𝒞 ′.
Lemma 20. Let 𝒞 ⊆ 𝒞 ′ ⊆ F𝑛𝑞 be two anticodes of measures 𝜇, 𝜇′ respectively, and
72
consider noise stability with respect to the 𝑞-SC(𝜖). Then we have the estimate
|Stab𝜖(𝑛, 𝒞 ′) − Stab𝜖(𝑛, 𝒞)| ≤ 4(𝜇′ − 𝜇).
Proof of Lemma 20. As usual, we define ℎ(𝑖) = (1 − 𝜖)𝑛(
𝜖(𝑞−1)(1−𝜖)
)𝑖. Observe that
each codeword in 𝒞 ′ ∖𝒞 can have at most(𝑛𝑖
)(𝑞− 1)𝑖 codewords at Hamming distance
𝑖 from it. From this observation, we can bound the difference of energy with respect
to ℎ and hence the noise stability (by Lemma 14) by
Stab𝜖(𝑛, 𝒞 ′) − Stab𝜖(𝑛, 𝒞) ≤ 2(𝜇′ − 𝜇) + 2𝑞−𝑛
𝑛∑𝑖=0
ℎ(𝑖)|𝒞 ′ ∖ 𝒞|(𝑛
𝑖
)(𝑞 − 1)𝑖
= 2(𝜇′ − 𝜇) + 2(1 − 𝜖)𝑛(𝜇′ − 𝜇)𝑛∑
𝑖=0
(𝜖
1 − 𝜖
)𝑖(𝑛
𝑖
)= 4(𝜇′ − 𝜇). (2.55)
We may apply complements as in the proof of Proposition 5 to get the symmetric
version of (2.55), namely
Stab𝜖(𝑛, 𝒞 ′) − Stab𝜖(𝑛, 𝒞) = −(Stab𝜖(𝑛, 𝒞) − Stab𝜖(𝑛, 𝒞 ′))
≥ 4(𝜇− 𝜇′).
This completes the proof.
All the pieces are now in play to prove Theorem 4. We remark that all estimates
we use here are effective unlike Proposition 5 since we focus on finite but large enough
𝑛, though we prefer not to quantify the required parameters for simplicity.
Proof of Theorem 4. Proposition 6 guarantees the existence of a cube size 𝑘(𝑞), a
suitably large 𝑛0(𝑞), an anticode 𝒞(𝑞), an interval 𝐼(𝑞) = (𝜖(𝑞), 1 − 1/𝑞), and a lower
bound 𝑐(𝑞) > 0 such that for all 𝜖 ∈ 𝐼(𝑞),
Stab𝜖(𝑛0(𝑞), 𝒞(𝑞)) − Stab𝜖(𝑛0(𝑞), 𝒞𝑞,𝑘(𝑞)) ≥ 𝑐(𝑞). (2.56)
73
For 𝑛′ ≥ 𝑛0(𝑞), considering the product anticode
𝒟(𝑛′, 𝑠, 𝑞) = 𝒞(𝑞) × L(F𝑛′−𝑛0(𝑞)𝑞 , 𝑠)
for any 1 ≤ 𝑠 ≤ 𝑞𝑛′−𝑛0(𝑞), we get by Lemma 15 and (2.56),
Stab𝜖(𝑛′,𝒟(𝑛′, 𝑠, 𝑞)) − Stab𝜖(𝑛
′,L(F𝑛′𝑞 , 𝑠𝑞
𝑘(𝑞))) =
𝑐(𝑞)
2(Stab𝜖(𝑛
′ − 𝑛0(𝑞),L(F𝑛′−𝑛0(𝑞)𝑞 , 𝑠)) + 2𝑠𝑞𝑛0(𝑞)−𝑛′ − 1). (2.57)
We can lower bound the right hand side of (2.57) by Corollary 4 to get
Stab𝜖(𝑛′,𝒟(𝑛′, 𝑠, 𝑞)) − Stab𝜖(𝑛
′,L(F𝑛′𝑞 , 𝑠𝑞
𝑘(𝑞))) ≥ 𝑐(𝑞)𝜇′(𝑞, 𝑠, 𝑛′)2, (2.58)
where 𝜇′(𝑞, 𝑠, 𝑛) = 𝑠/𝑞𝑛′−𝑛0(𝑞). The goal is a uniform lower bound, so we specialize to
𝑠 an integer satisfying
𝑞𝑛′−𝑛0(𝑞)−𝑛1(𝑞) ≤ 𝑠 ≤ 𝑞𝑛
′−𝑛0(𝑞).
Letting
𝜇(𝑞) , 𝑞𝑘(𝑞)−𝑛0(𝑞), 𝜈(𝑞) = 𝑞𝑘(𝑞)−𝑛0(𝑞)−𝑛1(𝑞),
we have by the above (2.58) a uniform lower bound (call it 𝑐′(𝑞)) for measures
𝜈(𝑞), 𝜈(𝑞) + 𝑞𝑘(𝑞)−𝑛′, 𝜈(𝑞) + 2𝑞𝑘(𝑞)−𝑛′
, . . . , 𝜇(𝑞).
We now interpolate to the full range of integers in [𝜈(𝑞), 𝜇(𝑞)] by using the above
measures as anchors. We do this by using a random subanticode for the first term
on the left of (2.58) (cf. Lemma 18), and the estimate for the difference of noise
stability between lex-sets for the second term (cf. Lemma 20). As 𝜈(𝑞), 𝜇(𝑞) are fixed
and bounded away from 0, there exists an 𝑛2(𝑞) such that for any 𝑛′ ≥ 𝑛2(𝑞) and
74
cardinality 𝑟 ∈ [𝜈(𝑞)𝑞𝑛′, 𝜇(𝑞)𝑞𝑛
′], there exists a subanticode 𝒟′(𝑛′, 𝑟, 𝑞) with
|Stab𝜖(𝒟′(𝑛′, 𝑟, 𝑞)) − Stab𝜖(𝒟(𝑛′, 𝑠(𝑟), 𝑞))| < 𝑐′(𝑞)
3, (2.59a)
|Stab𝜖(𝑛′,L(F𝑛′
𝑞 , 𝑠(𝑟)𝑞𝑘(𝑞))) − Stab𝜖(𝑛
′,L(F𝑛′𝑞 , 𝑟))| <
𝑐′(𝑞)
3. (2.59b)
Here 𝑠(𝑟) denotes the largest integer of the form 𝑠𝑞𝑘(𝑞) at or below 𝑟. The esti-
mates eqs. (2.58) and (2.59) yield
Stab𝜖(𝑛′,𝒟′(𝑛′, 𝑟, 𝑞)) − Stab𝜖(𝑛
′,L(F𝑛′𝑞 , 𝑟)) >
𝑐′(𝑞)
3
∀𝑟 ∈ [𝜈(𝑞)𝑞𝑛′, 𝜇(𝑞)𝑞𝑛
′], 𝑛′ ≥ 𝑛2(𝑞), 𝜖 ∈ 𝐼(𝑞). (2.60)
We may now stack 𝒟′(𝑛′, 𝑟, 𝑞) on top of ℬ = L(F𝑛𝑞 , 𝑟
′) to get an anticode 𝒜(𝑛, 𝑟+𝑟′, 𝑞)
that does better than L(F𝑛𝑞 , 𝑟+ 𝑟′) for 𝜖 ∈ 𝐼(𝑞) as long as the conditions of Lemma 19
are met.
One simple set of sufficient conditions on the cardinality 𝑟 and size 𝑛 is the fol-
lowing. Let 𝑟 =∑𝑛−1
𝑖=0 𝑟𝑖𝑞𝑛−𝑖 be the base 𝑞 expansion of 𝑟. Suppose we ignore the last
𝑛2(𝑞) digits, and focus on (𝑟0, 𝑟1, . . . , 𝑟𝑛−𝑛2(𝑞)−1). Let 𝑎(𝑞) ≤ 𝑏(𝑞) be two integers such
that 1 + log𝑞(𝜇(𝑞)−1) ≤ 𝑎(𝑞) ≤ 𝑏(𝑞) ≤ −1 + log𝑞(𝜈(𝑞)−1). Note that this is always
possible as we had flexibility in the choice of 𝜈(𝑞), indeed we can make [𝑎(𝑞), 𝑏(𝑞)]
have arbitrarily long, but constant (independent of 𝑛) length.
Suppose that there is a run of zeros in (𝑟0, . . . , 𝑟𝑛−𝑛2(𝑞)−1) of length 𝑙 with 𝑎(𝑞) ≤
𝑙 ≤ 𝑏(𝑞). Then, there is an anticode 𝒜 ⊆ F𝑛𝑞 with |𝒜| = 𝑟 that does better than
L(F𝑛𝑞 , 𝑟) for noise stability in the range 𝐼(𝑞). In particular, for these cardinalities
we do not have universal optimality of lex-sets, which are the only candidates for
universal optimality. It is also clear that the set of such cardinalities 𝒯 is contained
in 𝒮, and that it also has measure |𝒯 |/𝑞𝑛 ≥ 1− 𝑜 (𝑐′′(𝑞)𝑛), for some 𝑐′′(𝑞) < 1, simply
because we have to avoid a finite pattern. This completes the proof.
75
2.8 Open problems
Many problems here remain open. What we find most attractive is the one we raise in
Conjecture 1, as it would complete the classification of which subcubes are universally
optimal for noise stability. One can also ask analogous questions for general 𝑞.
We also lay out a far more general version of this problem, where we ask for
characterizing the sharp value of
𝑠*𝑞(𝜇, 𝜖) , Stab*𝜖(∞, 𝜇), (𝜇, 𝜖) ∈
[0,
1
2
]×[0, 1 − 1
𝑞
].
Other 𝜇 may be obtained by complementing, and other 𝜖 no longer reflect an anti-
coding problem. We give our speculations regarding the 𝑞 = 2 case based on what
we understood from proving Theorem 4. Let us consider a dyadic 𝜇 = 0.𝑎1𝑎2 . . . 𝑎𝑘,
where the 𝑎𝑘 denote bits in a binary expansion. By the stacking construction used in
proving Theorem 4, we know that with a sufficiently long run of zeros in the 𝑎𝑖, and
large enough 𝜖, we can stack some anticode on top of a lex-set to do better than a
lex-set of the same augmented cardinality in terms of noise stability. The anticodes
we used in the proof were bootstrapped from a finite phenomenon(e.g. Proposition 6,
Remark 4), with the finite phenomenon being given by an almost-Hamming ball. It
is perhaps more natural to stack almost-Hamming balls directly, and in fact it may
be possible to prove Theorem 4 by such an approach. However, it does not seem
easy to express the noise stability of a Hamming ball with measure different from 1/2
in convenient form, with even the degree-1 Fourier weight (that describes the limit
𝜖 → 1/2) involving the Gaussian isoperimetric function, and the full noise stability
expression involving the Gaussian quadrant probability (see e.g. [89, Ex. 5.32]. On
the other hand, it is possible that the focus on 𝑛 = ∞ helps with some technical is-
sues, such as the fact that we don’t know of a closed form for the distance distribution
of a Hamming ball, even for 𝑞 = 2.
As our focus was on a finite 𝑛 statement, for the proof, we favored the above
bootstrapping approach. Nevertheless, it is possible that examining the set bits of
𝜇’s binary expansion, and using the most favorable stacking of an almost-Hamming
76
ball out of them on top of a lex-set will yield 𝑠*2(𝜇, 𝜖). We note that the relevant
distance distributions and noise stability may be computed in polynomial time (in 𝑛)
for a fixed cardinality. With numerical simulation, we were unable to come up with
any better construction of anticodes for large 𝑛. We note that our “stacking” pro-
posal here is to some extent anticipated by the reviewer comments described in [50].
Basically, the proposal of [50] was to simply use lex-sets without taking into account
the ball/subcube interplay that was pointed out by the reviewer. Thus much about
𝑠*𝑞(𝜇, 𝜖) remains open; indeed Theorem 2 characterizes just a couple of lines in the
(𝜇, 𝜖) square, while Theorem 4 shows that for a.s 𝜇, we lack universally optimal an-
ticodes.
Lastly, we find the use of Fourier analytic techniques for isodiametry intriguing,
and wish to return to this theme in future work. We also refer the reader to more
general remarks in our concluding Chapter 5 regarding the use of Fourier analytic
techniques for geometric questions.
77
78
Chapter 3
Near-Optimal Coded Apertures for
Imaging via Nazarov’s Theorem
Given a convex set 𝐵 ∈ R2 (or R𝑛 , for Bang’s solution it is all the
same), is it possible to cover it by several strips of total width less than
the width of 𝐵 ? (The width of a convex body is defined as the width of
the narrowest strip containing it).
The conjectured answer was “no”, but it took about 40 years to prove
that. The proof, when found, was . . . 2 pages long and required from the
reader only basic knowlege of elementary geometry. For reader’s
convenience it is included into this paper as an appendix.
Of course, to solve the coefficient problem as it was stated above you
cannot just apply the result (. . . ), but it turns out that a minor
modification of the proof is enough. (So minor that I actually even do
not pretend to be an author of the next two sections; rather I act there
like a shadow that enters and goes over many strange places which
completely eliminate the attention of his master just passing by).
Fedor Nazarov, 1997
79
3.1 Introduction
In Chapter 2, we saw the role that Fourier analysis on Hamming space played in
the resolution of a question in theoretical computer science. Along the way we also
developed some non Fourier-analytic machinery towards answering natural follow-up
questions, such as:
1. What happens for channels that are not BSC(𝜖)?
2. Can we completely classify the universally optimal anticodes in Hamming space?
In this chapter, we return to Fourier analysis, this time on finite cyclic groups
Z/𝑛Z, or in terms more familiar for engineers, the DFT (discrete Fourier transform).
To fix notation and our choice of normalization, let us first define for convenience
𝑒(𝑧) , 𝑒2𝜋𝑖𝑧 as is commonly done in say analytic number theory. We then define the
DFT of a length 𝑛 vector = (𝑎0, 𝑎1, . . . , 𝑎𝑛−1) to be a length 𝑛 vector 𝑎 given by
𝑎𝑘 =𝑛∑
𝑗=0
𝑗𝑒
(−𝑗𝑘𝑛
).
It is also convenient for us to write
1 , (1, 1, . . . , 1).
Some illustrative examples to make sure that we are on the same page regarding
the choice of normalization are
= (𝑛, 0, 0, . . . , 0) ⇔ 𝑎 = (𝑛, 𝑛, . . . , 𝑛),
= (1, 1, 1, . . . , 1) ⇔ 𝑏 = (𝑛, 0, . . . , 0).
With this notational clarification, let us examine a problem that arises in the
context of computational imaging. Certain modern imaging systems, especially those
operating at high frequencies, use coded apertures. In these systems, a spatial mask
that selectively blocks light from reaching the sensor is used as opposed to a traditional
80
Figure 3-1: “De Radio Astronomica et Geometrica”, Gemma Frisius, 1545
Figure 3-2: Proposal of [37]
lens. The scene is then recovered by suitable post-processing. Perhaps the earliest
and simplest instance of coded aperture imaging is the pinhole structure (c. 470-391
BCE, Mozi in China [87, pp. 97-99]); see, e.g., [126] for a survey and Figure 3-1 for
an illustration.
The development of X-ray and gamma-ray astronomy gave rise to more sophis-
ticated coded apertures [1, 37] to get around the lack of lenses and mirrors in such
settings. For the reader curious as to why it can be difficult to fabricate lenses and
mirrors for such high frequencies, it turns out that most materials end up having
a refractive index near 1 at such frequencies. This fact may be understood from
standard dipole/induced dipole and associated oscillator analysis, and an elementary
treatment may be found in e.g. [45, Ch. 32].
Both Ables and Dicke [1, 37] proposed using random blockage patterns with a
specified mean transmittance as a method to increase the aperture size as compared
to the classical pinhole while retaining its resolution benefits. Dicke’s construction
is shown in Figure 3-2. Naturally, this also requires a certain nontrivial decoding
procedure to recover the image from the superposition formed from the various copies
81
Figure 3-3: Maximum spectral magnitude of random on-off, normalized
0 2.5× 104 5.0× 104 7.5× 104 1.0× 105
0.9
1.0
1.1
1.2
1.3
1.4
𝑛
𝑀()√𝑛 log(𝑛)
across the different holes. The decoding may done (classically) in analog, or digitally
with a computer. In particular, Dicke [37, Fig 2-4] proposed various beautiful analog
decoders.
A more modern development of special importance to this chapter is the usage
of uniformly redundant arrays (URA) to improve upon random on-off patterns [44].
At a high level, the reason for their superior performance is that the DFT of such a
pattern is “spectrally flat”, that is |𝑎𝑖| is constant across all 𝑖 = 0. “Spectral flatness”
is not even close to being achieved with a random on-off pattern drawn from the i.i.d.
Bernoulli(0.5) ensemble across the 𝑛 components, as can be verified numerically by
plotting e.g. 𝑀 (𝑎) , max𝑖 =0 |𝑎𝑖| for 𝑖 = 0 versus 𝑚(𝑎) , min𝑖 =0 |𝑎𝑖| for a random on-
off pattern (a random (0, 1, 1, 0, 0, 0, . . . , 1) vector of length 𝑛). For a spectrally flat
pattern with∑
𝑖 𝑖 = Θ(𝑛), 𝑀 (𝑎),𝑚(𝑎) = Θ(√𝑛). As can be seen in Figures 3-3, 3-4,
for a random on-off pattern, this is certainly not the case.
From a historical perspective, this mathematical phenomenon was studied in great
depth by Salem and Zygmund in their paper on trigonometric sums with random
82
Figure 3-4: Minimum spectral magnitude of random on-off, normalized
0 2.5× 104 5.0× 104 7.5× 104 1.0× 105
0.00
0.05
0.10
0.15
𝑛
𝑚()√𝑛
signs [101, Ch. 4]. In particular, [101, Ch. 4] elucidates the 𝑀 (𝑎) = Θ(√𝑛 log(𝑛))
behavior observed in the plot.
Other modern developments include anti-pinhole imaging [25], as well as the com-
bining of mask and lens in order to, e.g., facilitate depth estimation [75], deblur
out-of-focus elements in an image [131], enable motion deblurring [95], and/or re-
cover 4D lightfields [119]. Even more recent work seeks to forgo lenses altogether to
decrease costs and meet physical constraints [40, 11]. Understanding coded apertures
is also relevant in non-line-of-sight applications where masks naturally occur as scene
occlusions [115, 113].
In light of the increased importance of coded apertures, prior work [125] described
a model under which they can be analyzed. This model uses far-field geometric optics
to model light propagation and a sensor model that includes thermal and shot noise
components. Our analysis [8] is performed with respect to a slight refinement of the
model of [125]. The basic element at play is an aperture, which we model as a discrete
vector of length 𝑛. Our choice of a discrete vector reflects a one-dimensional imaging
83
model, and is done purely for notational convenience and clarity. We request patience
from the reader who does not find such a restriction reasonable, since all will become
clear eventually. See in particular Sec. 3.4 for remarks on this aspect.
We denote the aperture by ∈ R𝑛, and we assume it satisfies ≥ 0 entry-wise.
Basically, the aperture entries model how much incoming power is sent onwards. For
example, a larger entry means that more light is let through at that location of the
aperture, and an entry of zero means that all light is blocked. Thus, a key notion
that both [8, 125] account for is that of transmissivity.
Definition 14. For an aperture , we define its transmissivity 𝜌(𝑎) by
𝜌(𝑎) ,1
𝑛
∑𝑖
𝑖.
Together with mutual information (MI) as a performance metric, [125] compared
the classical random on-off apertures [1, 37] of varying transmissivity (think of i.i.d.
Bernoulli(𝑝) ensemble to target a transmissivity 𝑝) to the “spectrally flat” patterns
with transmissivity 1/2 (same as the URA of [44])1. Among other things, the analysis
showed that when shot noise dominates thermal noise, randomly generated masks
with lower transmissivity than 1/2 offered greater performance compared to spectrally
flat patterns of transmissivity 1/2.
Our work here extends the work of [125] in multiple respects that may be broadly
grouped into the following three main contributions.
First, we refine the prior model of [125] by incorporating exposure time. The
effects of exposure time are illustrated well in the original proposal of [37], and ac-
cordingly the model of [44] (based off the PhD thesis [23]) incorporates it. Our model
is essentially equivalent to that of [44]. At a high level, a lengthier exposure time
clearly improves the signal to noise ratio (SNR), though the exact correspondence
between the two depends on the sources of noise (thermal vs shot noise) and their
statistics. However, this has associated costs, such as the inability to capture motion
1Technically, the transmissivity of the URA is 1/2 +𝑂 (1/𝑛). We omit this extra term in subse-quent nontechnical discourse.
84
accurately. For background on these sources of noise and how they affect reconstruc-
tion, we recommend [44] and the references therein.
Our high level goal is to design apertures that minimize the exposure time needed
for a given target reconstruction quality. There are a variety of reconstruction quality
metrics one can use, and unfortunately it is usually the case that the most percep-
tually meaningful metrics are not convenient for mathematical analysis. For exam-
ple, in the very popular context of video coding, a good state of the art metric
is “Video Multimethod Assessment Fusion” (VMAF) developed by Netflix (see e.g.
https://github.com/Netflix/vmaf and references therein), while a far more conve-
nient metric used classically and in more theoretical literature is peak signal-to-noise-
ratio (PSNR). As our focus is on mathematical development, we analyze analytically
convenient metrics, specifically mean squared error (MSE). Even that is not enough
for our purposes. For example, it is sensitive to the noise model, and for many noise
models is quite intractable. We therefore analyze linearly-constrained minimum mean
square error (LMMSE) estimation. Separate from considerations of analytical conve-
nience, we defend this choice since most practical decoders (analog/digital) for coded
apertures use linear procedures. We note that the prior model of [125] used MI under
a Gaussian noise model as a quality measure. At a high level, such a measure may
also be written in terms of the spectrum of the aperture, and our analysis is suffi-
ciently general to cover such a measure as well. Broadly and imprecisely, our methods
extend to a vast range of quality measures, as long as they can be written in terms
of the “content” of the aperture with respect to an orthonormal basis. We elaborate
upon these aspects in Sec. 3.4.
Second, we note that the classical URA based apertures have transmissivity 1/2.
Although the spectrum of such apertures is flat in magnitude, a fixed transmissivity
performs suboptimally under varying shot noise (proportional to the transmissivity)
versus thermal noise levels. As such, we describe how one can construct spectrally
flat sequences with transmissivities 1/8, 1/4 in addition to 1/2. The spectrally flat
sequences with these transmissivities 1/8, 1/4, when combined with the classical URA
work, extend the range of parameters where we have a sharp characterization of
85
optimal coded apertures in our framework. Furthermore, these sequences allow us to
obtain a tight answer to the problem of optimal coded apertures for i.i.d. scenes; see
Props. 8, 9 for precise statements. Broadly, what we mean by “tight” is in the sense
of being within a constant multiplicative factor of optimal for the exposure time.
Equivalently, one can express this as a guarantee of being within a constant number
of dB in SNR of optimality.
Third, we provide optimal (again up to a constant) coded apertures, both in 1D
as well as in 2D, applicable for any prior on the spectrum of the scene at hand.
The sense of tightness of the optimality is given precisely in Prop. 10. The priors
on the scene spectrum include (but are not limited to) the naturally occuring power
law [83](𝑓−𝛾-prior). Our aperture design naturally varies depending on the choice of
prior, and we provide a (heuristically) efficient greedy algorithm for their generation.
As in [125], we use a 1D model to simplify the exposition of concepts and results. We
emphasize that all of the results of this chapter generalize naturally to the analogous
2D model, whose discussion we defer to Sec. 3.4.
Essentially all the required mathematical results stem from a beautiful and pow-
erful theorem of Nazarov [86, p. 5] combined with classical waterfilling for spectrum
allocation. We find it remarkable here firstly for its mathematical generality and el-
egance, which among other things allows the analysis to carry over to other quality
measures of possible interest such as MI. Secondly, Nazarov’s theorem is “effective” in
the sense of leading to aperture designs that can be computed in a reasonable amount
of time by our greedy algorithm.
To the best of our knowledge, our work here represents the first detailed study of
Nazarov’s theorem in an applied context. However, we do note that [21, pp. 9-11] has
identified other applied problems for which Nazarov’s theorem provides conceptual
clarity and/or solutions.
86
3.2 Model
We first describe our model, and discuss how it differs from that in [125]. We use
the standard Poisson model of classical optics for photon counting, and emphasize its
dependence on the exposure time 𝑡. Another perspective is that of far-field, incoherent
illumination and the study of the intensity/power transfer, see e.g. [67, Chapter 7]
for more information and physical justification. The analysis of MI under Poisson
models is cumbersome, and even with mean square error (MSE) it is often unclear
how to achieve optimal MSE in practice. As such, the standard estimation process is
linear; indeed, the work of [1, 37] used correlation decoders. In fact, both [1, 37] give
beautiful analog realizations of such a decoder. Accordingly, we emphasize LMMSE.
We note that if one used a Gaussian model instead, LMMSE is the same as MSE.
Furthermore, under a Gaussian model, MSE is in turn essentially equivalent to MI in
the low SNR limit [106, 59]. LMMSE depends purely on first and second moments,
so in our mathematical study we do not emphasize the specific Poisson statistics.
Let 𝑓 denote the intensities of the unknown 1D scene of length 𝑛 of expected total
power 𝐽 . Let E[𝑓 ] = (𝐽/𝑛)1,Cov[𝑓 ] = Q. We assume Q is circulant and diagonalized
as Q = F*𝑛DF𝑛; F𝑛 is the unitary discrete Fourier transform (DFT) matrix and
D = diag(𝑑). The measurements at the imaging plane are denoted 𝑗, 𝑗 ∈ [𝑛] and
the 𝑛 × 𝑛 transfer matrix A models the aperture. We assume its entries all satisfy
0 ≤ A𝑗𝑖 ≤ 1/𝑛 to model that the light can not be redirected, and∑
𝑗 A𝑗𝑖 ≤ 1 to
model local conservation of power. An ideal, perfectly focused, lens may be treated
in this setup by A = I, as it redirects light perfectly.
We assume A𝑗𝑖 = (1/𝑛)𝑎𝑖−𝑗 (mod 𝑛) for a ≥ 0, i.e. A is circulant. Let 𝜌(𝑎) =
(1/𝑛)∑
𝑖 𝑖 be the transmissivity of the aperture. The noise component is denoted
by and its statistics are given by E[] = 0,Cov[] = (𝑡(𝑊 + 𝐽𝜌)/𝑛)I, where 𝑊,𝐽
correspond to thermal and shot noise respectively, and 𝑡 is the exposure time. With
these, our measurement model is then given by 𝑗 = 𝑡∑
𝑖A𝑗𝑖𝑓𝑖 + 𝑗, which leads to
87
the following expression for the LMMSE of estimating 𝑓 from .
𝑚(𝑛, 𝑡,𝑊, 𝐽, 𝑑, ) =𝑛−1∑𝑖=0
1
1
𝑑𝑖+ 𝑡|𝑎𝑖|2
𝑛(𝑊+𝐽𝜌())
. (3.1)
Here, 𝑎 is the DFT of .
3.2.1 Background on linear estimation
We note that (3.1) is an entirely routine derivation for a reader well versed in estima-
tion/statistical signal processing. For a general reader, we provide some background
on minimum mean squared error estimation and then derive (3.1). Let x,y be a pair
of vector valued random variables. Let their means be well-defined and denoted by
x,y respectively. The goal is to estimate x from y, and the minimum squared error is
achieved by the conditional expectation E[x|y]. As we have discussed, this expression
is often intractable, and it is often convenient to restrict the form of the estimator.
We study linear estimators here, i.e. estimators of the form x = 𝑊 (y − y) + 𝑏, and
we wish to minimize the expected squared error E[(x− x)𝑇 (x− x)].
In the development we shall assume x,y have finite second moments as well.
This assumption in turn ensures that the covariance matrices 𝐶x , E[(x − x)(x −
x)𝑇 ], 𝐶x,y , E[(x − x)(y − y)𝑇 ], 𝐶y,x , 𝐶𝑇x,y, 𝐶y , E[(y − y)(y − y)𝑇 ] are all well-
defined. We shall also assume that 𝐶y is nonsingular.
We remind the reader of the well-known “orthogonality principle” in standard
Hilbert space theory. The orthogonality principle implies that the estimation error
of the optimal estimator x − x is uncorrelated with any function 𝑔(y) of finite sec-
ond moment. For a good reference, we recommend for instance the treatment by
Luenberger [80, Chapter 3,4].
We first note that E[x] = x since the desired estimator must be unbiased (take
𝑔 = 1). We immediately get
𝑏 = x−𝑊y. (3.2)
88
Thus it remains to determine 𝑊 . Taking 𝑔(y) = y − y, we have
E[(x− x)(y − y)𝑇 ] = 0,
⇒ E[(𝑊 (y − y) − (x− x))(y − y)𝑇 ] = 0,
⇒ 𝑊𝐶y − 𝐶x,y = 0,
⇒ 𝑊 = 𝐶x,y𝐶−1y .
We may now determine the error covariance matrix
𝐶𝑒 , E[(x− x)(x− x)𝑇 ]
= E[(x− x)(𝑊 (y − y) − (x− x))𝑇 ]
= 0 − E[(x− x)(x− x)𝑇 ]
= E[((x− x) −𝑊 (y − y))(x− x)𝑇 ]
= 𝐶x −𝑊𝐶y,x
= 𝐶x − 𝐶x,y𝐶−1y 𝐶y,x, (3.3)
where on the third line we used the orthogonality principle.
Let us now specialize to the situation here, where we have a linear observation
model y = 𝐴x + z, x, z are uncorrelated, and 𝐴 is a fixed matrix. Then, we have
𝐶y,x = E[(y − y)(x− x)𝑇 ]
= E[(𝐴(x− x) + (z− z))(x− x)𝑇 ]
= 𝐴E[(x− x)(x− x)𝑇 ] + E[(z− z)(x− x)𝑇 ]
= 𝐴𝐶x. (3.4)
89
Similarly,
𝐶y = E[(y − y)(y − y)𝑇 ]
= E[(𝐴(x− x) + (z− z))(𝐴(x− x) + (z− z))𝑇 ]
= 𝐴E[(x− x)(x− x)𝑇 ]𝐴𝑇 + E[(z− z)(z− z)𝑇 ]
= 𝐴𝐶x𝐴𝑇 + 𝐶z. (3.5)
Substituting eqs. (3.4) and (3.5) into (3.3), we have the formula
𝐶𝑒 = 𝐶x − 𝐶x𝐴𝑇 (𝐴𝐶x𝐴
𝑇 + 𝐶z)−1𝐴𝐶x. (3.6)
Now the expression for the LMMSE is simply tr(𝐶𝑒). In our context here, we
may use our assumption regarding the simultaneous diagonalization of all relevant
matrices by the Fourier basis to simplify (3.6) in terms of spectral content as follows.
Let us for convenience denote 𝛾 , 𝑡/(𝑛(𝑊 + 𝜌(𝑎)𝐽)). We take (3.6) and rewrite it in
terms of eigenvalues 𝜆(𝑖) as
𝑛−1∑𝑖=0
𝜆𝐶x(𝑖)𝜆𝐶z(𝑖)
𝜆𝐶x(𝑖)𝜆𝐴𝐴𝑇 (𝑖) + 𝜆𝐶z(𝑖). (3.7)
At this stage, we recall our assumptions and notation to see that 𝜆𝐶x(𝑖) = 𝑑𝑖 (the
prior on the scene), 𝜆𝐴𝐴𝑇 (𝑖) = 𝑡2|𝑎𝑖|2/𝑛2 (observation period of 𝑡, and the formula for
eigenvalues of 𝐴𝐴𝑇 in terms of those for 𝐴), and 𝜆𝐶z(𝑖) = 𝑡2/(𝑛2𝛾) by our definition
of the notational convenience 𝛾 and the formulation of the noise model.
Plugging in the above into (3.7), we finally obtain
𝑛−1∑𝑖=0
𝑑𝑖𝑡2
𝑛2𝛾
𝑑𝑖𝑡2|𝑎𝑖|2𝑛2 + 𝑡2
𝑛2𝛾
=𝑛−1∑𝑖=0
1
1
𝑑𝑖+ 𝑡|𝑎𝑖|2
𝑛(𝑊+𝐽𝜌())
,
which is nothing but (3.1) as claimed.
90
3.2.2 Model clarifications and intuition
In general, we assume 𝑑𝑖 = (1/𝑛)𝑑(𝑖/𝑛) are 𝑛 equally spaced samples from a nonneg-
ative, bounded, continuous function 𝑑(𝑥) on [0, 1] with symmetry 𝑑(𝑥) = 𝑑(1−𝑥) and
normalized so that 𝑑(0) = 𝜃. For example, i.i.d. scenes correspond to 𝑑(𝑥) = 𝜃. We
note that our main result, Prop. 10, holds in greater generality. The above restriction
on the form of 𝑑 simply ensures correct physical scaling (invariant with respect to 𝑛)
of the variance of total scene intensity coming from an arbitrary direction.
It is instructive to compare an ideal lens to a mask with respect to (3.1), as a
function of exposure time. An ideal lens satisfies A = I, (i.e., = (𝑛, 0, . . . , 0)).
Thus 𝑎 = (𝑛, 𝑛, . . . , 𝑛). Then from (3.1), it can be readily seen that for a 𝑡 growing
arbitrarily slowly with 𝑛 (say 𝑡 = log(𝑛)), the LMMSE decays to 0 as 𝑛 → ∞. On
the other hand, the entry-wise restriction ∈ [0, 1] that holds for a mask results
in a significant reduction in ‖𝑎‖2. Due to this, in order to get an LMMSE that is
bounded away from the trivial∫𝑑(𝑥) d𝑥 (corresponding to 𝑡 = 0, 𝑛→ ∞), one needs
an exposure time that is Ω(𝑛). Of course, this is not surprising; there are strong
benefits to lenses when they are available. The need for long exposure times for
coded apertures is also a known phenomenon, consistent with the emphasis of [37] on
“hypothesis tests” between scenes as opposed to resolving full detail.
One way to interpret increased 𝑡 is that it reduces noise relative to the signal. All
our main results established in the sequel ( eqs. (3.9), (3.10) and (3.13)) show that one
can construct apertures that are guaranteed to be tight within a constant factor of 𝑡.
Under the above interpretation, what we establish rigorously is that our results are
tight to within a constant number (≈ 18.30) of dB, regardless of the scene correlation
structure given by 𝑑. This factor may be read off from 2𝑀(𝑛)2 of Prop. 10.
91
3.3 Results
The goal of optimal aperture design (mathematically, optimal ) is to minimize the
LMMSE formula subject to the scene model, denoted as
𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑) , min𝑚(𝑛, 𝑡,𝑊, 𝐽, 𝑑, ).
Let us first understand why the minimization above is a challenging problem.
Consider the even simpler problem in which the optimal transmissivity, say 𝜌0, is given
to us. Then, although ∈ [0, 1], 𝜌(𝑎) = 𝜌0 is a convex constraint, the LMMSE (3.1)
which we wish to minimize is neither convex nor quasiconvex in , since 1/(1 + 𝑐𝑥2)
as a function of 𝑥 lacks any of these behaviors.
In order to solve this problem, our general approach is as follows. First, we
use Parseval’s identity that relates time and frequency space. Under a fixed power
budget, it is easy to solve for the optimal spectrum allocation |𝑎𝑖|2 by studying the
well-behaved and convex (as a function of 𝑥) 1/(1 + 𝑐𝑥) that has a solution given
by waterfilling (3.8). Next, we are faced with the “coefficient problem” of finding a
∈ [0, 1] with given spectrum allocation. To address this, we appropriately apply a
theorem of Nazarov [86, p. 5]. An exposition of Nazarov’s work together with the
context he draws from (e.g., the geometric ideas of [16], along with the analytic ideas
of [33]) may be found in [15].
3.3.1 Lower bound
For notational ease, we let 𝛾 , 𝑡/(𝑛(𝑊 + 𝐽𝜌)) throughout.
We first derive a lower bound for LMMSE (3.1) based on waterfilling (see, e.g.,
[92, Thm 19.7]).
Lemma 21. Consider the convex program
inf∑𝑃𝑖≤𝑃,𝑃𝑖≥0
𝑛−1∑𝑖=1
11
𝑑𝑖+ 𝛾𝑃𝑖
.
92
Then the solution is given by a “water-level” 𝑇 implicitly governed by setting the
optimal 𝑃𝑖 =(𝑇 − 1
𝑑𝑖
)+/𝛾, satisfying
∑𝑛−1𝑖=1 𝑃𝑖 = 𝑃 .
Proof of Lemma 21. Let us associate Lagrange multipliers 𝜇𝑖 for the constraints 𝑃𝑖 ≥
0, and 𝜆 for the constraint∑𝑃𝑖 ≤ 𝑃 . Then we wish to minimize the Lagrangian
𝜆
((𝑛−1∑𝑖=1
𝑃𝑖
)− 𝑃
)+
𝑛−1∑𝑖=1
(1
1
𝑑𝑖+ 𝛾𝑃𝑖
− 𝜇𝑖𝑃𝑖
).
First order conditions yield
𝜆− 𝜇𝑖 =𝛾(
1
𝑑𝑖+ 𝛾𝑃𝑖
)2 , 𝜇𝑖𝑃𝑖 = 0.
Thus, for all 𝑖 where 𝑃𝑖 is not zero, 1/𝑑𝑖 +𝛾𝑃𝑖 = 𝑇 for some constant “water-level”
𝑇 , or in other words, for such 𝑖,
𝑃𝑖 =𝑇 − 1
𝑑𝑖
𝛾.
𝜆 can’t be zero for a nonzero 𝑃 , so by complementary slackness we see that∑𝑃𝑖 = 𝑃 .
This completes the proof.
Remark 5. The above convex program structure and associated “waterfilling” solution
occur in a wide variety of problems and is well known. For example, the reference
we provided [92, Thm 19.7] studies a similar problem to determine the capacity of a
parallel AWGN channel. In the AWGN context, [93] attributes the derivation to [64].
Mathematically, all that we did was replace the convex − log(1 + 𝑎𝑥) occuring in the
capacity problem by the convex 1/(1 + 𝑎𝑥) occuring here. We also note that a similar
problem arises in the context of bit allocation for quantizers (see, e.g., [52, 8.2]), who
attribute the solution (without the nonnegativity constraint) to [65].
At a high level, what Lemma 21 allows us to do is solve for the best possible
spectral allocation for coded apertures, assuming that one were able to set the levels
freely modulo Parseval’s equality relating time and frequency.
93
What we do next is observe that the lack of redirection of light for an aperture
(essentially a 𝑙∞ constraint on ) allows one to write a natural upper bound on the
𝑙2 norm of . The assumption on the lack of redirection of light turns into an upper
bound on the 𝑙2 norm of 𝑎, and in turn allows us to apply Lemma 21. We give
an elementary and short treatment below that is sufficient for our purposes as will
become clear shortly. For the reader interested in the general conceptual framework,
we recommend the study of the Fourier operator norm in the 𝑙𝑝 → 𝑙𝑞 sense for which
there is a vast amount of material. One way to approach such a study is through
tracing the historical line through the classical Hausdorff-Young inequality [127], [61],
and the sharper Babenko-Beckner inequality [12], [17].
We accordingly have
Lemma 22. Let 𝑥𝑖 ∈ [0, 1], 1 ≤ 𝑖 ≤ 𝑛. Suppose∑𝑥𝑖 = 𝑟, where 0 ≤ 𝑟 ≤ 𝑛. Then,∑
𝑥2𝑖 ≤ ⌊𝑟⌋ + (𝑟 − ⌊𝑟⌋)2.
Proof of Lemma 22. Think of a “trading mass” operation that takes a pair of distinct
indices (𝑖, 𝑗), 𝑖 = 𝑗, with 𝑥𝑖 ≤ 𝑥𝑗 without loss of generality, and replaces 𝑥𝑖 by 𝑥𝑖 − 𝜖,
𝑥𝑗 by 𝑥𝑗 + 𝜖 where 𝜖 ≥ 0. This trading mass operation preserves the sum at 𝑟, but if
𝜖 > 0, the sum of squares is strictly increased.
(𝑥𝑖 − 𝜖)2 + (𝑥𝑗 + 𝜖)2 = (𝑥2𝑖 + 𝑥2𝑗) + 2𝜖(𝜖+ 𝑥𝑗 − 𝑥𝑖)
> 𝑥2𝑖 + 𝑥2𝑗 .
In such an operation, we may chose 𝜖 as large as possible until one of the 𝑥𝑖, 𝑥𝑗 escapes
[0, 1], and we term that as the “full mass trade” acting on (𝑖, 𝑗).
Clearly any 𝑥 maximizing the sum of squares must be invariant under the “full
mass trade” across all pairs (𝑖, 𝑗), 𝑖 = 𝑗. In particular, at optimality, we can have
at most one “intermediate” 𝑥𝑖 ∈ (0, 1). These properties characterize the possible
optimal vectors sufficiently to complete the proof.
For a less ad-hoc and more systematic proof, consider the set of vectors formed
by permuting the entries of (𝑟 − ⌊𝑟⌋, 1, 1, . . . , 1, 0, 0, . . . , 0) arbitrarily. These are the
extremal vectors of the convex hull formed by these vectors, which in fact encompasses
94
the entire constraints space. For a convex maximization problem, it is sufficient to
examine the extremal vectors, and so we are once again done. One can view the first
proof as an “algorithmic” variant of the second.
Lemma 22 gives us the “power upper bound” that we need in order to employ
waterfilling (Lemma 21) and prove
Proposition 7. Let satisfy 𝜌(𝑎) = 𝜌. Then
𝑚(𝑛, 𝑡,𝑊, 𝐽, 𝑑, ) ≥ 1𝑛𝜃
+ 𝛾𝑛2𝜌2+
𝑛−1∑𝑖=1
11
𝑑𝑖+ 𝛾𝑃𝑖
. (3.8)
Here 𝑃𝑖 = (1/𝛾)(𝑇 −1/𝑑𝑖)+ and total power 𝑃 =
∑𝑛−1𝑖=1 𝑃𝑖 = 𝑛(⌊𝑛𝜌⌋+(𝑛𝜌−⌊𝑛𝜌⌋)2)−
𝑛2𝜌2. Also note 𝑃 ≤ 𝑛2𝜌(1−𝜌). We remark that (3.8) is sharp if and only if |𝑎𝑖|2 = 𝑃𝑖
for 0 < 𝑖 < 𝑛.
Proof. Recall that 𝑑0 = 𝜃/𝑛, and that 𝑎0 = 𝑛𝜌. This takes care of the first term. We
now turn to the nonzero frequencies. First, note that∑
𝑖 2𝑖 ≤ 𝑛(⌊𝑛𝜌⌋+ (𝑛𝜌−⌊𝑛𝜌⌋)2)
by Lemma 22. By Parseval’s identity, we immediately have∑
𝑖𝑎2𝑖 ≤ 𝑛(⌊𝑛𝜌⌋ + (𝑛𝜌−
⌊𝑛𝜌⌋)2). We may substitute in 𝑎0 = 𝑛𝜌 to get the expression for 𝑃 above. The
waterfilling Lemma 21 then gives the final expression.
The bound 𝑃 ≤ 𝑛2𝜌(1− 𝜌) is convenient as it allows us to get rid of the floors for
analytical study. We may readily derive it by
𝑛(⌊𝑛𝜌⌋ + (𝑛𝜌− ⌊𝑛𝜌⌋)2) − 𝑛2𝜌2 ≤ 𝑛(⌊𝑛𝜌⌋ + 𝑛𝜌− ⌊𝑛𝜌⌋) − 𝑛2𝜌2 ≤ 𝑛2𝜌(1 − 𝜌).
We used 0 ≤ 𝑥− ⌊𝑥⌋ ≤ 1 together with 𝑥2 ≤ 𝑥 on [0, 1] above.
Note that minimizing the right hand side over 𝜌 gives a lower bound on𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑).
The minimization task is trivial numerically, but in general difficult analytically. We
denote this optimal 𝜌 by 𝜌* henceforth.
95
3.3.2 Upper bound
Our goal here has been set from (3.8). Conceptually, the design issue is finding a
∈ [0, 1] with prescribed lower bounds |𝑎𝑖|2 ≥ 𝑃𝑖. In general, it is impossible to find
such a given arbitrary 𝑃𝑖 satisfying the power bound of Proposition 7. Therefore,
our lower bound (3.8) is not sharp in all settings. However, it should be noted that
sharp cases do exist. Perhaps the conceptually simplest example is the analog of (3.8)
for a lens, where our bound is sharp.
Our general approach is to back off by a factor 𝐶 and obtain a ∈ [0, 1] with
|𝑎𝑖|2 ≥ 𝑃𝑖/𝐶. What we do next is address how we can guarantee such a 𝐶 universally
across 𝑛, 𝑑. We shall move from simpler to more complex situations, and accordingly
start off with i.i.d. scenes, where for infinitely many 𝑛 one does not need the full
generality of Nazarov’s solution.
i.i.d. scenes
As clarified in our model, i.i.d. scenes correspond to 𝑑(𝑥) = 𝜃, a constant. Examining
the lower bound (3.8), we see that the waterfilling solution prescribes a 0, 1 sequence
with uniform spectrum allocation after the DC term (“spectrally flat sequences”). As
already noted in [44, 125], one can certainly construct such spectrally flat sequences
for infinitely many values of 𝑛, as long as they are at least asymptotically “unbiased”
with 𝜌 = 1/2 − 𝑜(1). The performance of the apertures corresponding to these
spectrally flat sequences meets the lower bound (as 𝑛 → ∞) as long as the optimal
𝜌* is 1/2 − 𝑜(1) for the given 𝑡,𝑊, 𝐽 .
We now describe how one constructs spectrally flat sequences with 𝜌 = 1/2−𝑜(1).
Our treatment here is brief and non-comprehensive. The construction we present here
is based on elementary number theory, and can be attributed to Gauß [51].
First, let us reframe the problem of spectral flatness of a 0, 1 sequence of length
𝑛 that we denote . Observe that |𝑎𝑖|2 occurs as the Fourier transform of the (cyclic)
autocorrelation sequence 𝑏𝑗 ,∑𝑛−1
𝑖=0 𝑎𝑖𝑎𝑖−𝑗, where we wrap indices modulo 𝑛. It thus
clearly suffices to ensure that the autocorrelation sequence is constant for 1 ≤ 𝑗 ≤
96
𝑛 − 1. One can rephrase this as asking for a subset of 𝒮 ⊆ Z/𝑛Z (corresponding
to the ones) with the property that the nonzero pair differences 𝑖 − 𝑗 corresponding
to pairs (𝑖, 𝑗) ∈ 𝒮 × 𝒮 occur equally among 1, 2, . . . , 𝑛 − 1. For reasons that are
intuitively clear from the physical situation, and will become mathematically clear
shortly, we also want a “nontrivial” transmissivity, i.e. 𝜌 ∈ (0, 1) as 𝑛 grows. Indeed,
the construction of Gauß is “unbiased” and achieves 𝜌 = 1/2 − 𝑜(1). We will first
describe Gauß’s construction below, and then show how one can generalize it and
obtain 𝜌 ∈ 1/4, 1/8.
Let 𝑛 = 𝑝 be a prime. Let us call 𝑎 ∈ Z/𝑛Z a quadratic residue if 𝑥2 = 𝑎 has a
solution in Z/𝑛Z, otherwise we call 𝑎 a quadratic nonresidue. To make sure we are
on the same page, 0, 1 are always quadratic residues.
We assume the reader is familiar with Euler’s criterion.
Lemma 23 (Euler). Let 𝑎 = 0 ∈ Z/𝑝Z, where 𝑝 is an odd prime. Then, in Z/𝑝Z,
𝑎𝑝−12 =
⎧⎪⎨⎪⎩1 if a is a quadratic residue
−1 otherwise.
Proof. A proof may be found in almost any elementary number theory book, such as
the freely available [108, Propn. 4.2.1].
Using Euler’s criterion (Lemma 23), we may now easily prove the spectral flatness
of Gauß’s construction [51, Art. 356].
Theorem 7 (Gauß). Let 𝑛 be a prime 𝑝 of the form 4𝑘 + 3. Let 𝑖 = 1 if 𝑖 is a
quadratic residue and 𝑖 = 0, 0 otherwise. Then is spectrally flat, i.e. |𝑎𝑖| is constant
for 1 ≤ 𝑖 ≤ 𝑛− 1.
Proof of Theorem 7. Taking 𝑎 = −1 in Euler’s criterion, we see that −1 is a quadratic
nonresidue. Observe also that Euler’s criterion immediately implies that the product
of two quadratic nonresidues is a quadratic residue, a nonresidue and a residue is a
nonresidue, and finally a residue and a residue is a residue.
97
Recall by our general discussion regarding autocorrelations and spectral flatness
that our task amounts to showing that 𝑟𝑎 − 𝑟𝑏 = 𝜆 has the same number of solutions
in 𝑟𝑎, 𝑟𝑏 across all 𝜆 = 0 ∈ Z/𝑛Z, where 𝑟𝑎, 𝑟𝑏 = 0 are quadratic residues.
Suppose 𝜆 is a quadratic residue. Note that 𝑟𝑎 − 𝑟𝑏 = 𝜆 ↔ 𝑟𝑎𝑟−1𝑏 − 𝜆𝑟−1
𝑏 = 1,
where inverses are taken in the multiplicative group (Z/𝑛Z)*. This observation gives
a bijection between difference pairs for 𝜆 and pairs for 1. Thus the number of solutions
is the same across all 𝜆 that are quadratic residues.
Now, suppose 𝜆 is a quadratic nonresidue. Then −𝜆 is a quadratic residue, and
we have 𝑟𝑎 − 𝑟𝑏 = 𝜆 ↔ 𝑟𝑏 − 𝑟𝑎 = −𝜆. Thus the number of solutions for quadratric
residues is the same as those for nonresidues when we combine with the preceding
paragraph. This completes the proof.
Remark 6. The astute reader may have observed that we did not actually compute
the DFT of Gauß’s quadratic residue sequence, as we did not need to determine the
phase for checking spectral flatness. By comparsion, Gauß [51, Art. 356] in fact
computed the DFT of such a sequence (with the phase information) explicitly in both
the cases 𝑝 = 4𝑘+ 1 and 𝑝 = 4𝑘+ 3. A popular approach for performing the full DFT
computation due to Dirichlet is the Poisson summation formula, see, e.g., [32, Ch.
2].
Now that we have established the existence of “unbiased” spectrally flat sequences
(at least for some 𝑛), a natural question is how good is using an “unbiased” spectrally
flat sequence when 𝜌* = 1/2? The answer is given in the following
Proposition 8. Let 𝜃,𝑊, 𝐽 be fixed and let 𝑑 = (𝜃/𝑛)1. Then for infinitely many 𝑛,
there exists a ∈ [0, 1] such that
𝑚(𝑛, 2𝑡,𝑊, 𝐽, 𝑑, ) ≤ 𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑). (3.9)
In other words, “unbiased” spectrally flat sequences are always guaranteed to
achieve optimal LMMSE at the expense of increasing the exposure time 𝑡 by a factor
of at most 2. In the sequel, we show how one can reduce this factor even further.
98
The improvement of the constant factor 2 is achieved by using spectrally flat
sequences with 𝜌 = 1/8−𝑜(1) and 𝜌 = 1/4−𝑜(1) in combination with 𝜌 = 1/2−𝑜(1),
and allows us to refine 2 to 8/7. Intuitively, a lower 𝜌 helps with scenarios where shot
noise dominates over thermal noise, and a higher 𝜌 when thermal noise dominates
shot noise. One therefore expects that having multiple constructions with different 𝜌
helps with aperture selection, tailored to the specific shot noise versus thermal noise
scenario at hand. If one had other values of 𝜌 with spectrally flat constructions,
one could possibly reduce the constant factor further, depending on the calculation
presented in the upcoming Lemma 24.
The construction of spectrally flat sequences for 𝜌 ∈ 1/4, 1/8 is based on well-
established cyclotomic number computations originating in [51] and developed further
in [38] in number theory. 𝜌 = 1/4 corresponds to quartic residues [24], and 𝜌 = 1/8
corresponds to octic residues [74]. It should be emphasized, however, that in contrast
to the case in which 𝜌 = 1/2, the existence of such sequences for infinitely many values
of 𝑛 is not guaranteed, because no single-variable quadratic taking on infinitely many
primes is known [117].
Of perhaps greater importance is the fact that the octic residue constructions
of [74] rely upon primes that come from a second order linear recurrence with rather
large coefficients, arising as the solutions of Brahmagupta-Pell equations. There is
thus a paucity of such constructions; indeed [74] gives only two such 𝑛 below 109,
namely 𝑛 = 73 and 𝑛 = 26041. On the other hand, the quartic residue constructions
are reasonably numerous, with over 150 of them available below 107. Even restricting
ourselves to the quartic residues allows us to tighten from 2 to 4/3. Summarizing all
of the above, we have
Proposition 9. Let 𝜃,𝑊, 𝐽 be fixed and let 𝑑 = (𝜃/𝑛)1. Then for some values of 𝑛
that exist even beyond, e.g., 109, there exists a ∈ [0, 1] such that:
𝑚(𝑛, (8/7)𝑡,𝑊, 𝐽, 𝑑, ) ≤ 𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑). (3.10a)
Moreover, for many (> 150 for 𝑛 < 107) values of 𝑛 that exist even beyond, e.g., 109,
99
there exists a ∈ [0, 1] such that
𝑚(𝑛, (4/3)𝑡,𝑊, 𝐽, 𝑑, ) ≤ 𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑). (3.10b)
Before turning to the proof, we have a few words to say about the constants
2, 4/3, 8/7. The astute reader may have noticed that they were obtained for 𝜌 ∈
1/2, 1/4, 1/8, and may have also noticed the 2𝑘/(2𝑘 − 1) pattern. One may thus
hope for this pattern to continue if one could (hypothetically) construct spectrally
flat sequences with 𝜌 = 1/16 as well. Unfortunately, this is not the case. In fact, the
values of these “magic” constants come from a two variable optimization that is readily
carried out on a computer, but is somewhat painful and certainly unilluminating to
do by hand. We obtain the constants in
Lemma 24. Let 𝑎 > 0, and let
𝑓𝑎(𝜌) : [0, 1] → R ,𝜌(1 − 𝜌)
𝑎+ 𝜌.
Let
𝑀(𝑎,ℬ) , max𝑥∈[0,1]
𝑓𝑎(𝑥)
max𝜌∈ℬ 𝑓𝑎(𝜌).
Then
𝑀
(𝑎,
1
2
)≤ 2,
𝑀
(𝑎,
1
2,1
4
)≤ 4
3,
𝑀
(𝑎,
1
2,1
4,1
8
)≤ 8
7.
Proof of Lemma 24. See Appendix A.
We also note that the constants are rather small, especially compared with those
arising from the general solution based on Nazarov’s theorem as will become clear
later. A high level takeaway from Lemma 24 is that having different aperture designs
with different transmissivities allows one greater design flexibility, and thus allows one
100
to tailor the choice to the relative shot/thermal noise levels. The precise numerical
factors are much less important.
One may wonder what the optimal (in the min-max sense) choice of transmissivity
is within our model. After all, the mathematics of minimizing the multiplicative factor
subject to a single choice of 𝜌 is clear. Along the lines of the proof of Lemma 24, this
is a straightforward calculus exercise. The answer turns out to be 𝜌 = 1/4, yielding a
factor of 4/3. In other words, in the notation of Lemma 24, we have for any 𝜌 ∈ [0, 1],
𝑀
(𝑎,
1
4
)≤ 4
3≤ sup
𝑎𝑀 (𝑎, 𝜌) . (3.11)
We do not recommend that the reader pays too much attention to the above (3.11).
After all, in general, we believe such questions are best left to actual physical tests
as there are a number of factors our model ignores.
We now turn to the proof of Props. 8, 9.
Proof of Props. 8, 9. Recall that we wish to use spectrally flat sequences. First, we
note that the indicator/characteristic function of the “difference sets” of [24, 74] are in
our language spectrally flat sequences. The constant factor is given by the following
single variable optimization. In view of (3.8), let 𝑓𝑎(𝜌) = (𝜌(1 − 𝜌))/(𝑎 + 𝜌) defined
on [0, 1]; 𝑎 corresponds to 𝑊/𝐽 . The numerator comes from the power bound, the
denominator from the noise penalty. Then, 𝑀(𝑎, 𝜌) = sup𝑥 𝑓𝑎(𝑥)/𝑓𝑎(𝜌) is the multi-
plicative loss factor for a fixed 𝑊/𝐽 and fixed 𝜌 ∈ 0.125, 0.25, 0.5. One may then
optimize over 𝜌, 𝑎 to get the constant (3.10a) via Lemma 24. This proof, modified
to 𝜌 ∈ 0.25, 0.5 and 𝜌 ∈ 0.5, also yields (3.10b) and (3.9) respectively, again
by the computations of Lemma 24. The fact that there are infinitely many 𝑛 for
𝜌 = 0.5 − 𝑜(1) follows from the quadratic residue construction together with the well
known fact that there are infinitely many primes 𝑝 = 4𝑘+3 (see, e.g., [10, Ch. 7]).
Correlated scenes
We now turn to correlated scenes. Here the waterfilling is nontrivial, and prescribes an
unequal spectrum allocation. We therefore invoke Nazarov’s solution to the coefficient
101
problem [86, p. 5], and also provide a statement here specialized to the DFT and 𝑙∞
that we use.
Nazarov’s theorem [86, p. 5] is presented in a very elegant and concise manner.
We do not know of any nontrivial improvements to it, either in exposition or in power.
The proof is sufficiently short that we present it here. We caution the reader that
although short, this proof is certainly nontrivial and can be a bit mysterious. In fact,
a lot of Nazarov’s original paper [86] is devoted to “unravelling” the mysterious steps,
as alluded to in the epigraph.
Consider the problem stated in the epigraph, which is Tarski’s famous plank prob-
lem 2. A strip is a region enclosed between two parallel hyperplanes. The width of
a convex body is defined as the width of the narrowest strip containing it. Tarski
asked whether given a convex set 𝐵 ∈ R𝑛, is it possible to cover it by several strips
of total width less than the width of 𝐵? The answer is no, but it was surprisingly
difficult to prove. Nevertheless, Bang’s solution [16] is very short (2 pages) and com-
pletely elementary! Nazarov includes Bang’s solution in his paper [86] for the reader’s
convenience.
Let us think about the nature of the plank problem and why one might see some
links to the coefficient problem. A strip centered at the origin may be written as 𝑥 :
|⟨𝑥, 𝜓⟩| ≤ 𝑎, where 𝜓 is a unit vector. Thus a point 𝑦 in some convex set 𝐵 that isn’t
covered by a collection of strips with unit normals 𝜓1, . . . , 𝜓𝑛 and coefficients 𝑎1, . . . , 𝑎𝑛
must have large coefficients with respect to all these vectors simultaneously. As such,
it is not unreasonable to expect that the methods of Bang [16] that understood
(and in fact explicitly constructed) points 𝑦 that are not covered by the strips could
potentially be used to construct vectors that have large coefficients with respect to an
orthonormal basis. The problem of constructing such vectors with large coefficients
ties in well with the precise design problem we face here based on the belief that
waterfilling should guide the spectral allocation.
It is one thing to spot the above heuristic link between covering a set by strips and
2The problem apparently first appeared in print in 1932. See Bang’s original paper [16] for areference and historical remarks.
102
the coefficient problem, a nontrivial ask in of itself, and another to actually solidify
the link. Nazarov succeeded in [86], and we present a fruit of his labor in
Theorem 8 (Nazarov). Let 𝑇 be a measure space with probability measure 𝜇. Let
𝜓𝑗 : 𝑇 → R be an at most countable system of functions satisfying
∑
𝑗
𝑐𝑗𝜓𝑗
2
≤
(∑𝑗
𝑐2𝑗
) 12
,
for any 𝑐𝑗 ∈ R. Suppose 2 ≤ 𝑝 ≤ ∞, and let 𝑞 be the conjugate exponent to 𝑝, i.e.,
𝑝−1 + 𝑞−1 = 1. Assume
∀𝑗, |𝜓𝑗|𝑞 ≥ 𝛽 > 0.
Let 0 ≤ 𝑝𝑖 satisfy∑
𝑖 𝑝𝑖 = 1. Then there exists 𝑏 ∈ 𝑙𝑝(𝑇 ) with
|𝑏|𝑝 ≤(
3𝜋
2
)1− 2𝑝
𝛽−2,
such that
∀𝑗, |⟨𝑏, 𝜓𝑗⟩| ≥√𝑝𝑗.
Here, inner products are defined with respect to the probability measure 𝜇 by
⟨𝑓, 𝑔⟩ ,∫𝑇
𝑓𝑔𝑑𝜇.
Proof of Theorem 8. Let 𝜖 ∈ (±1,±1, . . . ) 3, and define
𝑓𝜖 ,∑𝑗
𝜖𝑗√𝑝𝑗𝜓𝑗.
Define Φ(𝑥) via
Φ′′(𝑥) = (1 + 𝑥2)2𝑝−1,Φ(0) = Φ′(0) = 0.
Such a function exists and is uniquely determined by the existence and uniqueness3Nazarov calls 𝜖 a “sign cortège”.
103
theorem for differential equations.
Now the integral 𝐼(𝑓) ,∫𝑇
Φ(𝑓)𝑑𝜇 is well defined and continuous in 𝑙2(𝑇 ). Since
the family 𝑓𝜖 is compact in the topology of 𝑙2(𝑇 ), one can find a cortège 𝜖* such
that 𝑓𝜖* maximizes 𝐼(𝑓) over all 𝑓𝜖.
Now consider the cortège obtained by “flipping” a single sign, and accordingly
define “bit-flipped” 𝑓𝑗 = 𝑓𝜖* − 2𝜖*𝑗√𝑝𝑗𝜓𝑗. By the mean value theorem and our choice
of 𝜖*, we have
0 ≤∫𝑇
(Φ(𝑓𝜖*) − Φ(𝑓𝑗))𝑑𝜇
=
∫𝑇
Φ′(𝑓𝜖*)(𝑓𝜖* − 𝑓𝑗)𝑑𝜇+ (1/2)
∫𝑇
Φ′′(𝑔)(𝑓𝜖* − 𝑓𝑗)2𝑑𝜇,
where 𝑔 lies between 𝑓, 𝑓𝑗 pointwise. Recalling the definition of 𝑓, 𝑓𝑗, we obtain∫𝑇
Φ′(𝑓𝜖*)𝜓𝑗𝑑𝜇
≥ √
𝑝𝑗
∫𝑇
Φ′′(𝑔)𝜓2𝑗𝑑𝜇. (3.12)
At this stage, the path forward is as follows. We will use a (possibly scaled)
version of Φ′(𝑓𝜖*) as the desired 𝑏. In order to do so, we will accomplish two tasks:
1. Give a uniform lower bound on∫𝑇
Φ′′(𝑔)𝜓2𝑗𝑑𝜇. We claim that 𝛽23
2𝑝−1 works.
2. Show that Φ′(𝑓𝜖*) ∈ 𝑙𝑝(𝑇 ). We claim that |Φ′(𝑓𝜖*)|𝑝 ≤(𝜋2
)1− 2𝑝 .
Let us look at the first task. Here, recall that Φ′′(𝑥) = (1 +𝑥2)2𝑝−1, so by Hölder’s
inequality 4, we have
(∫𝑇
Φ′′(𝑔)𝜓2𝑗𝑑𝜇
) 𝑞2(∫
𝑇
(1 + 𝑔2)𝑑𝜇
)1− 𝑞2
≥(∫
𝑇
|𝜓𝑗|𝑞𝑑𝜇).
But∫𝑇
(1 + 𝑔2)𝑑𝜇 ≤∫𝑇
(1 + 𝑓 2𝜖* + 𝑓 2
𝑗 )𝑑𝜇 = 3, so∫𝑇
Φ′′(𝑔)𝜓2𝑗𝑑𝜇 ≥ 𝛽23
2𝑝−1.
4Historical note: Although commonly called Hölder’s inequality, Rogers discovered it in 1888 [99]before Hölder in 1889 [63]. Henceforth we follow the common convention.
104
Now for the second task. We give an upper bound on |Φ′(𝑥)| by Hölder’s inequality.
|Φ′(𝑥)| =
∫ |𝑥|
0
(1 + 𝑠2)2𝑝−1𝑑𝑠 ≤
(∫ |𝑥|
0
𝑑𝑠
) 2𝑝(∫ |𝑥|
0
(1 + 𝑠2)−1𝑑𝑠
)1− 2𝑝
≤(𝜋
2
)1− 2𝑝 |𝑥|
2𝑝 .
Thus
|Φ′(𝑓𝜖)|𝑝 ≤(𝜋
2
)1− 2𝑝,
completing the second task.
Take 𝑏 = 31− 2𝑝𝛽−2Φ′(𝑓𝜖*) to complete the proof.
Remark 7. We note that the constant 3𝜋/2 for 𝑝 = ∞ is not sharp and may be
improved. A cheap way of doing this is studying the family of functions Φ𝑏(𝑥) governed
by Φ′′𝑏 (𝑥) = 1/(1 + 𝑏𝑥2) and repeating Nazarov’s argument. It turns out that 𝑏 = 1/2
optimizes the constant in this family, and yields the very modest improvement of
3𝜋/2 →√
2𝜋.
We return to our question of coded aperture design, the associated Fourier analysis
on Z/𝑛Z, and specialize Nazarov’s theorem 8 to such a situation. First, let us define
inner products with respect to the uniform probability distribution on 0, 1, . . . , 𝑛−1.
Let 0 ≤ 𝑖, 𝑗 ≤ 𝑛−1, and let 𝜓𝑗 be a orthonormal basis for the DFT on real sequences.
Explicitly, let ℎ = ⌈(𝑛 − 1)/2⌉, 𝜔 = 2𝜋/𝑛. Let 𝜓0(𝑖) = 1, 𝜓𝑗(𝑖) =√
2 cos(𝜔𝑗𝑖) for
0 < 𝑗 < ℎ, 𝜓𝑗(𝑖) =√
2 sin(𝜔𝑗𝑖) for ℎ < 𝑗 < 𝑛. If 𝑛 is even, let 𝜓ℎ(𝑖) = cos(𝜔ℎ𝑖),
otherwise 𝜓ℎ(𝑖) =√
2 cos(𝜔ℎ𝑖). Finally, let 𝛽(𝑛) = min𝑗 |𝜓𝑗|1.
Corollary 7 (Nazarov). Let 𝑀(𝑛) = ((3𝜋)/2)𝛽(𝑛)−2. Let 0 ≤ 𝑝0, 𝑝1, . . . , 𝑝𝑛−1 be
such that∑𝑝𝑗 = 1. Then there exists a ∈ [−𝑀(𝑛),𝑀(𝑛)] with |⟨𝑏, 𝜓𝑗⟩|2 ≥ 𝑝𝑗 for
all 0 ≤ 𝑗 ≤ 𝑛− 1.
Proof of Corollary 7. Take 𝑝 = ∞ and use 𝜓𝑖 as an orthonormal basis in Nazarov’s
Theorem 8.
With Corollary 7 in hand, we are able to reach a far more general version of
Prop. 8, 9 valid for any 𝑛 and any scene prior 𝑑. Also, in Sec. 3.3.3 we show how to
105
construct sequences that achieve our goal of being guaranteed to lie within a constant
(independent of 𝑛, 𝑑) factor of optimal sequences. At the moment, we know that their
existence is guaranteed by Corollary 7.
Proposition 10. For all 𝑛, 𝑡,𝑊, 𝐽, 𝑑, there exists a ∈ [0, 1] such that
𝑚(𝑛, 2𝑀(𝑛)2𝑡,𝑊, 𝐽, 𝑑, ) ≤ 𝑚*(𝑛, 𝑡,𝑊, 𝐽, 𝑑). (3.13)
Furthermore, we have
𝑀(𝑛) ∈ [(3𝜋3)/16 + 𝑜(1), 3𝜋 + 𝑜(1)]. (3.14)
The justification of the tightness of (3.13) lies in establishing (3.14), which we do
first. The phenomenon is captured by the factorization of 𝑛, with the best, that is
the largest, 𝛽 occurring for 𝑛 prime, and the worst occuring for 𝑛 divisible by 4.
In order to elucidate this behavior, we first establish the following
Lemma 25. Let 𝑛 ≥ 4 be a natural number, and let 𝜔 , 2𝜋/𝑛. Consider 𝐴𝑛, 𝐵𝑛
defined via
𝐴𝑛 ,1
𝑛
𝑛−1∑𝑘=0
| cos(𝜔𝑘)|
𝐵𝑛 ,1
𝑛
𝑛−1∑𝑘=0
| sin(𝜔𝑘)|.
Then, for 𝑛 ≥ 4,
𝐴𝑛, 𝐵𝑛 ≥ 1
2,
𝐴𝑛, 𝐵𝑛 =2
𝜋+𝑂
(1
𝑛
).
Furthemore, 1/2 is attained at 𝑛 = 4 only.
106
Proof of Lemma 25. We have the well known triangle inequality written as
||𝑥| − |𝑦|| ≤ |𝑥− 𝑦|.
Thus, we have
|| sin(𝑥)| − | sin(𝑦)|| ≤ | sin(𝑥) − sin(𝑦)| ≤(
sup𝑥
| sin′(𝑥)|)|𝑥− 𝑦| = |𝑥− 𝑦|.
Similarly, we get || cos(𝑥)| − | cos(𝑦)|| ≤ |𝑥 − 𝑦|. As such, by classical results on the
comparison of a (left) Riemann sum with the definite integral (see e.g. [100, Ch. 6]),
we get 𝐴𝑛 −
∫ 1
0
| cos(2𝜋𝑥)|𝑑𝑥≤ 2𝜋
𝑛,
𝐵𝑛 −∫ 1
0
| sin(2𝜋𝑥)|𝑑𝑥≤ 2𝜋
𝑛.
The conceptual part of the proof is completed by the above discussion. The rest
depends on the following exercise, that we recommend performing on a computer:
compute 𝐴𝑛, 𝐵𝑛 up to say 𝑛 = 10000, and check the minimum, which happens to
occur at 𝑛 = 4. For 𝑛 > 10000, 𝐴𝑛, 𝐵𝑛 are certainly within say 0.01 of the asymptotic
2/𝜋 ≈ 0.6366 established above.
Remark 8. Our published work [8] had some remarks regarding Euler-Maclaurin
summation in the proof sketch given there. The point we were emphasizing there
is that Lemma 25 and similar such statements are entirely routine matters that fall
under the general umbrella of Euler-Maclaurin summation, which provides an asymp-
totic expansion for the difference between a sum and an integral in terms of higher
derivatives evaluated at the endpoints. However, the function | cos(𝑥)| has annoying
discontinuities in the first derivative. Nevertheless, such matters are also entirely
routine and can be handled in numerous ways. Our work [8] gave an ad-hoc one via
smoothing out the discontinuities with splines.
We came up with the much clearer and simpler approach via the triangle inequal-
107
ity (or more generally the “modulus of continuity”) after publication. Nevertheless,
we still believe that Euler-Maclaurin summation is well worth understanding for the
following reasons.
1. The reader may justifiably view our use of the triangle inequality as a mere
trick that allowed us to get what we need and may thus yearn for a conceptual
framework. Euler-Maclaurin summation provides one route.
2. The reader may wish to obtain fine grained control on the error terms that
we simply lumped into the 𝑂(1/𝑛) of Lemma 25. Euler-Maclaurin gives a full
asymptotic expansion.
We therefore recommend [56, Sec. 9.5] for an easy to read treatment suitable for a wide
audience, and/or a blog post of Tao [111] for an exposition at a more sophisticated
level together with applications to number theory.
With the routine Lemma 25 in hand, we have the following Lemma which estab-
lishes (3.14).
Lemma 26. Let 𝛽(𝑛) denote the 𝑙1 lower bound of Nazarov’s theorem 8, specialized
to the real orthonormal basis 𝜓𝑗 for the DFT defined earlier. Then,
𝛽(𝑛) ∈
[1√2
+ 𝑜(1),2√
2
𝜋+ 𝑜(1)
]
as 𝑛→ ∞. Moreover, if we restrict to 𝑛 being prime,
𝛽(𝑛) =2√
2
𝜋+ 𝑜(1).
Proof of Lemma 26. Let us first examine the case in which 𝑛 = 𝑝, where 𝑝 is a prime.
Then, for any 𝑗 = 0, 𝑗𝑘 sweeps over 0, 1, . . . , 𝑝 − 1, modulo 𝑝 as 𝑘 sweeps over
0, 1 . . . , 𝑝− 1. Thus, we are examining the Riemann sum approximation
1
𝑝
𝑝−1∑𝑘=0
cos
(2𝜋𝑘
𝑝
)
108
to∫ 1
0| cos(2𝜋𝑥)|𝑑𝑥 = 2/𝜋. The 𝑙2 norm of cos(2𝜋𝑥) on [0, 1] is 1/
√2, and we may
invoke Lemma 25 to complete the proof.
The composite case is slightly more involved, as it needs to take into account the
divisor structure of 𝑛, which prevents such symmetry of the cosine vectors. What we
mean by this is the following. Consider the basic term 𝑗𝑘/𝑛, where 𝑗 is fixed, and 𝑘
varies over 0, 1, . . . , 𝑛−1. We may divide by the greatest common divisor (gcd) (𝑗, 𝑛)
to equivalently study 𝑗′𝑘/𝑛′, where 𝑗′, 𝑛′ are coprime ((𝑗′, 𝑛′) = 1). Now as 𝑘 varies
over 0, 1, . . . , 𝑛−1, 𝑘 varies over 0, 1, . . . , 𝑛′−1, modulo 𝑛′, with each residue occuring
the same number of times. As 𝑗′, 𝑛′ are coprime, 𝑗′𝑘 also varies over 0, 1, . . . , 𝑛′ − 1,
modulo 𝑛′, with each residue occuring the same number of times. Thus, one is simply
looking at a Riemann sum approximation to an integral number (𝑛/𝑛′) of periods of
| cos(2𝜋𝑥)|. Thus we may once again invoke Lemma 25 to complete the proof: the
role of 𝑛 is replaced by that of its possible divisors. In particular, the “worst” possible
divisor of 4 from Lemma 25 results in the “worst” 𝛽 for 𝑛 divisible by 4.
We emphasize that by Lemma 26 𝑀(𝑛) ≤ 𝐶 for some constant 𝐶 ≈ 9.4248, with
even better values available at, e.g., prime 𝑛 > 100. There, 𝐶 ≈ 5.8146 suffices.
Proof of Prop. 10. Corollary 7, with 𝑝0 = 0 and 𝑝𝑗 = 𝑃𝑗/∑
𝑗 𝑃𝑗 for 0 < 𝑗 < 𝑛 yields
a with |𝑏|∞ ≤ 𝑀(𝑛) and |⟨𝑏, 𝜓𝑗⟩|2 ≥ 𝑝𝑗 for 0 < 𝑗 < 𝑛. Without loss, we may
assume that ⟨𝑏, 𝜓0⟩ ≤ 0, else simply flip signs. The fact that we have the appropriate
frequency magnitudes for 𝜓𝑗 translates to an equivalent statement for the complex
exponentials by considering the real and imaginary parts separately. Recalling the
upper bound 𝑃 ≤ 𝑛2𝜌(1 − 𝜌), we obtain |𝑏𝑗|2 ≥ 𝑃𝑗/(𝜌(1 − 𝜌)). Now consider =
(𝑏+𝑀(𝑛))/2𝑀(𝑛). Then, ∈ [0, 1], 𝜌(𝑎) ≤ 0.5, and |𝑎𝑗|2 ≥ 𝑃𝑗/(4𝑀(𝑛)2𝜌(1− 𝜌)) for
0 < 𝑗 < 𝑛. We are now in a similar situation to that of Prop. 8, except with an extra
𝑀(𝑛)2 factor, and the fact that 𝜌(𝑎) ≤ 0.5 instead of 𝜌(𝑎) = 0.5 + 𝑜(1). The latter is
no problem, as lower 𝜌 only helps us with the shot noise term, and the former simply
multiplies the 2 of (3.9) by 𝑀(𝑛)2.
109
3.3.3 Greedy algorithm
Here we propose a (heuristically) efficient algorithm to construct vectors that satisfy
the conditions of Prop. 10. This algorithm has its roots in Nazarov’s original proof.
Recall that at a high level, Nazarov’s theoretical construction boils down to finding
a “sign cortège” [86, p. 6] that is globally optimal for a certain real-valued Boolean
function of 𝑛 signs, taking exponential time in the worst case. However, notice that in
the proof of Theorem 8, we emphasized the word “maximizes”. A closer examination
of the proof of Theorem 8 reveals that one simply needs a sign cortège that is locally
optimal in the sense of Hamming geometry for the proof to work! In less explicit terms,
one can simply replace the emphasized “maximizes” with “locally maximizes” and
the proof still goes through.
Our observation suggests a natural greedy algorithm where one starts with a ran-
dom cortège, and then flips one sign at a time if it improves the objective, repeating
until no further improvement is possible. We do not know of any theoretical justifi-
cation as to why this is a good algorithm: for instance, a function on the hypercube
may have only one local optimum.
Nevertheless, in our simulations 5 this runs very fast, and empirically takes time
cubic in 𝑛. For example, on our standard laptop, we can generate apertures for
𝑛 = 2000 in 4 seconds. Our situation superficially resembles the situation of the
simplex algorithm and the smoothed analysis of [105], or more directly recent work on
max-cut [9]. Direct application of the methods of [9] to obtain theoretical guarantees
runs into difficulties with the nonlinear change in objective with a single bit flip in
our setting, unlike the linear change for max-cut. As such, we defer theoretical study
of the greedy algorithm given here to future work.
3.3.4 Simulations
We give a simple illustration in Fig. 3-5 which confirms the following intuition based
on our main results eqs. (3.9), (3.10) and (3.13). With an i.i.d. scene prior, one
5Code:https://github.com/gajjanag/apertures
110
would prefer using the spectrally flat construction as opposed to the one coming
from Nazarov’s theorem due to the smaller constant. On the other hand, with a
strong prior—e.g., a bandlimited one—the waterfilling becomes highly skewed, and
one would favor the one coming from Nazarov’s theorem as it takes into account such
strong skewing of the desired spectrum and accordingly utilizes the spectrum better.
For completeness, we also include the performance of a random on-off sequence with
density 𝜌 [125], where 𝜌 is optimized over [0, 1] for each 𝑡.
In Fig. 3-5, we note that the Nazarov and lower bound plots are within a constant
distance of each other even as 𝑡 grows. The constant distance arises from our notion
of “near-optimality”, namely that the performance is within a constant multiplicative
factor of being optimal. That multiplicative factor translates to a constant on a
logarithmic scale. For the spectrally flat case, the plot does not diverge from the
lower bound as 𝑡 grows, though the gap between the two depends on the choice of
prior, and can be arbitrarily large given certain priors unlike the Nazarov plot. We also
note that the optimal random on-off (where 𝜌 is optimized for each value of 𝑡) diverges
from the lower bound. We may heuristically justify the phenomenon as follows. We
may assume that asymptotically, |𝑎𝑖|2 behaves like a 𝜒-squared random variable of
two degrees of freedom, and plug in its density into the LMMSE expression (3.1).
Upon performing the computation, this divergence ultimately comes from the fact
that ∫ ∞
0
𝑎𝑒−𝑥𝑑𝑥
1 + 𝑎𝑥= Θ(log(𝑎)), 𝑎→ ∞.
3.4 Discussion and Future Work
Our refined analysis of a model drawing heavily from [125] yields tight conclusions
across all scene correlation patterns and noise regimes, with sharp conclusions avail-
able in some specific scenarios. Moreover, we give heuristically efficient algorithms for
the generation of optimal coded apertures. We also note that similar conclusions to
our main results eqs. (3.9), (3.10) and (3.13) also hold for MI and Gaussian statistics
of [125], simply because of the form of the expression for MI. Basically, MI is another
111
functional that may be written in the form
𝑛−1∑𝑖=0
𝑓(|𝑎𝑖|),where 𝑓 is concave.
Nazarov’s theorem is sufficiently general to allow the analysis of such functionals
to a similar extent to what we did for LMMSE. Namely, although we can’t necessarily
give sharp answers (in our view, a difficult problem!), we can give answers that are
effectively constructible and guaranteed to be good in the sense of being a constant
factor away from optimal. Naturally, the precise sense of this, and the exact constant
factor, will depend on the choice of 𝑓 . We leave such exercises to the interested reader,
who may have his/her own favorite 𝑓 .
Furthermore, we note that our conclusions generalize naturally to 2D apertures,
and in particular we have a tight characterization of optimal coded apertures in that
setting. Concretely, one simply needs to take 𝛽(𝑛)2 as opposed to 𝛽(𝑛) due to the
squaring of the 𝑙1 lower bound for the 2D DFT. The rest of the analysis of Theorem 8
and Prop. 10 carries over naturally, with the orthogonal basis provided by products
𝜓𝑗 ⊗ 𝜓𝑘. We emphasize that this works regardless of the scene prior, even ones
which are not separable. With an i.i.d. prior, separable apertures are optimal up to
constants as in 1D, and in fact taking a product of spectrally flat apertures yields
natural analogs of Props. 8, 9. However, with other priors, it seems like one needs
the generality provided by Theorem 8. For separable priors, one can simply use a
product of apertures arising from the specialization of Nazarov’s theorem to the 1D
DFT that we described in this chapter. For general priors, one can repeat the analysis
of this chapter, applied to the 2D DFT instead with basis 𝜓𝑗 ⊗ 𝜓𝑘. Our work thus
also answers the question of 2D apertures raised in [125]. We also view experimental
verification of these ideas as a worthwhile task.
As noted in [125], [119] raises the question of whether continuous-valued masks
perform better than binary-valued ones. Our work sheds some light on this: the
solution of Nazarov which we have shown is tight does seem to use the flexibility of
112
the 𝑙∞ norm in an essential way; see, e.g., [58, p. 12] for more on this. And more
specifically, we have numerical evidence for finite 𝑛; to give a concrete example, for 𝑛 =
13, the mask [1, 0, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0] has optimal LMMSE for an i.i.d. scene over
binary-valued masks for 𝜌 = 6/13, 𝜃 = 0.01,𝑊 = 𝐽 = 0.001, 𝑡 = 130, but is improved
upon by the continuous-valued mask whose first entry is equal to 𝜖 and whose 𝑖th
entry is equal to 1 − 𝜖/6 if 𝑖 − 1 is a quadratic residue modulo 13, and 0 otherwise,
for 0.26 ≤ 𝜖 ≤ 0.34. We do view a full resolution of this question to be of significant
mathematical as well as engineering interest. The engineering interest stems from
our belief that partial occluders may be more difficult to synthesize than the classical
on-off apertures. The surrounding mathematical landscape is rich, and a touchstone
is perhaps provided by the recently resolved (by [13]) “Littlewood’s flatness” problem.
First raised by Erdős [41, Prob. 26] and later extended and popularized by Littlewood
in several of his papers such as [77] as well as his book [78], Littlewood’s problem is
very simple to state. In the original form proposed by Erdős [41, Prob. 26], we have
Theorem 9. There exists, for each 𝑛, a polynomial
𝑓𝑛(𝑧) =𝑛∑
𝑘=1
𝜖𝑛,𝑘𝑧𝑘 (𝜖𝑛,𝑘 = ±1),
such that, for all 𝜃, 𝑐1√𝑛 < |𝑓𝑛(𝑒𝑖𝜃)| < 𝑐2
√𝑛, where 𝑐1, 𝑐2 are positive constants
independent of 𝜃, 𝑛.
Note that what we ask here, for the spectrally flat case, is a weaker variant of the
above Theorem 9, where we restrict 𝜃 to 2𝜋𝑘/𝑛 with 𝑘 = 0, 1, . . . , 𝑛 − 1. However,
analogous questions can be asked in the spirit of Nazarov’s solution to the coefficient
problem, where one still restricts 𝜃 to 2𝜋𝑘/𝑛, but asks for a “shaped” magnitude
response across the unit circle. We also think that an efficient numerical procedure
(at least heuristically), both in the context of Littlewood flatness as well as a “shaped”
magnitude response in the spirit of Nazarov’s solution is an interesting challenge.
We note that our weaker variant of the flatness problem has some aspects that
may be understood without too much investment. For example, allowing the 𝜖𝑛,𝑘
to lie on the unit circle allows for a trivial solution to the weak variant above for
113
𝑛 = 𝑝 prime via the Gauß sum. One can also work with the standard FFT recursion
(Cooley/Tukey/Gauß) and construct solutions inductively to the weak variant for
𝑛 = 2𝑘 with 𝜖𝑛,𝑘 ∈ ±1,±𝑖. Naturally, the full resolution by [13] is much more
involved and we do not describe it here.
We also note a potentially interesting approach towards understanding the limits
to which the coefficient problem can be solved with binary vectors that draws a con-
nection with our work in the previous chapter, and linear programming bounds (2.12)
more specifically. Briefly, a 0, 1 vector of length 2𝑑 can be thought of as a code in
the Hamming space 0, 1𝑑. As the discussion in Remark 1 makes clear, the dual dis-
tance distribution of this code is nothing but the (binary) Fourier squared magnitude,
grouped by Hamming weight. Thus, constraints on the distance/dual distance distri-
bution translate into constraints on the (binary) Fourier coefficients. One may object
that the characters are not the same as those for Z/𝑛Z, however we emphasize that
Nazarov’s solution works for both and thus view this as a reasonable toy problem.
Thus, is it possible that the linear programming bounds yield something interesting
for the coefficient problem on 0, 12𝑑? A direct use does not, simply because we
have, in the language of the previous chapter, the following
Lemma 27. For any 𝑛, 𝑞 and any 𝑐𝑖 ≥ 0 for 1 ≤ 𝑖 ≤ 𝑛 with∑
𝑖 𝑐𝑖 = 1, (1, 𝑐1, 𝑐2, . . . , 𝑐𝑛)
is a valid quasicode.
Proof of Lemma 27. Follows immediately from positive definiteness of Krawtchouk
polynomials 𝐾𝑗(0) ≥ |𝐾𝑗(𝑖)|.
There are thus numerous possibilities that are perhaps worth exploring for future
research. We outline a few here.
1. Is it possible to “shape” the magnitudes of the binary Fourier coefficients arbi-
trarily in the sense of a multiplicative constant a la Nazarov once we “coalesce”
them by Hamming weight? We believe the answer is yes, but do not have a
construction at present.
2. It is also possible that the LP bounds are just not enough to give useful in-
formation about the coefficient problem. This lack of information from the LP
114
bounds is a mystery for us and we believe this phenomenon worth clarifying
further. For example, what happens with higher order SDP bounds and the
coefficient problem?
3. Can we use answers to the above to get useful information for other orthonormal
bases, such as Fourier on Z/2𝑑Z?
Although Prop. 10 shows universal tightness across all priors, even “extreme” ones
like bandlimited ones, the constant is worse than that for a spectrally flat construction
for i.i.d. scenes. The better performance of spectrally flat constructions over the ones
inspired by Nazarov’s theorem (in certain regimes) seems to extend to other “natural”
priors like the 𝑓−𝛾 one, as the waterfilling still yields something that is nearly “flat”.
It might be interesting to quantify and understand the “flatness” of the waterfilling
for “natural” priors. Such an analysis is conceptually simple given the contents of the
waterfilling Lemma 21 and associated bound Proposition 7.
One issue that we have not addressed here or in [125] is the equal scaling of 𝑛 at
both sensor and scene. One natural way to address this is letting A be 𝑚 × 𝑛, or
alternatively one could study a continuous model. Another issue is obtaining a good
understanding of mask/lens combinations. Understanding such combinations will
require not only updates to the simple propagation model studied here and in [125],
but also a refined understanding of the cost tradeoffs between lenses and apertures.
Stepping back from imaging problems, one may ask the question of where else
Nazarov’s theorem can be used in applied contexts, something also raised implicitly
in [21]. For example, as Nazarov’s theorem does not care about orthogonality, but
merely a 𝑙2 estimate like Parseval’s theorem, one can use it for frames as well as
bases, or for anything satisfying a restricted isometry property. Another example
is the fact that we merely use the 𝑙∞ case of his theorem which works for all 𝑙𝑝
spaces. Furthermore, the astute reader would have noticed that much of the discussion
of this chapter relies only on just a few pieces that have a fair bit of slack. We
therefore firmly believe that Nazarov’s theorem and the (heuristically) efficient greedy
algorithm could play interesting roles elsewhere. An entertaining illustration is that
115
of constructing good lattice packings via Bang’s lemma [16] as Ball [14] describes.
We do not describe Ball’s construction here, but simply note that we still agree with
Ball’s assessment given at the end of his article [14]: “As with other “constructions”
of efficient packings, the simplicity here is probably an illusion”. Roughly speaking,
the construction relies on finding a sign cortège of length at least exponential in
𝑛. Without our observation regarding the sufficiency of “local” optimality, this would
require at least doubly exponential time in the dimension 𝑛. With it, we (heuristically)
get rid of one exponentiation to make it singly exponential.
116
−40 −20 0 20 40
−60
−40
−20
0
10 log10(𝑡/𝑛)
LMM
SE(d
B)
lower boundoptimal random on-off
spectrally flatNazarov
(a) i.i.d prior (𝑑(𝑥) = 𝜃 for 0 ≤ 𝑥 ≤ 1/2)
−40 −20 0 20 40
−100
−80
−60
−40
−20
10 log10(𝑡/𝑛)
LMM
SE(d
B)
lower boundoptimal random on-off
spectrally flatNazarov
(b) bandlimited prior (𝑑(𝑥) = 𝜃 for 0 ≤ 𝑥 ≤ 𝑠− 𝑟, 0 for 𝑥 ≥ 𝑠+ 𝑟, and𝜃(𝑠+ 𝑟 − 𝑥)/(2𝑟) otherwise for 0 ≤ 𝑥 ≤ 1/2)
Figure 3-5: 𝑛 = 677, 𝜃 = 1,𝑊 = 𝐽 = 0.001, 𝑠 = 0.02, 𝑟 = 0.005. We use the quarticresidue construction for spectrally flat. Jaggedness of the Nazarov plot comes fromthe fact that in general the spectrum allocation varies with 𝑡 and we randomly seedthe sign cortège.
117
118
Chapter 4
New lower bounds for the mean
squared error of vector quantizers
For twelve years I have been studying properties of parallelohedra. I can
say it is a thorny field for investigation, and the results which I obtained
and set forth in this memoir cost me dear. . .
Three-dimensional parallelohedra are now playing an important role in
the theory of crystalline bodies, and crystallographers have already paid
attention to properties of these strange polyhedra, but till now
crystallographers were satisfied with the description of parallelohedra
from a purely geometrical point of view. I noticed already long ago that
the task of dividing the 𝑛-dimensional analytical space into convex
congruent polyhedra is closely related to the arithmetic theory of
positive quadratic forms.
Georgy Voronoï, 1907
4.1 Introduction
Let us shift gears a bit and examine the problem of vector quantization in Euclidean
space. Links and interesting connections to material presented in the preceding chap-
ters will become clear as we proceed.
119
The problem of quantization is of fundamental importance to signal processing,
with a long and distinguished history [57]. Our focus in the current chapter is on
the mathematical theory. Specifically, we study lower bounds on the mean squared
error under the “high-resolution limit”, a study which originated in work on pulse-
coded modulation (PCM) [90]. We do note that there is an important complementary
perspective offered by rate distortion theory, see e.g. [57] or [92, Ch. 25-27] for more
information on this topic and the relation between these two perspectives. The focus
on mean squared error is for mathematical simplicity, though even in such a setting
fine-grained questions appear difficult. An astute reader will note that a nontrivial
amount of the discussion, especially the general setting explored in Theorem 10, is
generalizable to distortion measures that go beyond squared error.
It is well known that one can reduce the “high-resolution” problem for general
source probability distributions (under weak assumptions [129, Thm. 1]) to that of
studying a uniform source over a large region through the use of “companders” in
the scalar case [19, 91], and a point density function in the general case [128, 129]1. As such, the basic object of study may be defined as follows [30]. For points
𝑝1, 𝑝2, . . . , 𝑝𝑀 ∈ [0, 1]𝑑, define the normalized second moment (NSM), scaled down by
a factor of 𝑑 by
𝐺(𝑝1, . . . , 𝑝𝑀) ,1
𝑑
1𝑀
∑𝑀𝑖=1
∫𝑉 (𝑝𝑖)
|𝑥− 𝑝𝑖|2𝑑𝑥(1𝑀
∑𝑀𝑖=1 |𝑉 (𝑝𝑖)|
)1+ 2𝑑
, (4.1)
where 𝑉 (𝑝) denotes the Voronoï cell associated to 𝑝 restricted to [0, 1]𝑑, and | · |
denotes volume. The Voronoï cell 𝑉 (𝑝) is the set of points closer to 𝑝 than any other
point, that is
𝑉 (𝑝𝑖) , 𝑥 ∈ R𝑑 : ∀𝑗 = 𝑖, |𝑥− 𝑝𝑖| < |𝑥− 𝑝𝑗|.
Note that as defined above, the Voronoï cells do not strictly partition R𝑑 as the cell
boundaries are not included in any of them. Such issues do not concern us at the
moment, and play a minimal role in this chapter. One may ignore these issues in our
context simply because these boundaries have zero measure, and thus the boundaries
1For 𝑑 = 2, this is already implicitly present in [43].
120
do not affect “bulk” quantities like the NSM. We henceforth reserve the term NSM
to refer to the quantity that is not divided by 𝑑, so for instance in (4.1), the NSM is
𝑑𝐺(𝑝1, . . . , 𝑝𝑀). We shall call 𝐺 itself the per-dimensional NSM.
We also find it convenient to talk about the NSM of a body 𝐵 about a point 𝑣,
defined by
𝑁𝑆𝑀(𝐵, 𝑣) ,
∫𝐵|𝑥− 𝑣|2𝑑𝑥|𝐵|1+ 2
𝑑
.
When the choice of 𝑣 is clear from context, we may omit it. For example, when we
talk of the NSM of a ball, 𝑣 is implicitly the center of the ball.
Before proceeding further, we comment on the history of Voronoï cells. The
original mathematical impetus can arguably be attributed to Gauß, Hermite, and
Lagrange, who did a detailed study of quadratic forms in number theory. According
to [114, p. 7], Dirichlet delivered an interesting lecture in the physical-mathematical
class meeting of the Prussian Military Academy on the 31st of July, 1848 on the reduc-
tion of a positive quadratic form with three indeterminate integers. In his successful
effort at simplifying the work of Gauß and Seeber on this topic, Dirichlet introduced
what we now commonly call a Voronoï cell of a lattice (corresponding to the quadratic
form). Hence some authors also call this a “Dirichlet tesselation”. However, Voronoï
appears to have been the first to undertake a detailed study of such tessellations in
three outstanding papers in Crelle’s journal ( [122], [121], [123]). Henceforth we stick
to the common practice of talking about Voronoï cells, partitions, decompositions,
and tessellations. The middle two are exact synonyms, and the last one, which was
the principal object of Voronoï’s study, we reserve for the lattice case only, that is
when 𝑝𝑖 are elements of a lattice Λ ∈ R𝑑.
Definition 15. A lattice Λ ∈ R𝑑 is an additive subgroup of R𝑑 which is isomorphic
to the additive group Z𝑑, and which spans the real vector space R𝑑.
We also note that in applied contexts, the physician and father of modern epi-
demiology John Snow used a Voronoï partition to illustrate how most people who died
in the 1854 Broad Street cholera outbreak lived closer to the infected Broad street
pump than to any other pump. For a detailed account of how Snow collected his
121
data and constructed his map, we recommend [68]. Given the simplicity and central
importance of this idea, we think it likely that this idea was discussed by many others
as well.
Returning to mathematics, we may then define the minimal per-dimensional NSM
by
𝐺𝑑 , lim𝑀→∞
inf𝑝𝑖𝐺(𝑝1, . . . , 𝑝𝑀). (4.2)
Restricting to the special case where 𝑝𝑖 are points of a lattice Λ′, (4.1), (4.2) simplify,
and one may define a minimal per-dimensional NSM for lattices by
𝐺Λ,𝑑 , infΛ′
∫𝑉 (0)
|𝑥|2𝑑𝑥
𝑑|𝑉 (0)|1+ 2𝑑
. (4.3)
Zador [128, 129] proved
𝐺𝑑 ≥1
(𝑑+ 2)𝜋Γ
(𝑑
2+ 1
) 2𝑑
. (4.4)
Zador [128, 129] also obtained an asymptotically matching upper bound
𝐺𝑑 ≤1
𝑑𝜋Γ
(𝑑
2+ 1
) 2𝑑
Γ
(1 +
2
𝑑
),
and thus showed
lim𝑑→∞
𝐺𝑑 =1
2𝜋𝑒.
Zador’s work [128, 129] contains most of the foundational material on the mathe-
matics of vector quantization, see e.g. the survey of Gray and Neuhoff [57] for details.
Poltyrev [130, Lemma 1] observed that asymptotically good coverings result in
asymptotically good quantizers. In particular, one may use the good lattice coverings
of Rogers [98] and demonstrate
lim𝑑→∞
𝐺Λ,𝑑 =1
2𝜋𝑒.
122
4.2 Main results
The main results of this chapter are improvements on (4.4). To describe them, we
need some notation. Let
𝑉𝑑 ,𝜋
𝑑2
Γ(𝑑2
+ 1)
be unit ball volume. Then 𝐴𝑑 = 𝑑𝑉𝑑 is its surface area. Let
𝐼𝑥(𝑎, 𝑏) ,
∫ 𝑥
0𝑡𝑎−1(1 − 𝑡)𝑏−1𝑑𝑡∫ 1
0𝑡𝑎−1(1 − 𝑡)𝑏−1𝑑𝑡
be the regularized incomplete beta function. One may compute the solid angle and
NSM of a circular solid cone of diagonal 1, height ℎ about its vertex and obtain
Θ(𝑑, ℎ) =1
2𝐴𝑑𝐼1−ℎ2
(𝑑− 1
2,1
2
),
𝜉(𝑑, ℎ) =𝑑1+
2𝑑
𝑑+ 2
ℎ2 + (1 − ℎ2)𝑑−1𝑑+1
(ℎ𝑉𝑑−1(1 − ℎ2)𝑑−12 )
2𝑑
.
We then have the following conjecture:
Conjecture 2. Let 𝑑 ≥ 2. Let 𝐹𝑑 , 2(2𝑑−1), and let ℎ𝑑 be the root of Θ(𝑑, ℎ)− 𝐴𝑑
𝐹𝑑=
0. Then,
𝐺Λ,𝑑 ≥𝜉(𝑑, ℎ𝑑)
𝑑𝐹2𝑑𝑑
. (4.5)
Taking
𝐹𝑑,𝑘 , 2(2𝑑 − 1) + (𝑘 − 1)2𝑑 (4.6)
in the above bound as opposed to 𝐹𝑑, one obtains a lower bound on 𝐺𝑑 restricted to
quantizers formed by 𝑘 translates of a lattice.
We believe that we have firm evidence in favor of this Conjecture 2. The missing
ingredients can be captured by certain technical inequalities that may be readily
verified on a computer (such as (4.19)), but that we are unfortunately unable to
rigorously prove.
One of the chief aims of this chapter is to enable the reader to understand where
123
this conjecture comes from, and why we view it as extremely plausible. We also believe
that Conjecture 2 should be easier to prove than Conway and Sloane’s conjectured
bound [31], though we note that Conway and Sloane’s conjectured bound is sharper
and applies to all quantizers, not just lattice quantizers.
We are sympathetic to the reader who is dissatisfied with the state of affairs
regarding the missing technicalities required to prove Conjecture 2. Such readers may
find solace in the following rigorous improved lower bound valid for all quantizers,
and not just lattice ones:
Theorem 10. Define 𝜈(𝑑, 𝑟) for dimension 𝑑 ≥ 1, 0 ≤ 𝑟 ≤ 1 to be the NSM of a
truncated unit ball centered at the origin and computed about the origin. Here the ball
is truncated by intersecting it with a hyperplane at distance 𝑟 from the origin, such as
𝑥1 ≤ 𝑟. At the limit 𝑟 = 0, we have a hemisphere, and at 𝑟 = 1, we have the original
unit ball. Define
𝜅(𝑑, 𝑟) , min0≤𝑥≤𝑟
𝜈(𝑑, 𝑟).
Let 𝑐𝑑 = 𝜅(𝑑, 1) be the NSM of the unit ball, namely
𝑐𝑑 ,𝑑
(𝑑+ 2)𝜋Γ
(𝑑
2+ 1
) 2𝑑
.
Let 𝛾𝑑 , 1 − 32
(23
)𝑑, and let 𝑘𝑝 be the base of the exponent of the asymptotic sphere
packing bound of Kabatyanskii and Levenshtein [69], given by
𝑘𝑝 = 2−0.599... ≈ 0.6602.
Let 𝑑 ≥ 3 be sufficiently large. Let 𝑓(𝑧) be an 𝑛-level quantizer on [0, 1]𝑑. Then,
∫[0,1]𝑑
|𝑧 − 𝑓(𝑧)|2𝑑𝑧 ≥ max
(𝑛− 2
𝑑 𝑐𝑑1
2
((2
3
)1+ 2𝑑
+
(4
3
)1+ 2𝑑
),
𝑛− 2𝑑
((𝛾𝑑 −
1
2
)𝜅
(𝑑,
3
2𝑘𝑝
)− 𝑑2
+
(3
2− 𝛾𝑑
)𝑐− 𝑑
2𝑑
)− 2𝑑
+ 𝑜(𝑛− 2𝑑 )
),
as 𝑛→ ∞.
124
To the best of our knowledge, Theorem 10 represents the first rigorous improve-
ment over the bound of Zador (4.4).
4.3 Proofs
At a high level, our strategy for lattice quantizers may be described as follows. The
original “sphere bound” of Zador [128], [129] comes from the fact that each Voronoï
cell’s second moment, normalized by its volume, can’t be lower than that of a ball.
This trivially provable fact is very nice as it immediately yields Zador’s asymptotically
tight bound upon carrying out the computation. Our approach here is heavily inspired
by the work of Tóth/Newman [116], [88], who prove a sharp bound for 𝑑 = 2 by
utilizing an upper bound on the number of edges of the Voronoï polygons. This upper
bound on the number of edges, together with some calculus, convexity, and Hölder’s
inequality allows one to prove that the hexagonal lattice quantizer is optimal for 𝑑 = 2.
What we do is simply work with upper bounds on facet counts in higher dimensions
and appropriately generalize the machinery. There is one serious drawback of this
approach: there is a finite upper bound on facet counts only for the lattice case for
𝑑 ≥ 3, see e.g. work of Dolbilin and Tanemura [39, §5] for a construction for 𝑑 = 3
of arbitrarily large facet counts for a Voronoï cell in a non-lattice quantizer. Our
methods therefore bifurcate into separate lattice and general quantizer cases.
Although we find the above approach to lattice quantizers more interesting as
compared to our methods for the general case, we shall first study the general case
and prove Theorem 10. Our justification for this ordering is that it illustrates some
of the basic convexity and Hölder’s inequality machinery that plays a key role in the
lattice case as well. In fact, even before studying the general case, we give a brief
rederivation of Zador’s lower bound (4.4) using convexity. We have not found this
approach in the literature.
125
4.3.1 Rederivation of Zador’s lower bound
Let us for simplicity look at lattice quantizers Λ with |Λ| = 1 in R𝑑. By |Λ| = 1 we
mean that the volume of a fundamental cell of the lattice is 1. We have the basic
∀𝑥,∀𝛽 > 0, min𝑣∈Λ
|𝑥− 𝑣|2 ≥ − 1
𝛽log
(∑𝑣∈Λ
𝑒−𝛽|𝑥−𝑣|2)
(4.7)
Let the quantizer value (second moment) be denoted 𝑁𝑆𝑀(Λ); we have ensured
normalization by the assumption that |Λ| = 1. We have by the concavity of log
and (4.7)
𝑁𝑆𝑀(Λ) =
∫𝑥∈𝑉 (0)
min𝑣∈Λ
|𝑥− 𝑣|2𝑑𝑥
≥ − 1
𝛽
∫𝑥∈𝑉 (0)
log
(∑𝑣∈Λ
𝑒−𝛽|𝑥−𝑣|2)𝑑𝑥
≥ − 1
𝛽log
(∫𝑥∈𝑉 (0)
∑𝑣∈Λ
𝑒−𝛽|𝑥−𝑣|2𝑑𝑥
)
= 𝑑
(1
2𝜋
log(𝛽𝜋
)𝛽𝜋
).
Optimizing over 𝛽, one picks 𝛽 = 𝜋𝑒. This choice of 𝛽 yields 𝑁𝑆𝑀(Λ) ≥ 𝑑2𝜋𝑒
,
which is in fact the asymptotic value (𝑑 → ∞) of the minimum NSM, as proved by
Zador [129]. Can we do better and actually recover the non-asymptotic lower bound
of Zador (4.4)? It turns out that we can!
More abstractly, let us consider what we need for the above argument. Let us
consider a radial function 𝑓(𝑟2) satisfying:
𝑓 ≥ 0, 𝑓 ′ ≤ 0, 𝑓 ′′ ≥ 0.
We also certainly want 𝑓 ∈ 𝑙1(R𝑑), and the inverse to make sense on the infinite sum∑𝑣∈Λ 𝑓(|𝑥− 𝑣|2). With these conditions met, we may replace 𝑒−𝛽𝑟2 by 𝑓 .
It may be checked that the exact optimal 𝑓 meeting these conditions is a “tent”
function of the appropriate scale, where a “tent” function is of the form 𝑓(𝑟) =
126
(𝑎 − 𝑏𝑟)+ with 𝑥+ = max(𝑥, 0) and 𝑎, 𝑏 > 0. This choice of 𝑓 may be justified as
follows. The tent functions are the extremal rays of the cone given above, and it is
easily checked that using a convex combination of functions for the above argument
does no better than the best extremal endpoint. All that remains is to pick the right
scale for the tent. Upon optimizing the scale of the tent and performing the relevant
computation, one recovers Zador’s non-asymptotic lower bound (4.4).
The above argument generalizes to quantizers formed by 𝑘 translates of a lattice,
where 𝑘 is a finite number. Such quantizers can come arbitrarily close in performance
to optimal quantizers by standard limiting arguments. For example, just consider
quantizing the unit cube at arbitrarily fine resolution, and then replicating the quan-
tization points contained in the unit cube across space by translating them by Z𝑑.
Thus in fact the argument of this section is a genuine rederivation of Zador’s lower
bound (4.4).
4.3.2 The general case
Hopefully the reader is now convinced of the value of convexity considerations in the
study of the minimal NSM of quantizers. We now perform a more detailed study and
prove Theorem 10.
We start off with a basic inequality.
Lemma 28. Let 𝑎𝑖, 𝑏𝑖 ≥ 0,∑
𝑖 𝑎𝑖 = 1, 𝑝 > 1, and let 𝑏𝑖 ≥ 𝑐 > 0 for all 𝑖. Then, we
have ∑𝑖
𝑎𝑝𝑖 𝑏𝑖 ≥ max
⎛⎝𝑐∑𝑖
𝑎𝑝𝑖 ,
(∑𝑖
𝑏1
1−𝑝
𝑖
)1−𝑝⎞⎠ .
Proof of Lemma 28. The first term in the maximum is trivial, and the second term
is immediate from Hölder’s inequality.
In our application of Lemma 28, 𝑐 will be the NSM of a ball, 𝑏𝑖 will be the NSM
of Voronoï cells, 𝑎𝑖 will be the volumes of the Voronoï cells in a decomposition of the
unit square, and finally 𝑝 = 1 + 2/𝑑 where 𝑑 is the dimension of the vector quantizer.
127
Plugging in 𝑏𝑖 ≥ 𝑐 into Lemma 28, and dropping the first term out of the maximum
immediately yields Zador’s “sphere bound” (4.4) once the number of points 𝑛 → ∞
as we defined earlier. Plugging in 𝑎𝑖 = 1/𝑛 and dropping the second term also yields
Zador’s sphere bound. Thus our goal is to somehow get a nontrivial trade-off between
the two terms. Quantifying this trade-off amounts to the task of showing that at least
one of the following phenomena must take place for any vector quantizer:
1. A nontrivial amount of “dispersion” in the volumes of the Voronoï cells.
2. A nontrivial fraction of the 𝑏𝑖 are bounded away from the ball’s NSM.
Let us first understand why one would might expect this. Consider the “extreme” case
for the first item, which happens with lattice quantizers where all Voronoï cells are
identical. Each point of the lattice has a “close” nearest neighbor since lattices can’t
have packing density exceeding standard sphere packing density bounds. Heuristi-
cally, a packing of spheres is just a non-overlapping collection of equally sized balls
in Euclidean space. Its density is given by the limiting ratio of the volume occupied
by the balls to the volume of a large bounded region, with the limit taken as the
region grows to infinity. As we are not proving statements about sphere packing
here, we simply refer the reader to e.g. [30, Ch. 1] for a rigorous definition of sphere
packing density. The current best known (asymptotic) upper bound is the classi-
cal 2(−0.599···+𝑜(1))𝑑 due to Kabatyanskii and Levenshtein [69], who used Delsarte’s
linear programming method [35] together with a suitably modified version (for the
Euclidean sphere) of the construction of McEliece, Rodemich, Rumsey, and Welch
(MRRW) [82] done for Hamming space. The sphere packing bound implies that the
𝑏𝑖 must be bounded away from 𝑐: at best their NSM matches that of a ball cut off by
a hyperplane governed by the packing radius. The sphere packing bounds apply not
just to lattices, but also arbitrary point configurations! We may therefore proceed
further.
To mathematically quantify this phenomenon, the following is useful.
Lemma 29. Let 𝑥, 𝑦 ≥ 0, and 𝑝 > 1, and let 𝑤, 𝑎, 𝑏 ∈ (0, 1). Consider the optimiza-
128
tion problem
min𝐹 , 𝑤𝑥𝑝 + (1 − 𝑤)𝑦𝑝,
subject to 𝑤𝑥+ (1 − 𝑤)𝑦 = 1,
𝑥 ≤ 𝑎,
𝑤 ≥ 𝑏.
Then the minimum is attained at 𝑥 = 𝑎, 𝑤 = 𝑏.
Proof of Lemma 29. First, let 𝑤 be fixed, and consider optimizing over 𝑥. Setting
𝑦 = 1−𝑤𝑥1−𝑤
, we get𝜕𝐹
𝜕𝑥= −𝑝𝑤
((1 − 𝑤𝑥
1 − 𝑤
)𝑝−1
− 𝑥𝑝−1
).
As 𝑥 < 1, 𝑝 > 1, we see that 𝐹 is decreasing in 𝑥. Thus for a fixed 𝑤, we must set
𝑥 = 𝑎 to minimize 𝐹 .
Now, setting 𝑥 = 𝑎, we claim that 𝐹 is increasing in 𝑤. Showing this would
complete the proof. Once again, studying 𝜕𝐹𝜕𝑤
, we reduce our task to showing:
(1 − 𝑎𝑤)𝑝−1[1 − 𝑎𝑤 − (1 − 𝑎)𝑝] ≤ (𝑎− 𝑎𝑤)𝑝. (4.8)
We have equality at 𝑎 = 1, suggesting the following proof of (4.8). We may
rewrite (4.8) as
(1 − 𝑎𝑤)𝑝 − (𝑎− 𝑎𝑤)𝑝 ≤ (1 − 𝑎)𝑝(1 − 𝑎𝑤)𝑝−1.
But observe that 𝑥𝑝 is convex since 𝑝 > 1, and so we have:
(1 − 𝑎𝑤)𝑝 − (𝑎− 𝑎𝑤)𝑝 ≤ [(1 − 𝑎𝑤) − (𝑎− 𝑎𝑤)]𝑝(1 − 𝑎𝑤)𝑝−1,
as desired.
We may now proceed with the Proof of Theorem 10 along the sketched direction
above.
129
Proof of Theorem 10. Let the points of the quantizer be 𝜆1, 𝜆2, . . . , 𝜆𝑛. Let the min-
imum distance for each of these points to their nearest neighbors be denoted by 𝑟𝑖.
Define 𝑟𝑑(𝑣) to be the radius of the ball in 𝑑 dimensions of volume 𝑣. By the sphere
packing consideration, we see that for any 0 < 𝛾 < 1, and sufficiently large 𝑑, 𝛾𝑛+𝑜(𝑛)
of the 𝜆𝑖 must have
𝑟𝑖 ≤(
1
1 − 𝛾
) 1𝑑
𝑘𝑝𝑟𝑑
(1
𝑛
). (4.9)
For if this statement was not true, we could restrict our attention to the (1−𝛾) fraction
of points with largest minimum distance and violate the sphere packing density upper
bound.
Here 𝑘𝑝 is the base of the exponent in an asymptotic upper bound for sphere
packing density of the form ∆𝑑 ≤ 𝑘𝑑+𝑜(𝑑)𝑝 , and we may take by [69]
𝑘𝑝 = 2−0.599... <2
3< 1.
We emphasize that the argument does not rely on the precise numerics of 𝑘𝑝, though
our choice of certain parameters here does. For example, at a certain step we chose
2/3 simply because (3/2)𝑘𝑝 < 1 and it is a simple fraction. All one really needs is
𝑘𝑝 < 1. For a reader interested in elementary arguments that still yield a nontrivial
𝑘𝑝, we recommend the work of Blichfeldt [20] who obtained
∆𝑑 ≤𝑑+ 2
2
(√1
2
)𝑑
,
valid for all 𝑑 ≥ 1.
Now let 𝑓 denote the fraction of the number of points 𝜆𝑖 with associated Voronoï
cell volumes |𝑉 (𝜆𝑖)| ≥ 2/(3𝑛). Now we divide into two cases depending on the value
of 𝑓 .
1. Suppose 𝑓 ≤ 1/2. Here we have sufficient “dispersion” in the volumes of the
130
Voronoï cells. We may apply Lemma 29 to immediately get a lower bound
∫[0,1]𝑑
|𝑧 − 𝑓(𝑧)|2 ≥ 𝑛−2𝑑 𝑐𝑑
1
2
((2
3
)1+ 2𝑑
+
(4
3
)1+ 2𝑑
), (4.10)
where 𝑐𝑑 is the NSM of the ball. This covers the first term of our basic approach
outlined in Lemma 28.
2. Now suppose 𝑓 > 1/2. Here we no longer rely on “dispersion” of the volumes,
but instead use the fact that we have a nontrivial fraction of sufficiently “large”
Voronoï cells. We also know by (4.9) that a large fraction of points have short
minimum distance. These two sets must have a nontrivial intersection.
Mathematically, we need to choose 𝛾 appropriately. What we have is:
𝑟𝑖 ≤ 𝑟𝑑
(1
1 − 𝛾𝑘𝑑𝑝
1
𝑛
),
and we will make the argument of 𝑟𝑑 match [2/(3𝑛)] ((3/2)𝑘𝑝)𝑑 for example. We
chose 3/2 simply because (3/2)𝑘𝑝 < 1. We therefore take
𝛾𝑑 , 1 − 3
2
(2
3
)𝑑
≥ 5
9,
since we may take 𝑑 ≥ 3. For sufficiently large 𝑑, we thus get
∫[0,1]𝑑
|𝑧−𝑓(𝑧)|2 ≥ 𝑛− 2𝑑
((𝛾𝑑 +
1
2− 1)𝜅
(𝑑,
3
2𝑘𝑝
)− 𝑑2
+
(3
2− 𝛾𝑑
)𝑐− 𝑑
2𝑑
)− 2𝑑
+𝑜(𝑛− 2𝑑 ),
(4.11)
where the little 𝑜 is with respect to 𝑛 and 𝑑 is fixed.
Combining (4.10) and (4.11) gives us Theorem 10.
Remark 9. No attempt has been made to optimize the parameters of Theorem 10.
Part of the difficulty lies in understanding quantities like 𝜅(𝑑, 𝑟). The astute reader
may, for example, conjecture that 𝜅(𝑑, 𝑟) = 𝜈(𝑑, 𝑟), which would follow from the
natural conjecture that 𝜈(𝑑, 𝑟) is decreasing in 𝑟 for 𝑟 ∈ [0, 1]. We do not have a proof
131
of this statement. Indeed, more generally, we do not have a good grasp on the NSM of
objects derived in natural fashions from balls, such as the 𝜉(𝑑, ℎ) of Conjecture 2. This
difficulty extends to our subsequent discussion in subsection 4.3.4 on lattice quantizers
as well. It is also clear that 𝑑 being sufficiently large can be replaced by 𝑑 ≥ 2 as long
as one uses a non-asymptotic sphere packing bound.
Let us now turn to the study of lattice quantizers, where we generalize the work of
Tóth/Newman on the optimality of the hexagonal lattice quantizer for 𝑑 = 2 among
all quantizers, and not just lattice ones!
We believe that the best way to understand our generalization of is by first ex-
amining the original work of Tóth/Newman [116], [88]. More precisely, we follow
Newman’s presentation essentially verbatim as it is both mathematically simpler and
also available readily in English. We do not know of any essential simplification to
his argument, presented in the following subsection 4.3.3.
4.3.3 Tóth/Newman’s hexagon theorem
Theorem 11. Let 𝑆 be the unit square [0, 1]2, and 𝑓(𝑍) any function on 𝑆 taking at
most 𝑛 distinct values, i.e. an 𝑛-level quantizer. Then,
∫𝑆
|𝑧 − 𝑓(𝑧)|2𝑑𝑧 > 𝜎
𝑛, where 𝜎 =
5
18√
3.
Recall that Voronoï cells form a partition of 𝑆, bounded by straight line segments.
We prove a Lemma regarding the number of edges formed by any partition of 𝑆 into
convex polygons (not necessarily Voronoï!) as a direct consequence of Euler’s theorem
on planar graphs:
Lemma 30. Let 𝑆 be divided into 𝑘 convex polygons in any manner, and let 𝐸 denote
the total number of edges. Then 𝐸 ≤ 3𝑘 + 1.
Proof of Lemma 30. At each vertex 𝑣 we let 𝑚𝑣 be the number of edges meeting at
it. By double counting, 2𝐸 =∑
𝑣𝑚𝑣. Furthermore, 𝑚𝑣 ≥ 3 at all 𝑣 except possibly
132
at the corners of the square 𝑆 due to convexity of the polygons. Thus,
2𝐸 =∑𝑣
𝑚𝑣 ≤ 2 × 4 + 3
[∑𝑣
(𝑚𝑣 − 2)
]= 6𝐸 − 6𝑉 + 8,
where 𝑉 is the total number of vertices. By Euler’s theorem, 𝐸 − 𝑉 = 𝑘 − 1, so the
proof is complete.
We now apply Lemma 30 to a slightly modified Voronoï diagram, where we par-
tition the Voronoï cells further into right and obtuse angled triangles according to a
certain procedure described below and prove the following Lemma.
Lemma 31. Let 𝜆𝑖, 1 ≤ 𝑖 ≤ 𝑛 denote the quantizer outputs. Let 𝑉 (𝜆𝑖), 1 ≤ 𝑖 ≤ 𝑛
denote their respective Voronoï cells. For each 𝑉 (𝜆𝑖), draw in it all the line segments
from 𝜆𝑖 to the vertices of 𝑉 (𝜆𝑖). Also draw in all the perpendiculars from 𝜆𝑖 to the
edges of 𝑉 (𝜆𝑖), possibly extended, and terminate these at the boundary of 𝑉 (𝜆𝑖). This
construction procedure subdivides the Voronoï cells into triangles, each of which has
a right or obtuse angle.
On performing this procedure over all the 𝑉 (𝜆𝑖), we end up with triangles 𝑇1, 𝑇2, . . . , 𝑇𝑁 ,
and vertex angles formed at the corresponding 𝜆𝑖 that we call 𝜃1, 𝜃2, . . . , 𝜃𝑁 . Then,
𝑁 ≤ 12𝑛− 4.
Proof of Lemma 31. Let 𝑁𝑖 be the number of triangles 𝑇𝑘 formed through the subdi-
vision process given above on 𝑉 (𝜆𝑖). Let 𝐸𝑖 be the number of edges of 𝑉 (𝜆𝑖). Once
again, by a double counting argument (taking care of the boundary of 𝑆), we have
∑𝑖
𝐸𝑖 = 2𝐸 − (number of edges on boundary of 𝑆) ≤ 2𝐸 − 4.
But then
𝑁 =∑𝑖
𝑁𝑖 = 2∑𝑖
𝐸𝑖 ≤ 4𝐸 − 8 ≤ 12𝑛− 4,
where in the last inequality we used Lemma 30.
133
We now show that the NSM of a triangle 𝑇𝑘 about its vertex is lower bounded by
that of a right angled triangle with the same vertex angle 𝜃𝑘.
Lemma 32. Define
𝜑(𝜃) ,3 tan(𝜃)
3 + tan2(𝜃). (4.12)
If 𝑇 is any right or obtuse triangle and 𝜆 is a vertex of 𝑇 with acute vertex angle 𝜃,
then ∫𝑇
|𝑧 − 𝜆|2𝑑𝑧 ≥ |𝑇 |2
𝜑(𝜃).
Proof of Lemma 32. The proof is essentially by a geometric comparison with a suit-
able right angled triangle with the same vertex, same vertex angle, and same area.
The details are given below.
Let 𝑒 be the edge of 𝑇 whose endpoints are 𝜆 and the right or obtuse vertex of 𝑇 ,
let 𝑓 be the other edge of 𝑇 containing 𝜆, and finally let 𝑔 be the third edge. From
some point 𝑥 on 𝑓 , we drop a perpendicular to 𝑒 (possibly extended) and call the foot
of the perpendicular 𝑦. We choose 𝑥 such that the right angled triangle formed by
𝜆, 𝑥, 𝑦 has the same area as 𝑇 ; this is what we mean by the “geometric comparison”
alluded to above. Let us call this right angled triangle 𝑈 . Let the intersection of 𝑥𝑦
with 𝑔 be 𝑧0.
It is clear that
∀𝑧 ∈ 𝑇 − 𝑈, |𝑧 − 𝜆| ≥ |𝑧0 − 𝜆|,
∀𝑧 ∈ 𝑈 − 𝑇, |𝑧 − 𝜆| ≤ |𝑧0 − 𝜆|.
Thus, with the integrand |𝑧 − 𝜆|2 and the implicit Lebesgue measure, we have
∫𝑇
=
∫𝑇∩𝑈
+
∫𝑇−𝑈
≥∫𝑇∩𝑈
+
∫𝑈−𝑇
=
∫𝑈
=|𝑈 |2
𝜑(𝜃),
where for the last equality one performs a direct calculation and is indeed how one
comes across 𝜑 in the first place. The proof is complete since by construction |𝑈 | =
|𝑇 |.
134
We now explicitly note that 𝜑 is concave:
Lemma 33. 𝜑 governed by (4.12) is concave on(0, 𝜋
2
).
Proof of Lemma 33. One may readily calculate
𝜑′′(𝜃) =−48 sin3(𝜃) cos(𝜃)
(cos(2𝜃) + 2)3,
and the nonpositivity on (0, 𝜋/2) is obvious.
Remark 10. We are not satisfied with the proof of Lemma 33 above as it is not
conceptual enough and relies on a direct calculation. The reader may not view this
as a defect right now since the differentiation and associated concavity check is easily
observed above. However, in our efforts to generalize this method to 𝑑 ≥ 3, we run
into a nontrivial assertion of concavity that we are unable to prove (Conjecture 4),
but can simulate to whatever precision we want on a computer. The nontriviality of
Conjecture 4 is ultimately why we must still leave Conjecture 2 as a Conjecture, and
not a Theorem. We note that neither Newman [88] (who asserted the concavity of 𝜑
as defined above) nor Tóth [116] (who had a slightly different 𝜑 but relied on concavity
as well) have a more elegant proof. We therefore consider it worthwhile to find an
alternative proof.
All the pieces are now in play to prove the “hexagon” Theorem 11:
Proof of Theorem 11. Consider the equation 𝜑(𝜃) = 𝜃2𝜋𝜎
. Equality holds at 𝜃 = 0 and
𝜃 = 𝜋6, so by concavity, we immediately get
𝜑(𝜃) <𝜃
2𝜋𝜎(4.13)
on(𝜋6, 𝜋2
). We may thus write, by Jensen’s inequality,
𝑁∑𝑘=1
𝜑(𝜃𝑘) ≤ 𝑁𝜑
(∑𝜃𝑘𝑁
)= 𝑁𝜑
(2𝜋𝑛
𝑁
)< 𝑁
𝑛
𝜎𝑁=𝑛
𝜎. (4.14)
135
In the second inequality we used Lemma 31 to write 𝑁 < 12𝑛, and we combined this
with the above (4.13).
We complete the proof by an application of Cauchy-Schwarz. Using the bound (4.14),
we have ∫𝑆
|𝑧 − 𝑓(𝑧)|2 ≥𝑁∑𝑘=1
|𝑇𝑘|2
𝜑(𝜃𝑘)≥ (∑
|𝑇𝑘|)2∑𝜑(𝜃𝑘)
≥ 𝜎
𝑛.
4.3.4 Conjectured bound for lattices
As remarked upon earlier, our approach to improved lower bounds for lattice quan-
tizers is best described as a generalization of the preceding subsection 4.3.3. Our
thinking may be described as follows:
1. We can not rely on Euler’s theorem to get upper bounds on facet counts for
𝑑 ≥ 3. Our solution is to focus in on lattice quantizers only, and rely on the
original work of Minkowski [84, §6] and independently Voronoï [121, §48] for an
upper bound on the facet counts in this case. We note that one can generalize
this to a union of 𝑘 translates of a lattice quantizer by the work of Delone and
Sandakova [34], indeed this is where the 𝐹𝑑,𝑘 of (4.6) comes from.
2. We need to think of a relevant decomposition of Voronoï polyhedra to which we
can apply the concavity considerations described in the proof of Theorem 11.
There are two natural candidates.
One is closer in spirit to the work of Newman [88], and decomposes the Voronoï
polyhedra into orthosimplices. Here, one needs upper bounds not just on facet
counts, but also lower dimensional faces of the polytope. These are also due
to Voronoï [121, §65], and we give these bounds in a remark later 11. The
decomposition into orthosimplices was our first attempt. However, the technical
difficulties of coming up with a good lower bound on the second moment with
respect to the vertex of the orthosimplex, subject to fixed solid angle (as opposed
to regular angle) and analogous to Lemma 32, defeated us. More importantly,
136
even if we could resolve these difficulties, we believe that the conjectured bound
following such an approach is weaker than that of the second candidate described
below.
Our second candidate is closer in spirit to the earlier work of Tóth [116]. Here,
we do not construct perpendiculars, but simply use the edges connecting the
quantizer points to the vertices of the Voronoï cells. We do not have orthosim-
plices, but rather pyramids on top of each of the facets of the Voronoï cells
for 𝑑 ≥ 3. In Tóth’s case for 𝑑 = 2, these are triangles. Tóth showed by
a geometric comparison inequality similar in spirit to Newman’s above in the
proof of Lemma 32 that one could lower bound the NSM about the vertex by
that of an isosceles triangle with the same vertex angle. Our candidate for the
appropriate generalization of the isosceles triangle to 𝑑 ≥ 3 is a right circular
cone of the appropriate solid angle (Conjecture 3). Here, we are able to obtain
much more significant progress, such as a reduction of this Conjecture 3 to a
technical inequality given in Conjecture 5. However, even if we could prove
Conjecture 5, we are still currently defeated by the analog of the concavity of
𝜑 (Lemma 33) for 𝑑 ≥ 3, expressed as Conjecture 4. The rest of the machinery
carries through naturally, with Cauchy-Schwarz replaced by Hölder’s inequal-
ity. The end result of this reasoning is our conjectured lattice NSM lower bound
given by Conjecture 2.
Let us now spell out some of the details. We emphasize the conceptual aspects
and leave the missing technicalities as conjectures (specifically Conjectures 3 and 4).
In the subsequent subsection 4.3.5, we detail some nontrivial progress towards these
conjectures and also justify why they may be verified readily on a computer even if
we do not have a proof yet.
For lattices, one may examine a single Voronoï cell. By examining Z𝑑 mod 2
component-wise, Minkowski [84, §6] and Voronoï [121, §48] independently proved
Lemma 34 (Minkowski/Voronoï upper bound on facet count). For a lattice Λ, the
137
number of facets of a Voronoï cell 𝐹 (Λ) satisfies
𝐹 (Λ) ≤ 𝐹𝑑 , 2(2𝑑 − 1).
In order to prove Lemma 34, we have
Definition 16. A relevant vector 𝑣 for a lattice Λ is a vector in Λ such that 𝑣 is a
normal for one of the faces of the Voronoï cell 𝑉 (0).
We now prove a Lemma that characterizes relevant vectors. One can consult the
originals [84], [121] for a proof. Instead we follow the exposition of the far more
readily available book by Conway and Sloane [30, Thm. 10].
Lemma 35 (Minkowski/Voronoï characterization of relevant vectors). A nonzero
vector 𝑣 ∈ Λ is relevant iff ±𝑣 are the only shortest vectors in the coset 𝑣+ 2Λ. Here,
2Λ is the dilation of Λ by the factor 2.
Proof of Lemma 35. Note that every nonzero 𝑣 ∈ Λ determines a halfspace
𝐻𝑣 , 𝑥 ∈ R𝑑 : ⟨𝑥, 𝑣⟩ ≤ 1
2⟨𝑣, 𝑣⟩,
and the Voronoï cell 𝑉 (0) is the intersection of all 𝐻𝑣. In fact it is the intersection of
just the 𝐻𝑣 where 𝑣 is relevant by the definition of relevance.
Let us prove the “only if” direction first. Suppose 𝑣 ≡ 𝑤 (mod 2Λ), 𝑣 = ±𝑤,
and suppose without loss of generality that |𝑤| ≤ |𝑣|. Then 𝑡 = (𝑣 + 𝑤)/2, 𝑢 =
(𝑣 − 𝑤)/2 are also nonzero vectors in Λ. Now, if 𝑥 ∈ 𝐻𝑡 ∩𝐻𝑢, then we have ⟨𝑥, 𝑡⟩ ≤
(1/2)⟨𝑡, 𝑡⟩, ⟨𝑥, 𝑢⟩ ≤ (1/2)⟨𝑢, 𝑢⟩. Adding, expanding, and using ⟨𝑤,𝑤⟩ ≤ ⟨𝑣, 𝑣⟩, we get
⟨𝑥, 𝑣⟩ ≤ (1/2)⟨𝑣, 𝑣⟩. Thus 𝐻𝑣 is not needed to define the cell 𝑉 (0), and so 𝑣 is not
relevant.
Now let us prove the “if” part. Suppose 𝑣 is not relevant. Then, 𝑣/2 must lie on
or outside some 𝐻𝑤 for a nonzero 𝑤 = 𝑣 ∈ Λ. In other words, ⟨𝑣, 𝑤⟩ ≥ ⟨𝑤,𝑤⟩. This
expression can be rewritten as |𝑣 − 2𝑤|2 ≤ |𝑣|2. But 𝑣 − 2𝑤 = ±𝑣, and it is also in
the coset 𝑣 + 2Λ.
138
Using Lemma 35, we may easily complete the proof of Lemma 34:
Proof of Lemma 34. Let 𝑣1, 𝑣2, . . . , 𝑣𝑑 be a basis for Λ. Then, every 𝑣 ∈ Λ can be
associated with a vector (𝑎1, 𝑎2, . . . , 𝑎𝑑) ∈ Z𝑑 by its basis expansion 𝑣 =∑𝑑
𝑖=1 𝑎𝑖𝑣𝑖.
By the above Lemma 35, we see that we can have at most two relevant vectors
(corresponding to the sign choice in the ±) to each nonzero (mod 2) residue class.
There are 2𝑑 − 1 nonzero residue classes in Z𝑑, so we get the desired 𝐹𝑑 upper bound
on the face counts.
Remark 11. For a proof and generalization of the facet count upper bound to a union
of 𝑘 translates of Λ in English, see e.g., [39, p.181-183]. Alternatively, one can study
the original work of Delone and Sandakova [34]. Another interesting generalization
may be found in Voronoï’s original [121, §65], where he proves the upper bound
𝐾𝑣,𝑑 ≤ (𝑑+ 1 − 𝑣)∆𝑑−𝑣(𝑚𝑑)|𝑚=1,
where 𝐾𝑣,𝑑 is the number of faces of dimension 𝑣 for a Voronoï cell of a lattice Λ ⊂ R𝑑
and ∆(𝑓(𝑥)) = 𝑓(𝑥+ 1) − 𝑓(𝑥) is the finite difference operator. Taking 𝑣 = 𝑑− 1 we
recover the bound on the number of facets, which are nothing but 𝑑 − 1-dimensional
faces.
As an illustration of Lemma 34, consider 𝑑 = 2 where we get an upper bound of
6 on the number of sides of Voronoï polygons of a lattice. A very interesting aspect
that we saw earlier in Lemma 31 is that this upper bound of 6 carries over in the
“averaged” sense to non-lattice 2-dimensional vector quantizers, via an application
of Euler’s theorem for planar graphs. This aspect of Lemma 31 is the basic reason
why Tóth/Newman [116], [88] works as a bound for all quantizers for 𝑑 = 2, not just
lattice ones. We lose this generality when we go to 𝑑 ≥ 3.
The next ingredient we need is our hypothesis that right circular cones have min-
imal NSM about a vertex subject to a solid angle constraint.
Conjecture 3. Let 𝐻 be a fixed hyperplane not passing through 0 and 𝐵 ⊂ 𝐻 a
compact, convex set. Let the solid angle of the pyramid formed by 𝐵 and the vertex
139
0 about 0 be Θ. Then, the NSM of this pyramid is lower bounded by that of a right
circular cone with vertex 0 and the same solid angle Θ.
Let us denote the NSM of a right circular cone with solid angle Θ (in 𝑑 dimensions)
𝜑𝑑(Θ)−1, in analogy with Newman’s argument. As in Newman’s argument, we then
have a concavity hypothesis.
Conjecture 4. 𝜑𝑑(Θ)𝑑2 is concave on (0, 𝐴𝑑/2).
Assuming the veracity of Conjecture 3, we may proceed further with Newman’s
argument by replacing Cauchy-Schwarz with Hölder’s inequality and obtain
(𝑁∑𝑘=1
|𝐵𝑘|1+2𝑑
𝜑𝑑(Θ𝑘)
) 𝑑𝑑+2(
𝑁∑𝑘=1
𝜑𝑑(Θ𝑘)𝑑2
) 2𝑑+2
≥ 1.
Here, we are partitioning the Voronoï cells 𝑉 (𝜆1), 𝑉 (𝜆2), . . . , 𝑉 (𝜆𝑛) for the 𝑛 point
quantizer into pyramids formed by the 𝜆𝑖 and the 𝑑− 1–dimensional facets of 𝑉 (𝜆𝑖),
and call these 𝐵𝑘, 1 ≤ 𝑘 ≤ 𝑁 . Θ𝑘 are the respective solid angles of 𝐵𝑘 with their
associated 𝜆𝑗. The analog of 𝑁 ≤ 12𝑛 − 4 for Newman’s argument (Lemma 31) is
Lemma 34 which shows 𝑁 ≤ 2(2𝑑−1)𝑛+𝑜(𝑛). At 𝑑 = 2, we now have a coefficient of
6 in front of 𝑛 as opposed to 12, this is precisely because our partitioning is different
from that of Newman and is in fact more closely aligned with that of Tóth. The 𝑜(𝑛)
comes simply from boundary effects at the edge of the unit cube [0, 1]𝑑.
Finally, assuming the veracity of the concavity hypothesis (Conjecture 4), we may
then complete the argument for Conjecture 2.
4.3.5 Towards a proof for lattices
At this stage, we hope that the reader is convinced that the beef of our approach
towards lattices lies in establishing Conjectures 3 and 4. We now perform certain
reductions that may assist in proving these conjectures and may clarify certain con-
ceptual aspects. At the very least, they provide a method for computer verification.
Let us draw the normal 𝑛 to the hyperplane 𝐻 passing through the vertex 0, and
parametrize the convex body 𝐵 by a function 𝑓(𝜃), 0 ≤ 𝜃 < 𝜋2, 0 ≤ 𝑓 ≤ 1, and 𝑓
140
nonincreasing with 𝜃. Here, 𝜃 describes the circular locus of points 𝐻𝜃 lying on 𝐻
forming angle 𝜃 with the normal 𝑛. 𝑓(𝜃) denotes the relative measure of the body at
that angle, given by
𝑓(𝜃) =|𝐻𝜃 ∩𝐵||𝐻𝜃|
.
The above definition makes it clear why 0 ≤ 𝑓 ≤ 1. The fact that 𝑓 is nonincreasing
follows at once from the convexity of 𝐵 and the that we may assume without loss
of generality that the foot of 𝑛 on 𝐻 lies inside 𝐵. For if the foot was outside 𝐵,
we may translate 𝐵 so that its center of mass is the foot of the perpendicular and
thereby not increase the NSM.
Let us now write down various quantities of interest in terms of 𝑓 .
First, the solid angle Θ formed at the vertex is
Θ =
∫ 𝜋2
0
𝑓(𝜃)𝐴𝑑−1 sin𝑑−2(𝜃)𝑑𝜃. (4.15)
Next, the volume of the pyramid 𝑉 is
𝑉 =𝑑− 1
𝑑
∫ 𝜋2
0
𝑓(𝜃) tan𝑑−2(𝜃) sec2(𝜃)𝑑𝜃. (4.16)
And finally, the second moment 𝑀 is
𝑀 =𝑑− 1
𝑑+ 2
∫ 𝜋2
0
𝑓(𝜃) tan𝑑−2(𝜃) sec4(𝜃)𝑑𝜃. (4.17)
Observe that 𝑀,𝑉 are linear functionals in 𝑓 . Let us examine
𝑁𝑆𝑀− 𝑑2 =
𝑉𝑑+22
𝑀𝑑2
.
By Hölder’s inequality, it is immediately clear that
(𝑡𝑉1 + 𝑡𝑉2
) 𝑑+22(
𝑡𝑀1 + 𝑡𝑀2
) 𝑑2
≤ 𝑡𝑉
𝑑+22
1
𝑀𝑑21
+ 𝑡𝑉
𝑑+22
2
𝑀𝑑22
,
141
where 𝑡 ∈ [0, 1], 𝑡 = 1 − 𝑡. Thus, 𝑁𝑆𝑀− 𝑑2 is a convex function in 𝑓 , and we are
interested in maximizing it subject to fixed solid angle. Now note that the constraints
on 𝑓 (namely 0 ≤ 𝑓 ≤ 1,∫𝑓(𝜃)𝑑𝜇(𝜃) = 𝑐 for 𝜇 given by (4.15), and 𝑓 nonincreasing)
are all convex constraints, and thus we are maximizing a convex function over a
compact, convex, set. We may apply the classical Krein-Milman Theorem [73] to
therefore reduce the study to the extremal 𝑓 . We may focus on step functions without
loss as their closure coincides with that of the set of 𝑓 that we are interested in. The
extremal step functions are of the form
𝑓(𝑥) =
⎧⎪⎪⎪⎪⎪⎨⎪⎪⎪⎪⎪⎩1, 0 ≤ 𝑥 < 𝑐1,
𝑎, 𝑐1 ≤ 𝑥 < 𝑐2,
0, 𝑐2 ≤ 𝑥,
(4.18)
where 𝑐1 ≥ 0, 𝑐1 ≤ 𝑐2 <𝜋2, 0 < 𝑎 < 1, and 𝑓 must integrate out to the desired solid
angle Θ. At this stage one can certainly simulate the minimal NSM problem on a com-
puter, using the above consideration together with the equations (4.15), (4.16), (4.17).
We have thus numerically verified the truth of Conjecture 3 for various choices of 𝑑.
The concavity hypothesis Conjecture 4 has a direct route to numerical verification,
and we have performed this verification as well.
We now outline a possible route towards an analytical proof, at least for Conjec-
ture 3. A natural guess is that reducing the length of the [𝑐1, 𝑐2] interval in (4.18) and
increasing 𝑎 accordingly could monotonically decrease the NSM as 𝑎 increases. Once
translated into mathematics, we have the following conjecture (that would therefore
imply Conjecture 3). Conjecture 5 given below is one of the technical inequalities
alluded to earlier. The other inequality corresponds to verifying the concavity hy-
pothesis given in Conjecture 4.
Conjecture 5. Let 0 ≤ 𝑎 < 𝑏 < 𝜋2, and let 𝑑 ≥ 2. Then
∫ 𝑏
𝑎sin(𝑡)𝑑−2(sec(𝑏)𝑑+2 − sec(𝑡)𝑑+2)𝑑𝑡∫ 𝑏
𝑎sin(𝑡)𝑑−2(sec(𝑏)𝑑 − sec(𝑡)𝑑)𝑑𝑡
≥(
1 +2
𝑑
) ∫ 𝑏
𝑎sin(𝑡)𝑑−2 sec(𝑡)𝑑+2𝑑𝑡∫ 𝑏
𝑎sin(𝑡)𝑑−2 sec(𝑡)𝑑𝑑𝑡
(4.19)
142
Remark 12. A reader may wonder whether one needs to use the decreasing nature
of 𝑓 , which was slightly less trivial to see than 0 ≤ 𝑓 ≤ 1. Retracing our steps with
this larger convex set instead, one ends up with another inequality of similar flavor
to (4.19), given by
sec(𝑏)𝑑+2 − sec(𝑎)𝑑+2
sec(𝑏)𝑑 − sec(𝑎)𝑑≥ 𝑑+ 2
𝑑
∫ 𝑏
𝑎tan(𝑡)𝑑−2 sec(𝑡)4𝑑𝑡∫ 𝑏
𝑎tan(𝑡)𝑑−2 sec(𝑡)2𝑑𝑡
. (4.20)
Unfortunately, (4.20) is false for 𝑑 > 3, though it does appear to be true for
𝑑 = 2, 3! Geometrically, this means that the NSM of an annular cone does not
necessarily decrease as the annulus is brought towards the normal while keeping the
solid angle fixed for 𝑑 ≥ 4. We do note that, not surprisingly, (4.19) is a weaker
inequality than the (too strong and incorrect) (4.20). The fact that (4.19) can be
deduced from the (incorrect for 𝑑 > 3!) (4.20) comes from the fact that
sec𝑑+2(𝑏) − sec𝑑+2(𝑥)
sec𝑑(𝑏) − sec𝑑(𝑥)
is increasing in 𝑥.
In our attempts to prove Conjecture 5, we attempted proving a more general state-
ment that also appears to be true numerically: Let
𝑓(𝛼, 𝛽; 𝑎, 𝑏) ,
∫ 𝑏
𝑎sin(𝑡)𝛼(sec(𝑏)𝛽 − sec(𝑡)𝛽)𝑑𝑡
𝛽∫ 𝑏
𝑎sin(𝑡)𝛼 sec(𝑡)𝛽𝑑𝑡
. (4.21)
Then the claim is that 𝑓(𝛼, 𝛽; 𝑎, 𝑏) is monotonically increasing in 𝛽. One may also
favor a 𝑢 = sec(𝑡) substitution to convert this general statement into showing mono-
tonicity in 𝛽 for 1 ≤ 𝑎 ≤ 𝑏 <∞ of
𝑔(𝛼, 𝛽; 𝑎, 𝑏) ,
∫ 𝑏
𝑎(𝑢2 − 1)
𝛼−12 𝑢−𝛼−1(𝑏𝛽 − 𝑢𝛽)𝑑𝑢
𝛽∫ 𝑏
𝑎(𝑢2 − 1)
𝛼−12 𝑢𝛽−𝛼−1𝑑𝑢
. (4.22)
Finally, we do note that although our “monotonicity approach” outlined in Con-
jecture 5 does not prove the concavity hypothesis given in Conjecture 4, it does still
143
have at least one interesting consequence other than Conjecture 3, namely that
𝜑𝑑(Θ)𝑑2
Θ
is nonincreasing (take 𝑐1 = 0 in (4.18)). The astute reader may recall this as an
easy consequence of concavity (or equivalently a weaker variant thereof) that was also
utilized in Newman’s proof.
4.4 Discussion and Future Work
Our original impetus for undertaking the study of lower bounds on the minimum
mean squared error of vector quantizers was inspired by the following folklore
Conjecture 6. 𝐸8 and the Leech lattice are optimal vector quantizers in the mean
squared error, high resolution limit sense (𝐺𝑑) for 𝑑 = 8 and 𝑑 = 24 respectively.
Given the relatively recent resolution of the sphere packing problem and associ-
ated universal optimality phenomena for R𝑑, 𝑑 = 8, 24 [120], [27], [28], we believed
it worthwhile to study the main Conjecture 6. We note that there is fairly reason-
able numerical evidence in support of Conjecture 6 in [2], whose main contribution
is a gradient based optimization algorithm for quantizers, different from the classical
Lloyd-Max [79], [81] 2. Essentially, they perform a gradient descent on a parametriza-
tion of the space of lattices, and generalize the approach to quantizers formed by 𝑘
translates of a lattice. For 𝑑 = 8, 24, the algorithm ends up at 𝐸8 and the Leech
lattice for many choices of random starting point.
One may therefore view our contributions here as coming up with a different kind
of evidence based on more rigorous considerations. For example, as Table 4.4 shows,
the scaled NSM for 𝐸8 is ≈ 0.07168, while our conjectured lattice lower bound (4.5) is
≈ 0.07102. For 𝑑 = 24, the gap is larger, with our conjecture giving the lower bound
of ≈ 0.06561, and the Leech lattice yielding a performance of ≈ 0.06577.
2The Lloyd-Max algorithm goes back at least to Steinhaus [109].
144
𝑑 [128] l.b. conj. Λ l.b. (4.5) [31] conj. l.b. u.b. [128] u.b.1 .08333 .08333 .08333 .08333 .500002 .07958 .08019 .08019 .08019 .159153 .07697 .07773 .07787 .07854 .115804 .07503 .07580 .07609 .07660 .099745 .07352 .07425 .07465 .07563 .091326 .07230 .07298 .07347 .07424 .086087 .07130 .07192 .07248 .07273* .082488 .07045 .07102 .07163 .07168 .079829 .06973 .07026 .07090 .07110* .0777810 .06910 .06959 .07026 .07081 .0761416 .06657 .06689 .06759 .06830 .0705324 .06475 .06497 .06561 .06577 .06722∞ .05855 .05855 .05855 .05855 .05855
Table 4.1: Numerical values for various bounds on the NSM (divided by the dimension𝑑) up to a couple of decimal places. Bold face denotes rigorously known sharp values.* denotes non-lattice constructions [2]. “l.b.” and “u.b.” stand for “lower bound” and“upper bound” respectively.
However, there is still a fair bit of work to be done in order to make (4.5) fully
rigorous. The work there has been encapsulated in the form of two Conjectures 3, 4.
It is also clear that we are quite far from establishing the folklore Conjecture 6 that
provides the answer for 𝑑 = 8, 24. We believe that establishing Conjecture 6 requires
new methods, and it would be pleasant if such methods could also extend to other
values of 𝑑 and thereby supersede our conjectured lower bound (4.5), our rigorous
general lower bound 10 (with tuned parameters), and Conway and Sloane’s conjec-
tured lower bound [31]. We believe that this search for new methods may in fact be
the most fruitful approach to establishing these conjectures.
Stepping back from the considerations of mean squared error and the high res-
olution limit, it is interesting to understand to what extent these methods can be
applied to other distortion measures.
145
146
Chapter 5
Conclusion and Future Directions
When someone failed, another has succeeded; what was unknown in one
century, the next has discovered; science and the arts do not grind
themselves into uniformity, but gain shape and regularity by carving and
polishing repeatedly... What my own strength has not been able to
uncover, I cease not from working at and trying out and, by reshaping
and solidifying this new material, in moulding and heating it, I
bequeathe to him who follows some facility and make it the more supple
and malleable for him. The second will do the same for the third, which
is why difficulty does not make me despair, nor of my own weakness...
Michel de Montaigne, Les Essais, Livre II, Chapitre XII
Let us summarize the main contributions of this dissertation at a very high level.
We began by reviewing the notion of codes and anticodes in a metric space in Chap-
ter 2. We then proceeded to review the notions of pairwise potential energy, ground
states, and universal optimality. We also reviewed the notion of noise stability popu-
lar in theoretical computer science. We then reviewed and discussed Fourier analysis
on finite groups, and how one can derive certain linear programming bounds. We
used these bounds to prove that certain natural Boolean functions maximize noise
stability subject to an expected value constraint.
In Chapter 3, we considered the problem of maximizing the quality of reconstruc-
tion for a coded aperture imaging apparatus under a simple model. We described how
147
this problem boils down to the question of how and to what extent can one shape the
magnitudes of the Fourier coefficients on Z/𝑛Z subject to an 𝑙∞ constraint in time.
We thus constructed a link with the “coefficient problem” in harmonic analysis, and
accordingly utilized Nazarov’s solution [86] in the resolution. We also showed how
one can make Nazarov’s solution algorithmically effective.
In Chapter 4, we considered the problem of finding improved lower bounds on
the mean squared error of vector quantizers in the so-called “high-resolution limit”
where one lets the number of quantizer points per unit volume tend to infinity. We
developed two approaches: one to handle lower bounds on lattice quantizers, and the
other to handle lower bounds on general quantizers. The lower bound we obtain for
general quantizers is rigorous, while the one for lattices rests on certain plausible and
easily numerically verified conjectures 3, 4 that we are currently unable to prove.
Throughout this dissertation, we have been guided by a couple of principles and
themes. In particular, we have been inspired by the research philosophy of Yuri
Vladimirovich Linnik. According to [66]:
“Linnik often liked to say that when starting a new area of research one should
select in it a difficult and neatly formulated problem: in trying to solve it, new
problems will crop up and the problem itself will serve as a touchstone for the methods
being used. This would lead step-by-step to the creation of a theory and of general
methods.”
In order to execute upon this program, in this dissertation we focused on topics
and problems with a rich history. This has the positive effect of allowing one to
focus on coming up with syntheses of methods as well as formulating “new” methods.
However, this may come at a cost of not addressing the most relevant problems
of our current era. Our general response to such a criticism is twofold. First, we
believe it shortsighted to lay judgement upon relevance based on our current era
as it is close to impossible to anticipate the future. Second, ultimately progress is
achieved through new methods and ideas. Problems that are classical, difficult, and
neatly formulated have been proven over time to be essentially as fruitful towards this
ultimate goal as the formulation of new fields and disciplines. With this view in mind,
148
historical anecdotes and references are collectively another principal contribution of
this dissertation.
Each chapter of this dissertation closed with its own suggestions for future re-
search, and we do not wish to repeat specific technicalities here. As such, we close
with just a few, very high level, ideas:
1. One key underlying theme here was the use of Fourier analysis to attack prob-
lems of geometric character. Such a program has been pursued since the incep-
tion of Fourier analysis. However, we believe that we are still at a very early
stage here and anticipate substantial advances in the future. For example, the
linear programming bounds have been primarily used for coding theory ques-
tions. We have demonstrated in this dissertation that they can be used for
isodiametric questions (see also [124, 49]), and also vector quantization ques-
tions. Another illustration of the interplay is provided by hypercontractivity, a
topic which we do not explore in this dissertation. We look forward to a syn-
thesized, general point of view that encompasses both the linear programming
bounds and hypercontractivity.
2. Another theme here is the understanding of how “bulk” constraints in time
(such as 𝑙𝑝) translate into “fine-grained” constraints on frequency components.
For example, Nazarov’s theorem refers to the individual frequency components,
and not just their “bulk” characteristics that can be captured by e.g. 𝑙𝑞 norms.
These “bulk” quantities are covered by more classical theory on the 𝑙𝑝 → 𝑙𝑞
operator norms; see for example work on hypercontractivity. Can we obtain
further understanding of the fine-grained structure of the frequency compo-
nents? For example, can we usefully incorporate phase information instead of
just discussing magnitudes?
3. What are the broader scientific and engineering implications of answers and the
search for answers to the preceding items? We have no idea and we generally
devote greater energy to the preceding items instead as they are more easily
formulated. Nevertheless, we hope that the reader pleasantly surprises us!
149
Appendices
150
Appendix A
Proofs for Chapter 3
As remarked in the main text, Lemma 24 is really just a calculus exercise that offers
limited insight. For example, one can reduce this to studying a single variable function
of 𝑎, and finding its maximum. This may be easily done on a computer to whatever
degree of precision is desired, and such a numerical study has been performed in our
code:https://github.com/gajjanag/apertures.
Nevertheless, we give an unenlightening fully rigorous “analytical” proof below for
completeness. This argument naturally did not appear in our paper [8] due these
reasons as well as space constraints.
Proof of Lemma 24. First, one may compute
𝑓 ′𝑎(𝑥) =
−2𝑎𝑥− 𝑥2 + 𝑎
(𝑎+ 𝑥)2,
𝑓 ′′𝑎 (𝑥) = −2𝑎(𝑎+ 1)
(𝑎+ 𝑥)3.
Thus 𝑓𝑎 is a concave function since 𝑎 > 0. Furthermore, 𝑓𝑎(0) = 𝑓𝑎(1) = 0, so in fact
𝑓𝑎(𝑥) attains its maximum precisely at the root of 𝑓 ′𝑎(𝑥) = 0 lying in [0, 1], namely
𝑥* =√𝑎2 + 𝑎− 𝑎. Plugging in this value, we get
𝑓𝑎(𝑥*) = 2𝑎+ 1 − 2
√𝑎(𝑎+ 1). (A.1)
151
Let us first look at
𝑔(𝑎) ,𝑓𝑎(𝑥
*)
𝑓𝑎(12
) .Using (A.1), we have the explicit expression
𝑔(𝑎) = (4𝑎+ 2)(2𝑎+ 1 − 2√𝑎(𝑎+ 1)).
We may differentiate to show that 𝑔 is certainly decreasing, since
𝑔′(𝑎) = −2(2𝑎+ 1 − 2
√𝑎(𝑎+ 1))2√
𝑎(𝑎+ 1)< 0.
Thus, 𝑔(𝑎) < 𝑔(0) = 2. This proves 𝑀(𝑎, 1/2) ≤ 2.
For 𝜌 = 1/4, we have a similar function
ℎ(𝑎) ,𝑓𝑎(𝑥
*)
𝑓𝑎(14
) .Explicitly,
ℎ(𝑎) =1
3(16𝑎+ 4)(2𝑎+ 1 − 2
√𝑎(𝑎+ 1)).
Differentiating, we get
ℎ′(𝑎) = −4(−2𝑎− 1 + 2
√𝑎(𝑎+ 1))(−4𝑎− 1 + 4
√𝑎(𝑎+ 1))
3√𝑎(𝑎+ 1)
.
Examining the signs of the factors, we see that ℎ decreases on [0, 1/8], and then
increases. Thus on [0, 1/8], min(𝑔(𝑎), ℎ(𝑎)) ≤ min(ℎ(0), 𝑔(0)) = 4/3. For 𝑎 > 1/8, we
see that min(𝑔(𝑎), ℎ(𝑎)) ≤ 𝑔 (1/8) = 5/4 < 4/3. This proves 𝑀(𝑎, 1/4, 1/2) ≤ 4/3.
For 𝜌 = 1/8, we have a similar function
𝑘(𝑎) ,𝑓𝑎(𝑥
*)
𝑓𝑎(18
) .Explicitly,
𝑘(𝑎) =1
7(64𝑎+ 8)(2𝑎+ 1 − 2
√𝑎(𝑎+ 1)).
152
Differentiating, we get
𝑘′(𝑎) = −8(−2𝑎− 1 + 2
√𝑎(𝑎+ 1))(−8𝑎− 1 + 8
√𝑎(𝑎+ 1))
7√𝑎(𝑎+ 1)
.
Examining the signs of the factors, we see that 𝑘 decreases on [0, 1/48], and then
increases. Thus on [0, 1/48], min(𝑔(𝑎), ℎ(𝑎), 𝑘(𝑎)) ≤ min(𝑔(0), ℎ(0), 𝑘(0)) = 8/7.
On [1/48, 1/8], min(𝑔(𝑎), ℎ(𝑎), 𝑘(𝑎)) ≤ min(𝑔(1/48), ℎ(1/48)) = 13/12 < 8/7. On
[1/8, 3/4], we see that min(𝑔(𝑎), ℎ(𝑎), 𝑘(𝑎)) ≤ ℎ (3/4) = 1.113 · · · < 8/7 = 1.14 . . . .
For 𝑎 > 3/4, we see that min(𝑔(𝑎), ℎ(𝑎), 𝑘(𝑎)) ≤ 𝑔 (3/4) = 1.043 · · · < 8/7. This
proves 𝑀(𝑎, 1/8, 1/4, 1/2) ≤ 8/7. Note that there was nothing special about 3/4 in
the above proof. Any number in a certain interval tuned appropriately to the above
argument would work.
153
154
Bibliography
[1] J. G. Ables. Fourier transform photography: a new method for X-ray astronomy.Publications of the Astronomical Society of Australia, 1(4):172–173, 1968.
[2] Erik Agrell and Thomas Eriksson. Optimization of lattices for quantization.IEEE Transactions on Information Theory, 44(5):1814–1828, 1998.
[3] Rudolf Ahlswede. Towards a general theory of information transfer. https://www.youtube.com/watch?v=uQZBlcSH6gs, July 2006.
[4] Rudolf Ahlswede, Harout K. Aydinian, and Levon H. Khachatrian. On perfectcodes and related concepts. Designs, Codes and Cryptography, 22(3):221–237,2001.
[5] Rudolf Ahlswede and Vladimir Blinovsky. Lectures on advances in combina-torics. Springer, 2008.
[6] Rudolf Ahlswede and Levon H. Khachatrian. The complete intersection theoremfor systems of finite sets. European Journal of Combinatorics, 18(2):125–136,1997.
[7] Rudolf Ahlswede and Levon H. Khachatrian. The diametric theorem in Ham-ming spaces—optimal anticodes. Advances in Applied Mathematics, 20(4):429–449, 1998.
[8] G. Ajjanagadde, C. Thrampoulidis, A. Yedidia, and G. Wornell. Near-optimalcoded apertures for imaging via Nazarov’s theorem. In ICASSP 2019 - 2019IEEE International Conference on Acoustics, Speech and Signal Processing(ICASSP), pages 7690–7694, May 2019.
[9] Omer Angel, Sébastien Bubeck, Yuval Peres, and Fan Wei. Local max-cut insmoothed polynomial time. In Proceedings of the 49th Annual ACM SIGACTSymposium on Theory of Computing, pages 429–437. ACM, 2017.
[10] Tom M. Apostol. Introduction to analytic number theory. Springer Science &Business Media, 2013.
[11] M. Salman Asif, Ali Ayremlou, Aswin Sankaranarayanan, Ashok Veeraragha-van, and Richard G Baraniuk. Flatcam: Thin, lensless cameras using coded
155
aperture and computation. IEEE Transactions on Computational Imaging,3(3):384–397, 2017.
[12] Konstantin Ivanovich Babenko. An inequality in the theory of Fourier integrals.Izvestiya Rossiiskoi Akademii Nauk. Seriya Matematicheskaya, 25(4):531–542,1961.
[13] Paul Balister, Béla Bollobás, Robert Morris, Julian Sahasrabudhe, and MariusTiba. Flat Littlewood polynomials exist, 2019.
[14] Keith Ball. A lower bound for the optimal density of lattice packings. Interna-tional Mathematics Research Notices, 1992(10):217–221, 05 1992.
[15] Keith Ball. Convex geometry and functional analysis. Handbook of the geometryof Banach spaces, 1:161–194, 2001.
[16] Thøger Bang. A solution of the “plank problem”. Proceedings of the AmericanMathematical Society, 2(6):990–993, 1951.
[17] William Beckner. Inequalities in Fourier analysis. Annals of Mathematics,102(1):159–182, 1975.
[18] Itai Benjamini, Gil Kalai, and Oded Schramm. Noise sensitivity of Boolean func-tions and applications to percolation. Publications Mathématiques de l’Institutdes Hautes Études Scientifiques, 90(1):5–43, 1999.
[19] W. R. Bennett. Spectra of quantized signals. The Bell System Technical Jour-nal, 27(3):446–472, July 1948.
[20] Hans F. Blichfeldt. The minimum value of quadratic forms, and the closestpacking of spheres. Mathematische Annalen, 101(1):605–608, 1929.
[21] Holger Boche and Ezra Tampubolon. Mathematics of signal design for com-munication systems. In Mathematics and Society, pages 185–220. EuropeanMathematical Society Publishing House, 2016.
[22] Salomon Bochner. Vorlesungen über Fouriersche Integrale. Akademische Ver-lagsgesellschaft, 1932.
[23] Christopher M. Brown. Multiplex imaging and random arrays. PhD thesis,University of Chicago, 1972.
[24] S. Chowla. A property of biquadratic residues. Proc. Nat. Acad. Sci. India.Sect. A., 14:45–46, 1944.
[25] Adam Lloyd Cohen. Anti-pinhole imaging. Optica Acta: International Journalof Optics, 29(1):63–67, 1982.
[26] Henry Cohn and Abhinav Kumar. Universally optimal distribution of pointson spheres. Journal of the American Mathematical Society, 20(1):99–148, 2007.
156
[27] Henry Cohn, Abhinav Kumar, Stephen D. Miller, Danylo Radchenko, andMaryna Viazovska. The sphere packing problem in dimension 24. Annals ofMathematics, 185(3):1017–1033, 2017.
[28] Henry Cohn, Abhinav Kumar, Stephen D. Miller, Danylo Radchenko, andMaryna Viazovska. Universal optimality of the 𝐸8 and Leech lattices and in-terpolation formulas, 2019.
[29] Henry Cohn and Yufei Zhao. Energy-minimizing error-correcting codes. IEEETransactions on Information Theory, 60(12):7442–7450, 2014.
[30] J. H. Conway and N. J. A. Sloane. Sphere Packings, Lattices and Groups.Springer New York, New York, NY, 1999.
[31] John Conway and Neil Sloane. A lower bound on the average error of vectorquantizers (corresp.). IEEE Transactions on Information Theory, 31(1):106–109, 1985.
[32] Harold Davenport. Multiplicative number theory, volume 74. Springer Science& Business Media, 2013.
[33] Karel De Leeuw, Yitzhak Katznelson, and Jean-Pierre Kahane. Sur les co-efficients de Fourier des fonctions continues. CR Acad. Sci. Paris Sér. AB,285(16):A1001–A1003, 1977.
[34] Boris Nikolaevich Delone and Nina Nikolaevna Sandakova. Theory of stereohe-dra. Trudy Matematicheskogo Instituta imeni VA Steklova, 64:28–51, 1961. InRussian.
[35] Philippe Delsarte. An algebraic approach to the association schemes of codingtheory. Philips Res. Reports Suppls., 10, 1973.
[36] Philippe Delsarte and Vladimir I. Levenshtein. Association schemes and codingtheory. IEEE Transactions on Information Theory, 44(6):2477–2504, 1998.
[37] R.H. Dicke. Scatter-hole cameras for X-rays and gamma rays. The astrophysicaljournal, 153:L101, 1968.
[38] L. E. Dickson. Cyclotomy, higher congruences, and Waring’s problem. AmericanJournal of Mathematics, 57(2):391–424, 1935.
[39] Nikolai Dolbillin and Masaharu Tanemura. How many facets on average can atile have in a tiling. Forma, 21(3):177–196, 2006.
[40] Marco F. Duarte, Mark A. Davenport, Dharmpal Takhar, Jason N. Laska,Ting Sun, Kevin F. Kelly, and Richard G. Baraniuk. Single-pixel imaging viacompressive sampling. IEEE Signal Processing Magazine, 25(2):83–91, 2008.
[41] Paul Erdős. Some unsolved problems. Michigan Math. J., 4(3):291–300, 1957.
157
[42] Paul Erdős. Intersection theorems for systems of finite sets. Quart. J. Math.Oxford Ser.(2), 12:313–320, 1961.
[43] L. Fejes Tóth. Sur la représentation d’une population infinie par un nombre finid’éléments. Acta Mathematica Academiae Scientiarum Hungarica, 10(3):299–304, Sep 1959.
[44] Edward E. Fenimore and Thomas M. Cannon. Coded aperture imaging withuniformly redundant arrays. Applied optics, 17(3):337–347, 1978.
[45] Richard P. Feynman. Feynman lectures on physics. Volume 2: Mainly elec-tromagnetism and matter. Reading, Ma.: Addison-Wesley, 1964, edited byFeynman, Richard P.; Leighton, Robert B.; Sands, Matthew, 1964.
[46] Yuval Filmus. The weighted complete intersection theorem. Journal of Combi-natorial Theory, Series A, 151:84–101, 2017.
[47] G. David Forney. Transforms and Groups, pages 79–97. Springer US, Boston,MA, 1998.
[48] Jean-Baptiste Joseph Fourier. Théorie analytique de la chaleur. F. Didot, 1822.
[49] Péter Frankl and Richard M. Wilson. The Erdős-Ko-Rado theorem for vectorspaces. Journal of Combinatorial Theory, Series A, 43(2):228–236, 1986.
[50] Fang-Wei Fu, Victor K. Wei, and Raymond W. Yeung. On the minimum aver-age distance of binary codes: linear programming approach. Discrete AppliedMathematics, 111(3):263–281, 2001.
[51] Carl Friedrich Gauß. Disquisitiones Arithmeticae. Lipsiae In Commissis ApudGerh. Fleischer Jux., 1801.
[52] Allen Gersho and Robert M. Gray. Vector quantization and signal compression,volume 159. Springer Science & Business Media, 2012.
[53] Dion Gijswijt, Alexander Schrijver, and Hajime Tanaka. New upper bounds fornonbinary codes based on the Terwilliger algebra and semidefinite programming.Journal of Combinatorial Theory, Series A, 113(8):1719–1731, 2006.
[54] Ambros Gleixner, Leon Eifler, Tristan Gally, Gerald Gamrath, Patrick Geman-der, Robert Lion Gottwald, Gregor Hendel, Christopher Hojny, Thorsten Koch,Matthias Miltenberger, Benjamin Müller, Marc E. Pfetsch, Christian Puchert,Daniel Rehfeldt, Franziska Schlösser, Felipe Serrano, Yuji Shinano, Jan MerlinViernickel, Stefan Vigerske, Dieter Weninger, Jonas T. Witt, and Jakob Witzig.The SCIP optimization suite 5.0. Technical Report 17-61, ZIB, Takustr. 7,14195 Berlin, 2017.
[55] Ambros M. Gleixner, Daniel E. Steffy, and Kati Wolter. Iterative refinementfor linear programming. Technical Report 15-15, ZIB, Takustr. 7, 14195 Berlin,2015.
158
[56] Ronald L. Graham, Donald E. Knuth, and Oren Patashnik. Concrete Math-ematics: A Foundation for Computer Science (2nd Edition). Addison-WesleyProfessional, 2 edition, 3 1994.
[57] R. M. Gray and D. L. Neuhoff. Quantization. IEEE Transactions on Informa-tion Theory, 44(6):2325–2383, Oct 1998.
[58] Ben Green. Spectral structure of sets of integers. In Fourier analysis andconvexity, pages 83–96. Springer, 2004.
[59] Dongning Guo, Shlomo Shamai, and Sergio Verdú. Mutual information andminimum mean-square error in Gaussian channels. IEEE Transactions on In-formation Theory, 51(4):1261–1282, 2005.
[60] Lawrence Hueston Harper. Optimal assignments of numbers to vertices. Journalof the Society for Industrial and Applied Mathematics, 12(1):131–135, 1964.
[61] F. Hausdorff. Eine ausdehnung des parsevalschen satzes über fourierreihen.Mathematische Zeitschrift, 16:163–169, 1923.
[62] G. Herglotz. Über potenzreihen mit positivem, reelen teil im einheitskreis. Ber.Verhandl. Sachs Akad. Wiss. Leipzig, Math.-Phys. Kl., 63:501–511, 1911.
[63] Otto Hölder. Über einen mittelwertsatz. Göttinger Nacgrichten, pages 38–47,1889.
[64] Jerry Lee Holsinger. Digital communication over fixed time-continuous channelswith memory-with special application to telephone channels. Research Labora-tory of Electronics Technical Report, 430, 1964.
[65] J. Huang and P. Schultheiss. Block quantization of correlated Gaussian ran-dom variables. IEEE Transactions on Communications Systems, 11(3):289–296,1963.
[66] I. A. Ibragimov, S. M. Lozinskii, A. V. Malyshev, V. V. Petrov, Yu. V.Prokhorov, N. A. Sapogov, and D. K. Faddeev. Yurii Vladimirovich Linnik(obituary). Russian Mathematical Surveys, 28(2):197–215, April 1973.
[67] R. C. Jennison. Fourier transforms and convolutions for the experimentalist.Pergamon Press, Oxford, 1961.
[68] Steven Johnson. The ghost map: The story of London’s most terrifyingepidemic—and how it changed science, cities, and the modern world. Penguin,2006.
[69] Grigorii Anatol’evich Kabatyanskii and Vladimir Iosifovich Levenshtein. Onbounds for packings on a sphere and in space. Problemy Peredachi Informatsii,14(1):3–25, 1978.
159
[70] Yitzhak Katznelson. An Introduction to Harmonic Analysis. Cambridge Math-ematical Library. Cambridge University Press, 3 edition, 2004.
[71] Daniel J. Kleitman. On a combinatorial conjecture of Erdős. Journal of Com-binatorial Theory, 1(2):209–214, 1966.
[72] Emmanuel Kowalski. An introduction to the representation theory of groups,volume 155. American Mathematical Society, 2014.
[73] Mark Krein and David Milman. On extreme points of regular convex sets.Studia Mathematica, 9(1):133–138, 1940.
[74] Emma Lehmer. On residue difference sets. Canad. J. Math, 5:425–432, 1953.
[75] Anat Levin, Rob Fergus, Frédo Durand, and William T. Freeman. Image anddepth from a conventional camera with a coded aperture. ACM transactionson graphics (TOG), 26(3):70, 2007.
[76] John H. Lindsey. Assignment of numbers to vertices. The American Mathe-matical Monthly, 71(5):508–516, 1964.
[77] J. E. Littlewood. On Polynomials∑𝑛±𝑧𝑚,
∑𝑛 𝑒𝛼𝑚𝑖𝑧𝑚, 𝑧 = 𝑒𝜃𝑖. Journal of theLondon Mathematical Society, s1-41(1):367–376, 01 1966.
[78] John Edensor Littlewood. Some problems in real and complex analysis. DCHeath, 1968.
[79] Stuart P. Lloyd. Least squares quantization in PCM, 1957. unpublished BellLab. Techn. Note, portions presented at the Institute of Mathematical StatisticsMeet., Atlantic City, NJ, Sept. 1957. Also, IEEE Trans. Inform. Theory (SpecialIssue on Quantization), vol. IT-28, pp. 129-137, Mar. 1982.
[80] David G. Luenberger. Optimization by vector space methods. Decision andcontrol. Wiley, New York, NY, 1969.
[81] Joel Max. Quantizing for minimum distortion. IRE Transactions on Informa-tion Theory, 6(1):7–12, 1960.
[82] Robert McEliece, Eugene Rodemich, Howard Rumsey, and Lloyd Welch. Newupper bounds on the rate of a code via the Delsarte-MacWilliams inequalities.IEEE Transactions on Information Theory, 23(2):157–166, 1977.
[83] R. P. Millane, S. Alzaidi, and W. H. Hsiao. Scaling and power spectra of naturalimages. In Proc. Image and Vision Computing New Zealand, pages 148–153,2003.
[84] H. Minkowski. Allgemeine Lehrsätze über die convexen Polyeder. Nachr. Ges.Wiss. Göttingen, Math.-Phys. Kl., 1897:198–219, 1897.
160
[85] Maho Nakata, Bastiaan J. Braams, Katsuki Fujisawa, Mituhiro Fukuda,Jerome K. Percus, Makoto Yamashita, and Zhengji Zhao. Variational calcu-lation of second-order reduced density matrices by strong 𝑁 -representabilityconditions and an accurate semidefinite programming solver. The Journal ofChemical Physics, 128(16):164113, 2008.
[86] Fedor L’vovich Nazarov. The Bang solution of the coefficient problem. Algebrai Analiz, 9(2):272–287, 1997. English translation in St. Petersburg Math. J. 9(1998), no. 2, 407-419.
[87] Joseph Needham. Science and Civilisation in China: Physics and physicaltechnology: pt. 1. Physics, with the collaboration of Wang Ling and the specialco-operation of Kenneth Girdwood Robinson, volume 4. University Press, 1954.
[88] Donald Newman. The hexagon theorem. IEEE Transactions on informationtheory, 28(2):137–139, 1982.
[89] Ryan O’Donnell. Analysis of Boolean functions. Cambridge University Press,2014.
[90] B. M. Oliver, J. R. Pierce, and C. E. Shannon. The philosophy of PCM. Pro-ceedings of the IRE, 36(11):1324–1331, Nov 1948.
[91] P. F. Panter and W. Dite. Quantization distortion in pulse-count modulationwith nonuniform spacing of levels. Proceedings of the IRE, 39(1):44–48, Jan1951.
[92] Yury Polyanskiy and Yihong Wu. Lecture Notes on Information The-ory. http://people.lids.mit.edu/yp/homepage/data/itlectures_v5.pdf,2018. [Online; accessed 21-September-2018].
[93] John G. Proakis and Masoud Salehi. Digital communications, volume 4.McGraw-Hill New York, 2001.
[94] R. A. Rankin. On the closest packing of spheres in 𝑛 dimensions. Annals ofMathematics, 48(4):1062–1081, 1947.
[95] Ramesh Raskar, Amit Agrawal, and Jack Tumblin. Coded exposure photogra-phy: motion deblurring using fluttered shutter. ACM Transactions on Graphics(TOG), 25(3):795–804, 2006.
[96] Frédéric Riesz. Sur certains systèmes singuliers d’équations intégrales. Annalesscientifiques de l’École Normale Supérieure, 28:33–62, 1911.
[97] Alain Robert. Introduction to the Representation Theory of Compact and Lo-cally Compact Groups. London Mathematical Society Lecture Note Series. Cam-bridge University Press, 1983.
[98] C. A. Rogers. Lattice coverings of space. Mathematika, 6(1):33–39, 1959.
161
[99] Leonhard James Rogers. An extension of a certain theorem in inequalities.Messenger of Math., 17:145–150, 1888.
[100] Walter Rudin. Principles of mathematical analysis. McGraw-hill New York, 3edition, 1976.
[101] R. Salem and A. Zygmund. Some properties of trigonometric series whose termshave random signs. Acta Mathematica, 91(1):245–301, Dec 1954.
[102] Alexander Schrijver. New code upper bounds from the Terwilliger algebraand semidefinite programming. IEEE Transactions on Information Theory,51(8):2859–2866, 2005.
[103] Jean-Pierre Serre. Linear representations of finite groups, volume 42. Springer,1977.
[104] Igor Shinkar. Intersecting families, independent sets and coloring of certaingraph products. Master’s thesis, Weizmann Institute of Science, 2009.
[105] Daniel Spielman and Shang-Hua Teng. Smoothed analysis of algorithms: Whythe simplex algorithm usually takes polynomial time. In Proceedings of thethirty-third annual ACM symposium on Theory of computing, pages 296–305.ACM, 2001.
[106] A. J. Stam. Some inequalities satisfied by the quantities of information of Fisherand Shannon. Information and Control, 2(2):101–112, 1959.
[107] Elias M. Stein and Rami Shakarchi. Fourier analysis: an introduction, volume 1of Princeton Lectures in Analysis. Princeton University Press, 2011.
[108] William Stein. Elementary number theory: primes, congruences, and secrets: acomputational approach. Springer Science & Business Media, 2008.
[109] Hugo Steinhaus. Sur la division des corp materiels en parties. Bull. Acad.Polon. Sci, IV(C1. III):801–804, 1956.
[110] Michel Talagrand. How much are increasing sets positively correlated? Com-binatorica, 16(2):243–258, 1996.
[111] Terence Tao. The Euler-Maclaurin formula, Bernoulli numbers, the zeta func-tion, and real-variable analytic continuation. https://tinyurl.com/ybweghs5.[Online; accessed 02-October-2018].
[112] Audrey Terras. Fourier Analysis on Finite Groups and Applications. LondonMathematical Society Student Texts. Cambridge University Press, 1999.
[113] Christos Thrampoulidis, Gal Shulkind, Feihu Xu, William T. Freeman, Jef-frey H. Shapiro, Antonio Torralba, Franco N. C. Wong, and Gregory W. Wor-nell. Exploiting occlusion in non-line-of-sight active imaging. IEEE Transac-tions on Computational Imaging, 4(3):419–431, 2018.
162
[114] Kit Tiyapan. Voronoi Translated: Introduction to Voronoi Tessellation andEssays by G.L. Dirichlet and G.F. Voronoi. God’s Ayudhya’s Defence, 2010.
[115] Antonio Torralba and William T. Freeman. Accidental pinhole and pinspeckcameras: Revealing the scene outside the picture. In Computer Vision andPattern Recognition (CVPR), 2012 IEEE Conference on, pages 374–381. IEEE,2012.
[116] László Fejes Tóth. Lagerungen in der Ebene auf der Kugel und im Raum.Springer Berlin Heidelberg, Berlin, Heidelberg, 1953.
[117] Stefan Kohl (https://mathoverflow.net/users/28104/stefan-kohl). Exis-tence of polynomials of degree ≥ 2 which represent infinitely many prime num-bers. https://mathoverflow.net/q/208614. [Online; accessed 02-October-2018].
[118] Jacobus Hendricus Van Lint. Introduction to coding theory, volume 86. SpringerScience & Business Media, 2012.
[119] Ashok Veeraraghavan, Ramesh Raskar, Amit Agrawal, Ankit Mohan, and JackTumblin. Dappled photography: Mask enhanced cameras for heterodyned lightfields and coded aperture refocusing. In ACM transactions on graphics (TOG),volume 26, page 69. ACM, 2007.
[120] Maryna S. Viazovska. The sphere packing problem in dimension 8. Annals ofMathematics, 185(3):991–1015, 2017.
[121] Georges Voronoï. Nouvelles applications des paramètres continus à la théoriedes formes quadratiques. II: Recherches sur les paralléloèdres primitifs. J. ReineAngew. Math., 134:198–287, 1908.
[122] Georges Voronoï. Nouvelles applications des paramètres continus à la théoriedes formes quadratiques. premier mémoire. sur quelques propriétés des formesquadratiques positives parfaites. J. Reine Angew. Math., 133:97–102, 1908.
[123] Georges Voronoï. Nouvelles applications des paramètres continus à théorie desformes quadratiques. deuxième mémoire. recherches sur les paralléloèdres prim-itifs. Seconde partie. J. Reine Angew. Math., 136:67–182, 1909.
[124] Richard M. Wilson. The exact bound in the Erdős-Ko-Rado theorem. Combi-natorica, 4(2-3):247–257, 1984.
[125] Adam Yedidia, Christos Thrampoulidis, and Gregory Wornell. Analysis andoptimization of aperture design in computational imaging. In 2018 IEEE In-ternational Conference on Acoustics, Speech and Signal Processing (ICASSP),pages 4029–4033. IEEE, 2018.
[126] M. Young. Pinhole optics. Applied Optics, 10(12):2763–2767, 1971.
163
[127] W. H. Young. On the Determination of the Summability of a Function by Meansof its Fourier Constants. Proceedings of the London Mathematical Society, s2-12(1):71–88, 01 1913.
[128] P. Zador. Development and evaluation of procedures for quantizing multivariatedistributions. PhD thesis, Stanford University, 1963.
[129] P. Zador. Asymptotic quantization error of continuous signals and the quanti-zation dimension. IEEE Transactions on Information Theory, 28(2):139–149,March 1982.
[130] R. Zamir and M. Feder. On lattice quantization noise. IEEE Transactions onInformation Theory, 42(4):1152–1159, July 1996.
[131] Changyin Zhou, Stephen Lin, and Shree Nayar. Coded aperture pairs for depthfrom defocus and defocus deblurring. International journal of computer vision,93(1):53–72, 2011.
164