Decomposing a factorial into large factors (second version)

Boris Alexeev, Evan Conway, Matthieu Rosenfeld, Andrew Sutherland, Markus Uhr, Kevin Ventullo, and I have uploaded to the arXiv a second version of our paper “Decomposing a factorial into large factors“. This is a completely rewritten and expanded version of a previous paper of the same name. Thanks to many additional theoretical and numerical contributors from the other coauthors, we now have much more precise control on the main quantity studied in this paper, allowing us to settle all the previous conjectures about this quantity in the literature.

As discussed in the previous post, denotes the largest integer such that the factorial can be expressed as a product of factors, each of which is at least . Computing is a special case of the bin covering problem, which is known to be NP-hard in general; and prior to our work, was only computed for ; we have been able to compute for all . In fact, we can get surprisingly sharp upper and lower bounds on for much larger , with a precise asymptotic

for an explicit constant , which we conjecture to be improvable to

for an explicit constant : … For instance, we can demonstrate numerically that

As a consequence of this precision, we can verify several conjectures of Guy and Selfridge, namely

• for all .
• for all .
• for all . (In fact we show this is true for , and that this threshold is best possible.)
  

Guy and Selfridge also claimed that one can establish for all large purely by rearranging factors of and from the standard factorization of , but surprisingly we found that this claim (barely) fails for all :

The accuracy of our bounds comes from several techniques:

• Greedy algorithms, in which one allocates the largest prime factors of first and then moves to smaller primes, provide quickly computable, though suboptimal, lower bounds on for small, medium, and moderately large values;
• Linear programming and integer programming methods provides extremely accurate upper and lower bounds on for small and medium values of ;
• Rearrangement methods can be analyzed asymptotically via linear programming, and work well for large ; and
• The modified approximate factorization strategy, discussed in the previous post is now sharpened by using -smooth numbers (products of and ) as the primary “liquidity pool” to reallocate factors of , as opposed to the previous approach of only using powers of .
  

To me, the biggest surprise was just how stunningly accurate the linear programming methods were; the very large number of repeated prime factors here actually make this discrete problem behave rather like a continuous one.

(Comment on this)

On the number of exceptional intervals to the prime number theorem in short intervals

Ayla Gafni and I have just uploaded to the arXiv the paper “On the number of exceptional intervals to the prime number theorem in short intervals“. This paper makes explicit some relationships between zero density theorems and prime number theorems in short intervals which were somewhat implicit in the literature at present.

Zero density theorems are estimates of the form

for various , where is a parameter going to infinity, counts the number of zeroes of the Riemann zeta function of real part at least and imaginary part between and , and is an exponent which one would like to be as small as possible. The Riemann hypothesis would allow one to take for any , but this is an unrealistic goal, and in practice one would be happy with some non-trivial upper bounds on . A key target here is the density hypothesis that asserts that for all (this is in some sense sharp because the Riemann-von Mangoldt formula implies that ); this hypothesis is currently known for and , but the known bounds are not strong enough to establish this hypothesis in the remaining region. However, there was a recent advance of Guth and Maynard, which among other things improved the upper bound on from to , marking the first improvement in this bound in over four decades. Here is a plot of the best known upper bounds on , either unconditionally, assuming the density hypothesis, or the stronger Lindelöf hypothesis:

One of the reasons we care about zero density theorems is that they allow one to localize the prime number theorem to short intervals. In particular, if we have the uniform bound for all , then this leads to the prime number theorem

holding for all if , and for almost all (possibly excluding a set of density zero) if . For instance, the Guth-Maynard results give a prime number theorem in almost all short intervals for as small as , and the density hypotheis would lower this just to .

However, one can ask about more information on this exceptional set, in particular to bound its “dimension” , which roughly speaking amounts to getting an upper bound of on the size of the exceptional set in any large interval . Based on the above assertions, one expects to only be bounded by for , be bounded by for , but have some intermediate bound for the remaining exponents.

This type of question had been studied in the past, most direclty by Bazzanella and Perelli, although there is earlier work by many authors om some related quantities (such as the second moment of prime gaps) by such authors as Selberg and Heath-Brown. In most of these works, the best available zero density estimates at that time were used to obtain specific bounds on quantities such as , but the numerology was usually tuned to those specific estimates, with the consequence being that when newer zero density estimates were discovered, one could not readily update these bounds to match. In this paper we abstract out the arguments from previous work (largely based on the explicit formula for the primes and the second moment method) to obtain an explicit relationship between and , namely that

where

Actually, by also utilizing fourth moment methods, we obtain a stronger bound

where

and is the exponent in “additive energy zero density theorems”

where is similar to , but bounds the “additive energy” of zeroes rather than just their cardinality. Such bounds have appeared in the literature since the work of Heath-Brown, and are for instance a key ingredient in the recent work of Guth and Maynard. Here are the current best known bounds:

These explicit relationships between exponents are perfectly suited for the recently launched Analytic Number Theory Exponent Database (ANTEDB) (discussed previously here), and have been uploaded to that site.

This formula is moderately complicated (basically an elaborate variant of a Legendre transform), but easy to calculate numerically with a computer program. Here is the resulting bound on unconditionally and under the density hypothesis (together with a previous bound of Bazzanella and Perelli for comparison, where the range had to be restricted due to a gap in the argument we discovered while trying to reproduce their results):

For comparison, here is the situation assuming strong conjectures such as the density hypothesis, Lindelof hypothesis, or Riemann hypothesis:

(Comment on this)

A Lean companion to “Analysis I”

Almost 20 years ago, I wrote a textbook in real analysis called “Analysis I“. It was intended to complement the many good available analysis textbooks out there by focusing more on foundational issues, such as the construction of the natural numbers, integers, rational numbers, and reals, as well as providing enough set theory and logic to allow students to develop proofs at high levels of rigor.

While some proof assistants such as Coq or Agda were well established when the book was written, formal verification was not on my radar at the time. However, now that I have had some experience with this subject, I realize that the content of this book is in fact very compatible with such proof assistants; in particular, the ‘naive type theory’ that I was implicitly using to do things like construct the standard number systems, dovetails well with the dependent type theory of Lean (which, among other things, has excellent support for quotient types).

I have therefore decided to launch a Lean companion to “Analysis I”, which is a “translation” of many of the definitions, theorems, and exercises of the text into Lean. In particular, this gives an alternate way to perform the exercises in the book, by instead filling in the corresponding “sorries” in the Lean code. (I do not however plan on hosting “official” solutions to the exercises in this companion; instead, feel free to create forks of the repository in which these sorries are filled in.)

Currently, the following sections of the text have been translated into Lean:

• Section 2.1: The natural numbers
• Section 2.2: Addition
• Section 2.3: Multiplication
• Chapter 2 epilogue: Isomorphism with the Mathlib natural numbers
• Section 3.1: Basic set theory
• Section 4.1: The integers

The formalization has been deliberately designed to be separate from the standard Lean math library Mathlib at some places, but reliant on it at others. For instance, Mathlib already has a standard notion of the natural numbers . In the Lean formalization, I first develop “by hand” an alternate construction Chapter2.Nat of the natural numbers (or just Nat, if one is working in the Chapter2 namespace), setting up many of the basic results about these alternate natural numbers which parallel similar lemmas about that are already in Mathlib (but with many of these lemmas set as exercises to the reader, with the proofs currently replaced with “sorries”). Then, in an epilogue section, isomorphisms between these alternate natural numbers and the Mathlib natural numbers are established (or more precisely, set as exercises). From that point on, the Chapter 2 natural numbers are deprecated, and the Mathlib natural numbers are used instead. I intend to continue this general pattern throughout the book, so that as one advances into later chapters, one increasingly relies on Mathlib’s definitions and functions, rather than directly referring to any counterparts from earlier chapters. As such, this companion could also be used as an introduction to Lean and Mathlib as well as to real analysis (somewhat in the spirit of the “Natural number game“, which in fact has significant thematic overlap with Chapter 2 of my text).

The code in this repository compiles in Lean, but I have not tested whether all of the (numerous) “sorries” in the code can actually be filled (i.e., if all the exercises can actually be solved in Lean). I would be interested in having volunteers “playtest” the companion to see if this can actually be done (and if the helper lemmas or “API” provided in the Lean files are sufficient to fill in the sorries in a conceptually straightforward manner without having to rely on more esoteric Lean programming techniques). Any other feedback will of course also be welcome.

[UPDATE, May 31: moved the companion to a standalone repository.]

(Comment on this)

Some variants of the periodic tiling conjecture

Rachel Greenfeld and I have just uploaded to the arXiv our paper Some variants of the periodic tiling conjecture. This paper explores variants of the periodic tiling phenomenon that, in some cases, a tile that can translationally tile a group, must also be able to translationally tile the group periodically. For instance, for a given discrete abelian group , consider the following question:

Question 1 (Periodic tiling question) Let be a finite subset of . If there is a solution to the tiling equation , must there exist a periodic solution to the same equation ?

We know that the answer to this question is positive for finite groups (trivially, since all sets are periodic in this case), one-dimensional groups with finite, and in , but it can fail for for certain finite , and also for for sufficiently large ; see this previous blog post for more discussion. But now one can consider other variants of this question:

• Instead of considering level one tilings , one can consider level tilings for a given natural number (so that every point in is covered by exactly translates of ), or more generally for some periodic function .
• Instead of requiring and to be indicator functions, one can allow these functions to be integer-valued, thus we are now studying convolution equations where are given integer-valued functions (with periodic and finitely supported).

We are able to obtain positive answers to three such analogues of the periodic tiling conjecture for three cases of this question. The first result (which was kindly shared with us by Tim Austin), concerns the homogeneous problem . Here the results are very satisfactory:

Theorem 2 (First periodic tiling result) Let be a discrete abelian group, and let be integer-valued and finitely supported. Then the following are equivalent.

• (i) There exists an integer-valued solution to that is not identically zero.
• (ii) There exists a periodic integer-valued solution to that is not identically zero.
• (iii) There is a vanishing Fourier coefficient for some non-trivial character of finite order.

By combining this result with an old result of Henry Mann about sums of roots of unity, as well as an even older decidability result of Wanda Szmielew, we obtain

Corollary 3 Any of the statements (i), (ii), (iii) is algorithmically decidable; there is an algorithm that, when given and as input, determines in finite time whether any of these assertions hold.

Now we turn to the inhomogeneous problem in , which is the first difficult case (periodic tiling type results are easy to establish in one dimension, and trivial in zero dimensions). Here we have two results:

Theorem 4 (Second periodic tiling result) Let , let be periodic, and let be integer-valued and finitely supported. Then the following are equivalent.

• (i) There exists an integer-valued solution to .
• (ii) There exists a periodic integer-valued solution to .

Theorem 5 (Third periodic tiling result) Let , let be periodic, and let be integer-valued and finitely supported. Then the following are equivalent.

• (i) There exists an indicator function solution to .
• (ii) There exists a periodic indicator function solution to .

In particular, the previously established case of periodic tiling conjecture for level one tilings of , is now extended to higher level. By an old argument of Hao Wang, we now know that the statements mentioned in Theorem 5 are now also algorithmically decidable, although it remains open whether the same is the case for Theorem 4. We know from past results that Theorem 5 cannot hold in sufficiently high dimension (even in the classic case ), but it also remains open whether Theorem 4 fails in that setting.

Following past literature, we rely heavily on a structure theorem for solutions to tiling equations , which roughly speaking asserts that such solutions must be expressible as a finite sum of functions that are one-periodic (periodic in a single direction). This already explains why tiling is easy to understand in one dimension, and why the two-dimensional case is more tractable than the case of general dimension. This structure theorem can be obtained by averaging a dilation lemma, which is a somewhat surprising symmetry of tiling equations that basically arises from finite characteristic arguments (viewing the tiling equation modulo for various large primes ).

For Theorem 2, one can take advantage of the fact that the homogeneous equation is preserved under finite difference operators : if solves , then also solves the same equation . This freedom to take finite differences one to selectively eliminate certain one-periodic components of a solution to the homogeneous equation until the solution is a pure one-periodic function, at which point one can appeal to an induction on dimension, to equate parts (i) and (ii) of the theorem. To link up with part (iii), we also take advantage of the existence of retraction homomorphisms from to to convert a vanishing Fourier coefficient into an integer solution to .

The inhomogeneous results are more difficult, and rely on arguments that are specific to two dimensions. For Theorem 4, one can also perform finite differences to analyze various components of a solution to a tiling equation , but the conclusion now is that the these components are determined (modulo ) by polynomials of one variable. Applying a retraction homomorphism, one can make the coefficients of these polynomials rational, which makes the polynomials periodic. This turns out to reduce the original tiling equation to a system of essentially local combinatorial equations, which allows one to “periodize” a non-periodic solution by periodically repeating a suitable block of the (retraction homomorphism applied to the) original solution.

Theorem 5 is significantly more difficult to establish than the other two results, because of the need to maintain the solution in the form of an indicator function. There are now two separate sources of aperiodicity to grapple with. One is the fact that the polynomials involved in the components may have irrational coefficients (see Theorem 1.3 of our previous paper for an explicit example of this for a level 4 tiling). The other is that in addition to the polynomials (which influence the fractional parts of the components ), there is also “combinatorial” data (roughly speaking, associated to the integer parts of ) which also interact with each other in a slightly non-local way. Once one can make the polynomial coefficients rational, there is enough periodicity that the periodization approach used for the second theorem can be applied to the third theorem; the main remaining challenge is to find a way to make the polynomial coefficients rational, while still maintaining the indicator function property of the solution .

It turns out that the restriction homomorphism approach is no longer available here (it makes the components unbounded, which makes the combinatorial problem too difficult to solve). Instead, one has to first perform a second moment analysis to discern more structure about the polynomials involved. It turns out that the components of an indicator function can only utilize linear polynomials (as opposed to polynomials of higher degree), and that one can partition into a finite number of cosets on which only three of these linear polynomials are “active” on any given coset. The irrational coefficients of these linear polynomials then have to obey some rather complicated, but (locally) finite, sentence in the theory of first-order linear inequalities over the rationals, in order to form an indicator function . One can then use the Weyl equidistribution theorem to replace these irrational coefficients with rational coefficients that obey the same constraints (although one first has to ensure that one does not accidentally fall into the boundary of the constraint set, where things are discontinuous). Then one can apply periodization to the remaining combinatorial data to conclude.

A key technical problem arises from the discontinuities of the fractional part operator at integers, so a certain amount of technical manipulation (in particular, passing at one point to a weak limit of the original tiling) is needed to avoid ever having to encounter this discontinuity.

(Comment on this)

A tool to verify estimates, II: a flexible proof assistant

In a recent post, I talked about a proof of concept tool to verify estimates automatically. Since that post, I have overhauled the tool twice: first to turn it into a rudimentary proof assistant that could also handle some propositional logic; and second into a much more flexible proof assistant (deliberately designed to mimic the Lean proof assistant in several key aspects) that is also powered by the extensive Python package sympy for symbolic algebra, following the feedback from previous commenters. This I think is now a stable framework with which one can extend the tool much further; my initial aim was just to automate (or semi-automate) the proving of asymptotic estimates involving scalar functions, but in principle one could keep adding tactics, new sympy types, and lemmas to the tool to handle a very broad range of other mathematical tasks as well.

The current version of the proof assistant can be found here. (As with my previous coding, I ended up relying heavily on large language model assistance to understand some of the finer points of Python and sympy, with the autocomplete feature of Github Copilot being particularly useful.) While the tool can support fully automated proofs, I have decided to focus more for now on semi-automated interactive proofs, where the human user supplies high-level “tactics” that the proof assistant then performs the necessary calculations for, until the proof is completed.

It’s easiest to explain how the proof assistant works with examples. Right now I have implemented the assistant to work inside the interactive mode of Python, in which one enters Python commands one at a time. (Readers from my generation may be familiar with text adventure games, which have a broadly similar interface.) I would be interested developing at some point a graphical user interface for the tool, but for prototype purposes, the Python interactive version suffices. (One can also run the proof assistant within a Python script, of course.)

After downloading the relevant files, one can launch the proof assistant inside Python by typing from main import * and then loading one of the pre-made exercises. Here is one such exercise:

>>> from main import *
>>> p = linarith_exercise()
Starting proof.  Current proof state:
x: pos_real
y: pos_real
z: pos_real
h1: x < 2*y
h2: y < 3*z + 1
|- x < 7*z + 2

This is the proof assistant’s formalization of the following problem: If are positive reals such that and , prove that .

The way the proof assistant works is that one directs the assistant to use various “tactics” to simplify the problem until it is solved. In this case, the problem can be solved by linear arithmetic, as formalize dby the Linarith() tactic:

>>> p.use(Linarith())
Goal solved by linear arithmetic!
Proof complete!

If instead one wanted a bit more detail on how the linear arithmetic worked, one could have run this tactic instead with a verbose flag:

>>> p.use(Linarith(verbose=true))
Checking feasibility of the following inequalities:
1*z > 0
1*x + -7*z >= 2
1*y + -3*z < 1
1*y > 0
1*x > 0
1*x + -2*y < 0
Infeasible by summing the following:
1*z > 0 multiplied by 1/4
1*x + -7*z >= 2 multiplied by 1/4
1*y + -3*z < 1 multiplied by -1/2
1*x + -2*y < 0 multiplied by -1/4
Goal solved by linear arithmetic!
Proof complete!

Sometimes, the proof involves case splitting, and then the final proof has the structure of a tree. Here is one example, where the task is to show that the hypotheses and imply :

>>> from main import *
>>> p = split_exercise()
Starting proof.  Current proof state:
x: real
y: real
h1: (x > -1) & (x < 1)
h2: (y > -2) & (y < 2)
|- (x + y > -3) & (x + y < 3)
>>> p.use(SplitHyp("h1"))
Decomposing h1: (x > -1) & (x < 1) into components x > -1, x < 1.
1 goal remaining.
>>> p.use(SplitHyp("h2"))
Decomposing h2: (y > -2) & (y < 2) into components y > -2, y < 2.
1 goal remaining.
>>> p.use(SplitGoal())
Split into conjunctions: x + y > -3, x + y < 3
2 goals remaining.
>>> p.use(Linarith())
Goal solved by linear arithmetic!
1 goal remaining.
>>> p.use(Linarith())
Goal solved by linear arithmetic!
Proof complete!
>>> print(p.proof())
example (x: real) (y: real) (h1: (x > -1) & (x < 1)) (h2: (y > -2) & (y < 2)): (x + y > -3) & (x + y < 3) := by
  split_hyp h1
  split_hyp h2
  split_goal
  . linarith
  linarith

Here at the end we gave a “pseudo-Lean” description of the proof in terms of the three tactics used: a tactic cases h1 to case split on the hypothesis h1, followed by two applications of the simp_all tactic to simplify in each of the two cases.

The tool supports asymptotic estimation. I found a way to implement the order of magnitude formalism from the previous post within sympy. It turns out that sympy, in some sense, already natively implements nonstandard analysis: its symbolic variables have an is_number flag which basically corresponds to the concept of a “standard” number in nonstandard analysis. For instance, the sympy version S(3) of the number 3 has S(3).is_number == True and so is standard, whereas an integer variable n = Symbol("n", integer=true) has n.is_number == False and so is nonstandard. Within sympy, I was able to construct orders of magnitude Theta(X) of various (positive) expressions X, with the property that Theta(n)=Theta(1) if n is a standard number, and use this concept to then define asymptotic estimates such as (implemented as lesssim(X,Y)). One can then apply a logarithmic form of linear arithmetic to then automatically verify some asymptotic estimates. Here is a simple example, in which one is given a positive integer and positive reals such that and , and the task is to conclude that :

>>> p = loglinarith_exercise()
Starting proof.  Current proof state:
N: pos_int
x: pos_real
y: pos_real
h1: x <= 2*N**2
h2: y < 3*N
|- Theta(x)*Theta(y) <= Theta(N)**4
>>> p.use(LogLinarith(verbose=True))
Checking feasibility of the following inequalities:
Theta(N)**1 >= Theta(1)
Theta(x)**1 * Theta(N)**-2 <= Theta(1)
Theta(y)**1 * Theta(N)**-1 <= Theta(1)
Theta(x)**1 * Theta(y)**1 * Theta(N)**-4 > Theta(1)
Infeasible by multiplying the following:
Theta(N)**1 >= Theta(1) raised to power 1
Theta(x)**1 * Theta(N)**-2 <= Theta(1) raised to power -1
Theta(y)**1 * Theta(N)**-1 <= Theta(1) raised to power -1
Theta(x)**1 * Theta(y)**1 * Theta(N)**-4 > Theta(1) raised to power 1
Proof complete!

The logarithmic linear programming solver can also handle lower order terms, by a rather brute force branching method:

>>> p = loglinarith_hard_exercise()
Starting proof.  Current proof state:
N: pos_int
x: pos_real
y: pos_real
h1: x <= 2*N**2 + 1
h2: y < 3*N + 4
|- Theta(x)*Theta(y) <= Theta(N)**3
>>> p.use(LogLinarith())
Goal solved by log-linear arithmetic!
Proof complete!

I plan to start developing tools for estimating function space norms of symbolic functions, for instance creating tactics to deploy lemmas such as Holder’s inequality and the Sobolev embedding inequality. It looks like the sympy framework is flexible enough to allow for creating further object classes for these sorts of objects. (Right now, I only have one proof-of-concept lemma to illustrate the framework, the arithmetic mean-geometric mean lemma.)

I am satisfied enough with the basic framework of this proof assistant that I would be open to further suggestions or contributions of new features, for instance by introducing new data types, lemmas, and tactics, or by contributing example problems that ought to be easily solvable by such an assistant, but are currently beyond its ability, for instance due to the lack of appropriate tactics and lemmas.

(Comment on this)

Orders of infinity

Many problems in analysis (as well as adjacent fields such as combinatorics, theoretical computer science, and PDE) are interested in the order of growth (or decay) of some quantity that depends on one or more asymptotic parameters (such as ) – for instance, whether the quantity grows or decays linearly, quadratically, polynomially, exponentially, etc. in . In the case where these quantities grow to infinity, these growth rates had once been termed “orders of infinity” – for instance, in the 1910 book of this name by Hardy – although this term has fallen out of use in recent years. (Hardy fields are still a thing, though.)

In modern analysis, asymptotic notation is the preferred device to organize orders of infinity. There are a couple of flavors of this notation, but here is one such (a blend of Hardy’s notation and Landau’s notation). Formally, we need a parameter space equipped with a non-principal filter that describes the subsets of parameter space that are “sufficiently large” (e.g., the cofinite (Fréchet) filter on , or the cocompact filter on ). We will use to denote elements of this filter; thus, an assertion holds for sufficiently large if and only if it holds for all in some element of the filter . Given two positive quantities that are defined for sufficiently large , one can then define the following notions:

• (i) We write , , or (and sometimes ) if there exists a constant such that for all sufficiently large .
• (ii) We write , , or (and sometimes ) if, for every , one has for all sufficiently large .
• (iii) We write (and sometimes or ) if , or equivalently if there exist constants such that for all sufficiently large .

We caution that in analytic number theory and adjacent fields, the slightly different notation of Vinogradov is favored, in which would denote the concept (i) instead of (ii), and would denote a fourth concept instead of (iii). However, we will use the Hardy-Landau notation exclusively in this blog post.

Anyone who works with asymptotic notation for a while will quickly recognize that it enjoys various algebraic properties akin to the familiar algebraic properties of order on the real line. For instance, the symbols behave very much like , , , , with properties such as the following:

• If and , then .
• if and only if .
• If is restricted to a sequence, then (after passing to a subsequence if necessary), exactly one of , , and is true.

One also has the “tropical” property , making asymptotic notation a kind of “tropical algebra“.

However, in contrast with other standard algebraic structures (such as ordered fields) that blend order and arithmetic operations, the precise laws of orders of infinity are usually not written down as a short list of axioms. Part of this is due to cultural differences between analysis and algebra – as discussed in this essay by Gowers, analysis is often not well suited to the axiomatic approach to mathematics that algebra benefits so much from. But another reason is due to our orthodox implementation of analysis via “epsilon-delta” type concepts, such as the notion of “sufficiently large” used above, which notoriously introduces a large number of both universal and existential quantifiers into the subject (for every epsilon, there exists a delta…) which tends to interfere with the smooth application of algebraic laws (which are optimized for the universal quantifier rather than the existential quantifier).

But there is an alternate approach to analysis, namely nonstandard analysis, which rearranges the foundations so that many of quantifiers (particularly the existential ones) are concealed from view (usually via the device of ultrafilters). This makes the subject of analysis considerably more “algebraic” in nature, as the “epsilon management” that is so prevalent in orthodox analysis is now performed much more invisibly. For instance, as we shall see, in the nonstandard framework, orders of infinity acquire the algebraic structure of a totally ordered vector space that also enjoys a completeness property reminiscent, though not identical to, the completeness of the real numbers. There is also a transfer principle that allows one to convert assertions in orthodox asymptotic notation into logically equivalent assertions about nonstandard orders of infinity, allowing one to then prove asymptotic statements in a purely algebraic fashion. There is a price to pay for this “algebrization” of analysis; the spaces one works with become quite large (in particular, they tend to be “inseparable” and not “countably generated” in any reasonable fashion), and it becomes difficult to extract explicit constants (or explicit decay rates) from the asymptotic notation. However, there are some cases in which the tradeoff is worthwhile. For instance, symbolic computations tend to be easier to perform in algebraic settings than in orthodox analytic settings, so formal computations of orders of infinity (such as the ones discussed in the previous blog post) could benefit from the nonstandard approach. (See also my previous posts on nonstandard analysis for more discussion about these tradeoffs.)

Let us now describe the nonstandard approach to asymptotic notation. With the above formalism, the switch from standard to nonstandard analysis is actually quite simple: one assumes that the asymptotic filter is in fact an ultrafilter. In terms of the concept of “sufficiently large”, this means adding the following useful axiom:

• Given any predicate , exactly one of the two statements “ holds for sufficiently large ” and “ does not hold for sufficiently large ” is true.

This can be compared with the situation with, say, the Fréchet filter on the natural numbers , in which one has to insert some qualifier such as “after passing to a subsequence if necessary” in order to make the above axiom true.

The existence of an ultrafilter requires some weak version of the axiom of choice (specifically, the ultrafilter lemma), but for this post we shall just take the existence of ultrafilters for granted.

We can now define the nonstandard orders of infinity to be the space of all non-negative functions defined for sufficiently large , modulo the equivalence relation defined previously. That is to say, a nonstandard order of infinity is an equivalence class

of functions defined on elements of the ultrafliter. For instance, if is the natural numbers, then itself is an order of infinity, as is , , , , and so forth. But we exclude ; it will be important for us that the order of infinity is strictly positive for all sufficiently large .

We can place various familiar algebraic operations on :

• Addition. We define . It is easy to see that this is well-defined, by verifying that if and , then . However, because of our positivity requirement, we do not define subtraction on .
• Multiplication and division We define and ; we do not need to worry about division by zero, thanks to the positivity requirement. Again it is easy to verify this is well-defined.
• Scalar exponentiation If is a real number, we define . Again, this is easily checked to be well-defined.
• Order We define if , and if . Again, this is easily checked to be well-defined. (And the ultrafilter axiom ensures that holds iff exactly one of and holds.)

With these operations, combined with the ultrafilter axiom, we see that obeys the laws of many standard algebraic structures, the proofs of which we leave as exercises for the reader:

• is a totally ordered set.
• In fact, is a totally ordered vector space, with playing the role of the zero vector, multiplication playing the role of vector addition, and scalar exponentiation playing the role of scalar multiplication. (Of course, division would then play the role of vector subtraction.) To avoid confusion, one might refer to as a log-vector space rather than a vector space to emphasize the fact that the vector structure is coming from multiplication (and exponentiation) rather than addition (and multiplication). Ordered log-vector spaces may seem like a strange and exotic concept, but they are actually already studied implicitly in analysis, albeit under the guise of other names such as log-convexity.
• is a semiring (albeit one without an additive identity element), which is idempotent: for all .
• More generally, addition can be described in purely order-theoretic terms: for all . (It would therefore be natural to call a tropical semiring, although the precise axiomatization of this term does not appear to be fully standardized currently.)

The ordered (log-)vector space structure of in particular opens up the ability to prove asymptotic implications by (log-)linear programming; this was implicitly used in my previous post. One can also use the language of (log-)linear algebra to describe further properties of various orders of infinity. For instance, if is the natural numbers, we can form the subspace

of consisting of those orders of infinity which are of polynomial type in the sense that for some ; this is then a (log)-vector subspace of , and has a canonical (log-)linear surjection that assigns to each order of infinity of polynomial type the unique real number such that , that is to say for all one has for all sufficiently large . (The existence of such an

(Comment on this)

A proof of concept tool to verify estimates

This post was inspired by some recent discussions with Bjoern Bringmann.

Symbolic math software packages are highly developed for many mathematical tasks in areas such as algebra, calculus, and numerical analysis. However, to my knowledge we do not have similarly sophisticated tools for verifying asymptotic estimates – inequalities that are supposed to hold for arbitrarily large parameters, with constant losses. Particularly important are functional estimates, where the parameters involve an unknown function or sequence (living in some suitable function space, such as an space); but for this discussion I will focus on the simpler situation of asymptotic estimates involving a finite number of positive real numbers, combined using arithmetic operations such as addition, multiplication, division, exponentiation, and minimum and maximum (but no subtraction). A typical inequality here might be the weak arithmetic mean-geometric mean inequality

where are arbitrary positive real numbers, and the here indicates that we are willing to lose an unspecified (multiplicative) constant in the estimates.

I have wished in the past (e.g., in this MathOverflow answer) for a tool that could automatically determine whether such an estimate was true or not (and provide a proof if true, or an asymptotic counterexample if false). In principle, simple inequalities of this form could be automatically resolved by brute force case splitting. For instance, with (1), one first observes that is comparable to up to constants, so it suffices to determine if

Next, to resolve the maximum, one can divide into three cases: ; ; and . Suppose for instance that . Then the estimate to prove simplifies to

and this is (after taking logarithms) a positive linear combination of the hypotheses , . The task of determining such a linear combination is a standard linear programming task, for which many computer software packages exist.

Any single such inequality is not too difficult to resolve by hand, but there are applications in which one needs to check a large number of such inequalities, or split into a large number of cases. I will take an example at random from an old paper of mine (adapted from the equation after (51), and ignoring some epsilon terms for simplicity): I wanted to establish the estimate

for any obeying the constraints

where , , and are the maximum, median, and minimum of respectively, and similarly for , , and , and . This particular bound could be dispatched in three or four lines from some simpler inequalities; but it took some time to come up with those inequalities, and I had to do a dozen further inequalities of this type. This is a task that seems extremely ripe for automation, particularly with modern technology.

Recently, I have been doing a lot more coding (in Python, mostly) than in the past, aided by the remarkable facility of large language models to generate initial code samples for many different tasks, or to autocomplete partially written code. For the most part, I have restricted myself to fairly simple coding tasks, such as computing and then plotting some mildly complicated mathematical functions, or doing some rudimentary data analysis on some dataset. But I decided to give myself the more challenging task of coding a verifier that could handle inequalities of the above form. After about four hours of coding, with frequent assistance from an LLM, I was able to produce a proof of concept tool for this, which can be found at this Github repository. For instance, to verify (1), the relevant Python code is

    a = Variable("a")
    b = Variable("b")
    c = Variable("c")
    assumptions = Assumptions()
    assumptions.can_bound((a * b * c) ** (1 / 3), max(a, b, c))

and the (somewhat verbose) output verifying the inequality is

Checking if we can bound (((a * b) * c) ** 0.3333333333333333) by max(a, b, c) from the given axioms.
We will split into the following cases:
[[b <~ a, c <~ a], [a <~ b, c <~ b], [a <~ c, b <~ c]]
Trying case: ([b <~ a, c <~ a],)
Simplify to proving (((a ** 0.6666666666666667) * (b ** -0.3333333333333333)) * (c ** -0.3333333333333333)) >= 1.
Bound was proven true by multiplying the following hypotheses :
b <~ a raised to power 0.33333333
c <~ a raised to power 0.33333333
Trying case: ([a <~ b, c <~ b],)
Simplify to proving (((b ** 0.6666666666666667) * (a ** -0.3333333333333333)) * (c ** -0.3333333333333333)) >= 1.
Bound was proven true by multiplying the following hypotheses :
a <~ b raised to power 0.33333333
c <~ b raised to power 0.33333333
Trying case: ([a <~ c, b <~ c],)
Simplify to proving (((c ** 0.6666666666666667) * (a ** -0.3333333
333333333)) * (b ** -0.3333333333333333)) >= 1.
Bound was proven true by multiplying the following hypotheses :
a <~ c raised to power 0.33333333
b <~ c raised to power 0.33333333
Bound was proven true in all cases!

This is of course an extremely inelegant proof, but elegance is not the point here; rather, that it is automated. (See also this recent article of Heather Macbeth for how proof writing styles change in the presence of automated tools, such as formal proof assistants.)

The code is close to also being able to handle more complicated estimates such as (3); right now I have not written code to properly handle hypotheses such as that involve complex expressions such as , as opposed to hypotheses that only involve atomic variables such as , , but I can at least handle such complex expressions in the left and right-hand sides of the estimate I am trying to verify.

In any event, the code, being a mixture of LLM-generated code and my own rudimentary Python skills, is hardly an exemplar of efficient or elegant coding, and I am sure that there are many expert programmers who could do a much better job. But I think this is proof of concept that a more sophisticated tool of this form could be quite readily created to do more advanced tasks. One such example task was the one I gave in the above MathOverflow question, namely being able to automatically verify a claim such as

for all . Another task would be to automatically verify the ability to estimate some multilinear expression of various functions, in terms of norms of such functions in standard spaces such as Sobolev spaces; this is a task that is particularly prevalent in PDE and harmonic analysis (and can frankly get somewhat tedious to do by hand). As speculated in that MO post, one could eventually hope to also utilize AI to assist in the verification process, for instance by suggesting possible splittings of the various sums or integrals involved, but that would be a long-term objective.

This sort of software development would likely best be performed as a collaborative project, involving both mathematicians and expert programmers. I would be interested to receive advice on how best to proceed with such a project (for instance, would it make sense to incorporate such a tool into an existing platform such as SageMATH), and what features for a general estimate verifier would be most desirable for mathematicians. One thing on my wishlist is the ability to give a tool an expression to estimate (such as a multilinear integral of some unknown functions), as well as a fixed set of tools to bound that integral (e.g., splitting the integral into pieces, integrating by parts, using the Hölder and Sobolev inequalities, etc.), and have the computer do its best to optimize the bound it can produce with those tools (complete with some independently verifiable proof certificate for its output). One could also imagine such tools having the option to output their proof certificates in a formal proof assistant language such as Lean. But perhaps there are other useful features that readers may wish to propose.

(Comment on this)

Stonean spaces, projective objects, the Riesz representation theorem, and (possibly) condensed mathematics

A basic type of problem that occurs throughout mathematics is the lifting problem: given some space that “sits above” some other “base” space due to a projection map , and some map from a third space into the base space , find a “lift” of to , that is to say a map such that . In many applications we would like to have preserve many of the properties of (e.g., continuity, differentiability, linearity, etc.).

Of course, if the projection map is not surjective, one would not expect the lifting problem to be solvable in general, as the map to be lifted could simply take values outside of the range of . So it is natural to impose the requirement that be surjective, giving the following commutative diagram to complete:

If no further requirements are placed on the lift , then the axiom of choice is precisely the assertion that the lifting problem is always solvable (once we require to be surjective). Indeed, the axiom of choice lets us select a preimage in the fiber of each point , and one can lift any by setting . Conversely, to build a choice function for a surjective map , it suffices to lift the identity map to .

Of course, the maps provided by the axiom of choice are famously pathological, being almost certain to be discontinuous, non-measurable, etc.. So now suppose that all spaces involved are topological spaces, and all maps involved are required to be continuous. Then the lifting problem is not always solvable. For instance, we have a continuous projection from to , but the identity map cannot be lifted continuously up to , because is contractable and is not.

However, if is a discrete space (every set is open), then the axiom of choice lets us solve the continuous lifting problem from for any continuous surjection , simply because every map from to is continuous. Conversely, the discrete spaces are the only ones with this property: if is a topological space which is not discrete, then if one lets be the same space equipped with the discrete topology, then the only way one can continuously lift the identity map through the “projection map” (that maps each point to itself) is if is itself discrete.

These discrete spaces are the projective objects in the category of topological spaces, since in this category the concept of an epimorphism agrees with that of a surjective continuous map. Thus can be viewed as the unique (up to isomorphism) projective object in this category that has a bijective continuous map to .

Now let us narrow the category of topological spaces to the category of compact Hausdorff (CH) spaces. Here things should be better behaved; for instance, it is a standard fact in this category that continuous bijections are homeomorphisms, and it is still the case that the epimorphisms are the continuous surjections. So we have a usable notion of a projective object in this category: CH spaces such that any continuous map into another CH space can be lifted via any surjective continuous map to another CH space.

By the previous discussion, discrete CH spaces will be projective, but this is an extremely restrictive set of examples, since of course compact discrete spaces must be finite. Are there any others? The answer was worked out by Gleason:

Proposition 1 A compact Hausdorff space is projective if and only if it is extremally disconnected, i.e., the closure of every open set is again open.

Proof: We begin with the “only if” direction. Let was projective, and let be an open subset of . Then the closure and complement are both closed, hence compact, subsets of , so the disjoint union is another CH space, which has an obvious surjective continuous projection map to formed by gluing the two inclusion maps together. As is projective, the identity map must then lift to a continuous map . One easily checks that has to map to the first component of the disjoint union, and ot the second component; hence , and so is open, giving extremal disconnectedness.

Conversely, suppose that is extremally disconnected, that is a continuous surjection of CH spaces, and is continuous. We wish to lift to a continuous map .

We first observe that it suffices to solve the lifting problem for the identity map , that is to say we can assume without loss of generality that and is the identity. Indeed, for general maps , one can introduce the pullback space

which is clearly a CH space that has a continuous surjection . Any continuous lift of the identity map to , when projected onto , will give a desired lift .

So now we are trying to lift the identity map via a continuous surjection . Let us call this surjection minimally surjective if no restriction of to a proper closed subset of remains surjective. An easy application of Zorn’s lemma shows that every continuous surjection can be restricted to a minimally surjective continuous map . Thus, without loss of generality, we may assume that is minimally surjective.

The key claim now is that every minimally surjective map into an extremally disconnected space is in fact a bijection. Indeed, suppose for contradiction that there were two distinct points in that mapped to the same point under . By taking contrapositives of the minimal surjectivity property, we see that every open neighborhood of must contain at least one fiber of , and by shrinking this neighborhood one can ensure the base point is arbitrarily close to . Thus, every open neighborhood of must intersect every open neighborhood of , contradicting the Hausdorff property.

It is well known that continuous bijections between CH spaces must be homeomorphisms (they map compact sets to compact sets, hence must be open maps). So is a homeomorphism, and one can lift the identity map to the inverse map .

Remark 2 The property of being “minimally surjective” sounds like it should have a purely category-theoretic definition, but I was unable to match this concept to a standard term in category theory (something along the lines of a “minimal epimorphism”, I would imagine).

In view of this proposition, it is now natural to look for extremally disconnected CH spaces (also known as Stonean spaces). The discrete CH spaces are one class of such spaces, but they are all finite. Unfortunately, these are the only “small” examples:

Lemma 3 Any first countable extremally disconnected CH space is discrete.

Proof: If such a space were not discrete, one could find a sequence in converging to a limit such that for all . One can sparsify the elements to all be distinct, and from the Hausdorff property one can construct neighbourhoods of each that avoid , and are disjoint from each other. Then and then are disjoint open sets that both have as an adherent point, which is inconsistent with extremal disconnectedness: the closure of contains but is disjoint from , so cannot be open.

Thus for instance there are no extremally disconnected compact metric spaces, other than the finite spaces; for instance, the Cantor space is not extremally disconnected, even though it is totally disconnected (which one can easily see to be a property implied by extremal disconnectedness). On the other hand, once we leave the first-countable world, we have plenty of such spaces:

Lemma 4 Let be a complete Boolean algebra. Then the Stone dual of (i.e., the space of boolean homomorphisms ) is an extremally disconnected CH space.

Proof: The CH properties are standard. The elements of give a basis of the topology given by the clopen sets . Because the Boolean algebra is complete, we see that the closure of the open set for any family of sets is simply the clopen set , which obviously open, giving extremal disconnectedness.

Remark 5 In fact, every extremally disconnected CH space <img src="https://s0.wp.com/latex.php?latex=%7BX%7D&amp;bg=ffffff&amp;fg=000000&amp;s=0&amp;c=20201002" srcset="https://s0.wp.com/latex.php?latex=%7BX%7D&amp;bg=ffffff&amp;fg=000000&amp;s=0&amp;c=20201002 1x, https://s0.wp.com/latex.php?latex=%7BX%7D&amp;bg=ffffff&amp;fg=000000&amp;s=0&amp;c

(Comment on this)

A suspicious conference

I have received multiple queries from colleagues who have been invited (from a non-academic email address) to speak a strange-sounding conference that is allegedly supported by major mathematical institutions, allegedly hosted at a prestigious university, and allegedly having myself (and two other Fields Medalists) as plenary speakers. The invitees are asked to pay “registration fees” upfront, with the promise of future reimbursement. (There is a bare-bones web site, which seems to be partially copy-pasted from some previous “conferences” in chemistry and physics, but I will not link to it here.)

I have not agreed (or even been asked) to participate in this event, and I can confirm the same for at least one of the other supposed plenary speakers. There is also no confirmation of the support or location claimed.

As such, this does not appear to be a legitimate scientific conference, and I would advise anyone receiving such an email to discard it.

EDIT: in order to have this post be picked up by appropriate search engine queries, the name of the alleged conference is “Infinity ’25: Horizons in Mathematical Thought”.

SECOND EDIT: I am *not* referring to the 2026 International Congress of Mathematicians (ICM), which is also sending out speaker invitations currently, and is of course an extremely legitimate conference.

(Comment on this)

Decomposing a factorial into large factors

I’ve just uploaded to the arXiv the paper “Decomposing a factorial into large factors“. This paper studies the quantity , defined as the largest quantity such that it is possible to factorize into factors , each of which is at least . The first few values of this sequence are

(OEIS A034258). For instance, we have , because on the one hand we can factor

but on the other hand it is not possible to factorize into nine factors, each of which is or higher.

This quantity was introduced by Erdös, who asked for upper and lower bounds on ; informally, this asks how equitably one can split up into factors. When factoring an arbitrary number, this is essentially a variant of the notorious knapsack problem (after taking logarithms), but one can hope that the specific structure of the factorial can make this particular knapsack-type problem more tractable. Since

for any putative factorization, we obtain an upper bound

thanks to the Stirling approximation. At one point, Erdös, Selfridge, and Straus claimed that this upper bound was asymptotically sharp, in the sense that

as ; informally, this means we can split into factors that are (mostly) approximately the same size, when is large. However, as reported in this later paper, Erdös “believed that Straus had written up our proof… Unfortunately Straus suddenly died and no trace was ever found of his notes. Furthermore, we never could reconstruct our proof, so our assertion now can be called only a conjecture”.

Some further exploration of was conducted by Guy and Selfridge. There is a simple construction that gives the lower bound

that comes from starting with the standard factorization and transferring some powers of from the later part of the sequence to the earlier part to rebalance the terms somewhat. More precisely, if one removes one power of two from the even numbers between and , and one additional power of two from the multiples of four between to , this frees up powers of two that one can then distribute amongst the numbers up to to bring them all up to at least in size. A more complicated procedure involving transferring both powers of and then gives the improvement . At this point, however, things got more complicated, and the following conjectures were made by Guy and Selfridge:

• (i) Is for all ?
• (ii) Is for all ? (At , this conjecture barely fails: .)
• (iii) Is for all ?

In this note we establish the bounds

as , where is the explicit constant

In particular this recovers the lost result (2). An upper bound of the shape

for some was previously conjectured by Erdös and Graham (Erdös problem #391). We conjecture that the upper bound in (3) is sharp, thus

which is consistent with the above conjectures (i), (ii), (iii) of Guy and Selfridge, although numerically the convergence is somewhat slow.

The upper bound argument for (3) is simple enough that it could also be modified to establish the first conjecture (i) of Guy and Selfridge; in principle, (ii) and (iii) are now also reducible to a finite computation, but unfortunately the implied constants in the lower bound of (3) are too weak to make this directly feasible. However, it may be possible to now crowdsource the verification of (ii) and (iii) by supplying a suitable set of factorizations to cover medium sized , combined with some effective version of the lower bound argument that can establish for all past a certain threshold. The value singled out by Guy and Selfridge appears to be quite a suitable test case: the constructions I tried fell just a little short of the conjectured threshold of , but it seems barely within reach that a sufficiently efficient rearrangement of factors can work here.

We now describe the proof of the upper and lower bound in (3). To improve upon the trivial upper bound (1), one can use the large prime factors of . Indeed, every prime between and divides at least once (and the ones between and divide it twice), and any factor that contains such a factor therefore has to be significantly larger than the benchmark value of . This observation already readily leads to some upper bound of the shape (4) for some ; if one also uses the primes that are slightly less than (noting that any multiple of that exceeds , must in fact exceed ) is what leads to the precise constant .

For previous lower bound constructions, one started with the initial factorization and then tried to “improve” this factorization by moving around some of the prime factors. For the lower bound in (3), we start instead with an approximate factorization roughly of the shape

where is the target lower bound (so, slightly smaller than ), and is a moderately sized natural number parameter (we will take , although there is significant flexibility here). If we denote the right-hand side here by , then is basically a product of numbers of size at least . It is not literally equal to ; however, an easy application of Legendre’s formula shows that for odd small primes , and have almost exactly the same number of factors of . On the other hand, as is odd, contains no factors of , while contains about such factors. The prime factorizations of and differ somewhat at large primes, but has slightly more such prime factors as (about such factors, in fact). By some careful applications of the prime number theorem, one can tweak some of the large primes appearing in to make the prime factorization of and agree almost exactly, except that is missing most of the powers of in , while having some additional large prime factors beyond those contained in to compensate. With a suitable choice of threshold , one can then replace these excess large prime factors with powers of two to obtain a factorization of into terms that are all at least , giving the lower bound.

The general approach of first locating some approximate factorization of (where the approximation is in the “adelic” sense of having not just approximately the right magnitude, but also approximately the right number of factors of for various primes ), and then moving factors around to get an exact factorization of , looks promising for also resolving the conjectures (ii), (iii) mentioned above. For instance, I was numerically able to verify that by the following procedure:

• Start with the approximate factorization of , by . Thus is the product of odd numbers, each of which is at least .
• Call an odd prime -heavy if it divides more often than , and -heavy if it divides more often than . It turns out that there are more -heavy primes than -heavy primes (counting multiplicity). On the other hand, contains powers of , while has none. This represents the (multi-)set of primes one has to redistribute in order to convert a factorization of to a factorization of <img src="https://s0.wp.com/latex.php?latex=%7BN%21%7D&#038;bg=ffffff&#038;fg=000000&#038;s=0&#038;c=20201002" srcset="https://s0.wp.com/latex.php?latex=%7BN%21%7D&#038;bg=ffffff

(Comment on this)

The three-dimensional Kakeya conjecture, after Wang and Zahl

There has been some spectacular progress in geometric measure theory: Hong Wang and Joshua Zahl have just released a preprint that resolves the three-dimensional case of the infamous Kakeya set conjecture! This conjecture asserts that a Kakeya set – a subset of that contains a unit line segment in every direction, must have Minkowski and Hausdorff dimension equal to three. (There is also a stronger “maximal function” version of this conjecture that remains open at present, although the methods of this paper will give some non-trivial bounds on this maximal function.) It is common to discretize this conjecture in terms of small scale . Roughly speaking, the conjecture then asserts that if one has a family of tubes of cardinality , and pointing in a -separated set of directions, then the union of these tubes should have volume . Here we shall be a little vague as to what means here, but roughly one should think of this as “up to factors of the form for any “; in particular this notation can absorb any logarithmic losses that might arise for instance from a dyadic pigeonholing argument. For technical reasons (including the need to invoke the aforementioned dyadic pigeonholing), one actually works with slightly smaller sets , where is a “shading” of the tubes in that assigns a large subset of to each tube in the collection; but for this discussion we shall ignore this subtlety and pretend that we can always work with the full tubes.

Previous results in this area tended to center around lower bounds of the form

for various intermediate dimensions , that one would like to make as large as possible. For instance, just from considering a single tube in this collection, one can easily establish (1) with . By just using the fact that two lines in intersect in a point (or more precisely, a more quantitative estimate on the volume between the intersection of two tubes, based on the angle of intersection), combined with a now classical -based argument of Córdoba, one can obtain (1) with (and this type of argument also resolves the Kakeya conjecture in two dimensions). In 1995, building on earlier work by Bourgain, Wolff famously obtained (1) with using what is now known as the “Wolff hairbrush argument”, based on considering the size of a “hairbrush” – the union of all the tubes that pass through a single tube (the hairbrush “stem”) in the collection.

In their new paper, Wang and Zahl established (1) for . The proof is lengthy (127 pages!), and relies crucially on their previous paper establishing a key “sticky” case of the conjecture. Here, I thought I would try to summarize the high level strategy of proof, omitting many details and also oversimplifying the argument at various places for sake of exposition. The argument does use many ideas from previous literature, including some from my own papers with co-authors; but the case analysis and iterative schemes required are remarkably sophisticated and delicate, with multiple new ideas needed to close the full argument.

A natural strategy to prove (1) would be to try to induct on : if we let represent the assertion that (1) holds for all configurations of tubes of dimensions , with -separated directions, we could try to prove some implication of the form for all , where is some small positive quantity depending on . Iterating this, one could hope to get arbitrarily close to .

A general principle with these sorts of continuous induction arguments is to first obtain the trivial implication in a non-trivial fashion, with the hope that this non-trivial argument can somehow be perturbed or optimized to get the crucial improvement . The standard strategy for doing this, since the work of Bourgain and then Wolff in the 1990s (with precursors in older work of Córdoba), is to perform some sort of “induction on scales”. Here is the basic idea. Let us call the tubes in “thin tubes”. We can try to group these thin tubes into “fat tubes” of dimension for some intermediate scale ; it is not terribly important for this sketch precisely what intermediate value is chosen here, but one could for instance set if desired. Because of the -separated nature of the directions in , there can only be at most thin tubes in a given fat tube, and so we need at least fat tubes to cover the thin tubes. Let us suppose for now that we are in the “sticky” case where the thin tubes stick together inside fat tubes as much as possible, so that there are in fact a collection of fat tubes , with each fat tube containing about of the thin tubes. Let us also assume that the fat tubes are -separated in direction, which is an assumption which is highly consistent with the other assumptions made here.

If we already have the hypothesis , then by applying it at scale instead of we conclude a lower bound on the volume occupied by fat tubes:

Since , this morally tells us that the typical multiplicity of the fat tubes is ; a typical point in should belong to about fat tubes.

Now, inside each fat tube , we are assuming that we have about thin tubes that are -separated in direction. If we perform a linear rescaling around the axis of the fat tube by a factor of to turn it into a tube, this would inflate the thin tubes to be rescaled tubes of dimensions , which would now be -separated in direction. This rescaling does not affect the multiplicity of the tubes. Applying again, we see morally that the multiplicity of the rescaled tubes, and hence the thin tubes inside , should be .

We now observe that the multiplicity of the full collection of thin tubes should morally obey the inequality

since if a given point lies in at most fat tubes, and within each fat tube a given point lies in at most thin tubes in that fat tube, then it should only be able to lie in at most tubes overall. This heuristically gives , which then recovers (1) in the sticky case.

In their previous paper, Wang and Zahl were roughly able to squeeze a little bit more out of this argument to get something resembling in the sticky case, loosely following a strategy of Nets Katz and myself that I discussed in this previous blog post from over a decade ago. I will not discuss this portion of the argument further here, referring the reader to the introduction to that paper; instead, I will focus on the arguments in the current paper, which handle the non-sticky case.

Let’s try to repeat the above analysis in a non-sticky situation. We assume (or some suitable variant thereof), and consider some thickened Kakeya set

where is something resembling what we might call a “Kakeya configuration” at scale : a collection of thin tubes of dimension that are -separated in direction. (Actually, to make the induction work, one has to consider a more general family of tubes than these, satisfying some standard “Wolff axioms” instead of the direction separation hypothesis; but we will gloss over this issue for now.) Our goal is to prove something like for some , which amounts to obtaining some improved volume bound

that improves upon the bound coming from . From the previous paper we know we can do this in the “sticky” case, so we will assume that is “non-sticky” (whatever that means).

A typical non-sticky setup is when there are now fat tubes for some multiplicity (e.g., for some small constant ), with each fat tube containing only thin tubes. Now we have an unfortunate imbalance: the fat tubes form a “super-Kakeya configuration”, with too many tubes at the coarse scale for them to be all -separated in direction, while the thin tubes inside a fat tube form a “sub-Kakeya configuration” in which there are not enough tubes to cover all relevant directions. So one cannot apply the hypothesis efficiently at either scale.

This looks like a serious obstacle, so let’s change tack for a bit and think of a different way to try to close the argument. Let’s look at how intersects a given -ball . The hypothesis suggests that might behave like a -dimensional fractal (thickened at scale ), in which case one might be led to a predicted size of of the form . Suppose for sake of argument that the set was denser than this at this scale, for instance we have

for all and some . Observe that the -neighborhood is basically , and thus has volume by the hypothesis (indeed we would ev

(Comment on this)

Closing the “green gap”: from the mathematics of the landscape function to lower electricity costs for households

I recently returned from the 2025 Annual Meeting of the “Localization of Waves” collaboration (supported by the Simons Foundation, with additional related support from the NSF), where I learned (from Svitlana Mayboroda, the director of the collaboration as well as one of the principal investigators) of a remarkable statistic: net electricity consumption by residential customers in the US has actually experienced a slight decrease in recent years:

The decrease is almost entirely due to gains in lighting efficiency in households, and particularly the transition from incandescent (and compact fluorescent) light bulbs to LED light bulbs:

Annual energy savings from this switch to consumers in the US were already estimated to be $14.7 billion in 2020 – or several hundred dollars per household – and are projected to increase, even in the current inflationary era, with the cumulative savings across the US estimated to reach $890 billion by 2035.

What I also did not realize before this meeting is the role that recent advances in pure mathematics – and specifically, the development of the “landscape function” that was a primary focus of this collaboration – played in accelerating this transition. This is not to say that this piece of mathematics was solely responsible for these developments; but, as I hope to explain here, it was certainly part of the research and development ecosystem in both academia and industry, spanning multiple STEM disciplines and supported by both private and public funding. This application of the landscape function was already reported upon by Quanta magazine at the very start of this collaboration back in 2017; but it is only in the last few years that the mathematical theory has been incorporated into the latest LED designs and led to actual savings at the consumer end.

LED lights are made from layers of semiconductor material (e.g., Gallium nitride or Indium gallium nitride) arranged in a particular fashion. When enough of a voltage difference is applied to this material, electrons are injected into the “n-type” side of the LED, while holes of electrons are injected into the “p-type” side, creating a current. In the active layer of the LED, these electrons and holes recombine in the quantum wells of the layer, generating radiation (light) via the mechanism of electroluminescence. The brightness of the LED is determined by the current, while the power consumption is the product of the current and the voltage. Thus, to improve energy efficiency, one seeks to design LEDs to require as little voltage as possible to generate a target amount of current.

As it turns out, the efficiency of an LED, as well as the spectral frequencies of light they generate, depend in many subtle ways on the precise geometry of the chemical composition of the semiconductors, the thickness of the layers, the geometry of how the layers are placed atop one another, the temperature of the materials, and the amount of disorder (impurities) introduced into each layer. In particular, in order to create quantum wells that can efficiently trap the electrons and holes together to recombine to create light of a desired frequency, it is useful to introduce a certain amount of disorder into the layers in order to take advantage of the phenomenon of Anderson localization. However, one cannot add too much disorder, lest the electron states become fully bound and the material behaves too much like an insulator to generate appreciable current.

One can of course make empirical experiments to measure the performance of various proposed LED designs by fabricating them and then testing them in a laboratory. But this is an expensive and painstaking process that does not scale well; one cannot test thousands of candidate designs this way to isolate the best performing ones. So, it becomes desirable to perform numerical simulations of these designs instead, which – if they are sufficiently accurate and computationally efficient – can lead to a much shorter and cheaper design cycle. (In the near future one may also hope to accelerate the design cycle further by incorporating machine learning and AI methods; but these techniques, while promising, are still not fully developed at the present time.)

So, how can one perform numerical simulation of an LED? By the semiclassical approximation, the wave function of an individual electron should solve the time-independent Schrödinger equation

where is the wave function of the electron at this energy level, and is the conduction band energy. The behavior of hole wavefunctions follows a similar equation, governed by the valence band energy instead of . However, there is a complication: these band energies are not solely coming from the semiconductor, but also contain a contribution that comes from electrostatic effects from the electrons and holes, and more specifically by solving the Poisson equation

where is the dielectric constant of the semiconductor, are the carrier densities of electrons and holes respectively, , are further densities of ionized acceptor and donor atoms, and are physical constants. This equation looks somewhat complicated, but is mostly determined by the carrier densities , which in turn ultimately arise from the probability densities associated to the eigenfunctions via the Born rule, combined with the Fermi-Dirac distribution from statistical mechanics; for instance, the electron carrier density is given by the formula

with a similar formula for . In particular, the net potential depends on the wave functions , turning the Schrödinger equation into a nonlinear self-consistent Hartree-type equation. From the wave functions one can also compute the current, determine the amount of recombination between electrons and holes, and therefore also calculate the light intensity and absorption rates. But the main difficulty is to solve for the wave functions for the different energy levels of the electron (as well as the counterpart for holes).

One could attempt to solve this nonlinear system iteratively, by first proposing an initial candidate for the wave functions , using this to obtain a first approximation for the conduction band energy and valence band energy , and then solving the Schrödinger equations to obtain a new approximation for , and repeating this process until it converges. However, the regularity of the potentials plays an important role in being able to solve the Schrödinger equation. (The Poisson equation, being elliptic, is relatively easy to solve to high accuracy by standard methods, such as finite element methods.) If the potential is quite smooth and slowly varying, then one expects the wave functions to be quite delocalizated, and for traditional approximations such as the WKB approximation to be accurate.

However, in the presence of disorder, such approximations are no longer valid. As a consequence, traditional methods for numerically solving these equations had proven to be too inaccurate to be of practical use in simulating the performance of a LED design, so until recently one had to rely primarily on slower and more expensive empirical testing methods. One real-world consequence of this was the “green gap“; while reasonably efficient LED designs were available in the blue and red portions of the spectrum, there was not a suitable design that gave efficient output in the green spectrum. Given that many applications of LED lighting required white light that was balanced across all visible colors of the spectrum, this was a significant impediment to realizing the energy-saving potential of LEDs.

Here is where the landscape function comes in. This function started as a purely mathematical discovery: when solving a Schrödinger equation such as

(where we have now suppressed all physical constants for simplicity), it turns out that the behavior of the eigenfunctions at various energy levels is controlled to a remarkable extent by the landscape function , defined to be the solution to the equation

As discussed in this previous blog post (discussing a paper on this topic I wrote with some of the members of this collaboration), one reason for this is that the Schrödinger equation can be transformed after some routine calculations to

thus making an effective potential for the Schrödinger equation (and also being the coefficients of an effective geometry for the equation). In practice, when is a disordered potential, the effective potential tends to be behave like a somewhat “smoothed out” or “homogenized” version of that exhibits superior numerical performance. For instance, the classical Weyl law predicts (assuming a smooth confining potential ) that the density of states up to energy – that is to say, the number of bound states up to – should asymptotically behave like . This is accurate at very high energies , but when is disordered, it tends to break down at low and medium energies. However, the landscape function makes a prediction for this density of states that is significantly more accurate in practice in these regimes, with a mathematical justification (up to multiplicative constants) of this accuracy obtained in this paper of David, Filoche, and Mayboroda. More refined predictions (again with some degree of theoretical support from mathematical analysis) can be made on the local integrated density of states, and with more work one can then also obtain approximations for the carrier density functions mentioned previously in terms of the energy band level functions , . As the landscape function is relatively easy to compute (coming from solving a single elliptic equation), this gives a very practical numerical way to carry out the iterative procedure described previously to model LEDs in a way that has proven to be both numerically accurate, and significantly faster than empirical testing, leading to a significantly more rapid design cycle.

In particular, recent advances in LED technology have largely closed the “green gap” by introducing designs that incorporate “-defects”: -shaped dents in the semiconductor layers of the LED that create lateral carrier injection pathways and modify the internal electric field, enhancing hole transport into the active layer. The ability to accurately simulate the effects of these defects has allowed researchers to largely close this gap:

My understanding is that the major companies involved in developing LED lighting are now incorporating landscape-based methods into their own proprietary simulation models to achieve similar effects in commercially produced LEDs, which should lead to further energy savings in the near future.

Thanks to Svitlana Mayboroda and Marcel Filoche for detailed discussions, comments, and corrections of the material here.

(Comment on this)

Cosmic Distance Ladder videos with Grant Sanderson (3blue1brown): commentary and corrections

Grant Sanderson (who runs, and creates most of the content for, the website and Youtube channel 3blue1brown) has been collaborating with myself and others (including my coauthor Tanya Klowden) on producing a two-part video giving an account of some of the history of the cosmic distance ladder, building upon a previous public lecture I gave on this topic, and also relating to a forthcoming popular book with Tanya on this topic. The first part of this video is available here; the second part is available here.

The videos were based on a somewhat unscripted interview that Grant conducted with me some months ago, and as such contained some minor inaccuracies and omissions (including some made for editing reasons to keep the overall narrative coherent and within a reasonable length). They also generated many good questions from the viewers of the Youtube video. I am therefore compiling here a “FAQ” of various clarifications and corrections to the videos; this was originally placed as a series of comments on the Youtube channel, but the blog post format here will be easier to maintain going forward. Some related content will also be posted on the Instagram page for the forthcoming book with Tanya.

Questions on the two main videos are marked with an appropriate timestamp to the video.

Comments on part 1 of the video

• 4:26 Did Eratosthenes really check a local well in Alexandria?
  
  This was a narrative embellishment on my part. Eratosthenes’s original work is lost to us. The most detailed contemperaneous account, by Cleomedes, gives a simplified version of the method, and makes reference only to sundials (gnomons) rather than wells. However, a secondary account of Pliny states (using this English translation), “Similarly it is reported that at the town of Syene, 5000 stades South of Alexandria, at noon in midsummer no shadow is cast, and that in a well made for the sake of testing this the light reaches to the bottom, clearly showing that the sun is vertically above that place at the time”. However, no mention is made of any well in Alexandria in either account.
• 4:50 How did Eratosthenes know that the Sun was so far away that its light rays were close to parallel?
  
  This was not made so clear in our discussions or in the video (other than a brief glimpse of the timeline at 18:27), but Eratosthenes’s work actually came after Aristarchus, so it is very likely that Eratosthenes was aware of Aristarchus’s conclusions about how distant the Sun was from the Earth. Even if Aristarchus’s heliocentric model was disputed by the other Greeks, at least some of his other conclusions appear to have attracted some support. Also, after Eratosthenes’s time, there was further work by Greek, Indian, and Islamic astronomers (such as Hipparchus, Ptolemy, Aryabhata, and Al-Battani) to measure the same distances that Aristarchus did, although these subsequent measurements for the Sun also were somewhat far from modern accepted values.
  
• 5:17 Is it completely accurate to say that on the summer solstice, the Earth’s axis of rotation is tilted “directly towards the Sun”?
  
  Strictly speaking, “in the direction towards the Sun” is more accurate than “directly towards the Sun”; it tilts at about 23.5 degrees towards the Sun, but it is not a total 90-degree tilt towards the Sun.
  
• 5:39 Wait, aren’t there two tropics? The tropic of Cancer and the tropic of Capricorn?
  
  Yes! This corresponds to the two summers Earth experiences, one in the Northern hemisphere and one in the Southern hemisphere. The tropic of Cancer, at a latitude of about 23 degrees north, is where the Sun is directly overhead at noon during the Northern summer solstice (around June 21); the tropic of Capricorn, at a latitude of about 23 degrees south, is where the Sun is directly overhead at noon during the Southern summer solstice (around December 21). But Alexandria and Syene were both in the Northern Hemisphere, so it is the tropic of Cancer that is relevant to Eratosthenes’ calculations.
  
• 5:41 Isn’t it kind of a massive coincidence that Syene was on the tropic of Cancer?
  
  Actually, Syene (now known as Aswan) was about half a degree of latitude away from the tropic of Cancer, which was one of the sources of inaccuracy in Eratosthenes’ calculations.  But one should take the “look-elsewhere effect” into account: because the Nile cuts across the tropic of Cancer, it was quite likely to happen that the Nile would intersect the tropic near some inhabited town.  It might not necessarily have been Syene, but that would just mean that Syene would have been substituted by this other town in Eratosthenes’s account.  
  
  On the other hand, it was fortunate that the Nile ran from North to South, so that distances between towns were a good proxy for the differences in latitude.  Apparently, Eratosthenes actually had a more complicated argument that would also work if the two towns in question were not necessarily oriented along the North-South direction, and if neither town was on the tropic of Cancer; but unfortunately the original writings of Eratosthenes are lost to us, and we do not know the details of this more general argument. (But some variants of the method can be found in later work of Posidonius, Aryabhata, and others.)
  
  Nowadays, the “Eratosthenes experiment” is run every year on the March equinox, in which schools at the same longitude are paired up to measure the elevation of the Sun at the same point in time, in order to obtain a measurement of the circumference of the Earth.  (The equinox is more convenient than the solstice when neither location is on a tropic, due to the simple motion of the Sun at that date.) With modern timekeeping, communications, surveying, and navigation, this is a far easier task to accomplish today than it was in Eratosthenes’ time.
  
• 6:30 I thought the Earth wasn’t a perfect sphere. Does this affect this calculation?
  
  Yes, but only by a small amount. The centrifugal forces caused by the Earth’s rotation along its axis cause an equatorial bulge and a polar flattening so that the radius of the Earth fluctuates by about 20 kilometers from pole to equator. This sounds like a lot, but it is only about 0.3% of the mean Earth radius of 6371 km and is not the primary source of error in Eratosthenes’ calculations.
  
• 7:27 Are the riverboat merchants and the “grad student” the leading theories for how Eratosthenes measured the distance from Alexandria to Syene?
  
  There is some recent research that suggests that Eratosthenes may have drawn on the work of professional bematists (step measurers – a precursor to the modern profession of surveyor) for this calculation. This somewhat ruins the “grad student” joke, but perhaps should be disclosed for the sake of completeness.
  
• 8:51 How long is a “lunar month” in this context? Is it really 28 days?
  
  In this context the correct notion of a lunar month is a “synodic month” – the length of a lunar cycle relative to the Sun – which is actually about 29 days and 12 hours. It differs from the “sidereal month” – the length of a lunar cycle relative to the fixed stars – which is about 27 days and 8 hours – due to the motion of the Earth around the Sun (or the Sun around the Earth, in the geocentric model). [A similar correction needs to be made around 14:59, using the synodic month of 29 days and 12 hours rather than the “English lunar month” of 28 days (4 weeks).]
  
• 10:47 Is the time taken for the Moon to complete an observed rotation around the Earth slightly less than 24 hours as claimed?
  
  Actually, I made a sign error: the lunar day (also known as a tidal day) is actually 24 hours and 50 minutes, because the Moon rotates in the same direction as the spinning of Earth around its axis. The animation therefore is also moving in the wrong direction as well (related to this, the line of sight is covering up the Moon in the wrong direction to the Moon rising at around 10:38).
  
• 14:49 I thought the sine function was introduced well after the ancient Greeks.
  
  It’s true that the modern sine function only dates back to the Indian and Islamic mathematical traditions in the first millennium CE, several centuries after Aristarchus.  However, he still had Euclidean geometry at his disposal, which provided tools such as similar triangles that could be used to reach basically the same conclusions, albeit with significantly more effort than would be needed if one could use modern trigonometry.
  
  On the other hand, Aristarchus was somewhat hampered by not knowing an accurate value for , which is also known as Archimedes’ constant: the fundamental work of Archimedes on this constant actually took place a few decades after that of Aristarchus!
  
• 15:17 I plugged in the modern values for the distances to the Sun and Moon and got 18 minutes for the discrepancy, instead of half an hour.
  
  Yes; I quoted the wrong number here. In 1630, Godfried Wendelen replicated Aristarchus’s experiment. With improved timekeeping and the then-recent invention of the telescope, Wendelen obtained a measurement of half an hour for the discrepancy, which is significantly better than Aristarchus’s calculation of six hours, but still a little bit off from the true value of 18 minutes. (As such, Wendelinus’s estimate for the distance to the Sun was 60% of the true value.)
  
• 15:27 Wouldn’t Aristarchus also have access to other timekeeping devices than sundials?
  
  Yes, for instance clepsydrae (water clocks) were available by that time; but they were of limited accuracy. It is also possible that Aristarchus could have used measurements of star elevations to also estimate time; it is not clear whether the astrolabe or the armillary sphere was available to him, but he would have had some other more primitive astronomical instruments such as the dioptra at his disposal. But again, the accuracy and calibration of these timekeeping tools would have been poor.
  
  However, most likely the more important limiting factor was the ability to determine the precise moment at which a perfect half Moon (or new Moon, or full Moon) occurs; this is extremely difficult to do with the naked eye. (The telescope would not be invented for almost two more millennia.)
  
• 17:37 Could the parallax problem be solved by assuming that the stars are not distributed in a three-dimensional space, but instead on a celestial sphere?
  
  Putting all the stars on a fixed sphere would make the parallax effects less visible, as the stars in a given portion of the sky would now all move together at the same apparent velocity – but there would still be visible large-scale distortions in the shape of the constellations because the Earth would be closer to some portions of the celestial sphere than others. (This problem would be solved if the celestial sphere was somehow centered around the moving Earth rather than the fixed Sun, but then this basically becomes the geocentric model with extra steps.)
  
• 18:29 Did nothing of note happen in astronomy between Eratosthenes and Kepler?
  
  Not at all! There were significant mathematical, technological, theoretical, and observational advances by astronomers from many cultures (Greek, Islamic, Indian, Chinese, European, and others) during this time, for instance improving some of the previous measurements on the distance ladder, a better understanding of eclipses, axial tilt, and even axial precession, more sophisticated trigonometry, and the development of new astronomical tools such as the astrolabe. See for instance this “deleted scene” from the video, as well as the FAQ entry for 14:49. But in order to make the overall story of the cosmic distance ladder fit into a two-part video, we chose to focus primarily on the first time each rung of the ladder was climbed.
  
• 18:30 Is that really Kepler’s portrait?
  
  We have since learned that this portrait was most likely painted in the 19th century, and may have been based more on Kepler’s mentor, Michael Mästlin. A more commonly accepted portrait of Kepler may be found at his current Wikipedia page.
  
• 19:07 Isn’t it tautological to say that the Earth takes one year to perform a full orbit around the Sun?
  
  Technically yes, but this is an illustration of the philosophical concept of “referential opacity“: the content of a sentence can change when substituting one term for another (e.g., “1 year” and “365 days”), even when both terms refer to the same object. Amusingly, the classic illustration of this, known as Frege’s puzzles, also comes from astronomy: it is an informative statement that Hesperus (the evening star) and Phosphorus (the morning star, also known as Lucifer) are the same object (which nowadays we call Venus), but it is a mere tautology that Hesperus and Hesperus are the same object: changing the reference from Phosphorus to Hesperus changes the meaning.
  
• 19:10 How did Copernicus figure out the crucial fact that Mars takes 687 days to go around the Sun? Was it directly drawn from Babylonian data?
  
  Technically, Copernicus drew from tables by European astronomers that were largely based on earlier tables from the Islamic golden age, which in turn drew from earlier tables by Indian and Greek astronomers, the latter of which also incorporated data from the ancient Babylonians, so it is more accurate to say that Copernicus relied on centuries of data, at least some of which went all the way back to the Babylonians. Among all of this data was the times when Mars was in opposition to the Sun; if one imagines the Earth and Mars as being like runners going around a race track circling the Sun, with Earth on an inner track and Mars on an outer track, oppositions are analogous to when the Earth runner “laps” the Mars runner. From the centuries of observational data, such “laps” were known to occur about once every 780 days (this is known as the synodic period of Mars). Because the Earth takes roughly 365 days to perform a “lap”, it is possible to do a little math and conclude that Mars must therefore complete its own “lap” in 687 days (this is known as the sidereal period of Mars). (See also this post on the cosmic distance ladder Instagram for some further elaboration.)
  
• 20:52 Did Kepler really steal data from Brahe?
  
  The situation is complex. When Kepler served as Brahe’s assistant, Brahe only provided Kepler with a limited amount of data, primarily involving Mars, in order to confirm Brahe’s own geo-heliocentric model. After Brahe’s death, the data was inherited by Brahe’s son-in-law and other relatives, who intended to publish Brahe’s work separately; however, Kepler, who was appointed as Imperial Mathematician to succeed Brahe, had at least some partial access to the data, and many historians believe he secretly copied portions of this data to aid his own research before finally securing complete access to the data from Brahe’s heirs after several years of disputes. On the other hand, as intellectual property rights laws were not well developed at this time, Kepler’s actions were technically legal, if ethically questionable.
  
• 21:39 What is that funny loop in the orbit of Mars?
  
  This is known as retrograde motion. This arises because the orbital velocity of Earth (about 30 km/sec) is a little bit larger than that of Mars (about 24 km/sec). So, in opposition (when Mars is in the opposite position in the sky than the Sun), Earth will briefly overtake Mars, causing its observed position to move westward rather than eastward. But in most other times, the motion of Earth and Mars are at a sufficient angle that Mars will continue its apparent eastward motion despite the slightly faster speed of the Earth.
  
• 21:59 Couldn’t one also work out the direction to other celestial objects in addition to the Sun and Mars, such as the stars, the Moon, or the other planets?  Would that have helped?
  
  Actually, the directions to the fixed stars were implicitly used in all of these observations to determine how the celestial sphere was positioned, and all the other directions were taken relative to that celestial sphere.  (Otherwise, all the calculations would be taken on a rotating frame of reference in which the unknown orbits of the planets were themselves rotating, which would have been an even more complex task.)  But the stars are too far away to be useful as one of the two landmarks to triangulate from, as they generate almost no parallax and so cannot distinguish one location from another.
  
  Measuring the direction to the Moon would tell you which portion of the lunar cycle one was in, and would determine the phase of the Moon, but this information would not help one triangulate, because the Moon’s position in the heliocentric model varies over time in a somewhat complicated fashion, and is too tied to the motion of the Earth to be a useful “landmark” to one to determine the Earth’s orbit around the Sun.
  
  In principle, using the measurements to all the planets at once could allow for some multidimensional analysis that would be more accurate than analyzing each of the planets separately, but this would require some sophisticated statistical analysis and modeling, as well as non-trivial amounts of compute – neither of which were available in Kepler’s time.
  
• 22:57 Can you elaborate on how we know that the planets all move on a plane?
  
  The Earth’s orbit lies in a plane known as the ecliptic (it is where the lunar and solar eclipses occur). Different cultures have divided up the ecliptic in various ways; in Western astrology, for instance, the twelve main constellations that cross the ecliptic are known as the Zodiac. The planets can be observed to only wander along the Zodiac, but not other constellations: for instance, Mars can be observed to be in Cancer or Libra, but never in Orion or Ursa Major. From this, one can conclude (as a first approximation, at least), that the planets all lie on the ecliptic.
  
  However, this isn’t perfectly true, and the planets will deviate from the ecliptic by a small angle known as the ecliptic latitude. Tycho Brahe’s observations on these latitudes for Mars were an additional useful piece of data that helped Kepler complete his calculations (basically by suggesting how to join together the different “jigsaw pieces”), but the math here gets somewhat complicated, so the story here has been somewhat simplified to convey the main ideas.
  
• 23:28 Can one work out the position of Earth from fixed locations of the Sun and Mars when the Sun and Mars are in conjunction (the same location in the sky) or opposition (opposite locations in the sky)?
  
  Technically, these are two times when the technique of triangulation fails to be accurate; and also in the former case it is extremely difficult to observe Mars due to the proximity to the Sun. But again, following the Universal Problem Solving Tip from 23:07, one should initially ignore these difficulties to locate a viable method, and correct for these issues later.
  
• 24:04 So Kepler used Copernicus’s calculation of 687 days for the period of Mars. But didn’t Kepler discard Copernicus’s theory of circular orbits?
  
  Good question! It turns out that Copernicus’s calculations of orbital periods are quite robust (especially with centuries of data), and continue to work even when the orbits are not perfectly circular. But even if the calculations did depend on the circular orbit hypothesis, it would have been possible to use the Copernican model as a first approximation for the period, in order to get a better, but still approximate, description of the orbits of the planets. This in turn can be fed back into the Copernican calculations to give a second approximation to the period, which can then give a further refinement of the orbits. Thanks to the branch of mathematics known as perturbation theory, one can often make this type of iterative process converge to an exact answer, with the error in each successive approximation being smaller than the previous one. (But performing such an iteration would probably have been beyond the computational resources available in Kepler’s time; also, the foundations of perturbation theory require calculus, which only was developed several decades after Kepler.)
  
• 24:21 Did Brahe have exactly 10 years of data on Mars’s positions?
  
  Actually, it was more like 17 years, but with many gaps, due both to inclement weather, as well as Brahe turning his attention to other astronomical objects than Mars in some years; also, in times of conjunction, Mars might only be visible in the daytime sky instead of the night sky, again complicating measurements. So the “jigsaw puzzle pieces” in 25:26 are in fact more complicated than always just five locations equally spaced in time; there are gaps and also observational errors to grapple with. But to understand the method one should ignore these complications; again, see “Universal Problem Solving Tip #1”. Even with his “idea of true genius” (which, incidentally one can find in Einstein’s introduction to Carola Baumgardt’s “Life of Kepler“), it took many years of further painstaking calculation for Kepler to tease out his laws of planetary motion from Brahe’s messy and incomplete observational data.
  
• 26:44 Shouldn’t the Earth’s orbit be spread out at perihelion and clustered closer together at aphelion, to be consistent with Kepler’s laws?
  
  Yes, you are right; there was a coding error here.

Comments on the “deleted scene” on Al-Biruni

• Was Al-Biruni really of Arab origin?
  
  Strictly speaking; no; his writings are all in Arabic, and he was nominally a subject of the Abbasid Caliphate whose rulers were Arab; but he was born in Khwarazm (in modern day Uzbekistan), and would have been a subject of either the Samanid empire or the Khrawazmian empire, both of which were largely self-governed and primarily Persian in culture and ethnic makeup, despite being technically vassals of the Caliphate. So he would have been part of what is sometimes called “Greater Persia” or “Greater Iran”.
  
  Another minor correction: while Al-Biruni was born in the tenth century, his work on the measurement of the Earth was published in the early eleventh century.
• Is really called the angle of declination?
  
  This was a misnomer on my part; this angle is more commonly called the dip angle.
• But the height of the mountain would be so small compared to the radius of the Earth! How could this method work?
  
  Using the Taylor approximation , one can approximately write the relationship between the mountain height , the Earth radius , and the dip angle (in radians) as . The key point here is the inverse quadratic dependence on , which allows for even relatively small values of to still be realistically useful for computing . Al-Biruni’s measurement of the dip angle was about radians, leading to an estimate of that is about four orders of magnitude larger than , which is within ballpark at least of a typical height of a mountain (on the order of a kilometer) and the radius of the Earth (6400 kilometers).
• Was the method really accurate to within a percentage point?
  
  This is disputed, somewhat similarly to the previous calculations of Eratosthenes. Al-Biruni’s measurements were in cubits, but there were multiple incompatible types of cubit in use at the time. It has also been pointed out that atmospheric refraction effects would have created noticeable changes in the observed dip angle . It is thus likely that the true accuracy of Al-Biruni’s method was poorer than 1%, but that this was somehow compensated for by choosing a favorable conversion between cubits and modern units.

Comments on the second part of the video

• 8:53 What does it mean for the transit of Io to be “twenty minutes ahead of schedule” when Jupiter is in opposition (Jupiter is opposite to the Sun when viewed from the Earth)?
  
  Actually, it should be halved to “ten minutes ahead of schedule”, with the transit being “ten minutes behind schedule” when Jupiter is in conjunction, with the net discrepancy being twenty minutes (or actually closer to 16 minutes when measured with modern technology). Both transits are being compared against an idealized periodic schedule in which the transits are occuring at a perfectly regular rate (about 42 hours), where the period is chosen to be the best fit to the actual data. This discrepancy is only noticeable after carefully comparing transit times over a period of months; at any given position of Jupiter, the Doppler effects of Earth moving towards or away from Jupiter would only affect the period between consecutive transits by just a few seconds, with the delays or accelerations only becoming cumulatively noticeable after many such transits.
  
  Also, the presentation here is oversimplified: at times of conjunction, Jupiter and Io are too close to the Sun for observation of the transit. Rømer actually observed the transits at other times than conjunction, and Huygens used more complicated trigonometry than what was presented here to infer a measurement for the speed of light in terms of the astronomical unit (which they had begun to measure a bit more accurately than in Aristarchus’s time; see the FAQ entry for 15:17 in the first video).
• 10:05 Are the astrological signs for Earth and Venus swapped here?
  
  Yes, this was a small mistake in the animation.
  
• 18:19 How did astronomers know that the Milky Way was only a small portion of the entire universe?
  
  This issue was known as the “Great Debate” in the early twentieth century. It was only with the work of Leavitt measuring Cepheids in Magellanic clouds and “spiral nebulae” (that we now know to be other galaxies), together with Hubble’s further analysis of distances to these Cepheids, that it was conclusively established that these clouds and nebulae in fact were at much greater distances than the diameter of the Milky Way.
  
• 20:46 Does general relativity alone predict an uniformly expanding universe?
  
  This was an oversimplification. Einstein’s equations of general relativity contain a parameter , known as the cosmological constant, which currently is only computable from experimental data. But even with this constant fixed, there are multiple solutions to these equations (basically because there are multiple possible initial conditions for the universe). For the purposes of cosmology, a particularly successful family of solutions are the solutions given by the Lambda-CDM model. This family of solutions contains additional parameters, such as the density of dark matter in the universe. Depending on the precise values of these parameters, the universe could be expanding or contracting, with the rate of expansion or contraction either increasing, decreasing, or staying roughly constant. But if one fits this model to all available data (including not just red shift measurements, but also measurements on the cosmic microwave background radiation and the spatial distribution of galaxies), one deduces a version of Hubble’s law which is nearly linear, but with an additional correction at very large scales; see the next item of this FAQ.
• 21:07 Is Hubble’s original law sufficiently accurate to allow for good measurements of distances at the scale of the observable universe?
  
  Not really; as mentioned in the end of the video, there were additional efforts to cross-check and calibrate Hubble’s law at intermediate scales between the range of Cepheid methods (about 100 million light years) and observable universe scales (about 100 billion light years) by using further “standard candles” than Cepheids, most notably Type Ia supernovae (which are bright enough and predictable enough to be usable out to about 10 billion light years), the Tully-Fisher relation between the luminosity of a galaxy and its rotational speed, and gamma ray bursts. It turns out that due to the accelerating nature of the universe’s expansion, Hubble’s law is not completely linear at these large scales; this important correction cannot be discerned purely from Cepheid data, but also requires the other standard candles, as well as fitting that data (as well as other observational data, such as the cosmic microwave background radiation) to the cosmological models provided by general relativity (with the best fitting models to date being some version of the Lambda-CDM model).
• 21:15 Where did this guess of the observable universe being about 20% of the full universe come from?
  
  There are some ways to get a lower bound on the size of the entire universe that go beyond the edge of the observable universe. One is through analysis of the cosmic microwave background radiation (CMBR), that has been carefully mapped out by several satellite observatories, most notably WMAP and Planck. Roughly speaking, a universe that was less than twice the size of the observable universe would create certain periodicities in the CMBR data; such periodicities are not observed, so this provides a lower bound (see for instance this paper for an example of such a calculation). The 20% number was a guess based on my vague recollection of these works, but there is no consensus currently on what the ratio truly is; there are some proposals that the entire universe is in fact several orders of magnitude larger than the observable one.
• 23:49 Where can I learn more about this 10% discrepancy in Hubble’s law?
  
  This is known as the Hubble tension: roughly speaking, the various measurements of Hubble’s constant (either from climbing the cosmic distance ladder, or by fitting various observational data to standard cosmological models) tend to arrive at one of two values, that are about 10% apart from each other. The values based on gravitational wave observations are currently consistent with both values, due to significant error bars in this extremely sensitive method; but other more mature methods are now of sufficient accuracy that they are basically only consistent with one of the two values. Currently there is no consensus on the origin of this tension: possibilities include systemic biases in the observational data, subtle statistical issues with the methodology used to interpret the data, a correction to the standard cosmological model, the influence of some previously undiscovered law of physics, or some partial breakdown of the Copernican principle.
  

(Comment on this)

New exponent pairs, zero density estimates, and zero additive energy estimates: a systematic approach

Timothy Trudgian, Andrew Yang and I have just uploaded to the arXiv the paper “New exponent pairs, zero density estimates, and zero additive energy estimates: a systematic approach“. This paper launches a project envisioned in this previous blog post, in which the (widely dispersed) literature on various exponents in classical analytic number theory, as well as the relationships between these exponents, are collected in a living database, together with computer code to optimize the relations between them, with one eventual goal being to automate as much as possible the “routine” components of many analytic number theory papers, in which progress on one type of exponent is converted via standard arguments to progress on other exponents.

The database we are launching concurrently with this paper is called the Analytic Number Theory Exponent Database (ANTEDB). This Github repository aims to collect the latest results and relations on exponents such as the following:

• The growth exponent of the Riemann zeta function at real part (i.e., the best exponent for which as );
• Exponent pairs (used to bound exponential sums for various phase functions and parameters );
• Zero density exponents (used to bound the number of zeros of of real part larger than );

etc.. These sorts of exponents are related to many topics in analytic number theory; for instance, the Lindelof hypothesis is equivalent to the assertion .

Information on these exponents is collected both in a LaTeX “blueprint” that is available as a human-readable set of web pages, and as part of our Python codebase. In the future one could also imagine the data being collected in a Lean formalization, but at present the database only contains a placeholder Lean folder.

As a consequence of collecting all the known bounds in the literature on these sorts of exponents, as well as abstracting out various relations between these exponents that were implicit in many papers in this subject, we were then able to run computer-assisted searches to improve some of the state of the art on these exponents in a largely automated fashion (without introducing any substantial new inputs from analytic number theory). In particular, we obtained:

• four new exponent pairs;
• several new zero density estimates; and
• new estimates on the additive energy of zeroes of the Riemann zeta function.

We are hoping that the ANTEDB will receive more contributions in the future, for instance expanding to other types of exponents, or to update the database as new results are obtained (or old ones added). In the longer term one could also imagine integrating the ANTEDB with other tools, such as Lean or AI systems, but for now we have focused primarily on collecting the data and optimizing the relations between the exponents.

(Comment on this)

Quaternions and spherical trigonometry

Hamilton’s quaternion number system is a non-commutative extension of the complex numbers, consisting of numbers of the form where are real numbers, and are anti-commuting square roots of with , , . While they are non-commutative, they do keep many other properties of the complex numbers:

• Being non-commutative, the quaternions do not form a field. However, they are still a skew field (or division ring): multiplication is associative, and every non-zero quaternion has a unique multiplicative inverse.
• Like the complex numbers, the quaternions have a conjugation

  

  although this is now an antihomomorphism rather than a homomorphism: . One can then split up a quaternion into its real part and imaginary part by the familiar formulae

  

  (though we now leave the imaginary part purely imaginary, as opposed to dividing by in the complex case).
• The inner product

  

  is symmetric and positive definite (with forming an orthonormal basis). Also, for any , is real, hence equal to . Thus we have a norm

  

  Since the real numbers commute with all quaternions, we have the multiplicative property . In particular, the unit quaternions (also known as , , or ) form a compact group.
• We have the cyclic trace property

  

  which allows one to take adjoints of left and right multiplication:

  

• As are square roots of , we have the usual Euler formulae

  

  for real , together with other familiar formulae such as , , , etc.

We will use these sorts of algebraic manipulations in the sequel without further comment.

The unit quaternions act on the imaginary quaternions by conjugation:

This action is by orientation-preserving isometries, hence by rotations. It is not quite faithful, since conjugation by the unit quaternion is the identity, but one can show that this is the only loss of faithfulness, reflecting the well known fact that is a double cover of .

For instance, for any real , conjugation by is a rotation by around :

Similarly for cyclic permutations of . The doubling of the angle here can be explained from the Lie algebra fact that is rather than ; it also closely related to the aforementioned double cover. We also of course have acting on by left multiplication; this is known as the spinor representation, but will not be utilized much in this post. (Giving the right action of makes it a copy of , and the spinor representation then also becomes the standard representation of on .)

Given how quaternions relate to three-dimensional rotations, it is not surprising that one can also be used to recover the basic laws of spherical trigonometry – the study of spherical triangles on the unit sphere. This is fairly well known, but it took a little effort for me to locate the required arguments, so I am recording the calculations here.

The first observation is that every unit quaternion induces a unit tangent vector on the unit sphere , located at ; the third unit vector is then another tangent vector orthogonal to the first two (and oriented to the left of the original tangent vector), and can be viewed as the cross product of and . Right multplication of this quaternion then corresponds to various natural operations on this unit tangent vector:

• Right multiplying by does not affect the location of the tangent vector, but rotates the tangent vector anticlockwise by in the direction of the orthogonal tangent vector , as it replaces by .
• Right multiplying by advances the tangent vector by geodesic flow by angle , as it replaces by , and replaces by .

Now suppose one has a spherical triangle with vertices , with the spherical arcs subtending angles respectively, and the vertices subtending angles respectively; suppose also that is oriented in an anti-clockwise direction for sake of discussion. Observe that if one starts at with a tangent vector oriented towards , advances that vector by , and then rotates by , the tangent vector now at and pointing towards . If one advances by and rotates by , one is now at pointing towards ; and if one then advances by and rotates by , one is back at pointing towards . This gives the fundamental relation

relating the three sides and three equations of this triangle. (A priori, due to the lack of faithfulness of the action, the right-hand side could conceivably have been rather than ; but for extremely small triangles the right-hand side is clearly , and so by continuity it must be for all triangles.) Indeed, a moments thought will reveal that the condition (4) is necessary and sufficient for the data to be associated with a spherical triangle. Thus one can view (4) as a “master equation” for spherical trigonometry: in principle, it can be used to derive all the other laws of this subject.

Remark 1 The law (4) has an evident symmetry , which corresponds to the operation of replacing a spherical triangle with its dual triangle. Also, there is nothing particularly special about the choice of imaginaries in (4); one can conjugate (4) by various quaternions and replace here by any other orthogonal pair of unit quaternions.

Remark 2 If we work in the small scale regime, replacing by for some small , then we expect spherical triangles to behave like Euclidean triangles. Indeed, (4) to zeroth order becomes

which reflects the classical fact that the sum of angles of a Euclidean triangle is equal to . To first order, one obtains

which reflects the evident fact that the vector sum of the sides of a Euclidean triangle sum to zero. (Geometrically, this correspondence reflects the fact that the action of the (projective) quaternion group on the unit sphere converges to the action of the special Euclidean group on the plane, in a suitable asymptotic limit.)

The identity (4) is an identity of two unit quaternions; as the unit quaternion group is three-dimensional, this thus imposes three independent constraints on the six real parameters <img src="https://s0.wp.com/latex.php?latex=%7Ba%2Cb%2Cc%2C%5Calpha%2C%5Cbeta%2C%5Cgamma%7D&#038;bg=ffffff&#038;fg=000000&#038;s=0&#038;c=20201002" srcset="https://s0.wp.com/latex.php?latex=%7Ba%2Cb%2Cc%2C%5Calpha%2C%5Cbeta%2C%5Cgamma%7D&#038;bg=ffffff&#038;fg=000000&#038;s=0&#038;c=20201002 1x, https://s0.wp.com/latex.php?latex=%7Ba%2Cb%2Cc%2C%5Calpha%2C%5Cbeta%2C%5Cgamma

(Comment on this)

On the distribution of eigenvalues of GUE and its minors at fixed index

I’ve just uploaded to the arXiv the paper “On the distribution of eigenvalues of GUE and its minors at fixed index“. This is a somewhat technical paper establishing some estimates regarding one of the most well-studied random matrix models, the Gaussian Unitary Ensemble (GUE), that were not previously in the literature, but which will be needed for some forthcoming work of Hariharan Narayanan on the limiting behavior of “hives” with GUE boundary conditions (building upon our previous joint work with Sheffield).

For sake of discussion we normalize the GUE model to be the random Hermitian matrix whose probability density function is proportional to . With this normalization, the famous Wigner semicircle law will tell us that the eigenvalues of this matrix will almost all lie in the interval , and after dividing by , will asymptotically be distributed according to the semicircle distribution

In particular, the normalized eigenvalue should be close to the classical location , where is the unique element of such that

Eigenvalues can be described by their index or by their (normalized) energy . In principle, the two descriptions are related by the classical map defined above, but there are microscopic fluctuations from the classical location that create subtle technical difficulties between “fixed index” results in which one focuses on a single index (and neighboring indices , etc.), and “fixed energy” results in which one focuses on a single energy (and eigenvalues near this energy). The phenomenon of eigenvalue rigidity does give some control on these fluctuations, allowing one to relate “averaged index” results (in which the index ranges over a mesoscopic range) with “averaged energy” results (in which the energy is similarly averaged over a mesoscopic interval), but there are technical issues in passing back from averaged control to pointwise control, either for the index or energy.

We will be mostly concerned in the bulk region where the index is in an inteval of the form for some fixed , or equivalently the energy is in for some fixed . In this region it is natural to introduce the normalized eigenvalue gaps

The semicircle law predicts that these gaps have mean close to ; however, due to the aforementioned fluctuations around the classical location, this type of claim is only easy to establish in the “fixed energy”, “averaged energy”, or “averaged index” settings; the “fixed index” case was only achieved by myself as recently as 2013, where I showed that each such gap in fact asymptotically had the expected distribution of the Gaudin law, using manipulations of determinantal processes. A significantly more general result, avoiding the use of determinantal processes, was subsequently obtained by Erdos and Yau.

However, these results left open the possibility of bad tail behavior at extremely large or small values of the gaps ; in particular, moments of the were not directly controlled by previous results. The first result of the paper is to push the determinantal analysis further, and obtain such results. For instance, we obtain moment bounds

for any fixed , as well as an exponential decay bound

for , and a lower tail bound

for any . We also obtain good control on sums of consecutive gaps for any fixed , showing that this sum has mean and variance . (This is significantly less variance than one would expect from a sum of independent random variables; this variance reduction phenomenon is closely related to the eigenvalue rigidity phenomenon alluded to earlier, and reflects the tendency of eigenvalues to repel each other.)

A key point in these estimates is that no factors of occur in the estimates, which is what one would obtain if one tried to use existing eigenvalue rigidity theorems. (In particular, if one normalized the eigenvalues at the same scale at the gap , they would fluctuate by a standard deviation of about ; it is only the gaps between eigenvalues that exhibit much smaller fluctuation.) On the other hand, the dependence on is not optimal, although it was sufficient for the applications I had in mind.

As with my previous paper, the strategy is to try to replace fixed index events such as with averaged energy events. For instance, if and has classical location , then there is an interval of normalized energies of length , with the property that there are precisely eigenvalues to the right of and no eigenvalues in the interval , where

is an affine rescaling to the scale of the eigenvalue gap. So matters soon reduce to controlling the probability of the event

where is the number of eigenvalues to the right of , and is the number of eigenvalues in the interval . These are fixed energy events, and one can use the theory of determinantal processes to control them. For instance, each of the random variables , separately have the distribution of sums of independent Boolean variables, which are extremely well understood. Unfortunately, the coupling is a problem; conditioning on the event , in particular, affects the distribution of , so that it is no longer the sum of independent Boolean variables. However, it is still a mixture of such sums, and with this (and the Plancherel-Rotach asymptotics for the GUE determinantal kernel) it is possible to proceed and obtain the above estimates after some calculation.

For the intended application to GUE hives, it is important to not just control gaps of the eigenvalues of the GUE matrix , but also the gaps of the eigenvalues of the top left minor of . This minor of a GUE matrix is basically again a GUE matrix, so the above theorem applies verbatim to the ; but it turns out to be necessary to control the joint distribution of the and , and also of the interlacing gaps between the and . For fixed energy, these gaps are in principle well understood, due to previous work of Adler-Nordenstam-van Moerbeke and of Johansson-Nordenstam which show that the spectrum of both matrices is asymptotically controlled by the Boutillier bead process. This also gives averaged energy and averaged index results without much difficulty, but to get to fixed index information, one needs some universality result in the index . For the gaps of the original matrix, such a universality result is available due to the aforementioned work of Erdos and Yau, but this does not immediately imply the corresponding universality result for the joint distribution of and or . For this, we need a way to relate the eigenvalues of the matrix to the eigenvalues of the minors . By a standard Schur’s complement calculation, one can obtain the equation

for all , where is the bottom right entry of , and are complex gaussians independent of . This gives a random system of equations to solve for in terms of . Using the previous bounds on eigenvalue gaps (particularly the concentration results for sums of consecutive gaps), one can localize this equation to the point where a given is mostly controlled by a bounded number of nearby , and hence a single gap is mostly controlled by a bounded number of . From this, it is possible to leverage the existing universality result of Erdos and Yau to obtain universality of the joint distribution of and (or of ). (The result can also be extended to more layers of the minor process than just two, as long as the number of minors is held fixed.)

This at last brings us to the final result of the paper, which is the one which is actually needed for the application to GUE hives. Here, one is interested in controlling the variance of a linear combination of a fixed number of consecutive interlacing gaps , where the are arbitrary deterministic coefficients. An application of the triangle and Cauchy-Schwarz inequalities, combined with the previous moment bounds on gaps, shows that this randomv ariable has variance . However, this bound is not expected to be sharp, due to the expected decay between correlations of eigenvalue gaps. In this paper, I improve the variance bound to

for any , which is what was needed for the application.

This improvement reflects some decay in the covariances between distant interlacing gaps . I was not able to establish such decay directly. Instead, using some Fourier analysis, one can reduce matters to studying the case of modulated linear statistics such as for various frequencies . In “high frequency” cases one can use the triangle inequality to reduce matters to studying the original eigenvalue gaps , which can be handled by a (somewhat complicated) determinantal process calculation, after first using universality results to pass from fixed index to averaged index, thence to averaged energy, then to fixed energy estimates. For low frequencies the triangle inequality argument is unfavorable, and one has to instead use the determinantal kernel of the full minor process, and not just an individual matrix. This requires some classical, but tedious, calculation of certain asymptotics of sums involving Hermite polynomials.

The full argument is unfortunately quite complex, but it seems that the combination of having to deal with minors, as well as fixed indices, places this result out of reach of many prior methods.

(Comment on this)

AI for Math fund

Renaissance Philanthropy and XTX Markets have announced the launch of the AI for Math Fund, a new grant program supporting projects that apply AI and machine learning to mathematics, with a focus on automated theorem proving, with an initial $9.2 million in funding. The project funding categories, and examples of projects in such categories, are:

1. Production Grade Software Tools

• AI-based autoformalization tools for translating natural-language mathematics into the formalisms of proof assistants
• AI-based auto-informalization tools for translating proof-assistant proofs into interpretable natural-language mathematics
• AI-based models for suggesting tactics/steps or relevant concepts to the user of a proof assistant, or for generating entire proofs
• Infrastructure to connect proof assistants with computer algebra systems, calculus, and PDEs
• A large-scale, AI-enhanced distributed collaboration platform for mathematicians

2. Datasets

• Datasets of formalized theorems and proofs in a proof assistant
• Datasets that would advance AI for theorem proving as applied to program verification and secure code generation
• Datasets of (natural-language) mathematical problems, theorems, proofs, exposition, etc.
• Benchmarks and training environments associated with datasets and model tasks (autoformalization, premise selection, tactic or proof generation, etc.)

3. Field Building

• Textbooks
• Courses
• Documentation and support for proof assistants, and interfaces/APIs to integrate with AI tools

4. Breakthrough Ideas

• Expected difficulty estimation (of sub-problems of a proof)
• Novel mathematical implications of proofs formalized type-theoretically
• Formalization of proof complexity in proof assistants

The deadline for initial expressions of interest is Jan 10, 2025.

[Disclosure: I have agreed to serve on the advisory board for this fund.]

Update: See also this discussion thread on possible projects that might be supported by this fund.

(Comment on this)

On several irrationality problems for Ahmes series

Vjeko Kovac and I have just uploaded to the arXiv our paper “On several irrationality problems for Ahmes series“. This paper resolves (or at least makes partial progress on) some open questions of Erdős and others on the irrationality of Ahmes series, which are infinite series of the form for some increasing sequence of natural numbers. Of course, since most real numbers are irrational, one expects such series to “generically” be irrational, and we make this intuition precise (in both a probabilistic sense and a Baire category sense) in our paper. However, it is often difficult to establish the irrationality of any specific series. For example, it is already a non-trivial result of Erdős that the series is irrational, while the irrationality of (equivalent to Erdős problem #69) remains open, although very recently Pratt established this conditionally on the Hardy–Littlewood prime tuples conjecture. Finally, the irrationality of (Erdős problem #68) is completely open.

On the other hand, it has long been known that if the sequence grows faster than for any , then the Ahmes series is necessarily irrational, basically because the fractional parts of can be arbitrarily small positive quantities, which is inconsistent with being rational. This growth rate is sharp, as can be seen by iterating the identity to obtain a rational Ahmes series of growth rate for any fixed .

In our paper we show that if grows somewhat slower than the above sequences in the sense that , for instance if for a fixed , then one can find a comparable sequence for which is rational. This partially addresses Erdős problem #263, which asked if the sequence had this property, and whether any sequence of exponential or slower growth (but with convergent) had this property. Unfortunately we barely miss a full solution of both parts of the problem, since the condition we need just fails to cover the case , and also does not quite hold for all sequences going to infinity at an exponential or slower rate.

We also show the following variant; if has exponential growth in the sense that with convergent, then there exists nearby natural numbers such that is rational. This answers the first part of Erdős problem #264 which asked about the case , although the second part (which asks about ) is slightly out of reach of our methods. Indeed, we show that the exponential growth hypothesis is best possible in the sense a random sequence that grows faster than exponentially will not have this property, this result does not address any specific superexponential sequence such as , although it does apply to some sequence of the shape .

Our methods can also handle higher dimensional variants in which multiple series are simultaneously set to be rational. Perhaps the most striking result is this: we can find an increasing sequence of natural numbers with the property that is rational for every rational (excluding the cases to avoid division by zero)! This answers (in the negative) a question of Stolarsky Erdős problem #266, and also reproves Erdős problem #265 (and in the latter case one can even make grow double exponentially fast).

Our methods are elementary and avoid any number-theoretic considerations, relying primarily on the countable dense nature of the rationals and an iterative approximation technique. The first observation is that the task of representing a given number as an Ahmes series with each lying in some interval (with the disjoint, and going to infinity fast enough to ensure convergence of the series), is possible if and only if the infinite sumset

to contain , where . More generally, to represent a tuple of numbers indexed by some set of numbers simultaneously as with , this is the same as asking for the infinite sumset

to contain , where now

So the main problem is to get control on such infinite sumsets. Here we use a very simple observation:

Proposition 1 (Iterative approximation) Let be a Banach space, let be sets with each contained in the ball of radius around the origin for some with convergent, so that the infinite sumset is well-defined. Suppose that one has some convergent series in , and sets converging in norm to zero, such that

for all . Then the infinite sumset contains .

Informally, the condition (2) asserts that occupies all of “at the scale “.

Proof: Let . Our task is to express as a series with . From (2) we may write

for some and . Iterating this, we may find and such that

for all . Sending , we obtain

as required.

In one dimension, sets of the form are dense enough that the condition (2) can be satisfied in a large number of situations, leading to most of our one-dimensional results. In higher dimension, the sets lie on curves in a high-dimensional space, and so do not directly obey usable inclusions of the form (2); however, for suitable choices of intervals , one can take some finite sums which will become dense enough to obtain usable inclusions of the form (2) once reaches the dimension of the ambient space, basically thanks to the inverse function theorem (and the non-vanishing curvatures of the curve in question). For the Stolarsky problem, which is an infinite-dimensional problem, it turns out that one can modify this approach by letting grow slowly to infinity with .

(Comment on this)

Higher uniformity of arithmetic functions in short intervals II. Almost all intervals

Kaisa Matomäki, Maksym Radziwill, Fernando Xuancheng Shao, Joni Teräväinen, and myself have (finally) uploaded to the arXiv our paper “Higher uniformity of arithmetic functions in short intervals II. Almost all intervals“. This is a sequel to our previous paper from 2022. In that paper, discorrelation estimates such as

were established, where is the von Mangoldt function, was some suitable approximant to that function, was a nilsequence, and was a reasonably short interval in the sense that for some and some small . In that paper, we were able to obtain non-trivial estimates for as small as , and for some other functions such as divisor functions for small values of , we could lower somewhat to values such as , , of . This had a number of analytic number theory consequences, for instance in obtaining asymptotics for additive patterns in primes in such intervals. However, there were multiple obstructions to lowering much further. Even for the model problem when , that is to say the study of primes in short intervals, until recently the best value of available was , although this was very recently improved to by Guth and Maynard.

However, the situation is better when one is willing to consider estimates that are valid for almost all intervals, rather than all intervals, so that one now studies local higher order uniformity estimates of the form

where and the supremum is over all nilsequences of a certain Lipschitz constant on a fixed nilmanifold . This generalizes local Fourier uniformity estimates of the form

There is particular interest in such estimates in the case of the Möbius function (where, as per the Möbius pseudorandomness conjecture, the approximant should be taken to be zero, at least in the absence of a Siegel zero). This is because if one could get estimates of this form for any that grows sufficiently slowly in (in particular ), this would imply the (logarithmically averaged) Chowla conjecture, as I showed in a previous paper.

While one can lower somewhat, there are still barriers. For instance, in the model case , that is to say prime number theorems in almost all short intervals, until very recently the best value of was , recently lowered to by Guth and Maynard (and can be lowered all the way to zero on the Density Hypothesis). Nevertheless, we are able to get some improvements at higher orders:

• For the von Mangoldt function, we can get as low as , with an arbitrary logarithmic saving in the error terms; for divisor functions, one can even get power savings in this regime.
• For the Möbius function, we can get , recovering our previous result with Tamar Ziegler, but now with type savings in the exceptional set (though not in the pointwise bound outside of the set).
• We can now also get comparable results for the divisor function.

As sample applications, we can obtain Hardy-Littlewood conjecture asymptotics for arithmetic progressions of almost all given steps , and divisor correlation estimates on arithmetic progressions for almost all .

Our proofs are rather long, but broadly follow the “contagion” strategy of Walsh, generalized from the Fourier setting to the higher order setting. Firstly, by standard Heath–Brown type decompositions, and previous results, it suffices to control “Type II” discorrelations such as

for almost all , and some suitable functions supported on medium scales. So the bad case is when for most , one has a discorrelation

for some nilsequence that depends on .

The main issue is the dependency of the polynomial on . By using a “nilsequence large sieve” introduced in our previous paper, and removing degenerate cases, we can show a functional relationship amongst the that is very roughly of the form

whenever (and I am being extremely vague as to what the relation “” means here). By a higher order (and quantitatively stronger) version of Walsh’s contagion analysis (which is ultimately to do with separation properties of Farey sequences), we can show that this implies that these polynomials (which exert influence over intervals ) can “infect” longer intervals with some new polynomials and various , which are related to many of the previous polynomials by a relationship that looks very roughly like

This can be viewed as a rather complicated generalization of the following vaguely “cohomological”-looking observation: if one has some real numbers and some primes with for all , then one should have for some , where I am being vague here about what means (and why it might be useful to have primes). By iterating this sort of contagion relationship, one can eventually get the to behave like an Archimedean character for some that is not too large (polynomial size in ), and then one can use relatively standard (but technically a bit lengthy) “major arc” techiques based on various integral estimates for zeta and functions to conclude.

(Comment on this)

The elephant in the room

The day after the election, I found myself struggling with how to approach the complex analysis class I was teaching. Could I ignore the (almost literal) elephant in the room? Would my students be in the right mental state to learn math? Would I be in the right mental state to teach it?

I opened with the statement that usually in math we have the luxury of working in abstractions far removed from the real world. We are familiar with addressing mathematical problems with the (inessential) connections to the real world stripped away, leaving only the essential features to focus one’s attention. An election, for instance, might be treated as the outcome of people, each of which has a probability of voting for one candidate, and for another… and one can then try to analyze the problem from a dispassionate mathematical perspective. This type of mindset can be illuminating in many contexts. Real world events have real consequences, however, and in light of an event as consequential as the last election, a math lecture on contour integration or the central limit theorem may seem meaningless.

But there is one precious thing mathematics has, that almost no other field currently enjoys: a consensus on what the ground truth is, and how to reach it. Because of this, even the strongest differences of opinion in mathematics can eventually be resolved, and mistakes realized and corrected. This consensus is so strong, we simply take it for granted: a solution is correct or incorrect, a theorem is proved or not proved, and when a problem is solved, we simply move on to the next one. This is, sadly, not a state of affairs elsewhere. But if my students can learn from this and carry these skills— such as distinguishing an overly simple but mathematically flawed “solution” from a more complex, but accurate actual solution—to other spheres that have more contact with the real world, then my math lectures have consequence. Even—or perhaps, especially—in times like these.

(Comment on this)