Ask Ethan: Is dark energy no longer a cosmological constant?

The story of the expanding Universe has been a back-and-forth one over the past 110 years: ever since general relativity was first introduced. Initially, Einstein introduced the notion of a cosmological constant — a form of energy inherent to the fabric of space itself — to prevent a matter-filled Universe from collapsing. When we discovered that the Universe was expanding, the constant disappeared, eventually leading Einstein to declare it his biggest blunder. Then in the 1990s, a surprising collection of data indicated that the Universe’s expansion was accelerating, a discovery that revived the cosmological constant. The combination of supernova, cosmic microwave background, and large-scale structure data all appeared to demand it.

But now, more than 25 years later, an interesting set of evidence has emerged, suggesting (but not proving) that what we observe as dark energy may not be a constant, but instead is changing over time. Others, however, including me, still prefer the “cosmological constant” interpretation, and remain skeptical of the idea of evolving dark energy. How can so many different people look at the same evidence and reach different conclusions? That’s what Robert Smičiklas wants to know, asking:

“There are a lot of news and research papers lately that claim that dark energy is dynamical and that it is getting weaker. Many of them claim that we now have 3, 4, or 5 sigma proof that it is dynamical and not a constant. Some even claim we will soon have 7-sigma proof that it is dynamical. Then there are scientists like George Efstathiou who claim there are mistakes in DESI data and there is no strong proof for dynamical dark energy over [a cosmological constant]. Is dark energy being dynamical the most likely scenario right now?”

This is a big question: the question over the nature of the dominant form of energy in the Universe. Let’s go through what we know, and help understand why there are multiple viable opinions and interpretations of where we are today, but no clear consensus.

Three different types of measurements, distant stars and galaxies (from supernovae), the large-scale structure of the Universe (from BAO), and the fluctuations in the CMB, tell us the expansion history of the Universe and its composition. Constraints on the total matter content (normal+dark, x-axis) and dark energy density (y-axis) from three independent sources: supernovae, the CMB (cosmic microwave background), and BAO (which is a wiggly feature seen in the correlations of large-scale structure).

What you can see, above, is the story of the Universe that we first uncovered in the 1990s, and then cemented during the 2000s: the fact that three independent lines of evidence all supported the same picture of the cosmos. We often call this our concordance cosmological model, or ΛCDM [for Lambda, which stands for dark energy as a cosmological constant, plus CDM, or cold dark matter], which is about 70% cosmological constant, 25% dark matter, and 5% normal (atom-based) matter. These three evidence are the big ones in cosmology, as they include:

• the CMB, or cosmic microwave background, which is the leftover glow from the Big Bang and which has tiny, one-part-in-30,000 fluctuations that detail the seeds of what will grow into the large-scale structure of our Universe,
• measurements of distances to and redshifts of type Ia supernovae, which are “standard candles” whose physics and intrinsic brightness is known, and so measuring them allows us to understand how the Universe has expanded since we observed them,
• and the large-scale structure of the Universe, which particularly measures how likely you are to find a galaxy located at a specific distance from another galaxy, with our cosmic expansion history revealed by measuring a specific feature known as “baryon acoustic oscillations” within this structure.

For a long time, these three pieces of evidence all appeared to converge on a single set of parameters for our Universe: our concordance, ΛCDM cosmology.

There are many possible ways to fit the data that tells us what the Universe is made of and how quickly it’s expanding, but these combinations all have one thing in common: they all lead to a Universe that’s the same age, as a faster-expanding Universe must have more dark energy and less matter, while a slower-expanding Universe requires less dark energy and greater amounts of matter.

But starting in the 2010s, cracks began to emerge in this picture. This is because measurements of all three data sets became more and more precise with the advent of both improved data and an improved understanding of our errors and uncertainties in that data.

• For the CMB, we had the Planck satellite, which measured these minuscule fluctuations in the CMB down to very tiny angular scales and helped us better understand the cosmic foregrounds (like galactic dust) that could pollute or bias these observations. It wound up converging on a picture where our Universe was expanding at 67 km/s/Mpc, with about 32% of our cosmic energy density being matter (including both dark and normal matter) and the other 68% being dark energy, consistent with a cosmological constant.
• For type Ia supernovae, we independently measured the expansion rate of the Universe to be 73 km/s/Mpc, with more supernovae and an improved understanding of the errors and uncertainties associated with not only those explosions, but of other calibration aspects of the cosmic distance ladder, bringing our uncertainty down to just ~1-2% in that measurement. This supports a Universe with slightly more dark matter and slightly less dark energy than the CMB indicates: about 34% matter and about 66% dark energy.
• And for the large-scale structure data, we’re conducting wide-field surveys where we map galaxies, with data from DESI, the Dark Energy Spectroscopic Instrument, providing the best measurements of the evolution of this baryon acoustic oscillation feature at present. It says very little about the expansion rate of the Universe on its own, but does point to a picture where the Universe is about 29% matter and about 71% dark energy.

In other words, as we’ve pinned down these measurements to greater and greater precision, we find that they don’t all point to the same exact picture, and instead, a series of tensions has emerged.

This graph shows a comparison between the value of H0, or the expansion rate today, as derived from Hubble space telescope Cepheids and anchors as well as other subsamples of JWST Cepheids (or other types of stars) and anchors. A comparison to Planck, which uses the early relic method instead of the distance ladder method, is also shown. Very clearly, the distance ladder and early relic methods do not yield mutually compatible results.

The big question is over how we interpret these tensions. Some choose to focus on a problem that we call the Hubble tension: the fact that the CMB and the supernova data seem to indicate different expansion rates for the Universe. Sure, 67 km/s/Mpc and 73 km/s/Mpc may not look particularly different from one another, but that’s because I didn’t show you the errors on those. For the CMB, the uncertainties on that figure are less than ±1 km/s/Mpc, and for the supernova data, the uncertainties, after a long time error-trapping each individual contribution, have been beaten down to be only about ±1 km/s/Mpc as well.

Now, fundamentally, these two methods aren’t measuring the same exact thing. The CMB takes a signal that was produced early on in cosmic history, was imprinted onto the face of the Universe back when only 380,000 years had elapsed since the Big Bang, and represents light that we observe after it’s been traveling through the Universe for around 13.8 billion years, being stretched and redshifted by cosmic expansion before it enters our instruments. It’s an early relic, left over from the Big Bang, observed today.

On the other hand, to measure the supernova data, we have to start here, at late times, and look outward: back to earlier times in cosmic history. This is an example of a distance ladder method, which starts at the present day and looks back. While the CMB and the supernova data aren’t mutually compatible, the large gap — in both distance and time — between the most distant supernova and the surface of the CMB means that perhaps the expansion rate has changed, or that perhaps the contents of the Universe have changed over that time.

A compilation of distance ladder measurements of H0 in comparison to the Pantheon+SH0ES data, which shows the smallest error bars of any distance ladder measurement. Here, the third rung of the distance ladder is redone using a variety of techniques, not just optical type Ia supernovae. The legend shows the different techniques included in constructing this figure. For comparison, the CMB-derived value is shown in the gray bar at left: quite distinct from all distance ladder measurements.

There are even some who work on the cosmic distance ladder who disbelieve the cited small error bars on the part of those working on type Ia supernovae, and instead assert that the Hubble parameter, rather than being ~73 km/s/Mpc, could actually be more like 70 or even a value in the high-60s, which would alleviate the tension and lead to a Universe that was entirely consistent with what the CMB data indicates. Many dispute that interpretation (for my part, I do as well), but it remains unclear what the resolution to this tension is.

• Perhaps the energy contents of the Universe have evolved over time.
• Perhaps dark energy is changing or decaying over time.
• Perhaps something weird is going on with the dark matter in the Universe.
• Or perhaps there are unidentified errors or uncertainties in our observations that we’re failing to account for.

At this point, we don’t know what the resolution to this puzzle is, but the Hubble tension isn’t going away anytime soon.

Now, however, we come to the third piece of the puzzle: the introduction of large-scale structure data, and in particular of the DESI data. DESI, to date, has been the largest, deepest, and most sensitive survey of the Universe’s large scale structure, comprehensively mapping galaxies across enormous swaths of the sky down to faint magnitudes and out to greater distances than any other survey that preceded it.

This slice of the DESI data maps celestial objects from Earth (center) to billions of light years away. Among the objects are nearby bright galaxies (yellow), luminous red galaxies (orange), emission-line galaxies (blue), and quasars (green). The large-scale structure of the universe is visible in the inset image, which shows the densest survey region and represents less than 0.1% of the DESI survey’s total volume.

If we try to look at the DESI data on its own, and ask questions like “what can it tell us about the expansion rate?” or “what can it tell us about the relative amounts of matter vs. dark energy density?” it turns out that even with this best-ever data set, it can’t tell us very much with any degree of confidence. If you look at the large-scale structure data, you have two approaches:

• you can start nearby and use some sort of “anchor point” and work outward from there, similar to the distance ladder method,
• or you can start with the spectrum of seed fluctuations that were in place at early times (such as at the time of the CMB) and work forward, toward us, from there, similar to the early relic method.

If you take the first method, you get something consistent with the (supernovae’s) distance ladder method; if you take the second method, you get something consistent with the (CMB’s) early relic method. For the expansion rate, it can’t tell us very much on its own.

Then you can go ahead and ask about the matter density versus the dark energy density. As you can see, below, DESI data indicates that only about 29% of the Universe is made of matter, versus 71% for dark energy, while the CMB indicates a Universe that’s 32% matter (and 68% dark energy) and supernova data, depending on which data set you use, yields a value that’s between 34-37% matter (and 63-66% dark energy). These values don’t miss each other by much, but they don’t converge to a single, unified picture either.

This figure, from the DESI collaboration’s second data release’s results paper, shows the different values of the matter density that are preferred by six different data sets: DESI’s first and second releases, the CMB, and the supernova samples of Pantheon+, Union, and DESY5. Note that BAO and supernova data sets are not really compatible with one another, and that the three different supernova data sets (Pantheon+, Union, and DESY) give wildly different results from one another.

So now we come to the big question: given all three lines of evidence — from the CMB, from supernovae (or other distance ladder methods), and from large-scale structure (such as the evolution of the baryon acoustic oscillation feature) — how can we make one coherent, unified picture of the Universe? Or, as some people are daring to ask, can you even make a coherent, non-contradictory picture of the Universe out of this, or does something have to give?

There are a few different approaches one can take. The most conservative approach, and I would argue that it’s too conservative, is to assume that there are still-unidentified errors or sources of bias in one or more of these data sets, and that there will be a combination of parameters for:

• the matter density,
• the dark energy density,
• and the expansion rate,

that will all fit the various data sets just fine, and that dark energy will remain a cosmological constant.

A less conservative approach is to declare that something fishy is likely going on, but that we have to be very cautious about assigning a cause as to what it is with the data we have right now. Many different aspects of the Universe could be changing or evolving, including:

• the dark energy density,
• the dark energy equation-of-state (i.e., the effective properties of dark energy),
• the dark matter density,
• or the properties of our Universe, including the possibility that we live in a region that arose from an unusual, statistically rare (but not impossible) fluctuation in initial density,

and that we’ll need to acquire superior data in order to distinguish between these various possibilities.

These graphs show the fit for evolving dark energy, in terms of the parameters w_0 and w_a, where a constant cosmological constant for dark energy corresponds to w_a = 0 and w_0 = -1, exactly. Note that the DESI data on its own is consistent with constant dark energy, but that when you combine CMB and supernova (for example, DESY5, as shown in the middle panel) data with it, it favors evolving dark energy instead.

That’s the approach that I personally favor, and I’m happy to elaborate as to why. When people say “dark energy is evolving, and I have evidence for it at 3-sigma, 4-sigma, 5-sigma, etc.,” you have to combine these different data sets that are all pointing to incompatible pictures of the Universe. For example, as I pointed out just a few months ago:

• When you look at the DESI data alone, it only favors evolving dark energy over the standard ΛCDM model at less than 2-sigma significance.
• When you look at the DESI data combined with Planck CMB data, there’s a 3.1-sigma preference for evolving dark energy over ΛCDM.
• And then if you look at combining DESI data with Planck CMB data and also supernova data, the significance of the preference for evolving dark energy over ΛCDM either increases (up to 3.8-sigma for Union data or 4.2-sigma for DESY5 data) or decreases (down to 2.8-sigma for Pantheon+ data) depending on which supernova data set you combine it with.

I will say that for me, it just doesn’t make sense to assume that the one thing that’s evolving is dark energy based on the data we have. What I’d want, in order to convince myself that was true, is to have any one line of evidence be so good in support of dark energy’s evolution that it’s compelling without needing to combine it with any other data set. While DESI has the best large-scale structure data, at present, there are at least four other missions that are going to gather superior data over the next few years:

In the aftermath of inflation, signatures are imprinted onto the Universe that are unmistakably inflationary in origin. While the CMB provides an early-time “snapshot” of these features, that’s just one moment in history. By probing the large variety of times/distances accessible to us throughout cosmic time, such as with large-scale structure, we can obtain information that would otherwise be obscure from any single snapshot.

I think when the data from these missions comes in, we’ll get to see whether the tentative evidence for evolving dark energy, suggested by DESI, actually holds up, or whether there’s a regression to the mean. Remember: DESI’s evidence for dark energy’s evolution only indicates a slight evolution and only at recent times, favoring a cosmological constant for the first 8-to-9 billion years of cosmic history and suggesting that the strength of dark energy has weakened by about 15-20% over the most recent 5 billion years or so. Less than 2-sigma evidence is what we call “no evidence” in physics and astronomy, so acquiring this superior, forthcoming data will really clarify what we’re looking at.

Nevertheless, a lot of people are making a lot of noise in the cosmology community by staking out the least conservative opinion: that dark energy is evolving, and that only old curmudgeons (like me) are either demanding better evidence or insisting that we look at the other aspects of the Universe that could be different from our assumptions. It’s only if you demand that:

• there are no other exotic, changing forms of matter or energy,
• that we don’t live in a statistically unusual pocket of the Universe,
• and insist that the Hubble tension can only be resolved by allowing dark energy to evolve,

that the case for evolving dark energy becomes strong. In other words, if our only test is an “evolving dark energy model” against “a standard ΛCDM model,” when we combine all the different lines of evidence together, then that’s when we’re going to conclude that dark energy is evolving with high significance. I think the below graphic, by large-scale structure specialist Dr. Claire Lamman, illustrates this possibility quite nicely.

This fun graphic illustrates the tension on Λ, Einstein’s cosmological constant, exerted by combining supernova data (right), baryon acoustic oscillations (left), and the cosmic microwave background (top). When all three data sets are combined, the idea of a cosmological constant struggles to hold together; it’s possible that something, but perhaps not necessarily Λ, is going to give.

However, as soon as we insist that evolving dark energy is the solution to all of our cosmic problems, we have to look at the rest of our derived parameters and ask whether we still have a consistent picture of reality. The answer, quite soberingly, is that we do not. If dark energy evolves as the combined large-scale structure, CMB, and supernova data indicates under that least conservative assumption, we encounter two pathologies. The first is that the sum of the neutrino masses, which affect the expansion of the Universe particularly early on in cosmic history, winds up being negative, which is unphysical. (From neutrino oscillation data, we are certain that sum is no less than 0.059 eV/c², which is positive and physically admissible.) And the second is that if dark energy evolves as the data indicates, it may be a form of phantom energy, which violates multiple energy conditions that the Universe is expected to obey.

None of this is evidence against dark energy’s evolution, but when you take all of it together, it fails to convince an unbiased (but knowledgeable) observer that evolving dark energy is the only, or even the best, solution to these cosmic puzzles and tensions. It’s perhaps telling that there’s an offset in magnitude between the differing supernova collaborations even for the same supernovae, which suggests that combining these different data sets together and drawing conclusions about our Universe from it is still a risky venture. I think the smartest approach is to keep an open mind about the possibility that dark energy evolves, but to demand better, higher-quality, and less ambiguous data before drawing a definitive conclusion. While many scientists are understandably racing to be the first to make a new discovery, the rest of the world should be cautious enough to demand that we take sufficient care to ensure that our early conclusions won’t simply crumble when better evidence arrives.

Send in your Ask Ethan questions to startswithabang at gmail dot com!