“Standard” Quantum Mechanics: A Risky Game with Reality

In the previous installment, we discussed Werner Heisenberg's initial formulation of quantum theory. We talked about his matrix mechanics, how he arrived at it by abandoning the search for electronic trajectories imposed by Niels Bohr and his atomic model, and how he chose to focus solely on the intensities that had been observed in the laboratory. We also mentioned that, despite its recognition, Heisenberg's solution was not well received by the scientific community of the time. It was a proposal based on a mathematics that physicists were not accustomed to, and, above all, it created discomfort because it was very difficult to reconcile this new mathematical formalism with a representation of physical reality, or rather, with the representation they were used to, which had been constructed from classical theories.

But the discomfort didn’t last long. Just six months passed before a new formulation, presented as a wave equation, came to light. Its author was Erwin Schrödinger, and the advantages of his proposal, in the eyes of physicists at the time, were obvious: the wave formulation not only avoided the discomfort of the new matrix mathematics and relied on a differential equation, which everyone knew how to work with, but it also (precisely because it employed familiar differential calculus) promised to restore the continuity characteristic of classical theories, avoiding the pesky elements of discontinuity that plagued Heisenberg's matrix mechanics. Thus, Schrödinger's formalism quickly became the preferred quantum framework for most physicists.

Schrödinger promised to restore the continuity characteristic of classical theories, avoiding the pesky elements of discontinuity that plagued Heisenberg's matrix mechanics.

But just as quickly, problems began to emerge, undermining the promise of restoring a classical representation. Schrödinger himself immediately recognized the difficulties: the domain of the quantum wave function, 𝚿, was not a three-dimensional mathematical space (like that which housed classical waves) but a configuration space, a space whose dimensions are determined by the system's degrees of freedom. These degrees of freedom are the ways in which the system can move, can act. For example: a wagon on a straight track has one degree of freedom because it can only move forward or backward, a flat robotic arm with two joints has two, but a spinning top has 6 degrees of freedom (3 degrees because it can move in x, y, z, and 3 more based on its ability to spin, tilt, and “wobble”). While the classical three-dimensional space functions as a sort of neutral and fixed stage, a container that remains always the same, whose dimensions do not vary regardless of what is placed within it, in a configuration space, this is no longer the case, and the number of dimensions of the space is equivalent to the degrees of freedom of the system considered, which completely ruins the possibility of a more or less classical image. This led Schrödinger to conclude:

All these assertions systematically contribute to abandoning the notions of 'electron position' and 'electron trajectory'. If one does not renounce them, contradictions persist. This contradiction has been felt so strongly that it has even led to doubts about whether what occurs in the atom could ever be described within the framework of space and time.

These difficulties, however, did not result in the rejection of Schrödinger's formulation; on the contrary, they were largely set aside following the probabilistic interpretation put forth by Max Born. Born's idea was that the quantum wave function proposed by Schrödinger should actually be interpreted as a wave of probabilities, through which what could be calculated was the probability of finding a particle in a given position. But the problems with this idea were also numerous. First, note that Born's interpretation suddenly introduces a reference to “particles,” for which there is no mathematical counterpart in a theory that, let’s remember, cannot even be understood in a classical space-time manner. It starts from a basic assumption that, as we hinted in the previous installment, is in no way justified by the mathematical formalism of the theory; it comes from outside, from an alien expectation, a hope (conscious or not) of somehow recovering a classical image of the physical world. Thus, by redirecting the quantum wave function towards “elementary particles,” Born turns the intensities (from which Heisenberg had managed to develop the quantum formalism) into secondary measures of something that did not appear in the theory, into probabilities related to an atomistic state of affairs.

Born turns the intensities into secondary measures of something that did not appear in the theory, into probabilities related to an atomistic state of affairs.

But beyond the reference to particles, the real problem was that the probability waves in Schrödinger's formalism could interact with each other, and consequently, an “ignorance-based” interpretation (like the one proposed by Born) made no sense. If we are talking about probabilities, and particularly about the probabilities of a particle being in one position or another, what does it mean for such probabilities to interact with each other as probabilities? What should be able to interact physically are the particles themselves, not the probabilities. What was this probability wave really?

Let’s delve a little deeper into this fundamental point, for which we first need to talk about what probability is. In physics, at least until quantum mechanics, the understanding of probability has always been determined by the attempt to describe a system despite the ignorance of the variables that characterize it. That’s why in classical physics we always talk about an “interpretation in terms of ignorance,” of an epistemic probability. In other words, probability is understood as a calculation made when there is no complete knowledge of the state of affairs one intends to describe, when we do not know exactly what the situation we are facing is, and we produce a calculation to hypothesize, given the knowledge we do have, how it could be the state of affairs. Something exists in a certain state, but we do not have enough information about it, and all we can do is provide probabilities for each possible state.

It is clear then that in the classical case, probability does not refer to something that actually exists (that’s why it’s called “epistemic probability”) but rather is a measure of our own ignorance, of our incomplete knowledge of the state of affairs we want to represent. Thus, for example, since we do not have complete knowledge of all the cards in a poker game, where we only know our own and those that have already been played on the table, we say there is a probability of, say, 0.27 that a certain player has a full house and 0.16 that they have a straight. But ultimately, the real state of affairs is always one; of all the probable states, there is only one that coincides with the current state. Each player actually has a specific set of cards; they either have or do not have a full house or a straight. It becomes evident that the classical notion of probability cannot be applied to quantum mechanics, precisely because quantum probability waves interact with each other. And what could this mean? Undoubtedly, this points to affirming their physical reality. But what is a “probability” that truly exists as such and interacts with other probabilities?

As often happens, despite the serious problems and evident incongruities (which many understood), Born's interpretation managed to establish itself within a community of physicists eager to recover something of the classical representation to which they had been accustomed for centuries. The price to pay for regaining a probabilistic interpretation related to particles was to accept that both particles and quantum probabilities were exceedingly “strange.” Born's interpretation allowed for a semblance of understanding where the concepts of particle and probability, while not at all the ones used until then, maintained the illusion of a return to an approximately classical representation. But along the way, too many problems were piling up: indefinable particles, impossible quantum leaps, multidimensional mathematical spaces, interacting probabilities.

Along the way, too many problems were piling up: indefinable particles, impossible quantum leaps, multidimensional mathematical spaces, interacting probabilities.

At this critical juncture in history, the one who somehow synthesized the situation, that is, who offered a theoretical framework that legitimized the use of classical concepts while simultaneously embracing their incongruence as a positive trait, was Niels Bohr, perhaps the most influential scientist of the 20th century. Bohr had managed to establish an enormous influence over the development of quantum physics in just a few years through his Theoretical Physics Institute at the University of Copenhagen. Through a strategy centered on collaboration and the provision of scholarships for research stays (often funded by the Rockefeller Foundation), the international prestige he had earned with the Nobel Prize in 1922 for his atomic model, and above all his incredible rhetorical and political skills, Bohr subtly directed a large part of the young researchers (most of whom passed through his Institute) towards the issues he considered priorities. The so-called Carlsberg Honorary Residence, inherited from its founder, was a palatial mansion where Niels Bohr lived from 1932 until his death and practically functioned as a diplomatic and social extension of his institute. The parties held there regularly combined high politics, royalty, and cutting-edge science. Bohr brilliantly utilized this social influence and his closeness to political and royal power to fund research, secure visas for foreign scientists persecuted during the war, and bolster the international prestige of his institute.

But what did Bohr say about quantum physics? This is the most interesting part. In principle, he stated that “it would be a mistake to believe that the difficulties of atomic theory could perhaps be avoided by replacing the concepts of classical physics with new conceptual forms.” Bohr maintained that the only possible language for physics is that of classical theories, and that this would always be the case: “the precise interpretation of any measurement must be framed essentially in terms of classical physical theories, and we could also say, in this sense, that the language of Newton and Maxwell will forever remain the language of physicists.”

To justify these (at first glance dogmatic) assertions, Bohr stated something that appears obvious but is subtly misleading. He asserted, as if it were a self-evident truth, the idea that the concepts of classical physics are those of our immediate, ordinary perception. As if the language of classical physics were by nature that of human perception. And that, as we perceive the measuring devices through which we obtain experimental observations via such immediate perception, we can only think about the results and talk about them with classical concepts. Thus, every physical theory must (and will always) inevitably conform to the concepts of classical physics.

We say this is misleading for several reasons. First, because the concepts of classical physics (established primarily by Newton just three centuries ago) were also, at first, something novel and counterintuitive compared to the dominant Aristotelian physics of that time, something that did not correspond with the way reality was perceived. To say that an object will move in the same uninterrupted way if no force is applied goes against our “common sense”; it’s something we only observe in movies where astronauts float without the resistance of gravity.

The concepts of classical physics (established primarily by Newton just three centuries ago) were also, at first, something novel and counterintuitive compared to the dominant Aristotelian physics.

And, moreover, it’s somewhat misleading because it inverts the relationship between theory and experience, between concept and observation. As we explained in the previous note, and as Einstein insisted time and again, it is only the theory that states what has been observed. I need atomistic concepts, like that of a particle, to say that when a “click” is heard in a detector or a spot appears on a photographic plate, I am “observing” a particle. It is evident that I did not directly observe any particle, and that I need a specific theoretical framework to give such meaning to what I observed. But Bohr acts as if it were the other way around: he promotes the idea that with each observation, classical concepts are attached, as something given.

In any case, Bohr is at times quite explicit regarding his program: what he wants to achieve is, in some way, to stretch classical physics (in his own words, “to generalize it”) to accommodate the quantum postulate. The guiding question for him is: how do we maintain the classical concepts, so successful until now, and so accessible to the thought and imagination of physicists, in a way that allows us to now also speak of what quantum mechanics addresses; that is, how to apply such concepts to quantum phenomena.

And that “how” he primarily answers with the creation of perhaps his most famous principle: that of complementarity. According to this principle, quantum objects require, for their complete representation, the use of mutually incompatible classical concepts, such as those of ‘wave’ and ‘particle’. A single state of affairs can thus be described in one experimental situation in terms of particles, and in another situation, in terms of waves. In other words, if we want to describe a quantum phenomenon with classical concepts, we must embrace a certain incongruence; we must accept the use, to talk about the same state of affairs, of concepts (and descriptions) that are incompatible with each other.

In this way, Bohr's proposal involved not only dismissing the question about the intrinsic nature of quantum phenomena and redirecting them to a classical language, but also rendering that classical language essentially inconsistent, as it allowed for the possibility of considering the same state in terms of two notions that are incompatible in the classical framework.

If we want to describe a quantum phenomenon using classical concepts, we must embrace a certain incongruity; we must accept the use of concepts that are incompatible with each other.

But that was not the only or the greatest sacrifice to be made to uphold the concepts of classical physics. Complementarity implied that the determination of the nature of the physical state considered (alternatively in terms of particles or waves) depended on the experimental arrangement used. In other words, the observer's decision regarding the experimental setup now defined the nature of the physical state itself. Measurement now 'created' the state of affairs.

This, which seems like a serious problem, Bohr presented not only as inevitable but directly as a positive trait, as the most important epistemological lesson of quantum physics. As he often repeated: 'We are not only spectators but also actors in the great drama of existence.' When we hear talk of spirituality or quantum 'esotericism,' and of things like how one can, with intention or thought, create one's own reality, these are perhaps exaggerated expressions, but ultimately anchored in this bold move by Bohr, which legitimized relativism, for the first time in the history of science, as valid within physics. From then on, the physical state was dependent on the perspective considered.

Moreover, as the different representations turned out to be incompatible with each other, it was no longer possible to conceive of a state that was independent of perspectives. Thus, objectivity in physics was lost. Between sacrificing classical concepts and sacrificing the objectivity of physical theories, Bohr opted for the latter (noting that this was not the only option). A sacrifice that, for some physicists, was undoubtedly unacceptable. In a letter to Schrödinger, Einstein writes:

You are the only contemporary physicist, besides Laue, who realizes that one cannot evade the assumption of reality (as long as one is honest). Most [physicists] simply do not realize the risky game they are playing with reality (reality is something independent of what is established experimentally).

But Einstein's reservations were seen as those of an old physicist, unable to grasp the revolution that was taking place, and Bohr's gamble paid off. It is precisely under the influence of Bohr's ideas that Paul Dirac developed, in his book The Principles of Quantum Mechanics, an axiomatic formulation that became the 'standard' interpretation of quantum physics, still taught in universities around the world.

Dirac appropriated Bohr's relativism to define, now in vector terms, the quantum 'state' of the physical system as dependent on the different reference systems considered (in technical terms, on the basis of the vector space). In this way, it was accepted that the determination of a physical state in quantum mechanics was dependent on the basis or context selected, that is, on the perspective considered. Thus, following Bohr, the physical state could no longer be conceived as something independent of perspectives.

But in addition to continuing and axiomatizing these Bohr ideas, Dirac introduced a new element: the projection postulate. This refers to a new physical phenomenon (which, by the way, has never been observed experimentally): the collapse of the wave function. This tells us that when measuring a property of a physical system (of the wave function), we produce a physical change that causes it to collapse, and what we observe in the measuring devices is the result of such a collapse produced by the observer. But why create a new physical phenomenon? Is this invented collapse really necessary?

Let's review what Dirac does. His starting assumption is, following Bohr, that quantum mechanics speaks of a microscopic world made up of 'quantum particles'. From there, and taking Born's interpretation as valid, he assumes that what the theory must account for is singular observations, like a unique 'click' in a detector. And this is because such a singular result would express the presence (or absence) of a supposed particle: a quantum particle hit the detector or the plate.

Thus, Dirac's main reference for quantum mechanics is singular events, binary observations. This is the key to a self-generated problem arising from the imposition of Bohr's classical concepts: by wanting to talk about particles, there is an emphasis on singular observation. But quantum mechanics does not refer to such singular events; its quantum formalism does not cooperate with such an assumption and does not tell us what singular observation we will obtain in each measurement. Quantum mechanics speaks of something else; it speaks, both in matrix mechanics and in wave formulation, of elements valued intensively and not binarily, of the interaction of 'probabilities' or of actions quantified intensively.

Quantum mechanics speaks of something else; it speaks, both in matrix mechanics and in wave formulation, of elements valued intensively and not binarily, of the interaction of 'probabilities' or of actions quantified intensively.

It thus speaks of intensive patterns that, if we wanted to measure them with experiments where we obtain a singular result, a click, at a time, we would of course need to repeat the experiment, producing a frequency that accounts for the intensive value. Quantum mechanics, in fact, predicts such intensive patterns with total certainty and accuracy (so it is not that quantum mechanics is uncertain, but rather that its certainty is, let’s say, intensive and not binary).

In this case, as can be seen, the singular observation is not the fundamental aspect to explain, but simply a partial measure, an incomplete piece of information, to be completed precisely by repetition. It is only if we think that the reference of the theory is singular observations (expressions of particles) that we have a problem, since between the intensive quantities given by the formalism and each singular result, an abyss opens up.

It is to somehow bridge that abyss that Dirac introduced the collapse of the wave function. Such a collapse, which we repeat, has no foundation in quantum formalism, and to this day no experimental proof has been offered, has nevertheless become a fundamental part of quantum axiomatics, of the standard formulation, in the form of a strange 'measurement postulate.' When you hear about the 'measurement problem,' which is still endlessly debated today, it is this problem they are referring to. A problem that, as we attempt to demonstrate, actually results from assumptions that are, at the very least, questionable.