(See Part 1 here. Also I am going to suspend the limit of three comments per post for this series of posts because it is a topic that benefits from back and forth discussions.)

It looks like I may have not been sufficiently precise in my first post, leading to some confusion, so I am going to take a slight detour from my series of posts on this topic to address an issue that came up in the comments about the nature of the probability and statistics that is used in quantum theory and how it differs from what we use in everyday life, in particular, the nature of the uncertainty in predicting outcomes. (As always with quantum mechanics, since the phenomena involved are invisible to our senses and often counter-intuitive, we have to use analogies and metaphors to try and bring out the ideas, with the caveat that those never exactly represent the reality.)

Let’s start with classical statistics that we use in everyday life in a situation where the results of a measurement are binary. Suppose that we want to know what percentage of a population has heights less that five feet. If we measure the height of a single person, that will be either more or less than five feet. It will not give us a probability. How do we find that? We take a random sample of people and measure their heights. From those results, we can calculate the fraction of people less than five feet by dividing the number in that category by the size of the sample. That fraction also now represents the probability that if we pick any future person at random, that person will be shorter than five feet. When we pick a random person, we do not know which category they will fall into but we do know that it will be either one or the other. What we also believe is, that in the classical world, each person’s height was fixed before we measured it. We just did not know it beforehand.

It is possible that we may already have a prediction for that probability because (say) the population that we are studying is a subset of a larger population where the result has already been determined and we have no reason to think that this subset differs in any significant way. So our study of the subset could be taken as a test of the hypothesis that its properties are the same as those of the larger population.

Moving on to the quantum case, we want to find out the probability that, on measurement, an electron (or photon) that was prepared in a superposition of spin up and spin down states, will be found to be in (say) a spin up state. A single measurement will again give us only a yes or no answer, not a probability. To get a probability, we need to make a large set of measurements. We cannot repeat the measurement on that same electron because once it is put into the spin up state, it will remain in that state. To get a probability, what we need to do is create a large number of identically prepared electrons, all in the same superposition of states, and take measurements on each. Some will be found spin up and others will be found spin down. From that information, we can calculate the probability of the result of any future measurement on another identically prepared state being found spin up. If the original superposition of states was also the solution of the Schrodinger equation (the quantum equivalent of Newton’s laws of motion in classical physics), it would predict the probability of getting the result and thus could serve as a test of our theories of quantum mechanics. Up until now, it has generally passed those tests, which is why we have such confidence in its correctness.

There are two major differences between the classical and quantum cases discussed above. One is that in the classical case, when we pick a random person and make the measurement of their height, we believe that the person already had that height (more or less than five feet) even before the measurement, while in the quantum case, we believe that the result only came into being as a result of the measurement. We did not know what result we would get, we only had probabilities. The second is that in the classical human case, each person is assumed to be different. If they were all identical, the results of measurements would also be identical. But in the quantum case, despite all the initial states being identically prepared, yet they give different results upon measurement. In other words, in the classical case, the population being measured consists of different states (i.e., heights). In the quantum case, the population being measured consists of identically prepared states.

A question that will naturally arise is how we know that the electron did not have the property prior to measurement, just like the humans. After all, all we have are the results of measurements, not anything prior to it. This was a very difficult question that was hotly debated from the inception of quantum mechanics in the first couple of decades of the twentieth century, and for the longest time there seemed to be no way of experimentally distinguishing the two views. It took the work of John S. Bell in 1964 to show that it was possible. He derived an inequality (now referred to as Bell’s theorem or Bell’s inequality) which could be used to distinguish the two views. However the experiments were very difficult to do and it took another two decades before results started coming in and they generally supported the idea that the measured state came into being as a result of the measurement and did not have it before. In other words, there is no objective reality in the quantum world. (I have to add that not everyone agrees with this. There have been some ingenious efforts to find ways of bringing back objective reality into the quantum world while being consistent with all the other results but these efforts have not gained broad support. Ironically, Bell himself was a supporter of those efforts, despite his theorem seeming to work against it)

I would like to add a personal note. I believe that John Bell was one of the most profound thinkers on quantum mechanics ever, and his sudden death in 1990 at the young age of 62 was a major blow. He was also an exceedingly modest and genial man, a quality that shines through in his writings. He was devoid of the massive ego and arrogance that other prominent physicists unfortunately often have. I had the good fortune to meet him at a physics conference a year before his death where he was given a major award. After he had given his talk, I encountered him in the lobby and asked him about some ideas he had presented and we sat down on some chairs and had a wonderful conversation. Even though he was famous and I was a mere unknown postdoctoral student, he seemed quite content to spend time with me and to also autograph my copy of the book Speakable and unspeakable in quantum mechanics which is a collection of all his published papers on the foundations of quantum mechanics. I am not a collector but that is the one book of mine that I will never lend to anyone for fear of losing it.