Thermal and Statistical Mechanics

This post gives a gentle introduction to thermal and statistical mechanics, including mysterious concepts like entropy, Boltzmann and quantum statistics, and Bose-Eeinstien condensation.

Entropy

The journey of the thermal and statistical mechanics starts with entropy, one of the most well-known and misunderstood concepts in Physics. Start from the basic definitions, we will see that it is actually very intuitive and can be explained with simple combinatorics.

Thermal and statistical mechanics studies complex systems of particles. Let’s first imagine that our system is consiting of coin-like particles, which have states of either heads or tails. If you find it too ridiculous, just think about electrons with spin up and down. Supppose that our system has $N$ such particles. The particles have identical properties, but if we virtually give each particle a label, there are $2^N$ possible states, which we call microstates. But since the particles are identical, many such states are indistinguishable. Actually, there are only 101 mascrostates, which is determined by the number of particles with heads, or equivalently by the number of particles with tails. Let’s denote the number of particles with heads as $n$, and we denote the number of microstates in a specific macrostate as $\Omega(N, n)$, or the multiplicity of the macrostate. In this case, $\Omega(N, n) = {N \choose n}$. The sum of the multiplicities should always equal the total number of microstates, in this case, $\sum_n=0^N \Omega(N, n) = \sum_{n=0}^N {N \choose n} = 2^N$.

Now here comes the the fundamental assumption of statistical mechanics:

In an isolated system in thermal equilibrium, all accessible microstates are equally probable.

In classical mechanics, physical interactions are deterministic, and we have a clockwork world. Quantum mechanics subverts this world view, and theorizes that many properties of particles are uncertain before measurements. Statistical mechancis forgoes determining the interactions of individual particles, because the systems of interest are too complex. Even though the fundamental assumption is probabilistic in nature, it is different from the probability wave in quantum mechanics (which comes after statistical mechanics).

The implication of this simple and intuitive assumption is profound. Since each microstate is equally probable, the macrostate with the greatest multiplicity is the most likely to occur, or “the multiplicity tends to increase”. Now we introduce the definition of entropy $S$,

where $k$ is the Boltzmann constant.

The “law of the increase of multiplicity” now becomes the “law of the increase of entropy”, one version of the well known second law of thermodynamics, which is actually just a strong statement about probabilities.

The second law of thermodynamics: Any large system equalibrium will be found in the state with largest multiplicity (largest entropy).

From here, we will consider different types of interactions between systems and derive familiar concepts like temperature and pressure.

Temperature

The concept of temperature arises when we consider thermal interactions, or the flow of heat between objects. We find that heat flows from objects with high temperature to objects with low temperature, and we will introduce a statistical mechanics definition in accordance to this behavior.

Suppose we have two systems $A$ and $B$, with entropy $S_A$ and $S_B$, energy $U_A$ and $U_B$. Let the total entropy be $S = S_A+S_B$. We will assume that only thermal interactions occur, and specifically, the total number of particles in each system and the volumes of the systems are constants. According to the second law, at the thermal equalibrium, the multiplicity or entropy is maximized, or

Finally, since energy is conserved, $U = U_A + U_B$ is cosntant, and $d U_A = -d U_B$, so

This suggests that temperature should somehow relates to $\frac{\partial S}{\partial U}$. In fact, the Boltzmann constant has the unit $J / K$, so $\frac{\partial S}{\partial U}$ has the unit $1/K$. This suggests the definition of temperature $T$ as

where the subscripts $N, V$ suggests that the number of particles and the volume are constants. In other words, heat tends to transfer to objects with larger $\frac{\partial S}{\partial U}$ or lower temperature.

Pressure

The concept of pressure arises when we consider mechanical interactions, or the change of volumes of two systems. We would expect systems with high pressure will tend to expand, or “receive volume” from systems with lower pressure. Suppose again we have two systems $A$ and $B$, with entropy $S_A$ and $S_B$, volumes $V_A$ and $V_B$. Similar to thermal interactions, we have that at the equalibrium

We can again play with dimensions of different quantities to realize that a reasonable combination to produce pressure is

We can actually show that this is consistent with the usual definition of pressure.

Chemical Potential

Chemical potential is a less well-known concept and arises when we consider diffusions, or the movement of particles between systems. We define chemical potential $\mu$ such that particles will flow from systems with larger chemical potential to systems with small potential.

The name chemical potential can be somewhat confusing since $\mu$ has the unit of energy.

The Thermodynamic Identity

Now we’ve introduced all the quantities relevant to entropy, we have the following identity:

and we can rearrange to derive the thermodynamic identity:

which summarizes all the physics we’ve entroduced so far.

Boltzmann Statistics

We’ve started with entropy, and connects it with usual physical properties. We will now use entropy or multiplicity to derive probability of microstates, and then connect these probabilities with the usual physical properties.

For now, let’s consider a system interacting with a reservoir. We assume that the change in temperature and volume of the reservoir is neglegible, and the system does not exchange particles with the reservoir. We use $\Omega_r(s_i)$ to denote the multiplicity of the reservoir when the system is at state $s_i$. Then we have that for two states $s_i, s_j$,

Now we will use the thermodynamic identity to connect entropy to energy of the system.

so we have

where $E(s_i)$ is the energy of the system at state $s_i$, and the last equality is due to energy conservation. Thus, we have that

We finally derived the Boltzmann statistics:

where $$Z = \sum_s e^{-E(s)/kT}$ is the normalizing factor known as the partition function* *and $$e^{-E(s)/kT}$ is known as the **Boltzmann factor.

We can also find the average values of the system, like the expected energy:

Define $\beta \equiv kT$, then we have that

so we have

Quantum Statistics

For the Boltzmann statistics, we assume that the system does not exchange particles with the reservoir. If we instead allow exchange of particles with the reservoir, we can use the thermodynamic identity to derive that

This leads to the quantum statistisc,

where $$Z = \sum_s e^{-E(s)-\mu N(s)/kT}$ is the normalizing factor known as the grand partition function or Gibbs Sum and $e^{-[E(s)-\mu N(s)]/kT}$ is known as the Gibbs factor.

Now here is the interesting part: from quantum mechanics (quantum field theory), we know that for fermions (e.g., electrons), a state can only be occupied by one particle, while for bosons, there is no such restriction. We will now discuss these two scenarios.

Fermi-Dirac Distrbution

For fermions, $N(s)$ is either equal to $0$ or $1$, so

where $\epsilon$ is the energy of a single particle. Thus, the average number of particles in the system is

Bose-Einstein Distrbution

For bosons, $N(s)$ can take arbitary values, so

And with some algebra, we can derive that

Blackbody Radiation

Now we have enough tools to explain blackbody radiation, which was one of the dark clouds above classical physics. In classical physics, we treat electromagnetic radiation as a continuous field that permeates all space, and we would expect a black box to produce infinite thermal energy, known as the “ultraviolet catastrophe”. To solve this problem, Planck decides that the photons have to come in quanta of energy unit $hf$, where $h$ is the Planck constant and $f$ is the frequency of the electromagnetic wave. He derived the Planck distribution

If you realize that $\epsilon = hf$ and let $\mu = 0$ for photons, this is exactly the Bose-Einstien distribution! This is in fact right, photons have zero chemical potential, which means that they can be emitted or absorbed in any quantity and the number of photons is not conserved.

Reference

[1] Schroeder, D. V. (1999). An introduction to thermal physics.