点击跳转到中文版（Chinese ver.)

Thermodynamics does not develop methods for the computations of thermodynamic parameters. Thermodynamics does not look fundamental and systematic, and all the important laws seem isolated from each other. Kinetic theory of molecules offers certain understanding, but not sufficiently general.

Postulates

Statistical mechanics:

• the equilibrium state
• non-equilibrium states

Statistical mechanics in this course is mainly concerned with the equilibrium state.

• Calculate macroscopic parameters from microstructures and interactions.
• Derive thermodynamics, e.g. define and calculate temperature, internal energy, entropy; prove the important laws in thermodynamics.
• Statistical mechanics generally goes beyond thermodynamics, e.g. compute correlation functions.

The system: a classical system isolated in the sense that the energy is conserved or nearly conserved, is composed of a large number $N$ of elements, typically

The thermodynamic limit:

but

Here $V$ is measured in a microscopic unit, e.g.

In other words, extremely dilute or dense systems are excluded in the thermodynamic limit.

The thermodynamic properties of the system may be tackled from three levels.

Fundamental level

Solve the microscopic equations of motion, such as Newton, Hamiltonian and Heisenberg equations.

Difficulties

• too many degrees of freedom
• microscopic initial conditions and boundary conditions
• irregular disturbance from environments

Here the time $t$ and space coordinate $q_i$ are microscopic.

Even if the fundamental equations could be solved, we still have to compute the macroscopic physical parameters from the microscopic solutions, based on certain statistical theories. Therefore, statistical mechanics does provide ‘fundamental’ methods for dealing with many-body systems.

Quasi-fundamental level

Do not trace the motion of each molecule, and consider only the probability distribution $\rho(p_i, q_i, t)$ with

being the probability of the system lying within a volume $\prod_i \dd^3 p_i \dd^3 q_i$ of the coordinate $q_i$ and momentum $p_i$.

Equations of motion:

• Liouville’s equation
• Boltzmann equation

These equations can be solved only in some simple cases, such as dilute gases. The time $t$ and space coordinate $q_i$ are mesoscopic. Otherwise there is no meaning for the probability distribution.

Statistical mechanics

Do not solve any equations of motion, but assume a form of $\rho(p_i, q_i, t)$ in the equilibrium state, i.e. $\rho(p_i, q_i, \infty) \equiv \rho(p,q)$.

• It can be tested by experiments.
• It can be derived from equations of motion in some special cases.

More strictly, this is the so-called ensemble theory. The equilibrium state is in the macroscopic sense, i.e. the time scale is macroscopic. For non-equilibrium states, one usually assumes certain effective equations of motion. The time scale is usually mesoscopic.

$\Gamma$ space: the phase space spanned by $(p,q)$; each point in $\Gamma$ space represents a microscopic state of the system.

An ensemble: A collection of systems, identical in composition and macroscopic conditions, existing in (different or the same) microscopic states, without interactions with each other. A system in the ensemble can be represented by a point in $\Gamma$ space, and these systems obey the probability distribution $\rho(p,q)$, i.e.

is the number of systems at the volume element $\dd p \dd q$.

In simple words, an ensemble is a set of microscopic states of the system with the probability distribution $\rho(p,q)$, and it is introduced to replace the dynamic evolution of the microscopic states. Physically, such a description is reasonable just because a macroscopic state may correspond to many microscopic states. Naturally, $\rho(p,q)$ should depend on the macroscopic environment of the system. The basic assumption in statistical mechanics starts from an isolated system.

Postulate of equal a priori probability

When an isolated system is in thermodynamic equilibrium, its microscopic state is equally likely to be any state satisfying the macroscopic conditions, i.e.

where $E$ is the energy, $\Delta \ll E$, $H$ is the Hamiltonian, and the ensemble described by this distribution is the so-called microcanonical ensemble.

Here it is important that the Hamiltonian is assumed to be known, which contains the microstructure and interactions of the system.

• Why is $\Delta$ introduced? Theoretically, it is convenient. Practically, isolation is not strict.
• Why is $\rho(p,q)$ a constant for all microscopic states with energy around $E$? Energy is assumed to be the most important quantity for the microscopic motion. If $\rho(p,q)$ is not a constant, another quantity should be specified.

One may understand the postulate from ergodicity, which assumes that the system experiences all the microscopic states in an infinitesimal macroscopic time. If there is ergodicity, it is natural that the real path of a system should be a simple loop in $\Gamma$ space. Therefore, the postulate holds.

If the path does not form a loop in an infinitesimal macroscopic time, the macroscopic state is not in equilibrium. If there are two loops, it is not ergodic. In fact, ergodicity is not strict. For example, if the walls of a cubic container are completely smooth, molecules of an ideal gas with parallel velocities will not change their velocities. But ergodicity may be assumed because of:

• the measure of non-ergodic states is negligibly small;
• the irregular disturbance from environments.

On the other hand, we may introduce the probability distribution $P(f)$ for the observable $f(p,q)$. The average of $f(p,q)$ can be computed by $\int \dd f\, f P(f)$. Obviously, $P(f)$ is different from $\rho(p,q)$.

As shown in Fig. 2.1, $P(f)$ is usually assumed to have a single sharp peak. We define the most probable value $f_0$ of $f(p,q)$ by the maximum probability $P(f)$.

The ensemble average $\langle f \rangle$ and the most probable value $f_0$ are nearly the same under the condition of Eq. (2.1.1), and should become identical in the thermodynamic limit. Otherwise, statistical mechanics should be questioned. But it does not mean that statistical mechanics is necessarily invalid; one should carefully study.

Question: in what case may Eq. (2.1.1) be not valid?

Answer: strongly correlated systems, for example, with long-range interactions, or around continuous phase transitions.

Microcanonical ensemble

To establish statistical mechanics, it is first important to define all the macroscopic physical parameters through the microscopic degrees of freedom, i.e. $f(p,q)$, especially for those typical thermodynamic observables. The average of the Hamiltonian $H$ is naturally defined as the internal energy. So, it is crucial how to define the entropy and temperature.

Let us denote the volume in $\Gamma$ space of the microcanonical ensemble by

$\Gamma(E)$ is understood to depend on $N, V$ and also $\Delta$. For example, the volume is shown for $H(p,q) = p^2 + q^2$ in Fig. 2.2.

1
\begin{tikzpicture}[>=stealth, thick, font=\small]
2
  \draw[->] (-2.8,0) -- (2.8,0) node[below] {$q$};
3
  \draw[->] (0,-2.8) -- (0,2.8) node[left] {$p$};
4
  \draw[thick] (0,0) circle (1.2);
5
  \draw[thick] (0,0) circle (1.6);
6
  \fill[gray!30] (0,0) circle (1.6);
7
  \fill[white] (0,0) circle (1.2);
8
  \node at (0,0) {$E$};
9
  \node at (1.4,1.4) {$E+\Delta$};
10
\end{tikzpicture}

Please note that $\Gamma(E)$ is the volume, not simply the number of states in $\Gamma$ space. To define the “number”, one should introduce the unit volume.

From experiments, the entropy is closely related to the number of microscopic states. The entropy is thus defined by

where $k$ is a universal constant eventually shown to be the Boltzmann constant. Such a definition is closely related to the fact that the entropy must be extensive.

To justify the definition of $S$, we should show:

(a) $S$ is extensive.
(b) $S$ satisfies the properties of the entropy as required by the second law of thermodynamics.

Proof:

(a) Let the system be divided into two subsystems:

1: $N_1, V_1, H_1$
2: $N_2, V_2, H_2$

with fixed $N_1, N_2, V_1$ and $V_2$, which satisfy

Here $(p_1,q_1)$ are the coordinates of particles in system 1, and $(p_2,q_2)$ are the coordinates of particles in system 2. Especially it is assumed that the interaction between two subsystems is negligibly small, although it must exist to allow the exchange of energy between two subsystems. For example, if the intermolecular potential is finite-range, and the surface-to-volume ratio of each subsystem is negligibly small.

If we define

we should show in the thermodynamic limit

If $E = E_1 + E_2$ is a given decomposition of the energy, i.e. $H_1$ fluctuates around $E_1$ and $H_2$ around $E_2$, the volume in $\Gamma$ space of the whole system is

In other words, the extensive property of $S$ has been proven if the two subsystems are simply added up.

The key point is that the possible decomposition of the energy $E = E_1 + E_2$ is not unique when the system is divided into two subsystems. In other words, both $E_1$ and $E_2$ may fluctuate from zero to $E$ although constrained by the energy conservation $E_1 + E_2 = E$.

Let us divide $E$ into $E/\Delta$ intervals with $E_i = i\Delta$, $i=0,1,\dots,E/\Delta$, then

We will show or have to show that only one term in the sum is dominating. This is the only possibility to set up the extensiveness of the entropy.

Let us assume that there uniquely exists a decomposition of energy $\bar{E}_1 + \bar{E}_2 = E$ with the largest term $\Gamma_1(\bar{E}_1)\Gamma_2(\bar{E}_2)$. Then

or

In the thermodynamic limit, we expect or assume

and

In addition, both $N_1$ and $N_2$ should be of order $N$, otherwise it may not be meaningful. Then

In thermodynamic equilibrium, the system is homogeneous, therefore

Thus $S$ is extensive up to the order of $N$, i.e. when omitting the terms of order $O(\log N)$.

In summary, the extensive property of $S$ is based on that a single decomposition $E = \bar{E}_1 + \bar{E}_2$ of the energy is dominating when the system is divided into two subsystems, with fixed $N_1, N_2$ and $V_1, V_2$. The existence of such a dominating decomposition will be discussed below.

$E \propto N_1 + N_2 = N$ is simply the extensive property of $E$. However, $\log \Gamma_1 \propto N_1 \propto V_1$ indicates already that $S_1$ is extensive. Therefore, the derivation with $E = \bar{E}_1 + \bar{E}_2$ sounds only to prove that if $S_1$ and $S_2$ are extensive then $S$ is also extensive.

Finally, we may notice that there is still another small parameter $\Delta$. However, this is a macroscopic parameter, and not necessarily infinitesimal in the microscopic sense. At least, one should not mix this small parameter with the thermodynamic limit.

Understanding:

Alternatively, we may consider $\bar{E}_1$ and $\bar{E}_2$ are the averaged energy of the two subsystems. In the thermodynamic limit, the fluctuation $\delta$ of $E_1$ around $\bar{E}_1$ should be much smaller than $\bar{E}_1$. Let us take $\delta < \Delta$, then only $\Gamma_1(\bar{E}_1)\Gamma_2(\bar{E}_2)$ is dominating:

Therefore

with $E = \bar{E}_1 + \bar{E}_2$.

This also explains why we can add two systems up in equilibrium, and divide a system into two parts in equilibrium.

In fact, following the discussions below Eq. (2.1.1), it is natural that the decomposition of energy with the largest term $\Gamma_1(E_1)\Gamma_2(E_2)$ should be identical to the decomposition of the average energy, since the former corresponds to the most probable values of energy for the two subsystems.

Question: why is it only “an understanding”?

Answer: $\Gamma_1(E_i)\Gamma_2(E-E_i)$ is just $P(E_1 = E_i)$, the probability that $E_1$ takes the value $E_i$. When we assume $\delta < \Delta$, it indicates already that all other terms are negligible.

In fact, $\Gamma_1(E_1)\Gamma_2(E-E_1)$ is usually a very sharp function of $E_1$, and such an understanding is reasonable. For example, suppose $\Gamma_1(E_1) = E_1^{N_1}$, $\Gamma_2(E-E_1) = (E-E_1)^{N_2}$ and $N_1 = N_2$, then

This function has a maximum at $E_1 = E/2$, and it is very sharp for a large $N_1$.

Reading materials: Why

e.g. $N=2$:

$\Gamma(E)$ is just the area of a circular ring. $\Gamma_1(E_1)\Gamma_2(E-E_1)$ is an element of the circular ring at $q = \sqrt{E_1}$. If $E_1$ is fixed, one may write

If not, one needs to integrate over $E_1$:

since $E_1$ takes values from 0 to $E$, corresponding to different states.

End reading materials.

In other words, $E = \bar{E}_1 + \bar{E}_2$ maximizes the function $\Gamma_1(E_1)\Gamma_2(E_2)$, i.e. also $\log[\Gamma_1(E_1)\Gamma_2(E_2)]$ with the restriction $\delta E = \delta E_1 + \delta E_2 = 0$, therefore,

This leads to

or

We define the temperature of any system by

Then

it is simply the zeroth law:

$T$ defined in this way is an intensive parameter, and $\partial S/\partial E = 1/T$ is also one of the Maxwell relations in thermodynamics. Therefore, if $S$ is correctly defined, $T$ should also be correct.

Thermodynamics

Let us define

Then

Then

It can be proved that up to an additive constant of order $O(\log N)$, the following definitions are equivalent:

Question: why?

For example, suppose $\Sigma(E) \propto E^N$, then $\Gamma(E) \propto N E^{N-1}$. Or, more generally, $\Gamma(E) < \Sigma(E) < \Gamma(E) \cdot E/\Delta$. The behaviors of $\log \Sigma(E)$ and $\log \Gamma(E)$ look similar, up to order $O(\log N)$.

From the definition itself, $S$ is a state function.

With the definition

it is easy to show that $S$ never decreases, i.e. the second law for a thermally isolated system in thermodynamics.

Proof: Consider a system described by the parameters $N, E, V$. By the definition of an isolated system, $N$ and $E$ cannot change, $V$ cannot decrease. Therefore, the second law here is simply stated as that $S$ is a non-decreasing function of $V$. This is obvious, for $\Sigma(E)$ is a non-decreasing function of $V$ by its definition.

Hint: an isolated gas can expand only, although it is somewhat less clear for liquid and solid.

For reversible thermodynamic transformations, the entropy should satisfy the fundamental equation of the second law. Assuming that the system is changed slowly by coupling the system to external environments, then it is a quasi-static process:

Here one again assumes that there are only two independent parameters.

Define the pressure of the system to be

then

or

Keeping in mind that $E$ is the internal energy, this looks like the first law.

Question: Is it reasonable to define

Hints:

1. $P$ is intensive.
2. Exercise: prove that $P$ defined above satisfies

Up to here, the definition of $P$ indeed is reasonable. The definition of $T$ is also correct if $S$ is really entropy.

Finally, $\dd Q = T \dd S$ is the heat absorbed by the system by assuming the first law; thus $S$ should be the entropy. In fact, one may show that $S$ exhibits all the properties of the entropy analyzed in thermodynamics. Here it is important that the first law is still an assumption in statistical mechanics!

If it is possible to compute $S(E,V)$, and to obtain the internal energy as $E = E(S,V)$, i.e. the characteristic function, one may derive all the thermodynamic quantities just by differentiation. But this can be done only in very simple cases.

Equipartition theorem

The Hamiltonian of the system is written as

e.g. for an ideal gas,

or a set of harmonic oscillators,

We will derive an equipartition theorem for a general Hamiltonian, and then apply it to a simple one.

Let $x_i$ be either $p_i$ or $q_i$:

The second equality is based on the definition of the integration.

The second equality is obtained through integration by parts. The first term in the last line reduces to a surface integral over the boundary of the region defined by $H<E$, i.e.

Question: how to describe this more convincingly?

Hint: suppose the volume $H<E$ is enclosed by the surface $H=E$, and the surface consists of the upper and lower sheets, i.e. $x_j = F_2(x_i,\dots)$ and $x_j = F_1(x_i,\dots)$.

(the second term is negligible, at least when compared to the first term)

If $i=j$,

If

where $P_i, Q_i$ are canonical conjugate variables, then one may easily show

Denote the number of degrees of freedom by $f$ and take the ensemble average of the above equation,

Exercise: Prove the equipartition theorem for

by explicitly calculating the ensemble average.

Classical ideal gas

The Hamiltonian of an ideal gas:

Then one may compute

Let

$\Omega_n(R)$ is the volume of an $n$-sphere of radius $R$:

$\Gamma(z)$ is the gamma function:

hence

From the above calculation, $\Omega_n(R) \propto R^n$ is essential, and $C_n$ is not much relevant.

Gibbs paradox

Consider the mixture of two ideal gases at the same temperature and the same density, with volumes $V_1$ and $V_2$, and particle numbers $N_1$ and $N_2$ respectively. Initially, two gases are separated by a partition wall, then the wall is removed and two gases mix.

If two gases are different, the entropy increases, since both gases expand. If two gases are identical, nothing is changed in thermodynamics, but the entropy seems increasing in statistical mechanics as described above, since both gases still expand.

To solve this problem, one may introduce a Gibbs factor in the computation of $\Sigma(E)$, i.e.

With this correction factor, and $\log N! \approx N \log N - N$, one obtains

Thus the entropy remains unchanged after the mixture of two identical gases. In other words, $S \sim Nk \log V$ is unreasonable, since $V$ is extensive. The Gibbs factor corrects it to $S \sim Nk \log(V/N)$. The Gibbs factor is needed whenever there exists an exchange of particles in different volumes.

In fact, it seems that the factor one needs is $N^N$ rather than $N!$. Why does one not directly introduce $N^N$? In quantum mechanics, an exchange of two identical particles does not yield a new quantum state, and $N!$ is the total number of such exchanges for $N$ particles.

Alternatively, when $V$ is decomposed into $V_1$ and $V_2$,

In the thermodynamic limit, however, only one term dominates the sum, which ensures the same density for both $V_1$ and $V_2$, i.e. $N_1/V_1 = N_2/V_2$. Therefore,

The factorial number looks intrinsic here.

Maxwell-Boltzmann distribution

There are $N$ identical molecules in volume $V$,

The system may be described by a microcanonical ensemble. If the molecules are quasi-independent,

where $h(p_i,q_i)$ represents the Hamiltonian of each molecule. There could be a number of atoms in a molecule, but the interactions between two molecules are negligible. For example, the simplest case is

$\mu$ space: the phase space spanned by a single-molecule coordinate $(p,q)$.

A microscopic state of a single molecule can be represented by a point in $\mu$ space. A microscopic state of the system is described by a set of such points.

Since the energy of a molecule is bounded by $E$, the points are confined to a finite region of $\mu$ space. We divide the region into $K$ elements of volume $\omega = \dd^3 p \dd^3 q$, and denote the number of molecules in the $l$-th element by $n_l$, then

Here the molecules are assumed to be quasi-independent, $\epsilon_l$ is the energy level of a single particle.

It is important to note that a microscopic state of the system may be described by a set of occupation numbers $\{n_l\}$, but a set of $\{n_l\}$ corresponds to not only one microscopic state, e.g. interchange of two molecules in different elements may lead to new states. That is, a given set of $\{n_l\}$ corresponds to a volume in $\Gamma$ space, which is called the volume occupied by $\{n_l\}$.

To describe a macroscopic state, we need to average over the microcanonical ensemble, i.e. all possible microscopic states. In other words, we should first obtain $\{\langle n_l \rangle\}$, then calculate all macroscopic quantities from this distribution. Here $\langle \dots \rangle$ represents the ensemble average.

However, it is difficult to perform this ensemble average.

We thus assume that the equilibrium state in the thermodynamic limit is described by the most probable distribution of $\{n_l\}$, which occupies the largest volume in $\Gamma$ space.

Why may such an assumption hold?

• This is somewhat similar to the case when we calculate $\Gamma(E) \simeq \Gamma_1(\bar{E}_1)\Gamma_2(\bar{E}_2)$ by dividing the system into two subsystems.
• For a fixed $l$, if the relative fluctuation

is still not small enough, we increase the total number $N$ to reduce it. Finally, $\langle n_l \rangle$ should be equal to the most probable $\bar{n}_l$.

The procedure:

(a) Calculate the volume of $\{n_l\}$.
(b) Maximize it to obtain the most probable $\{\bar{n}_l\}$.

The volume of $\{n_l\}$:

where $g_k$ is introduced for convenience, and may finally be put to 1. For example, the derivative over $g_k$ yields $n_k$. Alternatively, $g_k$ may be understood as the volume of the $k$-th element in $\mu$ space.

Understanding: There are $N!$ ways of distributing $N$ distinguishable molecules to $N$ positions. However, $N$ positions form $K$ groups with the distribution $\{n_l\}$. Inside a group, there are $n_l!$ ways of distributing $n_l$ molecules, which do not lead to new states in $\Gamma$ space. In other words, we consider that molecules in the same element are in the same state, since their contributions to the physical quantity $f(p,q)$ are the same.

For a large $n_l$,

(i.e. $n_l! \simeq n_l^{n_l}$)

Now we vary $\{n_l\}$ under the conditions $\sum_{l=1}^K n_l = N$ and $\sum_{l=1}^K \epsilon_l n_l = E$, to find the most probable $\{n_l\}$. We introduce the Lagrange multipliers $\alpha$ and $\beta$, and calculate

Now we consider all $n_l$ are independent of each other.

Finally, $\alpha$ and $\beta$ can be determined by the conservation of the total particles and the total energy.

To prove that $\{\bar{n}_l\}$ maximizes $\Omega(\{n_l\})$, we can simply calculate the second variation:

Note that $\bar{n}_l$ is only a function of $\epsilon_l = \epsilon(p,q)$, and does not generally depend on $(p,q)$. This is an important feature of the equilibrium state. The molecules tend to gather in the lower energy states.

Boltzmann statistical theory

We put $g_l = 1$.

Let us assume

The original subscript $l$ just labels $p$ and $q$, and $y = \{y_k\}$ represent macroscopic external parameters such as the external fields.

Define the partition function of a single particle:

then

Suppose the system is changed very slowly.