Entropy
≈ 28 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
Entropy is a thermodynamic state variable that quantifies the probabilistic distribution of accessible microstates in a system. The term and the concept are used in diverse fields, from classical thermodynamics (where it was first recognized), to the microscopic description of nature in statistical physics, and the principles of information theory.
It has far-ranging applications in chemistry and physics, in biological systems and their relation to life, in cosmology, economics, and information systems including the transmission of information in telecommunication. Entropy is central to the second law of thermodynamics, which states that the entropy of an isolated system left to spontaneous evolution cannot decrease with time. As a result, isolated systems evolve toward thermodynamic equilibrium, where the entropy is highest.
"High" entropy means that energy is more disordered or dispersed, while "low" entropy means that energy is more ordered or concentrated. A consequence of the second law of thermodynamics is that certain processes are irreversible. The thermodynamic concept was referred to by Scottish scientist and engineer William Rankine in 1850 with the names thermodynamic function and heat-potential.…
Entropy is a thermodynamic state variable that quantifies the probabilistic distribution of accessible microstates in a system. The term and the concept are used in diverse fields, from classical thermodynamics (where it was first recognized), to the microscopic description of nature in statistical physics, and the principles of information theory.
It has far-ranging applications in chemistry and physics, in biological systems and their relation to life, in cosmology, economics, and information systems including the transmission of information in telecommunication. Entropy is central to the second law of thermodynamics, which states that the entropy of an isolated system left to spontaneous evolution cannot decrease with time. As a result, isolated systems evolve toward thermodynamic equilibrium, where the entropy is highest.
"High" entropy means that energy is more disordered or dispersed, while "low" entropy means that energy is more ordered or concentrated. A consequence of the second law of thermodynamics is that certain processes are irreversible. The thermodynamic concept was referred to by Scottish scientist and engineer William Rankine in 1850 with the names thermodynamic function and heat-potential.
In 1865, German physicist Rudolf Clausius, one of the leading founders of the field of thermodynamics, defined it as the quotient of an infinitesimal amount of heat to the instantaneous temperature. He initially described it as transformation-content, in German Verwandlungsinhalt, and later coined the term entropy from a Greek word for transformation.
Austrian physicist Ludwig Boltzmann explained entropy as the measure of the number of possible microscopic arrangements or states of individual atoms and molecules of a system that comply with the macroscopic condition of the system.
He thereby introduced the concept of statistical disorder and probability distributions into a new field of thermodynamics, called statistical mechanics, and found the link between the microscopic interactions, which fluctuate about an average configuration, to the macroscopically observable behaviour, in the form of a simple logarithmic law, with a proportionality constant, the Boltzmann constant, which has become one of the defining universal constants for the modern International System of Units.
The entropy change \mathrm{d} S of a system can be well-defined as a small portion of heat \delta Q_{\mathsf{rev}} transferred from the surroundings to the system during a reversible process divided by the temperature T of the system during this heat transfer: \mathrm{d} S = \frac{\delta Q_\mathsf{rev}}{T} The reversible process is quasistatic (i.e., it occurs without any dissipation, deviating only infinitesimally from the thermodynamic equilibrium), and it may conserve total entropy.
For example, in the Carnot cycle, while the heat flow from a hot reservoir to a cold reservoir represents the increase in the entropy in a cold reservoir, the work output, if reversibly and perfectly stored, represents the decrease in the entropy which could be used to operate the heat engine in reverse, returning to the initial state; thus the total entropy change may still be zero at all times if the entire process is reversible. In contrast, an irreversible process increases the total entropy of the system and surroundings.
Any process that happens quickly enough to deviate from the thermal equilibrium cannot be reversible; the total entropy increases, and the potential for maximum work to be done during the process is lost. The concept of entropy arose from Rudolf Clausius's study of the Carnot cycle which is a thermodynamic cycle performed by a Carnot heat engine as a reversible heat engine.
In a Carnot cycle, the heat Q_\mathsf{H} is transferred from a hot reservoir to a working gas at the constant temperature T_\mathsf{H} during isothermal expansion stage and the heat Q_\mathsf{C} is transferred from a working gas to a cold reservoir at the constant temperature T_\mathsf{C} during isothermal compression stage. According to Carnot's theorem, a heat engine with two thermal reservoirs can produce a work W if and only if there is a temperature difference between reservoirs.
Originally, Carnot did not distinguish between heats Q_\mathsf{H} and Q_\mathsf{C} , as he assumed caloric theory to be valid and hence that the total heat in the system was conserved. But in fact, the magnitude of heat Q_\mathsf{H} is greater than the magnitude of heat Q_\mathsf{C} .
Through the efforts of Clausius and Kelvin, the work W done by a reversible heat engine was found to be the product of the Carnot efficiency (i.e., the efficiency of all reversible heat engines with the same pair of thermal reservoirs) and the heat Q_\mathsf{H} absorbed by a working body of the engine during isothermal expansion: W = \frac{ T_\mathsf{H} - T_\mathsf{C} }{ T_\mathsf{H} } \cdot Q_\mathsf{H} = \left( 1 - \frac{ T_\mathsf{C} }{ T_\mathsf{H} } \right) Q_\mathsf{H} To derive the Carnot efficiency Kelvin had to evaluate the ratio of the work output to the heat absorbed during the isothermal expansion with the help of the Carnot–Clapeyron equation, which contained an unknown function called the Carnot function.
The possibility that the Carnot function could be the temperature as measured from a zero point of temperature was suggested by Joule in a letter to Kelvin. This allowed Kelvin to establish his absolute temperature scale. It is known that a work W > 0 produced by an engine over a cycle equals to a net heat Q_\Sigma = \left\vert Q_\mathsf{H} \right\vert - \left\vert Q_\mathsf{C} \right\vert absorbed over a cycle.
Thus, with the sign convention for a heat Q transferred in a thermodynamic process ( Q > 0 for an absorption and Q for a dissipation) we get: W - Q_\Sigma = W - \left\vert Q_\mathsf{H} \right\vert + \left\vert Q_\mathsf{C} \right\vert = W - Q_\mathsf{H} - Q_\mathsf{C} = 0 Since this equality holds over an entire Carnot cycle, it gave Clausius the hint that at each stage of the cycle the difference between a work and a net heat would be conserved, rather than a net heat itself.
Which means there exists a state function U with a change of \mathrm{d} U = \delta Q - \mathrm{d} W . It is called an internal energy and forms a central concept for the first law of thermodynamics.
Finally, comparison for both the representations of a work output in a Carnot cycle gives us: \frac{\left\vert Q_\mathsf{H} \right\vert}{T_\mathsf{H}} - \frac{\left\vert Q_\mathsf{C} \right\vert}{T_\mathsf{C}} = \frac{Q_\mathsf{H}}{T_\mathsf{H}} + \frac{Q_\mathsf{C}}{T_\mathsf{C}} = 0 Similarly to the derivation of internal energy, this equality implies existence of a state function S with a change of \mathrm{d} S = \delta Q / T and which is conserved over an entire cycle. Clausius called this state function entropy.
In addition, the total change of entropy in both thermal reservoirs over Carnot cycle is zero too, since the inversion of a heat transfer direction means a sign inversion for the heat transferred during isothermal stages: - \frac{ Q_\mathsf{H} }{ T_\mathsf{H} } - \frac{ Q_\mathsf{C} }{ T_\mathsf{C} } = \Delta S_\mathsf{r, H} + \Delta S_\mathsf{r, C} = 0 Here we denote the entropy change for a thermal reservoir by \Delta S_{\mathsf{r}, i} = - Q_i / T_i , where i is either \mathsf{H} for a hot reservoir or \mathsf{C} for a cold one.
If we consider a heat engine which is less effective than Carnot cycle (i.e., the work W produced by this engine is less than the maximum predicted by Carnot's theorem), its work output is capped by Carnot efficiency as: W Substitution of the work W as the net heat into the inequality above gives us: \frac{Q_\mathsf{H}}{T_\mathsf{H}} + \frac{Q_\mathsf{C}}{T_\mathsf{C}} or in terms of the entropy change \Delta S_{\mathsf{r}, i} : \Delta S_\mathsf{r, H} + \Delta S_\mathsf{r, C} > 0 A Carnot cycle and an entropy as shown above prove to be useful in the study of any classical thermodynamic heat engine: other cycles, such as an Otto, Diesel or Brayton cycle, could be analysed from the same standpoint.
Notably, any machine or cyclic process converting heat into work (i.e., heat engine) that is claimed to produce an efficiency greater than the one of Carnot is not viable — due to violation of the second law of thermodynamics. For further analysis of sufficiently discrete systems, such as an assembly of particles, statistical thermodynamics must be used. Additionally, descriptions of devices operating near the limit of de Broglie waves, e.g. photovoltaic cells, have to be consistent with quantum statistics.
According to the Clausius equality, for a reversible cyclic thermodynamic process: \oint{\frac{\delta Q_\mathsf{rev}}{T}} = 0 which means the line integral \int_L{\delta Q_\mathsf{rev} / T} is path-independent. Thus we can define a state function S , called entropy: \mathrm{d} S = \frac{\delta Q_\mathsf{rev}}{T} Therefore, thermodynamic entropy has the dimension of energy divided by temperature, and the unit joule per kelvin (J/K) in the International System of Units (SI).
To find the entropy difference between any two states of the system, the integral must be evaluated for some reversible path between the initial and final states. Since an entropy is a state function, the entropy change of the system for an irreversible path is the same as for a reversible path between the same two states. However, the heat transferred to or from the surroundings is different as well as its entropy change. We can calculate the change of entropy only by integrating the above formula.
To obtain the absolute value of the entropy, we consider the third law of thermodynamics: perfect crystals at the absolute zero have an entropy S = 0 . From a macroscopic perspective, in classical thermodynamics the entropy is interpreted as a state function of a thermodynamic system: that is, a property depending only on the current state of the system, independent of how that state came to be achieved.
In any process, where the system gives up \Delta E of energy to the surrounding at the temperature T , its entropy falls by \Delta S and at least T \cdot \Delta S of that energy must be given up to the system's surroundings as a heat. Otherwise, this process cannot go forward.
In classical thermodynamics, the entropy of a system is defined if and only if it is in a thermodynamic equilibrium (though a chemical equilibrium is not required: for example, the entropy of a mixture of two moles of hydrogen and one mole of oxygen in standard conditions is well-defined).
Specifically, entropy is a logarithmic measure for the system with a number of states, each with a probability p_i of being occupied (usually given by the Boltzmann distribution): S = - k_\mathsf{B} \sum_i{p_i \ln{p_i}} where k_\mathsf{B} is the Boltzmann constant and the summation is performed over all possible microstates of the system.
In case states are defined in a continuous manner, the summation is replaced by an integral over all possible states, or equivalently we can consider the expected value of the logarithm of the probability that a microstate is occupied: S = - k_\mathsf{B} \left\langle \ln{p} \right\rangle This definition assumes the basis states to be picked in a way that there is no information on their relative phases.
In a general case the expression is: S = - k_\mathsf{B}\ \mathrm{tr}{\left( \hat{\rho} \times \ln{\hat{\rho}} \right)} where \hat{\rho} is a density matrix, \mathrm{tr} is a trace operator and \ln is a matrix logarithm. The density matrix formalism is not required if the system is in thermal equilibrium so long as the basis states are chosen to be eigenstates of the Hamiltonian. For most practical purposes it can be taken as the fundamental definition of entropy since all other formulae for S can be derived from it, but not vice versa.
In what has been called the fundamental postulate in statistical mechanics, among system microstates of the same energy (i.e., degenerate microstates) each microstate is assumed to be populated with equal probability p_i = 1 / \Omega , where \Omega is the number of microstates whose energy equals that of the system. Usually, this assumption is justified for an isolated system in a thermodynamic equilibrium.
Then in case of an isolated system the previous formula reduces to: S = k_\mathsf{B} \ln{\Omega} In thermodynamics, such a system is one with a fixed volume, number of molecules, and internal energy, called a microcanonical ensemble. The most general interpretation of entropy is as a measure of the extent of uncertainty about a system. The equilibrium state of a system maximizes the entropy because it does not reflect all information about the initial conditions, except for the conserved variables.
This uncertainty is not of the everyday subjective kind, but rather the uncertainty inherent to the experimental method and interpretative model. The interpretative model has a central role in determining entropy. The qualifier "for a given set of macroscopic variables" above has deep implications when two observers use different sets of macroscopic variables. For example, consider observer A using variables U , V , W and observer B using variables U , V , W , X .
Reception and honours
If observer B changes variable X , then observer A will see a violation of the second law of thermodynamics, since he does not possess information about variable X and its influence on the system. In other words, one must choose a complete set of macroscopic variables to describe the system, i.e. every independent parameter that may change during experiment. Entropy can also be defined for any Markov processes with reversible dynamics and the detailed balance property.
In Boltzmann's 1896 Lectures on Gas Theory, he showed that this expression gives a measure of entropy for systems of atoms and molecules in the gas phase, thus providing a measure for the entropy of classical thermodynamics.
Proofs of equivalence between the entropy in statistical mechanics — the Gibbs entropy formula: S = - k_\mathsf{B} \sum_i{p_i \ln{p_i}} and the entropy in classical thermodynamics: \mathrm{d} S = \frac{\delta Q_\mathsf{rev}}{T} together with the fundamental thermodynamic relation are known for the microcanonical ensemble, the canonical ensemble, the grand canonical ensemble, and the isothermal–isobaric ensemble.
These proofs are based on the probability density of microstates of the generalised Boltzmann distribution and the identification of the thermodynamic internal energy as the ensemble average U = \left\langle E_i \right\rangle . Thermodynamic relations are then employed to derive the well-known Gibbs entropy formula. However, the equivalence between the Gibbs entropy formula and the thermodynamic definition of entropy is not a fundamental thermodynamic relation but rather a consequence of the form of the generalized Boltzmann distribution.
Furthermore, it has been shown that the definitions of entropy in statistical mechanics is the only entropy that is equivalent to the classical thermodynamics entropy under the following postulates: In mechanics, the second law in conjunction with the fundamental thermodynamic relation places limits on a system's ability to do useful work. The entropy change of a system at temperature T absorbing an infinitesimal amount of heat \delta q in a reversible way, is given by \delta q / T .
More explicitly, an energy T_R S is not available to do useful work, where T_R is the temperature of the coldest accessible reservoir or heat sink external to the system. For further discussion, see Exergy. Statistical mechanics demonstrates that entropy is governed by probability, thus allowing for a decrease in disorder even in an isolated system. Although this is possible, such an event has a small probability of occurring, making it unlikely.
The applicability of a second law of thermodynamics is limited to systems in or sufficiently near equilibrium state, so that they have defined entropy. Some inhomogeneous systems out of thermodynamic equilibrium still satisfy the hypothesis of local thermodynamic equilibrium, so that entropy density is locally defined as an intensive quantity. For such systems, there may apply a principle of maximum time rate of entropy production. It states that such a system may evolve to a steady state that maximises its time rate of entropy production.
This does not mean that such a system is necessarily always in a condition of maximum time rate of entropy production; it means that it may evolve to such a steady state. The entropy of a system depends on its internal energy and its external parameters, such as its volume. In the thermodynamic limit, this fact leads to an equation relating the change in the internal energy U to changes in the entropy and the external parameters. This relation is known as the fundamental thermodynamic relation.
If external pressure p bears on the volume V as the only external parameter, this relation is: \mathrm{d} U = T\ \mathrm{d} S - p\ \mathrm{d} V Since both internal energy and entropy are monotonic functions of temperature T , implying that the internal energy is fixed when one specifies the entropy and the volume, this relation is valid even if the change from one state of thermal equilibrium to another with infinitesimally larger entropy and volume happens in a non-quasistatic way (so during this change the system may be very far out of thermal equilibrium and then the whole-system entropy, pressure, and temperature may not exist).
The fundamental thermodynamic relation implies many thermodynamic identities that are valid in general, independent of the microscopic details of the system. Important examples are the Maxwell relations and the relations between heat capacities. Thus, when one mole of substance at about is warmed by its surroundings to , the sum of the incremental values of q_\mathsf{rev} / T constitute each element's or compound's standard molar entropy, an indicator of the amount of energy stored by a substance at .
Entropy change also measures the mixing of substances as a summation of their relative quantities in the final mixture. Entropy is equally essential in predicting the extent and direction of complex chemical reactions.
For such applications, \Delta S must be incorporated in an expression that includes both the system and its surroundings: \Delta S_\mathsf{universe} = \Delta S_\mathsf{surroundings} + \Delta S_\mathsf{system} Via additional steps this expression becomes the equation of Gibbs free energy change \Delta G for reactants and products in the system at the constant pressure and temperature T : \Delta G = \Delta H - T\ \Delta S where \Delta H is the enthalpy change and \Delta S is the entropy change.
The spontaneity of a chemical or physical process is governed by the Gibbs free energy change (ΔG), as defined by the equation ΔG = ΔH − TΔS, where ΔH represents the enthalpy change, ΔS the entropy change, and T the temperature in Kelvin. A negative ΔG indicates a thermodynamically favorable (spontaneous) process, while a positive ΔG denotes a non-spontaneous one. When both ΔH and ΔS are positive (endothermic, entropy-increasing), the reaction becomes spontaneous at sufficiently high temperatures, as the TΔS term dominates.
Conversely, if both ΔH and ΔS are negative (exothermic, entropy-decreasing), spontaneity occurs only at low temperatures, where the enthalpy term prevails. Reactions with ΔH 0 (exothermic and entropy-increasing) are spontaneous at all temperatures, while those with ΔH > 0 and ΔS \dot{Q} , flow of shaft work \dot{W}_\mathsf{S} and pressure-volume work P \dot{V} across the system boundaries cause changes in the entropy of the system.
Heat transfer entails entropy transfer \dot{Q}/T , where T is the absolute thermodynamic temperature of the system at the point of the heat flow. If there are mass flows across the system boundaries, they also influence the total entropy of the system. This account, in terms of heat and work, is valid only for cases in which the work and heat transfers are by paths physically distinct from the paths of entry and exit of matter from the system.
To derive a generalised entropy balanced equation, we start with the general balance equation for the change in any extensive quantity \theta in a thermodynamic system, a quantity that may be either conserved, such as energy, or non-conserved, such as entropy.
The basic generic balance expression states that \mathrm{d} \theta / \mathrm{d} t , i.e. the rate of change of \theta in the system, equals the rate at which \theta enters the system at the boundaries, minus the rate at which \theta leaves the system across the system boundaries, plus the rate at which \theta is generated within the system.
For an open thermodynamic system in which heat and work are transferred by paths separate from the paths for transfer of matter, using this generic balance equation, with respect to the rate of change with time t of the extensive quantity entropy S , the entropy balance equation is: \frac{\mathrm{d} S}{\mathrm{d} t} = \sum_{k=1}^K{\dot{M}_k \hat{S}_k + \frac{\dot{Q}}{T} + \dot{S}_\mathsf{gen}} where \sum_{k=1}^K{\dot{M}_k \hat{S}_k} is the net rate of entropy flow due to the flows of mass \dot{M}_k into and out of the system with entropy per unit mass \hat{S}_k , \dot{Q} / T is the rate of entropy flow due to the flow of heat across the system boundary and \dot{S}_\mathsf{gen} is the rate of entropy generation within the system, e.g. by chemical reactions, phase transitions, internal heat transfer or frictional effects such as viscosity.
In case of multiple heat flows the term \dot{Q}/T is replaced by \sum_j{\dot{Q}_j/T_j} , where \dot{Q}_j is the heat flow through j -th port into the system and T_j is the temperature at the j -th port. The nomenclature "entropy balance" is misleading and often deemed inappropriate because entropy is not a conserved quantity. In other words, the term \dot{S}_\mathsf{gen} is never a known quantity but always a derived one based on the expression above.
Therefore, the open system version of the second law is more appropriately described as the "entropy generation equation" since it specifies that: \dot{S}_\mathsf{gen} \ge 0 with zero for reversible process and positive values for irreversible one.
For the expansion (or compression) of an ideal gas from an initial volume V_0 and pressure P_0 to a final volume V and pressure P at any constant temperature, the change in entropy is given by: \Delta S = n R \ln{\frac{V}{V_0}} = - n R \ln{\frac{P}{P_0}} Here n is the amount of gas (in moles) and R is the ideal gas constant. These equations also apply for expansion into a finite vacuum or a throttling process, where the temperature, internal energy and enthalpy for an ideal gas remain constant.
For pure heating or cooling of any system (gas, liquid or solid) at constant pressure from an initial temperature T_0 to a final temperature T , the entropy change is: \Delta S = n C_\mathrm{P} \ln{\frac{T}{T_0}} provided that the constant-pressure molar heat capacity (or specific heat) C_\mathrm{P} is constant and that no phase transition occurs in this temperature interval.
Similarly at constant volume, the entropy change is: \Delta S = n C_\mathrm{V} \ln{\frac{T}{T_0}} where the constant-volume molar heat capacity C_\mathrm{V} is constant and there is no phase change. At low temperatures near absolute zero, heat capacities of solids quickly drop off to near zero, so the assumption of constant heat capacity does not apply.
Since entropy is a state function, the entropy change of any process in which temperature and volume both vary is the same as for a path divided into two steps – heating at constant volume and expansion at constant temperature. For an ideal gas, the total entropy change is: \Delta S = n C_\mathrm{V} \ln{\frac{T}{T_0}} + n R \ln{\frac{V}{V_0}} Similarly if the temperature and pressure of an ideal gas both vary: \Delta S = n C_\mathrm{P} \ln{\frac{T}{T_0}} - n R \ln{\frac{P}{P_0}} Reversible phase transitions occur at constant temperature and pressure.
The reversible heat is the enthalpy change for the transition, and the entropy change is the enthalpy change divided by the thermodynamic temperature. For fusion (i.e., melting) of a solid to a liquid at the melting point T_\mathsf{m} , the entropy of fusion is: \Delta S_\mathsf{fus} = \frac{\Delta H_\mathsf{fus}}{T_\mathsf{m}}.
Similarly, for vaporisation of a liquid to a gas at the boiling point T_\mathsf{b} , the entropy of vaporisation is: \Delta S_\mathsf{vap} = \frac{\Delta H_\mathsf{vap}}{T_\mathsf{b}} \mathsf{Disorder} = \frac{C_\mathsf{D}}{C_\mathsf{I}} \mathsf{Order} = 1 - \frac{C_\mathsf{O}}{C_\mathsf{I}} Here, C_\mathsf{D} is the "disorder" capacity of the system, which is the entropy of the parts contained in the permitted ensemble, C_\mathsf{I} is the "information" capacity of the system, an expression similar to Shannon's channel capacity, and C_\mathsf{O} is the "order" capacity of the system.
A definition of entropy based entirely on the relation of adiabatic accessibility between equilibrium states was given by E. H. Lieb and J. Yngvason in 1999. This approach has several predecessors, including the pioneering work of Constantin Carathéodory from 1909 and the monograph by R. Giles. An equivalent approach that extends the operational definition of entropy to the entire nonequilibrium domain was derived from a rigorous formulation of the general axiomatic foundations of thermodynamics by J. H. Keenan, G. N. Hatsopoulos, E. P. Gyftopoulos, G.
P. Beretta, and E. Zanchini between 1965 and 2014. In the setting of Lieb and Yngvason, one starts by picking, for a unit amount of the substance under consideration, two reference states X_0 and X_1 such that the latter is adiabatically accessible from the former but not conversely.
Defining the entropies of the reference states to be 0 and 1 respectively, the entropy of a state X is defined as the largest number \lambda such that X is adiabatically accessible from a composite state consisting of an amount \lambda in the state X_1 and a complementary amount, (1 - \lambda) , in the state X_0 .
A simple but important result within this setting is that entropy is uniquely determined, apart from a choice of unit and an additive constant for each chemical element, by the following properties: it is monotonic with respect to the relation of adiabatic accessibility, additive on composite systems, and extensive under scaling.
In quantum statistical mechanics, the concept of entropy was developed by John von Neumann and is generally referred to as "von Neumann entropy": S = - k_\mathsf{B}\ \mathrm{tr}{\left( \hat{\rho} \times \ln{\hat{\rho}} \right)} where \hat{\rho} is the density matrix, \mathrm{tr} is the trace operator and k_\mathsf{B} is the Boltzmann constant.
This upholds the correspondence principle, because in the classical limit, when the phases between the basis states are purely random, this expression is equivalent to the familiar classical definition of entropy for states with classical probabilities p_i : S = - k_\mathsf{B} \sum_i{p_i \ln{p_i}} i.e. in such a basis the density matrix is diagonal. Von Neumann established a rigorous mathematical framework for quantum mechanics with his work .
He provided in this work a theory of measurement, where the usual notion of wave function collapse is described as an irreversible process (the so-called von Neumann or projective measurement). Using this concept, in conjunction with the density matrix he extended the classical concept of entropy into the quantum domain. When viewed in terms of information theory, the entropy state function is the amount of information in the system that is needed to fully specify the microstate of the system.
Entropy is the measure of the amount of missing information before reception. Often called Shannon entropy, it was originally devised by Claude Shannon in 1948 to study the size of information of a transmitted message. The definition of information entropy is expressed in terms of a discrete set of probabilities p_i so that: H(X) = - \sum_{i=1}^n{p(x_i) \log{p(x_i)}} where the base of the logarithm determines the units (for example, the binary logarithm corresponds to bits).
In the case of transmitted messages, these probabilities were the probabilities that a particular message was actually transmitted, and the entropy of the message system was a measure of the average size of information of a message. For the case of equal probabilities (i.e. each message is equally probable), the Shannon entropy (in bits) is just the number of binary questions needed to determine the content of the message.
Most researchers consider information entropy and thermodynamic entropy directly linked to the same concept, while others argue that they are distinct. Both expressions are mathematically similar. If W is the number of microstates that can yield a given macrostate, and each microstate has the same a priori probability, then that probability is p = 1/W .
The Shannon entropy (in nats) is: H = - \sum_{i=1}^W{p_i \ln{p_i}} = \ln{W} and if entropy is measured in units of k per nat, then the entropy is given by: H = k \ln{W} which is the Boltzmann entropy formula, where k is the Boltzmann constant, which may be interpreted as the thermodynamic entropy per nat. Some authors argue for dropping the word entropy for the H function of information theory and using Shannon's other term, "uncertainty", instead. The entropy of a substance can be measured, although in an indirect way.
The measurement, known as entropymetry, is done on a closed system with constant number of particles N and constant volume V , and it uses the definition of temperature in terms of entropy, while limiting energy exchange to heat \mathrm{d} U \rightarrow \mathrm{d} Q : T := {\left( \frac{\partial U}{\partial S} \right)}_{V, N}\ \Rightarrow\ \cdots\ \Rightarrow\ \mathrm{d} S = \frac{\mathrm{d} Q}{T} The resulting relation describes how entropy changes \mathrm{d} S when a small amount of energy \mathrm{d} Q is introduced into the system at a certain temperature T .
The process of measurement goes as follows. First, a sample of the substance is cooled as close to absolute zero as possible. At such temperatures, the entropy approaches zero due to the definition of temperature. Then, small amounts of heat are introduced into the sample and the change in temperature is recorded, until the temperature reaches a desired value (usually 25 °C). The obtained data allows the user to integrate the equation above, yielding the absolute value of entropy of the substance at the final temperature.
This value of entropy is called calorimetric entropy. Entropy and the Second Law of Thermodynamics – an A-level physics lecture with 'derivation' of entropy based on Carnot cycle 2.43 Advanced Thermodynamics - an MIT OCW course with emphasis on rigorous and general definitions taught by G. P.
Beretta, MIT, Spring 2024 Khan Academy: entropy lectures, part of Chemistry playlist Entropy Intuition More on Entropy Proof: S (or Entropy) is a valid state variable Reconciling Thermodynamic and State Definitions of Entropy Thermodynamic Entropy Definition Clarification The Discovery of Entropy by Adam Shulman. Hour-long video, January 2013. The Second Law of Thermodynamics and Entropy – Yale OYC lecture, part of Fundamentals of Physics I (PHYS 200)
Text derived from https://en.wikipedia.org/w/index.php?oldid=1376020512 under CC BY-SA 4.0; modifications have been made.