Shannon information

Shannon information is the quantitative measure of uncertainty reduction within an established probability distribution, introduced by Claude Shannon in 1948 and the foundational concept of classical information theory. Within the covolution framework, Shannon information is a complementary but distinct concept from the framework's own notion of information. The framework draws on Shannon-theoretic tools (transfer entropy, mutual information, channel capacity) for operational tests of its claims, while maintaining a broader ontological notion of information that Shannon's framework was not designed to capture. The two concepts coexist and do different work; conflating them produces incoherent claims about what the framework asserts.

This page defines Shannon information, defines the framework's own notion of information (which it calls covolutionary information or constitutive information), works through the differences and the relations, and provides guidance on when each concept applies. The principal claim is that Shannon bits measure information within established state-spaces, while covolutionary information is the structured distinguishability that constitutes the state-spaces and the objects within them.

What Shannon information is

For a discrete random variable X with possible outcomes {x_1, x_2, ,,,,, x_n} occurring with probabilities {p_1, p_2, ,,,,, p_n},
the Shannon entropy of the source is

measured in bits.
The mutual information quantifies how much knowing Y reduces uncertainty about X. The transfer entropy , a directional extension introduced by Schreiber in 2000, measures the influence of X's past on Y's future beyond Y's own past, and is widely used in systems biology and neuroscience to detect directed coupling.

Shannon information has four defining features.

It presupposes a probability distribution. Before any information can be measured, the possible states of the source and their probabilities must be specified. Information is not a property of states themselves but of the distribution from which states are drawn.

It is content-blind. The Shannon framework does not care what the bits mean, whether they refer to anything, or whether they constitute any kind of object. A string of bits encoding a literary work and a random string of the same length have the same Shannon entropy if drawn from distributions with the same statistics.

It is observer-relative. The same physical event can transmit different amounts of information depending on what the receiver already knows. A coin flip transmits one bit to someone who does not know the outcome and zero bits to someone who does.

It is communication-theoretic in origin. The framework was developed for problems of telegraph and radio transmission, and its native applications concern the transmission of messages across noisy channels. Extending Shannon information to biological, cognitive, or social systems is a real extension, not a trivial application.

Shannon information is a precise, mathematically rigorous, empirically applicable concept. It is also a narrow concept, doing exactly the work it was designed to do and not more. The framework treats Shannon information with respect, but does not treat it as the only legitimate notion of information.

What covolutionary information is

The framework's own notion of information, which can be called covolutionary information or constitutive information when contrast with Shannon is at issue, is the structured distinguishability that constitutes information objects and supports the switching operations among them. Where Shannon information measures uncertainty reduction within an established state-space, covolutionary information is what makes the state-space available in the first place.

Covolutionary information has four features that distinguish it from Shannon information.

It does not presuppose a probability distribution. The information that constitutes a gene as a gene is not the entropy of a distribution over possible nucleotide sequences; it is the specific sequence the gene has and the operational role that sequence plays. To make this Shannon-measurable would require specifying a probability distribution from which the actual sequence is taken to be one sample, but this is not what makes the gene the gene. The gene is what it is, not a sample from a distribution.

It is content-respecting. A gene encoding a functional protein and a random sequence of the same length and base composition are not the same information object under covolution, despite having similar Shannon entropy under most reasonable distribution assumptions. The framework's information concept distinguishes them because they participate in different switching networks and have different operational roles.

It is constitutive rather than communicative. Information in the framework's sense does not have to be transmitted to anyone. The genome of a stored seed has informational structure regardless of whether it is currently being read; the seed is an information object in storage. Shannon information is about transmission across channels; covolutionary information is about the structure that persists across time.

It depends on encapsulation and operational role. Constitutive information belongs to information objects, which require encapsulation in the framework's technical sense. Free-floating informational structure not belonging to any information object is, in the framework's vocabulary, a pattern rather than information proper.

These are differences in what the two concepts measure, not just terminological differences. The framework's information concept is genuinely distinct from Shannon's, and the framework's claims about organisms, lineages, institutions, and cognitive systems depend on the distinction.

A worked comparison: the lac operon

Consider the lac operon, a small regulatory region in E. coli that controls genes for lactose metabolism. Both concepts of information apply, but they apply differently.

Under Shannon information, one can specify a probability distribution over the bound and unbound states of the lac repressor, calculate the entropy of this distribution under various input conditions, and measure the mutual information between the operon's state and downstream gene expression. Empirical work in single-cell biology has done exactly this kind of analysis for NF-κB and other transcription factors, computing transfer entropies and mutual informations between regulatory states and gene expression outputs. The Shannon framework yields a precise number of bits flowing through this regulatory module per unit time, given a specified input distribution.

Under covolutionary information, the lac operon is an information object with specific structural features: a specific DNA sequence, a specific repressor binding affinity, a specific position in the genome, a specific regulatory logic. These features constitute the lac operon as the lac operon. Its informational content is not a number of bits but the structured organization that makes it a switch with definable inputs (lactose presence, glucose absence, repressor binding) and definable outputs (gene transcription, protein production). The covolutionary framework asks whether the lac operon satisfies the four-function switch test, whether it is part of an encapsulated information object (the cell), and whether it participates in the cell's covolutionary dynamics.

Both descriptions are correct. They do different work. The Shannon description quantifies information flow rates and bandwidth; the covolutionary description identifies what the operon is and what role it plays. Neither replaces the other.

A second worked comparison: a book and a random sheet

Consider two physical objects: a printed copy of a novel and a sheet of paper covered in randomly generated letters of the same length.

Under Shannon information, both objects have measurable entropy under standard letter-frequency distributions. The randomly generated sheet has higher entropy in most encodings, since it is closer to uniform letter distribution. The novel has lower entropy under monogram models because natural prose has the standard statistical structure of language. Under more sophisticated language models, the novel may still have lower entropy than uniform random text but might have higher entropy than the random sheet, depending on the model. The Shannon-theoretic question "how much information is in each object?" has answers that depend on the chosen distribution.

Under covolutionary information, the novel is an information object. It has informational structure (the specific arrangement of words encoding the narrative), is encapsulated by its binding and authorship and cultural recognition, participates in switching networks (readers' cognitive engagement, literary criticism, cultural reproduction), and is substrate-independent in principle (a digital copy is the same information object as a printed copy). The random sheet is not an information object: it has no specifiable structure beyond its substrate, no operational encapsulation (its physical boundary is incidental, not produced by any process the sheet performs), and no role in any switching network.

The two frameworks reach opposite conclusions about which object has more information. Shannon theory under uniform-distribution assumptions suggests the random sheet has more bits. Covolutionary theory says only the novel is an information object at all. Both conclusions are correct in their domains. Shannon is measuring uncertainty reduction; covolutionary information is identifying entity structure.

This is the deepest difference between the two concepts. A random configuration maximizes Shannon entropy and minimizes covolutionary information; a structured information object minimizes Shannon entropy relative to its substrate and maximizes covolutionary information. The two are often inversely related in natural cases, which is why conflating them produces incoherent claims about what "information" means.

Where the two frameworks meet

Despite the ontological differences, the two frameworks meet productively at the operational level. The four-function test for switches in the covolution framework is stated in terms that draw on Shannon-theoretic measures.

The respond criterion requires transfer entropy from the specifiable input class to the switch's state to be positive and to exceed transfer entropy from inputs outside the class: $T_{I \to S} > T_{\mathcal{I} \setminus I \to S} > 0$. The couple criterion requires transfer entropy from the switch's state to at least one downstream switch to be positive: $T_{S \to S'} > 0$. The distinguish criterion uses attractor structure that can be analyzed using Shannon-theoretic complexity measures. The hold criterion uses residence times that can be estimated from autocorrelation analyses informed by Shannon machinery.

This is why the framework can claim empirical tractability. At the operational level, where the framework's claims need to be tested against data, the relevant measurements use Shannon-theoretic tools. Transfer entropy is Shannon information adapted for directed coupling; the framework adopts this adaptation without modification because it works. Mutual information measures of regulatory networks are Shannon-theoretic, and the framework uses them.

The framework borrows the empirical machinery of Shannon information theory to test its own claims, while maintaining that what the claims are about is not Shannon information but the constitutive information of information objects. The Shannon tools test whether something is an operationally complete switch; the claim that something is an operationally complete switch is a claim about its status as part of an information object, which is a covolutionary claim.

Where the two frameworks part

The two frameworks part ways at the ontological level. Shannon information is a measure; it does not commit one to any view about what information objects are or whether they exist. Covolutionary information is constitutive; it commits the framework to a view about what kinds of entities the universe contains. A theorist could use Shannon information without believing in information objects in the framework's sense, but could not use covolutionary information without that commitment.

This is why "information" in the framework's claims should not be read as shorthand for Shannon information. When the framework says that organisms are informational structures, that the universe is informational, or that information accumulates under covolution, it is using "information" in the covolutionary sense. Readers trained in information theory will often hear Shannon by default and may dismiss the framework's claims as either trivially true (organisms have entropy) or vacuously vague (what does informational structure mean beyond entropy?). The framework needs to make the distinction explicit, both in its publications and in its wiki.

The cleanest statement of the relationship is one that has appeared elsewhere in the wiki and is worth preserving: Shannon bits measure information within established state-spaces; switches constitute the state-spaces. Covolutionary information is the structure that makes state-spaces possible in the first place, before any probability distribution can be defined over them.

How the two concepts apply across substrates

Both concepts apply across substrates, but with different strengths and weaknesses depending on the case.

In molecular and cellular biology, both concepts have substantial empirical purchase. Shannon-theoretic measures of regulatory network function are well-developed and yield specific numerical results. The covolutionary description of cells and regulatory elements as information objects is also well-grounded, since the four-function test can be applied with relative ease at this scale.

In cognitive science, both concepts apply but with greater empirical difficulty. Shannon information has been applied to neural coding with mixed success; transfer entropy measures of effective connectivity in brain imaging are growing in use but remain technically challenging. The covolutionary description of cognitive representations as information objects is conceptually clean but operationally underdeveloped, because the encapsulation and switching dynamics of cognitive objects are less well characterized than their molecular counterparts.

In social and cultural domains, both concepts apply only with substantial caveats. Shannon information measures of institutional communication or cultural transmission can be defined but are usually rough approximations whose interpretation depends heavily on the chosen probability model. The covolutionary description of institutions and cultural patterns as information objects is more natural but is empirically thinner, since the four-function test at this scale relies on judgment more than on direct measurement.

The framework is honest that operational maturity decreases as one moves from molecular to social scales. The Shannon concept and the covolutionary concept both share this limitation, though for different reasons. Shannon information becomes hard to apply because the relevant probability distributions become harder to define; covolutionary information becomes hard to apply because the four-function test becomes harder to operationalize. The two limitations are related but not identical.

How the framework uses both concepts

A coherent use of both concepts within the framework requires keeping the levels straight.

At the ontological level, the framework is committed to covolutionary information. The entities that exist, persist, and engage in covolution are information objects, and their constitutive informational structure is what makes them entities. This commitment is explicit in the Information object page and is the framework's distinctive ontological position.

At the operational level, the framework uses Shannon-theoretic measures to test its empirical claims. Transfer entropy is used to identify the input and output coupling of candidate switches. Mutual information is used to characterize regulatory networks. Channel capacity bounds are used when relevant. The framework does not reject Shannon information; it uses it.

The framework's distinctive contribution is the combination: it advances ontological claims (the universe contains information objects with specific structural properties) and tests them with empirical machinery (Shannon-theoretic measures of coupling and complexity). The combination is what makes the framework an empirical research program rather than a purely philosophical position.

When framework publications discuss information, they should make clear which concept is in play. In introductory and conceptual sections, "information" usually means covolutionary information. In operational and empirical sections, "information" often means Shannon information or a Shannon-derived quantity. The wiki should encourage this discipline rather than allowing the ambiguity that arises from using the unqualified word.

Limits

Several limitations of this distinction should be acknowledged.

The philosophical question of whether Shannon information and constitutive information are genuinely distinct concepts is contested.

Some philosophers of information (Floridi, Adriaans, others) have argued for unified accounts that subsume both concepts under a single framework. The covolution framework takes a position (the two are distinct but related) without resolving this larger debate. For wiki use and for empirical applications, this is acceptable. For peer-reviewed publication on the philosophical foundations of the framework, more engagement with this literature would be required.

The boundary between Shannon information and covolutionary information is fuzzy in the operational tests themselves. Transfer entropy is a Shannon-theoretic measure used to test covolutionary claims; whether this means the test is Shannon-theoretic or covolutionary is partly a matter of how the question is framed. The framework's position is that the tests use Shannon tools but test covolutionary properties, but this distinction may be harder to maintain in practice than in theory.

There is no fully developed mathematical theory of covolutionary information that parallels Shannon's theory of communication. The framework gestures toward such a theory through its operational definitions of switches and information objects, but a fully formalized account that would do for covolutionary information what Shannon did for communication does not yet exist. The framework should acknowledge this gap rather than pretending it is resolved.

The two concepts of information can be combined incoherently. A common error is to invoke Shannon information's mathematical rigor while making claims that require covolutionary information's broader scope, or to invoke covolutionary information's interpretive flexibility while making claims that require Shannon information's precision. The framework's discipline requires keeping the levels separate, and this discipline is harder to maintain than it appears.

Why this distinction matters

The distinction between Shannon and covolutionary information matters for three reasons within the framework.

It clarifies what the framework actually claims. Without the distinction, the framework's invocations of information are ambiguous, and critics can attack one reading while the framework retreats to the other. With the distinction explicit, the framework's claims are stable: it claims that information objects exist, that they have constitutive informational structure, and that this structure can be detected and characterized using Shannon-theoretic tools.

It positions the framework in relation to information theory. Covolution Theory is not a competitor to Shannon information theory; it is a research program that uses Shannon tools while advancing a different ontological position. This positioning protects the framework from the charge that it is reinventing information theory and clarifies what it is contributing.

It enables internal consistency in framework publications. When a framework paper uses Shannon-theoretic measurements, the distinction makes clear what those measurements are testing. When a paper makes ontological claims about information objects, the distinction makes clear that these claims are not reducible to Shannon-theoretic facts. The discipline of keeping the two concepts separate is what allows the framework to make both kinds of statement coherently.

Shannon information

Shannon information

What Shannon information is

What covolutionary information is

A worked comparison: the lac operon

A second worked comparison: a book and a random sheet

Where the two frameworks meet

Where the two frameworks part

How the two concepts apply across substrates

How the framework uses both concepts

Limits

Why this distinction matters

See also

댓글 0