20 results
Filters
Settings
Now showing1 - 10 of 20
- Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Unique Information Through the Lens of Channel Ordering: An Introduction and ReviewThe problem of constructing information measures with a well-defined interpretation is of fundamental significance in information theory. A good definition of an information measure entails certain desirable properties while also providing answers to operational problems. In this work, we investigate the properties of the unique information, an information measure that quantifies a deviation from the Blackwell order. Beyond providing an accessible introduction to the topic from a channel ordering perspective, we present a novel resource-theoretic characterization of unique information in a cryptographic task related to secret key agreement. Our operational view of unique information entails rich physical intuition that leads to new insights into secret key agreement in the context of non-negative decompositions of the mutual information into redundant and synergistic contributions. Through this lens, we illuminate new directions for research in partial information decompositions and information-theoretic cryptography.Publicationtype: Journal Article9 26 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Theory and application of the information bottleneck method(Multidisciplinary Digital Publishing Institute (MDPI), 2024-02-22); In 1999, Naftali Tishby et al [...]Publicationtype: Journal Article63 148 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Information bottleneck signal processing and learning to maximize relevant information for communication receivers(Multidisciplinary Digital Publishing Institute (MDPI), 2022-07-14); ; Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding. Technically, the receiver-side signal processing for conducting these tasks is complex and hence causes bottleneck situations in terms of power, delay and chip area. Typically, many bits per sample are required to represent and process the received signal in the digital receiver hardware accurately. In addition, demanding arithmetical operations are required in the signal processing algorithms. A popular recent trend is designing entire receiver chains or some of their crucial building blocks from an information theoretical perspective. Signal processing blocks with very simple mathematical operations can be designed to directly maximize the relevant information that flows through them. At the same time, a strong quantization reduces the number of bits processed in the receiver to further lower the complexity. The described system design approach follows the principle of the information bottleneck method. Different authors proposed various ideas to design and implement mutual information-maximizing signal processing units. The first important aim of this article is to explain the fundamental similarities between the information bottleneck method and the functionalities of communication receivers. Based on that, we present and investigate new results on an entire receiver chain that is designed following the information bottleneck design principle. Afterwards, we give an overview of different techniques following the information bottleneck design paradigm from the literature, mainly dealing with channel decoding applications. We analyze the similarities of the different approaches for information bottleneck signal processing. This comparison leads to a general view on information bottleneck signal processing which goes back to the learning of parameters of trainable functions that maximize the relevant mutual information under compression.Publicationtype: Journal ArticleScopus© Citations 1 92 158 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Complexity as causal information integrationComplexity measures in the context of the Integrated Information Theory of consciousness try to quantify the strength of the causal connections between different neurons. This is done by minimizing the KL-divergence between a full system and one without causal connections. Various measures have been proposed and compared in this setting. We will discuss a class of information geometric measures that aim at assessing the intrinsic causal influences in a system. One promising candidate of these measures, denoted by ΦCIS, is based on conditional independence statements and does satisfy all of the properties that have been postulated as desirable. Unfortunately it does not have a graphical representation which makes it less intuitive and difficult to analyze. We propose an alternative approach using a latent variable which models a common exterior influence. This leads to a measure ΦCII, Causal Information Integration, that satisfies all of the required conditions. Our measure can be calculated using an iterative information geometric algorithm, the em-algorithm. Therefore we are able to compare its behavior to existing integrated information measures.Publicationtype: Journal ArticleScopus© Citations 4 83 184 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Canonical divergence for flat α-connections: classical and quantumA recent canonical divergence, which is introduced on a smooth manifold M endowed with a general dualistic structure (g,∇,∇*), is considered for flat a-connections. In the classical setting, we compute such a canonical divergence on the manifold of positive measures and prove that it coincides with the classical α-divergence. In the quantum framework, the recent canonical divergence is evaluated for the quantum α-connections on the manifold of all positive definite Hermitian operators. In this case as well, we obtain that the recent canonical divergence is the quantum α-divergence.Publicationtype: Journal ArticleScopus© Citations 1 69 169 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Canonical divergence for measuring classical and quantum complexity(Multidisciplinary Digital Publishing Institute (MDPI), 2019-04-24); ; A new canonical divergence is put forward for generalizing an information-geometric measure of complexity for both, classical and quantum systems. On the simplex of probability measures it is proved that the new divergence coincides with the Kullback-Leibler divergence, which is used to quantify how much a probability measure deviates from the non-interacting states that are modeled by exponential families of probabilities. On the space of positive density operators, we prove that the same divergence reduces to the quantum relative entropy, which quantifies many-party correlations of a quantum state from a Gibbs family.Publicationtype: Journal ArticleScopus© Citations 8 70 216 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , How to Teach Heat Transfer More Systematically by Involving EntropyIn order to teach heat transfer systematically and with a clear physical background, it is recommended that entropy should not be ignored as a fundamental quantity. Heat transfer processes are characterized by introducing the so-called “entropic potential” of the transferred energy, and an assessment number is based on this new quantity.Publicationtype: Journal ArticleScopus© Citations 5 355 442 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: with files , Investigating the thermodynamic performances of TO-based metamaterial tunable cells with an entropy generation approach(Multidisciplinary Digital Publishing Institute (MDPI), 2017-10-13); ; ; Active control of heat flux can be realized with transformation optics (TO) thermal metamaterials. Recently, a new class of metamaterial tunable cells has been proposed, aiming to significantly reduce the difficulty of fabrication and to flexibly switch functions by employing several cells assembled on related positions following the TO design. However, owing to the integration and rotation of materials in tunable cells, they might lead to extra thermal losses as compared with the previous continuum design. This paper focuses on investigating the thermodynamic properties of tunable cells under related design parameters. The universal expression for the local entropy generation rate in such metamaterial systems is obtained considering the influence of rotation. A series of contrast schemes are established to describe the thermodynamic process and thermal energy distributions from the viewpoint of entropy analysis. Moreover, effects of design parameters on thermal dissipations and system irreversibility are investigated. In conclusion, more thermal dissipations and stronger thermodynamic processes occur in a system with larger conductivity ratios and rotation angles. This paper presents a detailed description of the thermodynamic properties of metamaterial tunable cells and provides reference for selecting appropriate design parameters on related positions to fabricate more efficient and energy-economical switchable TO devices.Publicationtype: Journal ArticleScopus© Citations 9 274 318 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Morphological computation: synergy of body and brain(Multidisciplinary Digital Publishing Institute (MDPI), 2017-08-31); ; There are numerous examples that show how the exploitation of the body's physicalproperties can lift the burden of the brain. Examples include grasping, swimming, locomotion,and motion detection. The term Morphological Computation was originally coined to describeprocesses in the body that would otherwise have to be conducted by the brain. In this paper, we arguefor a synergistic perspective, and by that we mean that Morphological Computation is a processwhich requires a close interaction of body and brain. Based on a model of the sensorimotor loop,we study a new measure of synergistic information and show that it is more reliable in cases inwhich there is no synergistic information, compared to previous results. Furthermore, we discuss analgorithm that allows the calculation of the measure in non-trivial (non-binary) systems.Publicationtype: Journal Article16 - Some of the metrics are blocked by yourconsent settings
Item type:Publication, Access status: without files , Comparing information-theoretic measures of complexity in Boltzmann machines(Multidisciplinary Digital Publishing Institute (MDPI), 2017-07-03); ; In the past three decades, many theoretical measures of complexity have been proposed to help understand complex systems. In this work, for the first time, we place these measures on a level playing field, to explore the qualitative similarities and differences between them, and their shortcomings. Specifically, using the Boltzmann machine architecture (a fully connected recurrent neural network) with uniformly distributed weights as our model of study, we numerically measure how complexity changes as a function of network dynamics and network parameters. We apply an extension of one such information-theoretic measure of complexity to understand incremental Hebbian learning in Hopfield networks, a fully recurrent architecture model of autoassociative memory. In the course of Hebbian learning, the total information flow reflects a natural upward trend in complexity as the network attempts to learn more and more patterns.Publicationtype: Journal Article10