Reduced-complexity optimization of distributed quantization using the information bottleneck principle

This paper addresses the optimization of distributed compression in a sensor network. A direct communication among the sensors is not possible so that noisy measurements of a single relevant signal have to be locally compressed in order to meet the rate constraints of the communication links to a common receiver. This scenario is widely known as the Chief Executive Officer (CEO) problem and represents a long-standing problem in information theory. In recent years significant progress has been achieved and the rate region has been completely characterized for specific distributions of involved processes and distortion measures. While algorithmic solutions of the CEO problem are principally known, their practical implementation quickly becomes challenging due to complexity reasons. In this contribution, an efficient greedy algorithm to determine feasible solutions of the CEO problem is derived using the information bottleneck (IB) approach. Following the Wyner-Ziv coding principle, the quantizers are successively designed using already optimized quantizer mappings as side-information. However, processing this side-information in the optimization algorithm becomes a major bottleneck because the memory complexity grows exponentially with number of sensors. Therefore, a sequential compression scheme leading to a compact representation of the side-information and ensuring moderate memory requirements even for larger networks is introduced. This internal compression is optimized again by means of the IB method. Numerical results demonstrate that the overall loss in terms of relevant mutual information can be made sufficiently small even with a significant compression of the side-information. The performance is compared to separately optimized quantizers and a centralized quantization. Moreover, the influence of the optimization order for asymmetric scenarios is discussed.

Subjects

Chief executive officer

distributed compression

distributed source coding

information bottleneck

DDC Class

004: Informatik

600: Technik

Funding Organisations

Deutsche Forschungsgemeinschaft (DFG)

More Funding Information

This work was supported in part by the University of Rostock, and in part by the German Research Foundation (DFG) in the funding programme Open Access Publishing under Grant 325496636.

Publication version

publishedVersion

Lizenz

https://creativecommons.org/licenses/by/4.0/

Name

Reduced-Complexity_Optimization_of_Distributed_Quantization_Using_the_Information_Bottleneck_Principle.pdf

Size

1.39 MB

Format

Adobe PDF

Options

Reduced-complexity optimization of distributed quantization using the information bottleneck principle