Before the official article, let’s share an interesting topic related to today’s content:
has 1,000 samples, one of which is unqualified. It is known that a detection instrument can simultaneously mix and detect any multiple samples (i.e. give whether there is one failure) and give accurate results within 15 minutes (the test does not destroy the sample, and the sample can be used for multiple instruments at the same time). If you require this unqualified sample to be found within 1 hour, at least how many testing instruments are needed?
Claude Elwood Shannon, 1916-2001
Image source: Internet
Claude Elwood Shannon (Claude Elwood Shannon, 1916-2001), an American mathematician and founder of information theory. The Shannon Prize, set to commemorate him, is the highest award in the field of communication theory, also known as the "Nobel Prize in the field of information".
1916, Shannon was born in a small town in , Michigan. Shannon's grandfather was a farmer and invented washing machines and many agricultural machinery. Under the influence of my grandfather, model aircraft, remote-controlled boats and radio stations are often made at home. Shannon worshipped Thomas Edison since he was a child. Interestingly, it turned out that Edison turned out to be his distant relative.
In 1932, Shannon entered University of Michigan and began to contact the theory of George Boole (George Boole , British mathematician) . When he graduated from college, he obtained two bachelor's degrees in electronic engineering and mathematics, and entered MIT for further studies. In 1938, Shannon obtained a master's degree in electrical engineering from the MIT. The master's thesis title was A Symbolic Analysis of Relay and Switching Circuits [1]. He pioneered the proposal to correspond to the "true" and "false" of Boolean algebra and the "on" and "off" of the circuit system, and used Boolean algebra analysis to optimize the switching circuit, laying the theoretical foundation for digital circuits. Professor Howard Gardner of Harvard once said, "This is probably the most important and famous master's thesis of this century." In 1940, Shannon won the Alfred Noble Award from the American Institute of Engineers for this achievement. However, even such achievements are far from being called Shannon's highest moment of glory.
1940, Shannon entered the Princeton Advanced Institute for after obtaining a doctorate in mathematics from MIT, and began to think about the issues of information theory and effective communication systems. After 8 years of hard work, Shannon published a far-reaching paper during his work in Bell Labs , A Mathematical Theory of Communication [2]. Shannon clearly explained the basic problems of communication, gave a model of the communication system, and proposed a mathematical expression of information quantity - information entropy , which became a milestone in the official birth of information theory. Information entropy is a tool that measures the amount of information, that is, the uncertainty of information , and the mathematical expression is:
. Simply put, the greater the uncertainty of information, the greater the information entropy calculated. If the calculation is based on b=2, then the calculated information entropy is in bits (bit). The emergence of "bits" marks that humans know how to measure the amount of information. (Think about how to use information entropy to solve the interesting problems above). In 1949, Shannon published another famous paper Communication in the Presence of Noise [3], solving a series of basic technical problems such as channel capacity , source statistical characteristics, source encoding, channel encoding, etc. It is worth mentioning that when we happily brush this article with our mobile phones, we have to thank Shannon for his achievements in this paper - Shannon's Second Theorem (Noisy Channel Coding Theorem). This theorem clearly defines what factors determine the transmission rate in the field of wireless communications and the quantitative relationship between them. It has played a huge role in promoting the development of wireless communication principles and technologies, and has pointed out the direction for people to use limited spectrum resources to transmit information faster and better.At the same time, it can be deduced from this formula that even if an infinite spectrum bandwidth is applied, the rate of information transmission is limited, which is the famous Shannon limit. This achievement was later successfully applied to telephones, fiber and wireless communications. For example, the spectrum efficiency of 5G has been largely close to or even reached Shannon's limit.
Shannon is a typical interest-driven scientist. He does not consider whether his research results are commercially valuable, or even does not care whether the final results are useful. He once said, "I spent a lot of time on completely useless things." In addition to the rich achievements in mathematics and communication fields, Shannon has also entered the fields of cryptography [4], artificial intelligence [5], and has achieved important results. For example, in 1949, his paper Programming a Computer for Playing Chess [5] was one of the pioneering work in the field of artificial intelligence.
Reference:
[1] Shannon C E. A Symbolic Analysis of Relay and Switching Circuits. Electrical Engineering, 1938, 57(12): 713-723.
[2] Shannon C E. A Mathematical Theory of Communication. The Bell system technical journal, 1948, 27(3): 379-423.
[3] Shannon C E. Communication in the Presence of Noise. Proceedings of the IRE, 1949, 37(1): 10-21.
[4] Shannon C E. Communication Theory of Secrecy Systems. The Bell system technical journal, 1949, 28(4): 656-715.
[5] CE S. Programming a Computer for Playing Chess. IEEE Press, 1993: 637-656.
text | Ding Qiming
cover image | Zhu Chengxuan
