The so-called pursuit of truth is the process of people starting from prejudice and moving step by step towards objectiveness.
Most people first heard that Bayesian theorem should be in middle school classroom. That formula doesn't seem complicated, and it seems ordinary among the fancy test points.
But soon, we knew what it means to hide deeply. From university textbooks to practical research, Bayes's three words frequently appear on important issues in information science, and it even turns from a theorem to a thought. Of course, such a transformation is not completed overnight. In history, Bayesians spent about two hundred years breaking through many obstacles to let people see the rich treasures contained in Bayes theorem.
01
3 New ideas that are not recognized
Bayes Theorem
Bayes theorem does not look amazing, and even Bayes himself, the proposer, did not pay much attention to this achievement, and even did not publish it in his lifetime. It was only two years after his death that Bayes theorem officially entered people's sight.
In fact, the first real Bayesianist was Laplace . In 1774, the mathematician and astronomer proposed the Laplace continuation rule , which could be introduced from Bayesian formula .
Simply put, this rule says that in a repeating event, people can predict future results based on the results that have occurred in the past. For example, in the past, the sun rose every day, then it will probably rise tomorrow.
But this law that embodies Bayesian wisdom has attracted a lot of ridicule. Until the 19th century, it was still criticized by mainstream academics.
Mathematician George Cristal even asserted, "(these probabilities) are dead, we should bury them decently in places that cannot be seen, rather than introducing them in textbooks and test questions... We should allow ourselves to quietly forget the recklessness of the great man." Why is this happening to
?
Because at that time, most people regarded objectivity as a principle that could not be touched. People think that if you have the statistics in place and the calculations are correct, you can find the answer, otherwise you will be wrong. However, the Laplace continuation law introduces a certain degree of subjectivity to probability.
Assuming we are giants of ancient times, we have only seen the sun rise five times. If the Laplace continuation law is applied at this time, then we will calculate a relatively small probability as to whether the sun will rise again. Our results will change over time. In the eyes of many people, this is really weird.
mainstream scholars who emphasize objectivity have formed a frequency school. Scholars who still believe that subjectivity contains great value in probability form Bayesianism.
Entering the 20th century, the Bayesians were excluding and suppressing so violently that Ronald Fisher, the representative of the frequency school, even used the extremely insulting words such as "fallacies".
Bayesians have to survive in the cracks. Even sometimes, Bayesian wisdom has been proven, and Bayesians still have no chance to come to the forefront.
WWIIhtml
During World War II, Allen Turing led the team to decipher the Ennig password of the German army and made great contributions. Bayes theorem played an important role in it. Unfortunately, this incident involved too many political factors, and after the war, the British government chose to continue to keep it confidential. Bayesianism lost a chance to be justified.
In the 1950s, British actuary Arthur Bailey discovered that some colleagues were using some "unknown" but particularly practical formulas.Bailey didn't know at that time that these formulas were closely related to Bayesian theorem, but the frequency faction dominated and kept them all outside the door of orthodox education.
Considering that Bayesianism has been rejected for more than two hundred years, it is difficult to estimate how many events there are in this type of event. But what is certain is that no matter how aggressive the frequency facts are, the frequency faction cannot change.
02
The era gave the answer
The fact is that Bayesians have their own tricks when it comes to the problem of the powerlessness of the frequency school. Although it is excluded by the small circle of statistics, the situation is quietly changing in areas where it pays more attention to practical work.
For example, Robert Schleiffer and Howard Laifa proposed the decision theory that contains uncertainty . Jerm Cohenfield practiced Bayesian statistics when studying the carcinogenic problem of tobacco and came to important conclusions.
NASA also hired institutions that master Bayesian tools to predict the probability of major accidents in rocket launches. In all these examples, Bayesian theory is more dynamic than frequency theory.
Finally, Bayesians waited for the best time to counterattack, that is, the birth and rise of computer science. In the 1960s, Ray Solomonov combined the computability theory of Turing with Bayesian formula to build the predecessor of the general framework of artificial intelligence .
In the 1980s, the Monte Carlo method brought revolution to the practical application of Bayesian formulas. A program called Gibbs sampling Bayesian inference even declared the victory of Bayesianism.
Bayesians usher in new development. Bayes is no longer equivalent to a formula or a proposition. Its methods are becoming more and more advanced, and its theoretical roots are becoming more and more developed.
There is a famous saying in the field of statistics: "All models of are wrong, but some are useful ." From George Box, a statistics master who has made important contributions in Bayesian inference and other aspects. In a sense, this sentence has also become an embodiment of Bayesian methodology.
frequency school is very similar to the classic science of old school, and likes to emphasize certainty, while Bayesian school is more like quantum mechanics , which also caused controversy in the 20th century. In the view of the frequency school, even if the parameters are unknown, they are objective, while the Bayesian school will say that the parameters are ever-changing. They know that everything has countless causes and consequences, and it is almost impossible to find absolutely objective.
Since this is the case, we should look at the model from a dynamic perspective, and even find ways to integrate as much information as possible and then produce results for more specific purposes. For pure Bayesians, incompatible models can not only exist at the same time, but also a good thing.
This kind of incredible method of acting shines with the help of a computer. Now, Bayesian wisdom has penetrated into all aspects of human society, and the fields involved include but are not limited to: Medical diagnosis, insurance, finance, artificial intelligence, neuroscience , advertising, logistics, signal processing, aerospace engineering ...
It can be said that No modern human life has nothing to do with Bayesians.
03
Looking the world with the eyes of Bayesians
What is even more amazing is the relationship between Bayesianism and some ultimate problems.
Turing-Church Thesis tells us that Nothing in the universe can complete the calculations that Turing machine cannot complete .This means that when you have enough data, a universal Turing machine can simulate the entire universe.
If that's the case, will it be possible for people to have the kind of machine that calculates everything in science fiction novels in the future? Can it be used to crack all the secrets in the universe?
Ray Solomonov answered this question in a Bayesian way. He proposed an interesting formula called "Solomonov Induction". This formula is also called the ultimate form of Bayes theorem by some young Bayesians.
Theoretically, if the input data contains a certain pattern, this formula will eventually find it, and the time required is proportional to the Solomonov complexity. This seems to be an optimistic conclusion, however Solomonov induction is uncalculable. The machine that calculates everything still needs to stay in people's fantasy.
So what is the use of this theory?
Some, It explains the contradiction between completeness and computability . Solomonov pointed out that all the computable knowledge philosophy of cannot detect all the laws in the data. In other words, any computable knowledge philosophy may continue to give wrong predictions under certain conditions, which forms a subtle reflection of Godel's theorem of incompleteness.
Even if you break through all the impossible, you have a Turing machine that meets the conditions and can run Solomonov induction, the universe also has a way to deceive us. As long as there is a piece of code, the complexity of Solomonov is greater than the amount of information provided. In this way, even with endless computing power, you cannot confirm any truth.
From another perspective, the laws that dominate nature and even human society also hide Bayesian logic. If we perform some algebraic operations on the Lotka-Volterra equation in biology and deduce the equation that governs the proportion changes of different variants in the population, we will find that it has surprising similarities with the Bayesian formula.
The evolutionary wisdom of nature created the human brain, and we are now starting to try to use artificial intelligence to imitate it. In the history of the evolution of human civilization, behind the formation of human concepts and customs of different ethnic groups and backgrounds, there is also a powerful guide to prior probability and posterior probability.
If that's the case, what kind of world do we live in?
From the Bayesian perspective, even if you say "our world is fake, it is a virtual program of God", this answer does not have to be labeled "absurd" immediately. Such a grand question may never be concluded, but if there is indeed a so-called God, then He must be a naughty and dynamic Bayesian.
04
Prejudice wisdom
Frequency school likes to say that Bayesian school is not objective, or holds prejudice, But Bayesian school's wisdom is precisely a kind of wisdom about prejudice.
In reality, no one is omniscient and omnipotent. Some people stop moving forward because they are afraid of making mistakes, while others simply deny the prevalence of prejudice and regard the principles that have been finitely verified as universal truth.
, and Bayesians accept the existence of prejudice. They believe that the so-called pursuit of truth is the process of people starting from prejudice and moving step by step towards objectiveness.
They are willing to look at the truth from a dynamic perspective, and are willing to continue to learn and constantly get out of prejudice —This is the spirit of the Bayesians.
There is still a lot of content worth learning about Bayesianism. For example, the Solomonov monsters created by Bayesianism, such as Bayesian methods and overfitting, and the Bayesian brain.
All of this content can be found in this book - "Bayes in Bayes"
