> The change is the gradual accumulation of statistics. These are relatively rare events. The LHC has been running, high-energy proton-proton collisions have been occurring, and the LHCb detector in this case has been measuring them. The statistics increase, and eventually the characteristic peaks of short-lived resonances can be identified above the noise of "background" collisions.
I think people are a bit spoiled by the Higgs leak/announcement/discovery timeline. I'm sure those in the know have known about this discovery for some time but, like you said, it takes some time to gather enough data to be confident (and to qualify as the mathematical standard set for "discovery").
Right, after the bump reaches 3x sigma they have some confidence it deserves their own attention, and at 4x they are sure, but the rule is you don't publish until you have enough data for a 5x sigma result, which just takes a lot more data and a lot longer.
"Sigma" here refers to standard deviations off the Gaussian normal mean. Zero means completely random. In psychology they publish at 2x sigma, 95%, which means 20:1 odds against a spurious result, and they publish a lot of spurious results because you can generate an unlimited number of hypotheses. In physics, things are considered more deterministic, and an experiment doesn't need to recruit undergrads to be data points, so you run your LHC for a few more months and avoid wasting people's attention.
The chance of falsely rejecting the null hypothesis increases as you gather more data. Put more simply, finding something that differs "significantly" from some distribution becomes easier as you gather more data. Imagine having only 3 psychology student in a study, the required effect size has to be huge for the test to say that it is significantly different.
However, the approach taken by CERN is of course right. They find a result at a certain significance level and then collect more data to verify the result. As long as there aren’t thousands of simultaneous verifications running, this approach is sound. Obviously yes, physicist’s know what they’re doing.
Having said that, please don’t read this comment as me approving of frequentists statistics. Bayesian or cross-validations are way easier to interpret where possible.
I think people are a bit spoiled by the Higgs leak/announcement/discovery timeline. I'm sure those in the know have known about this discovery for some time but, like you said, it takes some time to gather enough data to be confident (and to qualify as the mathematical standard set for "discovery").