The subatomic particle known as a neutrino is nicknamed the ghost particle. Every day trillions of them stream through the Earth without ever interacting with the matter around them. But scientists can detect neutrinos using specialized sensors deep underground.
To find the elusive particles, researchers have to sort through an absolutely astounding amount of data. The problem is even worse when you are looking for a particular type of neutrino. This is the case at the IceCube South Pole Neutrino Observatory, explains J.M. Porup for Motherboard. The IceCube is the world's largest particle detector, whose sensors are buried under a cubic kilometer of frozen water, seeking neutrinos.
The large amount of data pouring in adds up fast—terabytes of raw data every day. "In total, the IceCube project is storing around 3.5 petabytes (that's around 3.5 million gigabytes, give or take) in the UW-Madison data center as of [now]," Porup writes.
For some perspective: One petabyte, or 1,000 terabytes, is about the equivalent to a 32 year-long MP3 song and the amount of storage required for the 3D effects of the film Avatar.
But only a tiny fraction of this data is actually of interest. The IceCube detects about one neutrino produced by collisions that happen in the atmosphere every 10 minutes, but the high-energy neutrinos that the scientists are really interested in finding come from astronomical events far away in space, IceCube researcher Nathan Whitehorn tells Motherboard. These prize neutrinos are only detected about once a month.
This is a depressingly small amount: "Each particle interaction takes about 4 microseconds, so we have to sift through data to find the 50 microseconds a year of data we actually care about," Whitehorn tells Porup.
Why go to all the effort? These special neutrinos come from violent astrophysical happenings: Exploding stars, high energy bursts of gamma rays, events that happen in black holes and neutron stars. Studying neutrinos can give insights into these events as well as aid in the search for dark matter.
Data demands in physics aren’t new. The search for the Higgs Boson involved sifting through more than 800 trillion collisions at the CERN particle collider in Switzerland. CERN itself had collected about 200 petabytes of data by 2012 when the research team announced the Higgs discovery, reports Loraine Lawson for IT Business Edge.
For the IceCube project, storing and analyzing all that data is a monumental and expensive task, but it's worth the effort. Though scientists are only looking at a small fraction of the numbers now, the answers to many of the universe's mysteries may be lurking in those hard drives.