Decoded: How humans process large amount of data?

Humans can categorise data using less than 1% of the original information, say scientists, including those of Indian-origin, who have found an algorithm to explain human learning.

The method can also be used for machine learning, data analysis and computer vision, researchers said.

Humans learn to very quickly identify complex objects and variations of them. We generally recognise an 'A' no matter what the font, texture or background, for example, or the face of a coworker even if she puts on a hat or changes her hairstyle.

We also can identify an object when just a portion is visible, such as the corner of a bed or the hinge of a door.

"How do we make sense of so much data around us, of so many different types, so quickly and robustly?" said Santosh Vempala, from the Georgia Institute of Technology.

The researchers studied human performance in "random projection" tests to find how well humans learn an object.

They presented test subjects with original, abstract images and asked whether they could correctly identify that same image when randomly shown just a small portion of it.

"We hypothesised that random projection could be one way humans learn," said Rosa Arriaga, from Georgia Tech.

"The short story is, the prediction was right. Just 0.15% of the total data is enough for humans," she said.

The researchers then tested a computational algorithm to allow machines (very simple neural networks) to complete the same tests.

Machines performed as well as humans, which provides a new understanding of how humans learn.

The researchers wanted to come up with a mathematical definition of what typical and atypical stimuli look like and, from that, predict which data would be hardest for the human and the machine to learn.

Humans and machines performed equally well, demonstrating that indeed one can predict which data will be hardest to learn over time.

The researchers created three families of abstract images at 150x150 pixels, then very small 'random sketches' of those images. Test subjects were shown the whole image for 10 seconds, then randomly shown 16 sketches of each.

Using abstract images ensured that neither humans nor machines had any prior knowledge of what the objects were.

"We were surprised by how close the performance was between extremely simple neural networks and humans," Vempala said.

"This fascinating paper introduces a localised random projection that compresses images while still making it possible for humans and machines to distinguish broad categories," said Sanjoy Dasgupta, professor at the University of California San Diego.