DeepMind: Why is AI so good at language? It's something in language itself

Can the frequency of language, and qualities such as polysemy, affect whether a neural network can suddenly solve tasks for which it was not specifically developed, known as "few-shot learning"? DeepMind says yes.

Tiernan Ray for ZDNet

How is it that a program such as OpenAI's GPT-3 neural network can answer multiple choice questions, or write a poem in a particular style, despite never being programmed for those specific tasks?

It may be because human language has statistical properties that lead a neural network to expect the unexpected, according to new research by DeepMind, the AI unit of Google.   

Artificial Intelligence

• AI in 2023: A year of breakthroughs that left no human thing unchanged
• These are the jobs most likely to be taken over by AI
• AI at the edge: 5G and the Internet of Things see fast times ahead
• Almost half of tech executives say their organizations aren't ready for AI or other advanced initiatives

Natural language, when viewed from the point of view of statistics, has qualities that are "non-uniform," such as words that can stand for multiple things, known as "polysemy," like the word "bank," meaning a place where you put money or a rising mound of earth. And words that sound the same can stand for different things, known as homonyms, like "here" and "hear." 

Those qualities of language are the focus of a paper posted on arXiv this month, "Data Distributional Properties Drive Emergent Few-Shot Learning in Transformers," by DeepMind scientists Stephanie C.Y. Chan, Adam Santoro, Andrew K. Lampinen, Jane X. Wang, Aaditya Singh, Pierre H. Richemond, Jay McClelland, and Felix Hill.

Also: What is GPT-3? Everything you need to know about OpenAI's breakthrough AI language program

Garmin wins big at CES, launches new products and updates Garmin Connect From indoor solar to light-based internet, photonics offers a brighter future I saw Samsung and LG's new transparent TVs at CES, and there's a clear winner Have 10 hours? IBM will train you in AI fundamentals - for free Garmin wins big at CES, launches new products and updates Garmin Connect

From indoor solar to light-based internet, photonics offers a brighter future

I saw Samsung and LG's new transparent TVs at CES, and there's a clear winner

Have 10 hours? IBM will train you in AI fundamentals - for free

Editorial standards Show Comments