In his latest Forbes column, Radical Ventures Partner Rob Toews examines the legacy of the Transformer, the core technology powering the current wave of AI innovation, and explores what technology may one day replace it.
If modern artificial intelligence has a founding document, a sacred text, it is Google’s 2017 research paper “Attention Is All You Need.” This paper introduced a new deep learning architecture known as the transformer, which has gone on to revolutionize the field of AI over the past half-decade. The generative AI mania currently taking the world by storm can be traced directly to the invention of the transformer.
Every major AI model and product in the headlines today – ChatGPT, GPT-4, Midjourney, Stable Diffusion, GitHub Copilot, and so on – is built using transformers.
Transformers are remarkably general-purpose: while they were initially developed for language translation specifically, they are now advancing the state of the art in domains ranging from computer vision to robotics to computational biology. In short, transformers represent the undisputed gold standard for AI technology today
But no technology remains dominant forever.
It may seem surprising or strange, with transformers at the height of their influence, to contemplate what will come next. But in the fast-moving world of AI, it is both fascinating and advantageous to seek to “see around corners” and glimpse what the future holds before it becomes obvious.
In that spirit, let us ask:
What might replace transformers?
Read the article in full.
AI News This Week
-
How to use AI to talk to whales—and save life on Earth (Wired)
Projects like Earth Species are using AI to decode non-human communication and help humans understand animals. Beyond deciphering the syntax and semantics of animal communication, achieving a two-way dialogue also requires a mutual understanding of a world that is similar enough for ideas to be expressed through language. Beyond communicating with animals, conservationists are using algorithms to sift through historical recordings of animals that are now extinct, such as the Hawaiin crow, to revive their lost calls.
-
A watermark for AI images that you cannot edit out (The Verge)
Following a White House meeting in July, AI companies committed to watermarking audio and visual content to signify AI generation and mitigate the risks of deepfakes. Google DeepMind has released a tool designed to watermark AI-generated images with a digital watermark embedded within image pixels. Image watermarking that does not compromise image quality and cannot be easily removed through editing or compression is a difficult challenge. The technology is currently in closed beta.
-
Can AI crack comedy? (Financial Times)
Some comedians are embracing generative AI for joke creation, challenging conventional notions about the uniqueness of human humour and comedic creativity. Recent performances at the Edinburgh Festival and events like Improbotics explore AI’s role in humour by having human comedians react to AI-generated jokes, blurring the lines between human and AI comedy. While AI-generated comedy is still evolving and often relies on wordplay or templates, it presents possibilities for human-AI collaboration and has the potential to ease human-AI integration.
-
The 34 most promising AI startups of 2023, according to top VCs (Business Insider)
Among other AI companies, Radical portfolio companies CentML and Twelve Labs were recognized by venture capitalists as the most promising AI startups (where they have no financial ties). CentML is addressing the worldwide shortage of GPUs needed for training and inference of generative AI models by optimizing machine learning models to work faster to reduce compute costs. Twelve Labs is a cloud service for video search and understanding that helps developers build programs that can see, listen, and understand the world as we do through powerful video understanding infrastructure.
-
Research: AI-powered drone beats human champion pilots (Nature)
In a landmark achievement, a small drone equipped with an AI system defeated three world-champion drone pilots in an indoor race, marking the first time AI has outperformed human champions. Developed by University of Zurich and Intel researchers, the algorithm combined supervised learning for gate identification, conventional programming for position estimation, and deep reinforcement learning for virtual training on the race course. During races, the AI drone utilized its camera and neural network for gate recognition and onboard sensors for positioning, speed, and orientation. The AI drone won 15 out of 25 races and set a new record for the fastest lap time, opening up possibilities in fields such as structural inspections, space exploration, and search and rescue missions.
Radical Reads is edited by Leah Morris (Senior Director, Velocity Program, Radical Ventures).
