2022 will forever be remembered as the year Artificial Intelligence went viral.
The AI chatbot ChatGPT launched and grew to millions of users in days. Text-to-image generators such as DALL-E and Stable Diffusion captured attention by helping people create images based on whatever their imagination could dream up, while AI-assisted coding rapidly transformed what it means to develop software.
It seems that the adoption of AI was pulled forward by years. So what has changed? Did “generative AI,” materialize out of thin air as a new paradigm in technology? The answer is no.
Mainstream adoption of natural language as an interface for computing is driving the current frenzy of innovation. Underpinning this new modality is the Transformer model architecture, invented in 2017 by a group of then-Google Brain researchers, including Aidan Gomez, now CEO and Co-founder of Radical portfolio company Cohere.
For an illuminating overview of the technologies under the generative AI hood, Jeff Dean, Senior Fellow and SVP of Google Research, posted a blog post exploring the state of the art in 2022, and some of the breakthroughs he anticipates in the coming years, including:
Language models – The progress on larger and more powerful language models remains one of the most exciting areas of AI research. Important advances along the way have included approaches like sequence-to-sequence learning and Google’s development of the Transformer model.
Generative models – The quality and capabilities of generative models for imagery, video, and audio has shown extraordinary advances in 2022. For instance, Generative adversarial networks, developed in 2014, set up two models working against each other to get better and better at their tasks.
Multimodal models – Most multimodal work has focused on models that deal with a single modality of data (e.g. language models, image classification models, or speech recognition models), but expect advances in multi-modal models that can flexibly handle many different modalities simultaneously.
Responsible AI – Powerful language models and generative models can be used for amazing, useful, and creative purposes. However, leaders in machine learning and AI must lead not only in state-of-the-art technologies, but also in state-of-the-art approaches to responsibility and implementation.
As the language interface for computing continues to expand, access to increasingly sophisticated AI will be available to an even larger user base. With text-to-video and audio just around the corner, expect more viral moments in 2023.
Radical Reads for the week of January 22, 2023:
Google calls in Larry Page and Sergey Brin to tackle ChatGPT (The New York Times)
In the wake of the release of ChatGPT, Google’s CEO, Sundar Pichai, declared a “code red” for Google, and called in the company’s founders, Larry Page and Sergey Brin, to help make AI development Google’s #1 priority. ChatGPT’s release revealed a new way to think about search and human-computer interfaces, a revelation that has created a moment of significant vulnerability for Google. New companies, including You.com, a Radical portfolio company mentioned in the article, are already offering online search engines that let you ask questions through an online chatbot. ChatGPT is just the tip of the iceberg. Incumbent businesses in every industry will need to adopt new AI software to stay ahead of both their existing competitors and new entrants.
Autonomous trucking startup and Radical portfolio company, Waabi, announced Volvo’s venture capital arm as a strategic investor. “We’ve been selective in terms of who we bring on board as an investor, and this is the right time for Waabi to bring on a strategic original equipment manufacturer (OEM),” noted Waabi’s CEO and founder, Raquel Urtasun. The purpose-built for OEM integration means that Waabi’s Driver — which includes software, sensors and computer power—is manufactured directly into the vehicle.
Researchers used AI to discover a potential new cancer drug (The Toronto Star)
Using advances in AI, researchers from Insilico Medicine and the University of Toronto have shrunk what usually takes years or even decades in drug discovery to less than a month. The researchers include Radical Ventures’ Scientific Advisor, Alán Aspuru-Guzik. The study may be the world’s first to apply AlphaFold, the groundbreaking AI technology, to drug discovery research. Although their proposed drug still needs to pass clinical trials, the paper’s authors say their process demonstrates the revolutionary potential of AI in medical research. The study was published last week in the journal Chemical Science.
How AI is combatting burnout (The Financial Times)
Platforms for employee engagement and clinical screening increasingly include tools that detect sentiment from text and speech patterns. Depression is a major burden on the healthcare system worldwide. Most care for depression is delivered by general practitioners and misidentifications among those with symptoms are more likely than missed cases. AI could help with the nuances of diagnosis such as anxiety, burnout, and short-term stress responses that may look like depression among patients.
Research Paper: Mastering diverse domains through world models (arxiv)
Researchers at the University of Toronto and DeepMind have built DreamerV3, a general and scalable reinforcement learning (RL) algorithm that is outperforming previous approaches. DreamerV3 is one system that can be trained on different tasks without too much tinkering and tuning. It appears that DreamerV3’s three neural networks generalize widely and perform well. This could be a significant step forward as RL agents tend to either generalize widely but perform poorly (or inefficiently), or perform fantastically but generalize poorly. In comparison with previous algorithms, DreamerV3 has learned how to play Minecraft at a very high level of technical complexity.