The new year is here and we have no reason to believe that the pace of progress in AI will slow down. These are our 10 predictions on what 2024 will bring, along with some of the trending papers from the past month.
As part of our yearly tradition, we looked into the crystal ball of research papers, industry news, and social media, to see what significant trends will be driving AI in 2024.
Zeta Alpha's 10 Predictions for AI in 2024
OpenAI will soon respond to the competition. Is it going to be GPT-4.5 or GPT-5?
Legal challenges won't stop large-scale GenAI pre-training.
NVIDIA will start facing serious competition from AMD in AI training.
The best new models in 2024 will be trained on synthetic data, with techniques like self-play.
Text-to-video and multimodal models will finally have their ChatGPT moment.
LLMs will be available at everyone's fingertips ... and eyes?
We will have real foundational robotics models by the end of this year.
Everyone will have their own, personalized LLM.
AI4Science: Drugs, materials, and designs "invented" by AI will make a real-world impact.
Generative AI will start to contribute to real revenue and profit in companies.
Predicting the future in an exponentially growing space like AI is tough, but we are looking forward to seeing how many of our bets will be confirmed by the end of 2024.
If you are interested in the recently published papers that led us to some of these predictions, here is a reading list:
Trending AI papers for January 2024
Improving Text Embeddings with Large Language Models by Liang Wang et al. (Microsoft) - 31 Dec. 2023.
Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models by Zixiang Chen et al. (UCLA) - 2 Jan. 2024.
Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Models by Avi Singh et al. (Google Deepmind) - 22 Dec. 2023.
ReST meets ReAct: Self-Improvement for Multi-Step Reasoning LLM Agent by Renat Aksitov et al. (Google Research) - 15 Dec. 2023.
Generative Multimodal Models are In-Context Learners by Quan Sun et al. (BAAI) - 20 Dec. 2023.
VideoPoet: A Large Language Model for Zero-Shot Video Generation by Dan Kondratyuk et al. (Google Research) - 21 Dec. 2023.
LLM in a flash: Efficient Large Language Model Inference with Limited Memory by Keivan Alizadeh et al. (Apple) - 4 Jan. 2024.
PowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU by Yixin Song et al. (SJTU) - 16 Dec. 2023.
Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation by Zipeng Fu et al. (Stanford) - 4 Jan. 2024.
Retrieval-Augmented Generation for Large Language Models: A Survey by Yunfan Gao et al. (Tongji University) - 4 Jan. 2024.
A few runner-ups that are definitely worth checking out:
Beyond Chinchilla-Optimal: Accounting for Inference in Language Model Scaling Laws by Nikhil Sardana & Jonathan Frankle (MosaicML) - .31 Dec. 2023.
Mixtral of Experts by Albert Q. Jiang et al. (Mistral AI) - 8 Jan. 2024.
SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling by Dahyun Kim et al. (Upstage AI) - 29 Dec. 2023.
An In-depth Look at Gemini's Language Abilities by Syeda Nahida Akter et al. (Carnegie Mellon University) - 24 Dec. 2024.
You can find an annotated collection of these papers, and more, in Zeta Alpha, which allows you to effortlessly discover relevant literature and dive deeper into any of the topics that are close to your interests!
Finally, the full-length recording of our January 2024 Trends in AI webinar is available on YouTube. If you'd prefer a short video overview describing the trending papers, check this out:
To join us live for the next installment of the Trends in AI webinar, make sure to sign up here.