The annual NeurIPS conference is always a whirlwind of inspiration and intellectual overload. This year was no different, with the sheer volume of research pushing the boundaries of AI even further. The number of papers published at NeurIPS 2023 was more than 5x (!) the number published at NeurIPS 2017, where the seminal Attention Is All You Need paper introduced the Transformer architecture to the world, kickstarting the foundation model revolution. Just look at the graph below:

It's clear that the field is exploding, and one of the most exciting areas is the rise of large language models (LLMs). 2023 saw LLMs take center stage, not just in research papers, but also in real-world products that are finally putting the power of AI into the hands of everyday users. This productization of AI is a game-changer, and it's only going to get bigger in 2024.

At Arena, we're particularly interested in how foundation models can be applied to real-world problems. Here are three key takeaways from NeurIPS that resonated with us, and how they align with our vision for enterprise AI:

1. Multimodality is key: NeurIPS 2023 saw a surge of interest in multimodality, with models seamlessly integrating language, vision, and audio. We anticipate that multimodal intelligence will soon become the baseline expectation for any user interacting with AI.

2. From prediction to action: AI is no longer just about making predictions. In 2024, we expect to see a major shift towards "agentive AI" that can take actions in the real world to achieve specific goals. This is where things get really interesting, because it means AI can move beyond being a passive assistant to becoming an active collaborator.

3. Efficiency is the new black: Training massive models is impressive, but it's only half the battle. When you have millions of users interacting with your AI, inference costs become the real bottleneck. That's why we're excited about the focus on developing efficient AI architectures and algorithms. Techniques like Mixture of Experts models and sub-quadratic attention mechanisms are just the tip of the iceberg, and we expect to see even more innovation in this area in 2024.

At Arena, we're building enterprise AI solutions that are aligned with these trends. Our version of multimodality goes beyond the usual suspects, incorporating industry-specific modalities like customer purchase transaction history, mobile app clickstreams, multichannel sensor data, and more. We're also laser-focused on developing AI agents that can take actions in the real world, such as setting product promotion strategies, streamlining manufacturing production lines, or optimizing complex hardware systems. And finally, we're obsessed with efficiency, because we know that's what will allow us to scale our solutions and deliver real value to our customers.

2024 is shaping up to be a pivotal year for AI, and we at Arena are thrilled to be a part of it. We can't wait to see what the next NeurIPS brings, and we're confident that the innovations we see there will help us build the future of enterprise AI.

Stay tuned for more updates from Arena as we continue to push the boundaries of AI, and check out our open roles to join us!