Alibaba Cloud has integrated its Large Multimodal Model (LMM) solution with Nvidia’s automotive platform to improve smart mobility capabilities for Chinese automakers.
The collaboration, which also includes Alibaba’s smart cockpit solution provider Banma, aims to revolutionize in-car interactions and autonomous driving capabilities.
The collaboration was followed by the unveiling of a large multimodal model (LMM) solution by Alibaba Group Holding’s cloud computing services division. The LLM solution is tailored for automotive applications, which Alibaba Cloud developed in cooperation with Nvidia and start-up Banma Network Technology, Alibaba’s intelligent cockpit solution provider.
The partnership also includes Alibaba Cloud’s proprietary large-scale Qwen language model, including the Qwen2-7B and Qwen2-VL vision language models, incorporated into the NVIDIA DRIVE AGX Orin platform for autonomous vehicles. This integration, coupled with NVIDIA’s model acceleration technology, promises to reduce computational costs and minimize latency when processing complex real-time tasks.
This is the first integration of Alibaba Cloud’s large-scale AI models into the US semiconductor powerhouse’s Drive automotive platform. China’s primary electric vehicle (EV) manufacturers – including Li Auto, Great Wall Motors, Zeekr Intelligent Technology Geely Auto and EV unit Xiaomi – are using the platform to power their next-generation fleets.
Leveraging Qwen’s advanced capabilities, in-car voice assistance can engage in dynamic multi-turn conversations and offer recommendations. These range from providing information on nearby landmarks to suggesting when to turn on your headlights in adverse weather conditions.
The LMM solution also includes Alibaba Cloud’s Mobile Agent, a multi-modal artificial intelligence agent for edge devices. The technology is set to expand the capabilities of in-car voice assistants, allowing users to perform complex voice commands, such as ordering food through delivery apps, even while on the move.
NVIDIA’s involvement brings substantial benefits to the partnership. The company’s expertise in artificial intelligence and graphics processing has been instrumental in optimizing the performance of Alibaba Cloud models on automotive-grade hardware. This optimization ensures that AI-driven features can work effectively within the limitations of the vehicle’s computer systems.
Overall, the collaboration between Alibaba Cloud and Nvidia is expected to have implications for the wider automotive industry in China. By making this solution available to Chinese automakers, Alibaba Cloud and NVIDIA will potentially accelerate the adoption of AI-driven technologies in vehicles in one of the world’s largest automotive markets.
In the future, Alibaba Cloud has plans to expand cooperation with NVIDIA. The companies aim to adapt Qwen models for NVIDIA DRIVE Thor, NVIDIA’s next-generation centralized computing platform. This platform combines advanced driver assistance, autonomous driving and cockpit AI capabilities into a secure system.
Separately, Nvidia is also working with Alibaba Cloud on infrastructure “to meet people’s extreme computing power demands,” Nvidia global vice president He Tao said at a sub-forum hosted by the US company at the Apsara conference.
According to Nvidia, Nvidia and Alibaba Cloud are pursuing a number of LLM-related initiatives for accelerated computing. These initiatives use parallel processing to speed up work and increase energy efficiency across demanding applications. The two companies are also working to create customized solutions to help “traditional businesses” move their LLM operations to the cloud.
Sustainable will share more of his expertise at this year’s event Cyber Security & Cloud Expo Europe. Check out Tenable’s booth at booth #144 to learn more about how to keep your business safe.
Explore other upcoming enterprise technology events and webinars powered by TechForge here.
