Zoho partners with NVIDIA to build business-specific LLMs with NeMo technology

Zoho Corporation announced its plans to utilise NVIDIA's AI accelerated computing platform, including NVIDIA NeMo, to develop and implement large language models (LLMs) in its Software as a Service (SaaS) applications. This development was revealed at the NVIDIA AI Summit in Mumbai on October 24, 2024. The LLMs will be accessible to Zoho's extensive customer base of over 700,000 across ManageEngine and Zoho.com worldwide. Zoho has already invested over USD 10 million in NVIDIA's AI technology and GPUs and intends to invest an additional USD 10 million in the upcoming year.

Ramprakash Ramamoorthy, Director of AI at Zoho Corporation, highlighted the company's commitment to creating LLMs specifically tailored for various business applications. He stated that many existing LLMs are consumer-focused and lack business utility. Zoho's approach leverages its comprehensive tech stack to integrate context into AI, enhancing its effectiveness. The company emphasises user privacy by ensuring its models comply with privacy regulations from the outset, aiming for swift and effective return on investment for businesses through NVIDIA's AI software and accelerated computing.

Zoho has been integrating AI into its diverse range of over 100 products within the ManageEngine and Zoho divisions for more than a decade. The company adopts a multi-modal AI strategy to derive contextual intelligence, aiding users in making informed business decisions. Zoho is developing narrow, small, and medium-sized language models, distinct from LLMs, to address various use cases. These models offer businesses the flexibility to choose AI solutions that align with their needs, balancing performance and cost-effectiveness. Privacy remains a core focus, and Zoho's LLM models will not be trained on customer data.

Vishal Dhupar, Managing Director, Asia South at NVIDIA, commented on the collaboration, stating that Zoho's use of NVIDIA's AI software and accelerated computing platform enables the creation of a broad range of models to meet diverse business requirements. Zoho is enhancing its LLMs on NVIDIA's platform using NVIDIA Hopper GPUs and the NVIDIA NeMo end-to-end platform for custom generative AI development. The company is also testing NVIDIA TensorRT-LLM to optimise its LLMs for deployment, achieving a 60% increase in throughput and a 35% reduction in latency compared to previous open-source frameworks. Additionally, Zoho is accelerating workloads such as speech-to-text on NVIDIA's infrastructure.