Chinese tech unicorn 01.AI admits ‘oversight’ in changing name of AI model built on Meta Platforms’ Llama system

Beijing-based 01.AI said the company made several name changes in its open-source large language model’s code as part of experimental requirements
The firm has decided to change the so-called tensor name of its AI model Yi-34B to reflect that it was built on Meta Platforms’ Llama system

Artificial intelligence

Ben Jiangin Beijing

Published: 10:01pm, 15 Nov 2023

Why you can trust SCMP

A Chinese tech unicorn founded by Taiwanese venture capitalist and computer scientist Lee Kai-fu has decided to change the tensor name of its open-source artificial intelligence (AI) large language model (LLM) Yi-34B, following an inquiry that revealed it was based on the architecture of Meta Platforms’ Llama system.

Beijing-based 01.AI, whose value has reached more than US$1 billion in less than eight months since it was founded, on Tuesday described the matter as “an oversight” and that the tensor name of its LLM – the technology used to train intelligent chatbots like ChatGPT – would be changed to reflect that it was built on Meta’s LLM, according to a post on the Hugging Face open-source community platform by 01.AI’s open-source director Richard Lin, who responded to an earlier query by AI researcher Eric Hartford.

Tensors are data containers in the AI machine-learning process that hold and arrange information in a structured manner, making it easier for LLMs to understand and generate humanlike text.

LLMs are deep-learning AI algorithms that can recognise, summarise, translate, predict and generate content using very large data sets.

Lee Kai-fu, the co-founder, chairman and chief executive of Sinovation Ventures, in May founded Beijing-based start-up 01.AI, which reached a valuation of more than US$1 billion after its latest funding round. Photo: SCMP

“During extensive training experiments, we made several renamings in the code to meet experimental requirements,” 01.AI’s Lin said in his post on Tuesday. “But we kinda dropped the ball and didn’t switch them back before pushing out our release … We’re sorry for the confusion.”

The company said in a response to the Post via WeChat on Wednesday that it changed the tensor name of its Yi-34B LLM to “fully test the [Llama] model” and that there was no intention to mask the source of the AI model.

The oversight by 01.AI reflects the complex range of activities behind the current rush to develop various LLMs on the mainland, where a new government body has been set up by Beijing as part of plans to implement a national standard for AI models.

The incident happened at a time when Baidu chief executive Robin Li Yanhong has criticised Chinese tech companies’ frenzy to develop more LLMs as “a huge waste of resources”.

Hugging Face community member Hartford, who had raised more than a week ago questions about the tensor name of 01.AI’s Yi-34B, which was released on November 6, on Wednesday dismissed any oddity with using another company’s AI model to help develop its own LLM based on the open source community’s perspective.

China start-up 01.AI hits US$1 billion value with top-ranked open-source model

“In our open source community, we often share each other’s code,” Hartford, a senior researcher at conversational AI tech firm Convai, said. “We take ideas from different architectures and share ideas from different architectures. This is normal for us.”

Hartford said various AI models “may have the same architecture, but the data that they were trained with is completely different”.

Because of the investment and tooling around the Llama architecture, there is value in using the same names for the tensors, Hartford suggested in his earlier post on the Hugging Face platform.

There is a need “to better honour” the open source community’s conventions, according to a Hangzhou-based AI entrepreneur who requested anonymity owing to the sensitivity of the topic. That practice is akin to citing a research paper and giving the proper attribution, the entrepreneur said.

Backed by Alibaba Group Holding’s cloud unit and Sinovation Ventures, 01.AI is now one of the country’s largest AI unicorns, which include ex-Sogou founder Wang Xiaochuan’s Baichuan and state-backed ZhipuAI. Alibaba owns the Post.