Yuan 1.0 by Inspur
美国
人工智能GPT-3替代大语模型(LLMS)

Yuan 1.0 by Inspur 翻译站点

GPT-3等于中文

标签:
爱站权重:PC 百度权重移动 百度移动权重

中国服务器构建器启发训练了怪物文本生成神经网络
Yuan 1.0说要通过图灵测试,并且比GPT-3所需的GPU少得多。

Insers已将其手转向了AI,并声称它已经产生了一种比OpenAI生产的GPT-3优于GPT-3的文本和编码的机器学习模型。并使用gpus少得多。

Insers的模型称为Yuan 1.0,并以中文生产文本。中国服务器制造商表示,该型号具有2457亿个参数(GPT-3具有1.75亿个参数),声称它可以通过图灵测试,并认为它可以在成语阅读理解任务中击败人类。

__他们做了什么:__当您训练这方面的模型时,很多硬东西都是水暖的。您需要弄清楚如何在数千个GPU上构建良好的管道来训练模型,这涉及Salami切片模型培训的不同阶段,以最大程度地提高数据效率。同样,您需要以正确的顺序为这些GPU提供这些GPU,从而进一步提高效率。本文包括一些很好的讨论,讨论了启发研究人员如何尝试这样做。

__ compute:__他们使用2128 GPU训练245b型号,上下文长度为2048代币。

__DATA通过AI帮助AI:__训练模型,他们构建了5TB的主要中文文本数据集。 (相比之下,华为的GPT3等效pangu受过1.1TB的培训,Ernie 3.0接受了4TB的数据培训)。他们训练BERT风格的模型,以帮助对数据进行自动过滤。他们的数据来自常见的爬网,Sogou新闻,Sogout,百科全书和书籍。

__ __ yuan 1.0在各种标准基准测试方面做得很好。最有趣的结果是其文本生成的质量 - 在这里,作者采用了与原始GPT3论文中相同的方法,在该论文中,它们生成了不同形式的文本,并了解了人类如何将生成的文本与“真实”文本区分开来。结果令人惊讶 - 人类准确49.57%(与GPT3相比为52%),这意味着Yuan 1.0的输出非常好,与人类写的文本没有区别。

资料来源:
-https://www.theregister.com/2021/10/28/yuan_1_natural_language_model/
-https://jack-clark.net/2021/10/10/18/import-ai-270-inspur-makes-a-gpt3-microsofts-half-trimnion-parameter-model-model-plus-plus-plus-plus-a-a-fair-a-fair-surveillance-surillance-dataset /

原文:

Chinese server builder Inspur trained a monster text-generating neural network
Yuan 1.0 said to pass Turing test, and require many fewer GPUs than GPT-3.

Inspur has turned its hand to AI, and claims it has produced a text-and-code-generating machine-learning model superior to GPT-3 produced by OpenAI. And did so using significantly fewer GPUs.

Inspur's model is called Yuan 1.0 and produces text in Chinese. The Chinese server maker says the model has 245.7 billion parameters (GPT-3 has 175 billion), claims it can pass a Turing test, and reckons it can beat humans at an idiom-reading comprehension task.

__What they did:__ When you’re training models of this side, a lot of the hard stuff is plumbing – literally. You need to figure out how to build well-optimized pipelines for training your model on thousands of GPUs, which involves salami slicing different stages of model training to maximize data efficiency. Similarly, you need to feed these GPUs with data in the right order, further increasing efficiency. The paper includes some nice discussion of how the Inspur researchers tried to do this.

__Compute:__ They used 2128 GPUs to train the 245B model, with a context length of 2048 tokens.

__Data, via AI helping AI:__ To train the model, they build a dataset of 5TB of predominantly Chinese text. (By comparison, Huawei’s GPT3 equivalent PanGu was trained on 1.1TB of text, and ERNIE 3.0 was trained on 4TB of data). They train a BERT-style model to help do automatic filtering of the data. Their data comes from Common Crawl, Sogou News, SogouT, Encyclopedia, and Books.

__How good is it?__ Yuan 1.0 does well on a variety of standard benchmarks. The most interesting result is on the quality of its text generation – here, the authors adopt the same approach as in the original GPT3 paper, where they generate text of different forms and see how well humans can distinguish generated text from ‘real’ text. The results are striking – humans are 49.57% accurate (compared to 52% for GPT3), meaning the Yuan 1.0 outputs are so good they’re indistinguishable from human-written text.

Sources:
- https://www.theregister.com/2021/10/28/yuan_1_natural_language_model/
- https://jack-clark.net/2021/10/18/import-ai-270-inspur-makes-a-gpt3-microsofts-half-trillion-parameter-model-plus-a-fair-surveillance-dataset/

数据统计

数据评估

Yuan 1.0 by Inspur浏览人数已经达到239,如你需要查询该站的相关权重信息,可以点击"爱站数据""Chinaz数据"进入;以目前的网站数据参考,建议大家请以爱站数据为准,更多网站价值评估因素如:Yuan 1.0 by Inspur的访问速度、搜索引擎收录以及索引量、用户体验等;当然要评估一个站的价值,最主要还是需要根据您自身的需求以及需要,一些确切的数据则需要找Yuan 1.0 by Inspur的站长进行洽谈提供。如该站的IP、PV、跳出率等!

关于Yuan 1.0 by Inspur特别声明

本站GPT 案例导航提供的Yuan 1.0 by Inspur都来源于网络,不保证外部链接的准确性和完整性,同时,对于该外部链接的指向,不由GPT 案例导航实际控制,在2023年3月9日 下午10:31收录时,该网页上的内容,都属于合规合法,后期网页的内容如出现违规,可以直接联系网站管理员进行删除,GPT 案例导航不承担任何责任。

相关导航