Misplaced Pages

Qwen

Article snapshot taken from Wikipedia with creative commons attribution-sharealike license. Give it a read and then ask your questions in the chat. We can research this topic together.
Family of large language models by Alibaba

Qwen (also called Tongyi Qianwen) is a family of Large Language Models developed by Alibaba. In July of 2024 it ranked as the top Chinese language model in some benchmarks and third globally behind Anthropic and OpenAI.

Models

Alibaba first launched a beta of Qwen in April 2023 under the name Tongyi Qianwen. It was publicly released in September 2023 after receiving approval from the Chinese government. In December 2023 it released its 72B and 1.8B models as open source, while Qwen 7B was open sourced in August.

In June 2024 Alibaba launched Qwen 2 and in September it released some of its models as open source, while keeping its most advanced models proprietary.

In November 2024, QwQ-32B-Preview, a model focusing on reasoning similar to OpenAI's o1 was released as open source under the Apache 2.0 License, although only the weights were released, not the dataset or training method. QwQ has a 32,000 token context length and performs better than o1 on some benchmarks.

Alibaba has released several other model types such as Qwen-Audio, Qwen2-Math and Qwen-VL for vision. Alibaba also released Qwen-VL2 in 7 and 2 billion parameters. In total, it has released more than 100 models as open source, with its models having been downloaded more than 40 million times. Finetuned versions of Qwen have been developed by enthusiasts, such as “Liberated Qwen,” developed by San Francisco-based Abacus AI, which attempts to remove all safety guardrails from the model.

Architecture

Qwen 1 is based on llama developed by Meta AI with various modifications. Qwen 2 employs a Mixture of experts.

References

  1. Jiang, Ben (11 July 2024). "Alibaba's open-source AI model tops Chinese rivals, ranks 3rd globally". South China Morning Post.
  2. Chiang, Sheila (11 April 2023). "Alibaba to roll out its rival to ChatGPT across all its products". CNBC.
  3. Jiang, Ben (13 September 2023). "Alibaba opens Tongyi Qianwen model to public as new CEO embraces AI". South China Morning Post.
  4. Fan, Feifei (2023-12-01). "Alibaba unveils new Tongyi Qianwen AI language model". global.chinadaily.com.cn.
  5. Ye, Josh (August 3, 2023). "Alibaba rolls out open-sourced AI model to take on Meta's Llama 2". reuters.
  6. Jiang, Ben (7 June 2024). "Alibaba says new AI model Qwen2 bests Meta's Llama 3 in tasks like maths and coding". South China Morning Post.
  7. ^ Kharpal, Arjun (19 September 2024). "China's Alibaba launches over 100 new open-source AI models, releases text-to-video generation tool". CNBC.
  8. Dickson, Ben (29 November 2024). "Alibaba releases Qwen with Questions, an open reasoning model that beats o1-preview". VentureBeat.
  9. 故渊 (2024-11-28). "阿里通义千问 QwQ 登场：开源 AI 推理新王，MATH 测试超 OpenAI o1 模型 - IT之家". www.ithome.com.
  10. Wiggers, Kyle (27 November 2024). "Alibaba releases an 'open' challenger to OpenAI's o1 reasoning model". TechCrunch.
  11. Franzen, Carl (8 August 2024). "Alibaba claims no. 1 spot in AI math models with Qwen2-Math". VentureBeat.
  12. 沛霖 (2024-08-30). "阿里通义千问推出 Qwen2-VL：开源 2B / 7B 参数 AI 大模型，处理任意分辨率图像无需分割成块". ithome.com.
  13. Wang, Peng; Bai, Shuai; Tan, Sinan; Wang, Shijie; Fan, Zhihao; Bai, Jinze; Chen, Keqin; Liu, Xuejing; Wang, Jialin; Ge, Wenbin; Fan, Yang; Dang, Kai; Du, Mengfei; Ren, Xuancheng; Men, Rui; Liu, Dayiheng; Zhou, Chang; Zhou, Jingren; Lin, Junyang (September 18, 2024). "Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution". Cs.CV. arXiv:2409.12191.
  14. "Alibaba accelerates AI push by releasing new open-source models, text-to-video". Reuters. September 19, 2024.
  15. Mims, Christopher (April 19, 2024). "Here Come the Anti-Woke AIs". WSJ.
  16. Bai, Jinze; et al. (28 Sep 2023). "Qwen Technical Report". arXiv:2309.16609 .
  17. Yang, An; et al. (10 Sep 2024). "Qwen2 Technical Report". arXiv:2407.10671 .
Categories: