Technology Innovation Institute Announces Launch of NOOR, the World’s Largest Arabic NLP Model
Technology Innovation Institute (TII), a global research center and applied research pillar of Abu Dhabi’s Advanced Technology Research Council, today announced the launch of NOOR, the world’s largest Arabic natural language processing (NLP) model to date.
This press release features multimedia. View the full release here: https://www.businesswire.com/news/home/20220411005085/en/
TII’s team of advanced researchers and Artificial Intelligence (AI) specialists, has joined forces with LightOn, a technology company that unlocks extreme-scale machine intelligence for businesses, to transform the Arabic NLP model. The NOOR model has the capability to carry out tasks beyond the domain of language - offering end-to-end pipeline high quality data, including crawling, filtering, and curation at scale. The model facilitates extreme-scale distributed training and serving – to deliver applications with efficient inference and model specialization.
Dr. Ray O. Johnson, CEO, TII and ASPIRE, said: “With this development, we are well on track to enhance our research capabilities and credentials as well as elevate the status of Abu Dhabi and the UAE as a serious research ecosystem. Our expert teams have demonstrated yet again that this region can achieve breakthrough R&D outcomes to impact the world.”
Dr. Ebtesam Almazrouei, Director, AI Cross-Center Unit, TII, said: “Large language models have taken the world of natural language processing by storm, and we are proud to introduce this cutting-edge model with 10 billion parameters - the world’s largest Arabic NLP model. The uniquely large Arabic dataset collected to train the model is the result of months of work that included curating, scrapping, and filtering of varied sources. A special thank you to the entire team that worked on this project to make NOOR the go-to exploration model in Arabic for academicians and businesses everywhere.”
Speaking on the launch, Prof. Mérouane Debbah, Chief Researcher, Digital Science Research Center and AI Cross-Center Unit, TII, said: “With NOOR, TII has expanded the scope of the modern standard Arabic model by leveraging know-how in large language models to build cross-disciplinary, cutting-edge expertise in this new generation of AI research.”
To curate the world’s largest high-quality cross-domain Arabic datasets, NOOR’s unique dataset of more than 30 billion words combines web data with books, poetry, news articles, and technical information to significantly widen the applicability of the model.
Dr. Ebtesam Almazrouei said the NOOR model is based on the popular Transformer architecture. As a decoder-only model, similar in structure to GPT-3, it is programmed to tackle generative tasks with architecture upgraded to reflect the latest developments in the world of machine learning, including improvements such as better positional embeddings. To help ensure quality at scale in the NOOR dataset, the TII team designed an automated filtering pipeline based on machine learning techniques. These tools identify text like quality references and safeguard the model from exposure to spam content.
Leveraging state-of-the-art 3D parallelism, NOOR was trained on a High-Performance Computing resource with 128 A100 GPUs, allowing for the distribution of computations and ensuring efficient use of the available hardware resources.
The Director of the AI Cross-Center Unit noted that this was only the first step in the Unit’s efforts to contribute to the wider UAE Strategy for Artificial Intelligence.
Named for the Arabic word "light", the model has been so called to establish the correlation of the Arabic language model to enlightening the mind.
About Technology Innovation Institute (TII)
For more information, visit www.tii.ae
About Business Wire
101 California Street, 20th Floor
CA 94111 San Francisco
Subscribe to releases from Business Wire
Subscribe to all the latest releases from Business Wire by registering your e-mail address below. You can unsubscribe at any time.
Latest releases from Business Wire
NY-STARR/FPG17.5.2022 22:35:13 CEST | Press release
Starr Insurance Companies Expands in Thailand
NY-EMENDO-BIO17.5.2022 22:17:09 CEST | Press release
Emendo Biotherapeutics’ next generation CRISPR gene editing technologies achieve breakthrough results with allele-specific approach for ELANE-related Severe Congenital Neutropenia as presented at ASGCT Annual Meeting
MN-PIPER-SANDLER17.5.2022 20:01:06 CEST | Press release
Piper Sandler Advises PETRONAS Chemicals Group Berhad on Acquisition of Perstorp
MORINAGA-MILK-INDUSTRY17.5.2022 19:55:04 CEST | Press release
Morinaga Milk Obtains the Registration of "New Food Ingredient" in China for Use of Its Probiotic Bifidobacterium longum BB536 in Infant and Toddler Foods
HIRO-CAPITAL17.5.2022 19:32:07 CEST | Press release
Hiro Capital Joins Knollwood and Boost VC With Major Investment in Skybound Entertainment
In our pressroom you can read all our latest releases, find our press contacts, images, documents and other relevant information about us.Visit our pressroom