Inicio  /  Applied Sciences  /  Vol: 13 Par: 23 (2023)  /  Artículo
ARTÍCULO
TITULO

Contemporary Approaches in Evolving Language Models

Dina Oralbekova    
Orken Mamyrbayev    
Mohamed Othman    
Dinara Kassymova and Kuralai Mukhsina    

Resumen

This article provides a comprehensive survey of contemporary language modeling approaches within the realm of natural language processing (NLP) tasks. This paper conducts an analytical exploration of diverse methodologies employed in the creation of language models. This exploration encompasses the architecture, training processes, and optimization strategies inherent in these models. The detailed discussion covers various models ranging from traditional n-gram and hidden Markov models to state-of-the-art neural network approaches such as BERT, GPT, LLAMA, and Bard. This article delves into different modifications and enhancements applied to both standard and neural network architectures for constructing language models. Special attention is given to addressing challenges specific to agglutinative languages within the context of developing language models for various NLP tasks, particularly for Arabic and Turkish. The research highlights that contemporary transformer-based methods demonstrate results comparable to those achieved by traditional methods employing Hidden Markov Models. These transformer-based approaches boast simpler configurations and exhibit faster performance during both training and analysis. An integral component of the article is the examination of popular and actively evolving libraries and tools essential for constructing language models. Notable tools such as NLTK, TensorFlow, PyTorch, and Gensim are reviewed, with a comparative analysis considering their simplicity and accessibility for implementing diverse language models. The aim is to provide readers with insights into the landscape of contemporary language modeling methodologies and the tools available for their implementation.

Palabras claves

 Artículos similares

       
 
Dracos Vassalos and M. P. Mujeeb-Ahmed    
The paper provides a full description and explanation of the probabilistic method for ship damage stability assessment from its conception to date with focus on the probability of survival (s-factor), explaining pertinent assumptions and limitations and ... ver más

 
Jialu Zhang and Xiaotong Zhang    
Magnetic resonance imaging (MRI) integrates a static magnetic field, a time-varying gradient magnetic field at kHz and a radio-frequency (RF) magnetic field for non-invasive and real-time imaging; meanwhile, diffusion MRI (dMRI) pushes a further and clos... ver más
Revista: Applied Sciences

 
J.J. Sylvia IV    
This paper explores how the concepts of information and technics have been leveraged differently by a variety of philosophical and epistemological frameworks over time. Using the Foucauldian methodology of genealogical historiography, it analyzes how the... ver más
Revista: Information

 
Emanuel Marques Queiroga, João Ladislau Lopes, Kristofer Kappel, Marilton Aguiar, Ricardo Matsumura Araújo, Roberto Munoz, Rodolfo Villarroel and Cristian Cechinel    
Contemporary education is a vast field that is concerned with the performance of education systems. In a formal e-learning context, student dropout is considered one of the main problems and has received much attention from the learning analytics researc... ver más
Revista: Applied Sciences

 
Tim McCarthy, Lars Pforte and Rebekah Burke    
Urban airspace environments present exciting new opportunities for delivering drone services to an increasingly large global market, including: information gathering; package delivery; air-taxi services. A key challenge is how to model airspace environme... ver más
Revista: Aerospace