Machine Learning and Lexical Rule-Based Cost-Efficient Emotion Annotation of Hinglish Utterances

Summary

This study combines the rule-based strategy with machine learning and expert assistance to annotate the Hinglish and English text. The data is tested on 19,000 tweets with 81% accuracy, and it is much cheaper than doing it manually. It could be useful for tracking emotions during a crisis.

Abstract

Emotion annotation in code-mixed languages like Hinglish (Hindi-English) presents unique challenges due to linguistic complexity and resource constraints. This study introduces a hybrid active learning framework that combines lexical rules, machine learning, and iterative expert feedback to achieve cost-efficient, high-accuracy emotion annotation. Grounded in psychological theories of emotion, including Discrete Emotions Theory and Cognitive Appraisal Theory, the framework employs bilingual emotion dictionaries (e.g., mapping gussa and rage to anger), subword tokenization for compound terms (e.g., splitting into ), and active learning to prioritize ambiguous samples. Evaluated on a 19,000 war and conflict-related Hinglish tweets dataset, the framework achieved 81% accuracy (F-score: 0.76) while reducing operational costs by 40% compared to manual annotation. Lexical rules resolved 89% of code-switching ambiguities, and iterative refinements enabled incremental accuracy gains from 72% to 81%. The system's efficiency stems from limiting human effort to 73% of the dataset, with automated preprocessing of emojis, hashtags, and slang. This study is based on the hypothesis that integrating lexical rule-based methods with active learning and machine learning can enhance the accuracy of emotion annotation in Hinglish text, while simultaneously reducing the manual labeling and overall annotation effort.

Introduction

When two or more languages are mixed together in a single line or speech, this is called a code-mixed language. It is common in casual dialog like Hinglish. There are multiple ways human emotions can be understood, and to computationally model a series of emotional statements is to annotate them by the people who uttered those sentences. It can be understood in terms of biological, physiological, psychological levels, and so on. According to scientists such as Roger Penrose, many phenomena in our world are non-computational, and scientists such as Wolfram consider that everything (every phenomenon) can be modeled computationally¹. Penrose believes that consciousness involves processes (perhaps related to quantum mechanics within the brain) that go beyond what any step-by-step algorithmic procedure can achieve. He often cites Gödel's incompleteness theorems to support the idea that human mathematical insight, for instance, transcends formal systems². If consciousness is non-computational, then emotions, as a key aspect of conscious experience, might also have non-computational elements. Stephen Wolfram, known for Mathematica and his work on cellular automata, proposes the "Principle of Computational Equivalence." This suggests that even very complex systems, including potentially the universe itself and phenomena within it (like emotions), can ultimately be described and modeled by computational rules, even if those rules are very simple, generating complex behavior. But practically, this is not possible, and we need someone who is referred to as either an expert or simply an annotator who can do emotion analysis³.

In this research, we propagate the idea of building computational models. But that model will be quasi-computational. Our research in this context aims to be computational in form but might not capture all aspects perfectly, perhaps leaving room for complexities that are difficult or impossible to compute fully. Emotions are difficult to model computationally because they depend on subjective experiences, cultural context, and nuanced expressions that cannot be fully captured through fixed algorithms.

Therefore, for modeling human emotions using variable-based computational approaches, it is necessary to annotate human emotional utterances. This annotation should be performed by an expert or an annotator skilled in emotion analysis¹. Understanding the complexities of human emotions is not an easy task, particularly when dealing with mixed languages. Furthermore, problems related to scale mean that relying solely on manual annotation by humans is not a viable option. Recent research indicates a consistent need for a human-in-the-loop approach when building systems for such complex tasks. Consequently, a semi-automatic approach, which involves automating the more straightforward parts while reserving tasks requiring human nuance for annotators, appears most appropriate for developing natural language systems in this domain.

A human annotator will, of course, be doing work manually, and in the age of computation, this is not what is expected from contemporary scientists. If the annotator (manual, semi-automatic, or fully automatic) is able to intelligently guess the type of emotion embodied in the utterances, utterances that consist of multiple types of emotions expressed as symbols, with colloquialism or code-mixed and using multiple modalities, then the task is hard and easy at the same time. The complexity of emotion annotation in Hinglish utterances depends on the nature of the expression. When emotions are conveyed clearly using familiar words or emojis, annotation is relatively straightforward. However, the task becomes challenging when utterances involve multiple emotions, code-mixing, or ambiguous symbolic expressions. Therefore, annotation can be both easy and difficult, depending on how directly the emotion is expressed.

Contemporary approaches in the identification of emotions and sentiments deal with these challenges, including the subjective nature of emotions, the ambiguity in human expressions, the complexity of code-mixed languages like Hinglish, and the time-consuming and inconsistent nature of manual annotation. associated with constructing computational models and managing tedious annotation tasks. Recent research indicates that researchers are employing a diverse range of methods toward this goal, including machine learning, deep learning, and various hybrid approaches. Recent research shows that to overcome these issues, researchers are employing a variety of techniques, such as machine learning, deep learning, and hybrid models.

Recent research shows that researchers are employing all kinds of approaches, including machine learning, deep learning³, and hybrid approaches. The term sentiment analysis refers to a procedure used when the polarity of the emotions is believed to be a marker to understand the raw emotion of humans^3,4. The development of such technology has helped to recognize mood, sentiments, speech, facial emotions, and nonverbal cues, and has already made inroads into applications that allow real-time translation². A multimodal approach could be used to translate Hinglish into English and may be helpful in the future to make Indian cinema more accessible to remote societies^5,6. For example, in India, English is often the second language. Research in this context shows that this has improved the quality of English teaching by analyzing Indian speech (mix-code language) for the expressiveness, or degree of feeling and emotion, of each word.

Within this research context, the use of mixed-code language in conjunction with translation has been shown to enhance the quality of English teaching. This is accomplished through the analysis of Indian speech (mixed-code language) to determine the expressiveness, or emotional valence, of each word. Through the application of deep learning to train computers in speech interpretation, this research has already improved the accuracy of computerized speech analysis and facilitated a greater understanding of communication^4,5. According to the census results from 2001, Hinglish, a language that is a blend of Hindi and English, is currently used by an estimated 120 million people in India⁶.

From the contemporary landscape of learning algorithms, it is clear that active learning has emerged as a powerful tool for significantly reducing human effort in annotating large datasets, particularly in the domain of emotion identification and recognition. This iterative approach, which selectively annotates impactful annotations (with appropriate metrics), not only enhances annotation accuracy but also improves efficiency⁵. Previous studies have demonstrated its effectiveness in achieving substantial reductions in manual annotation workload while maintaining or even improving performance with smaller training datasets and proposing a cluster analysis-based method for informative instance selection^7,8. In the specific context of Hinglish emotion recognition, researchers have made valuable contributions through deep learning models and a multi-label annotated dataset^9,10,11. Previous studies^12,13 have introduced active learning and semi-supervised methods to minimize the dependence on human-labeled data, further enhancing efficiency and reducing annotation costs. Furthermore, active learning has been demonstrated in many projects to boost classification performance, particularly in multi-label emotion classification¹⁴.

The efficacy of active learning in improving classifier performance has been recognized across various machine-learning applications. Studies^15,16highlighted its crucial role in enhancing performance by focusing on educational applications. Similarly, an early study introduced a novel algorithm for active learning with support vector machines, significantly reducing the need for labeled instances¹⁷. Another work also explored its application in tasks involving structured instances, such as text classification¹⁸. Active learning's impact on emotion recognition tasks extends beyond efficiency gains, particularly in minimizing reliance on human-labeled data. One study introduced a multi-task framework for emotion classification and regression, surpassing the performance of single-task methods¹⁰.

Furthermore, researchers¹⁹made significant strides in speech and text emotion recognition using active learning, while demonstrating²⁰ its effectiveness in personalized music emotion classification. However, the process of categorizing and labelling emotions presents a significant challenge, as highlighted^21,22, particularly in sentiment analysis contexts. Notes that label usage can significantly influence emotion categorization, particularly for later-learned categories²³. To address these challenges, various algorithms, including keyword-based and learning-based methods, have been developed, achieving notable accuracy rates²⁴. Research on emotions based on written utterances and texts has been explored in numerous models, and approaches have implemented a dimensional model using normative databases for effective emotion detection²⁵. In another study²⁶, a cognitive emotion model enhanced a sequential method used for social emotion cause identification. The author provided a computational linguistic interpretation of the OCC emotion model, while a similar study²⁷proposed a system utilizing ontologies for representing word dependency relations and emotions. The authors of one study²⁸discussed the signals that correlate with emotional word processing, highlighting the brain's adaptation in expressing emotions in written language. Annotation of multiple arrays of raw emotions, including that of the multi-model data, is challenging. Nevertheless, investigating emotions related to war and conflict provides a scientific and systematic window into the human psyche under extreme circumstances, allowing us to better understand how individuals and communities cope with trauma, loss, and uncertainty⁵. Another study found that the annotation technique effectively enhanced genre classification, with the title feature playing a crucial role in the process²⁹. One study created a 44K vision-touch dataset with expert and GPT-4V to train a tactile encoder and a TVL model for text generation³⁰. Another study explored opinion and trend mining on political tweets, focusing on the active learning process to automatically annotate French-language tweets about politicians⁴¹. Another study introduced CloudFlows, a cloud-based scientific workflow platform designed for dynamic adaptive central analysis in data streams. It enables active learning to improve sentiment classification, allowing the algorithm to adapt to changes in real-time data⁴².

There is clear-cut tension between the complexity of human emotion and the desire for automated emotion analysis. An inherent tension exists between the complexity of human emotion and the objective of automated emotion analysis. Most of the contemporary work acknowledges the limitations of manual annotation and emphasizes the need for sophisticated computational methods to tackle the challenges of understanding emotions in diverse forms of communication. This ideal scenario is largely impractical, i.e., getting annotations from the people who wrote or spoke the sentences⁴³. The ideal scenario for obtaining data, specifically getting annotations directly from the individuals who wrote or spoke the sentences, is largely impractical. This impracticality stems from the impossibility of gathering and processing such personalized annotations on a large scale. Therefore, current efforts must rely on expert annotators or automated emotion detection algorithms to analyze and label emotions expressed in text. In this research work, we have attempted to overcome some aspects of these domain challenges. The key contributions in this problem area are presented hereafter⁴⁴.

Therefore, we need to rely on experts or annotators and emotion detection algorithms to analyze and label the emotions expressed in text. It's impossible to gather and process such personalized annotations on a large scale. Hence, in this research work, we have attempted to overcome some aspects of this domain knowledge. The following are the key contributions in this problem area.

The framework works together with rule-based methods like emotion tagging, code-mix detection, and emoji interpretation with machine learning techniques such as Random Forest and word embeddings, improving annotation accuracy while reducing manual effort. The Iterative Learning of the Classifier employs active learning as well as transfer learning to prioritize ambiguous feature samples, reducing the need for hard labor. This approach reduced operational costs by 40% compared to hard manual labeling.

To handle the nuances of Hinglish at a granular level, a custom context-sensitive tokenization method was developed. This approach processes code-mixed text by accounting for language switching, punctuation, emojis, and subword segmentation, enabling more accurate emotion annotation in mixed Hindi-English text. At a granular level, we developed custom context-sensitive tokenization for Hinglish text. The framework addresses the complexities of code-mixed text by incorporating bilingual emotion dictionaries, subword tokenization, and custom context-sensitive tokenization. Lexical rules resolved 89% of code-switching ambiguities.

Our work is grounded in established psychological theories of emotion, such as Discrete Emotions Theory and Cognitive Appraisal Theory. The research demonstrates the scalability of the approach for crisis response and social media monitoring, providing a blueprint for low-resource, multilingual NLP applications.

Table 1 explains the available studies for the same problem domain. From the literature survey and the tabulated summary, it can be inferred that most of the studies cannot escape doing some initial work on annotation using manual methods. Few researchers are following semi-automatic approaches⁴¹. However, the real difference in performance comes from the use of an effective learning model that can automate the process of annotation. The emotional content of the tweets must match theories that explain the pathways of humans' emotions and the organization of sentiments. The next section defines the problem based on limitations of existing approaches and empirical results of the papers.

Study	Dataset	Emotion	Methods	Domain	Labeling Process	Gaps			Future Scope
[31]	9,000,000 Tweets	tension, depression, anger, vigour, fatigue,	Confusion Profile of Mood States	English	No labeling	The study overlooks subtle emotional differences like surprise, joy, or fear, suggesting that emotion labelling can enhance the interpretability and granularity of sentiment trends, particularly in relation to socioeconomic events.			It could investigate how to better capture and examine a range of emotional expressions in social media data by utilising automated categorisation methods and well-established emotion taxonomies.
[32]	7000 Tweets	anger, disgust, fear, joy, love, sadness,	Support Vector Machine	English	Manual	The generalisability of the dataset is limited due to its topic-specificity and lack of representativeness of overall Twitter usage.  Due to subjective interpretation and minimal context, which is shown in modest inter-annotator agreement, it is challenging to annotate emotions in brief, casual tweets.			Future work will focus on developing improved emotion detection models by incorporating distinctions between topic-specific and emotion-specific linguistic styles, enabling more accurate classification in diverse tweet contexts.
[33]	21,000 Tweet	anger, disgust, fear, joy, sadness, surprise	Support Vector Machine	------	Using Hashtag	Existing emotion-labeled corpora are limited in size and domain, lacking large, diverse datasets for microblogs. Tweets are short, noisy, and context-limited, making accurate emotion detection and annotation difficult.			In future work study may includes expanding the emotion lexicon with synonyms and additional hashtags to improve coverage and detection accuracy.
[34]	16485 Tweets	anger, disgust, fear, joy, sadness, surprise	Support Vector Regression	Chinese	Manual	Traditional emotion classification methods often overlook the underlying cause of emotions, limiting feature quality. Accurately extracting emotion causes from short, informal microblog posts requires robust rule-based systems and domain knowledge.			Further exploration of emotion cause analysis can enhance emotion detection models and open new directions in textual emotion understanding.
[35]	10,040 Tweet	Fear, hope, joy anger, surprise,  sad, disgust	LDA, inter-rater agreement	Hinglish	Manual	There is a lack of publicly available, structured datasets for Hinglish, especially those capturing pragmatic and emotional nuances in crisis related content. Hinglish is a non-standard, code-mixed language, and regional variations complicate accurate sentiment analysis and annotation.			To expand multimodal datasets, integrate deep pragmatic analysis with machine learning models, and address scalability for real-time emotion tracking in conflict discourse.
[36]	134,000 tweets	active, inactive  happy, unhappy	Support Vector Machine and K-Nearest Neighbors	Hinglish	Using Hashtags	Manual emotion labeling of tweets is labor-intensive and inconsistent, limiting large-scale emotion classification efforts Crowdsourced annotations lack reliability, especially in identifying emotion arousal levels, highlighting subjectivity in emotion interpretation.			Focus on refining hashtag-based labeling and expanding emotion detection models for improved accuracy and generalizability across diverse emotional contexts.
[37]	3,000 students, psychologists and non-psychologistsfrom 37 countries	joy,fear, anger, sadness, disgust, shame, and guilt.	--	-----	Manual	Limited exploration of how cultural factors influence the regulation and expression of specific emotions across diverse societies. Balancing evidence for universal emotional patterns with culturally specific variations in emotion elicitation and interpretation remains complex.			Further studies should investigate the interaction between biological universality and cultural context in shaping emotional experience and communication
[38]	12000	Happiness, Sadness and Anger	inter-rater agreement	Hindi+English	Manual	Current research lacks a comprehensive, annotated dataset and standardized models for Hinglish emotion detection. The irregular grammar and code-mixed nature of social media texts make accurate emotion classification difficult.			Future work will  focus on expanding emotion categories and developing larger, multilingual code-mixed datasets.
[39]	2866	happiness, sadness, anger, surprise and sadness	Support Vector Machine	Hinglish (Hindi+English)	Manual	Lack of emotion-annotated code-mixed datasets.  Emotion expression in code-mixed text varies across languages and scripts, making annotation and classification complex.			Future work could expand the corpus to include more emotional diversity, integrate part-of-speech tagging, and explore multi-language code-mixed content.
[40]	13738	---	Machine Translation Google Translator	Hinglish	Manual	Existing machine translation systems lack accuracy on code-mixed social media data due to the absence of large, domain-specific parallel corpora. High spelling variation, informal structure, and ambiguity in language identification complicate translation of Romanized Hindi-English text.			The corpus can support development of code-mixed translation systems and be extended to other low-resource languages and NLP tasks like named entity recognition
[41]	11527	positive,very positive and negative,very negative	kNN-based classification,BOW representation	French politicians	Manual	Limited availability of high-quality annotated datasets for political opinion mining in non-English languages. Balancing annotation noise reduction with information retention and handling uneven label distribution in large-scale tweet datasets are key difficulties.			Future work may refine active learning methods to better preserve critical content while minimizing annotation noise in multilingual political discourse.
[42]	764,416	---	Kmeans Clustering, SVM	English	Semi supervised	Real-time labeling and model updating in sentiment analysis is constrained by data stream variability, labeling cost, and system scalability.			Future work will explore multi-class sentiment classification, integrate additional labeling strategies, and expand control over initial model generation

Table 1: Available studies with corresponding labeling methods. The table provides a complete comparative overview of the existing studies, addressing the emotion annotation and establishing the methodological landscape and conceptualizing the contribution of the present work within existing literature.

Problem statement
The most frequently studied emotions in annotation are heavily influenced by foundational psychological models like Ekman's and Plutchik's, primarily focusing on core categories like anger, fear, happiness, sadness, surprise, and so on⁴⁴. Hence, in this research work, we intend to work on well-established connotations of emotions. The challenge is to develop a dynamic computational framework, F, capable of accurately annotating Hinglish text instances (tᵢ ) from a corpus T focused on wars and conflicts with emotion labels (eᵢ) from a predefined set E = {e₁, e₂, ..., e₈}. This framework must synthesize principles from the Constructionist Theory of Emotion, Affective Events Theory (AET), Discrete Emotions Theory, and Cognitive Appraisal Theory to model the multifaceted emotional landscape of conflict-related discourse. Each text instance tᵢ in T is linguistically complex, blending Hindi (in Roman script), English, emojis, and symbols, necessitating a multi-layered approach to capture nuanced emotional expressions.

The computational model of emotions related to war (as a case study) may involve a multi-faceted approach, beginning with lexical rules that address Hinglish-based nuances. Tokenization, denoted as T, encompasses Roman scripts (Hindi written in Roman script), along with emojis and punctuation, forming the basis of language processing. emotion dictionaries, represented as D, map words across languages to specific emotions, such as anger, joy, and others, where each emotion_i has associated words_j in language_k. Subword decomposition, S, breaks down compound terms into their constituent subwords, enabling a deeper understanding of complex expressions. Subsequently, machine learning techniques, M, utilize embeddings, E, such as Word2Vec/fastText, to transform tokens into vector representations, vector_v, facilitating numerical analysis. Ensemble classifiers, C, like Random Forest, then predict emotion labels, emotion_label_p, from these vector sets. To iteratively improve the annotation learning model, an active learning mechanism, AL, is employed. Expert feedback, F, refines ambiguous cases, ambiguous_sample_q, by assigning refined_label_r, providing crucial corrections. Sample prioritization, P, focuses on low-confidence samples, low_confidence_sample_s, assigning them annotation_priority_t, thus optimizing the annotation process.

By integrating these components and theories, this framework aims to dynamically process Hinglish text, bridge linguistic and cultural nuances, and adaptively refine emotion annotations, offering a scalable solution for analyzing affective dimensions in conflict discourse.

Protocol

This section also explains how the multimodal framework for 8 emotion annotation has been constructed. The section begins with a discussion of the properties of the dataset, followed by the subsequent procedures. For a better understanding of the research procedure, refer to Figure 1.

Figure 1: Systematic framework of emotion annotation. The figure explains the effective emotion, annotation of hinglish text that combines machine learning, active learning, and dynamic lexicon rules through expert input, misclassified examples are progressively improved to increase the accuracy and lower the cost of annotation. Please click here to view a larger version of this figure.

Dataset preparation
Data collection begins by identifying a comprehensive list of keywords and hashtags related to war conflict and associated emotions. Resources such as academic literature, news articles, and social media trends were used to compile relevant and existing lists.

As per Figure 1, after the collection of tweets and pre-processing of the data is done, human experts are involved in manual labelling and making lexical rules that need to be incorporated to improve the annotation process. Using these keywords (conflict, war, crisis, gussa, etc.) an initial data set of 10,040 tweets was collected and served as the foundation for manual labelling where each tweet was annotated with eight predefined emotions (anger, fear, happiness, sadness, frustration, compassion, mixed, others relevant to the war and conflict domain. The manual labelling process was conducted by a team of experts proficient in both Hindi and English, ensuring that the nuances of English are accurately captured.

An example processing is described below.

Tokenization and Preprocessing:
Input Tweet: "Mujhe Bhayanak lag raha hai "
Tokenized Output: ["Mujhe", "Bhayanak", "lag", "raha", "hai", ""]
Roman Script Handling: Hindi words ("Mujhe", "Bhayanak") are retained in Roman script.
Emoji/Symbol Detection: "" is isolated as a symbolic token.

Emotion Dictionary Mapping (D):
Map tokens to emotions in E using bilingual (Hindi/English) emotion lexicons: "Bhayanak" (Hindi for "horrible") → Fear; "lag raha hai" (contextual phrase implying ongoing emotion) → Fear; "" → Anger

Subword Decomposition (S):
Decompose complex terms for deeper analysis: "Bhayanak" → ["Bhay" (fear), "anak" (suffix)] to clarify its semantic root in fear.

Embedding Generation (E):
Generate token embeddings using Word2Vec/fastText: Embeddings for ["Mujhe", "Bhayanak", "lag", "raha", "hai", ""] → vectors v₁, v₂, v₃, v₄, v₅, v₆.
Aggregation Rule: Average token embeddings to create a global semantic vector:
V_avg = (v₁ + v₂ + v₃ + v₄ + v₅ + v₆) / 6

Rule-Based Feature Extraction:
Extract auxiliary features for concatenation. Emotion Tag Counts: Fear: 2 instances ("Bhayanak", "lag raha hai"); Anger: 1 instance ("").
Code-Switch Flag: Binary flag = 1 (mixed Hindi and English tokens: "Mujhe" [Hindi], "lag", "raha", "hai" [Hindi-derived]).

Feature Fusion:
Combine aggregated embeddings and rule-based features into a unified input vector: Global Semantic Vector=V_avg(averaged embeddings),Emotion Counts=[Fear: 2, Anger: 1, Others: 0], Code-Switch Flag=1
Concatenation Rule Final Input Vector = V_avg [Fear: 2, Anger: 1, Others: 0] [1]
The machine learning algorithm processes this final vector, and the iterative process of annotation improvement begins. In the next section, we discuss the performance of the Active learning method adopted for this purpose.

After this, the dataset was then expanded to 19,000 tweets. This data set was curated using a combination of automated and semi-automated techniques, leveraging the insight gained from the initial manual annotation. The expanded data set was further refined from an iterative learning process, it involved selectively identifying and prioritizing ambiguous data/tweets for expert annotation and feedback from domain experts to improve accuracy, consistency, and efficiency of annotation. Throughout the data collection process, special attention was given to maintaining the balance between the different emotions, ensuring that the data set was representative of the diverse sentiments expressed about war and conflict. The resulting dataset is a valuable resource for analyzing Hinglish text. For better understanding, Figure 2 can be referred to.

Figure 2: Procedure of dataset collection. The figure depicts the dataset development from seed word identification to manual labelling, followed by active learning, to the final annotated dataset. Please click here to view a larger version of this figure.

The final preparation of the dataset is done after completing the iterative refinement process with the help of active learning. Active learning was used in a hybrid framework that included lexical rules, machine learning, and iterative expert input to annotate the emotions of Hinglish utterances. The steps were as follows:

The process starts with a manually labeled dataset. Using a Random Forest classifier, which was employed to identify ambiguous tweets that the machine learning model was uncertain about. Send these ambiguous samples for categorization to human experts. The model was repeatedly updated using the recently annotated data, which gradually improved accuracy and reduced misclassifications. Finalize the dataset and review the annotations to ensure accuracy. Prepare the dataset for analysis, ensuring that it is properly documented and formatted for future use to implement in downstream cases. However, it is important to investigate the patterns of emotions embedded in the utterances so that future steps become clearer in terms of implementation. Hence, in the next step, cluster analysis will be performed to find dominant emotions embedded in the dataset. This also helps in identifying the emotions that we are researching. 

Selection of emotion clusters
Table 2 shows the emotion groups and their Hinglish equivalents, along with the reason for selecting the respective emotions. From each group of emotions, a dominant emotion has been selected for further processing. These dominant emotions are selected from cluster analysis.

Emotion Group	Hinglish Equivalent	Justification of Selection
Fear (includes anxiety and panic)	Dar, khauf, Asahaj, Bekabu, Angadai, Chinta, tension, Fikr, Ashanka, Udaasi, Bechaini, Ghabrahat	Fear is a common emotion in war and conflict, as individuals face threats to their safety and well-being. This fear can manifest in various ways, such as anxiety, panic attacks, and hypervigilance.
Anger (includes irritation, hostility, frustration, and hurt)	Gussa, raag, Prakop, Raudra, Chidhaan, Shatruta, Krodh, Gussa dilana, apata, Atyachar, Khushfehmi, hairani, Bhayanak, Chakker Kathinaai	Anger is another prevalent emotion in war and conflict, often arising from feelings of injustice, betrayal, or loss. This anger can fuel aggression and violence, contributing to the destructive nature of these conflicts.
Sadness (includes grief, despair, and hopelessness)	Udaasi, gham, Shok, Bhavuk, Dukhi, Udas, Vismay, Nirasha, Shok, Dukhi, Vairagya, Aashank, Vishada, Bhavuk, Dukhi, Udas, Vinamrata, Bhavuk, Hridaytoda, Beumaar, nirasha, Vinaash, Bair, Nirasha, Asambhav, Haar	Sadness is a natural response to loss and grief, which are unfortunately common experiences in war and conflict. Soldiers may experience sadness over the loss of comrades, civilians may grieve for loved ones killed or displaced, and entire communities may mourn the loss of their homes and way of life. Feelings of despair and hopelessness can also arise due to the prolonged nature of the conflict and the seemingly endless cycle of violence.
Shame and Guilt	Sharm, lajjabari, Sharm, Laaj, Zillat, Afsos, Gunah, Afsos, Pashchatap, Laaj, Bechaini, Aatmasamarpan, Sharmindagi, Ashru, Pashchatap, Antaratma, Kasoor, gunaah, Khud ko doshi maana, Ninda, Dosha	Shame and guilt are complex emotions that can arise from feelings of personal or collective wrongdoing, inadequacy, or humiliation. In war and conflict, individuals may experience shame or guilt over their actions, their inability to prevent harm to others, or their survival while others have perished.
Disgust	Ghin, nafrat, Asahayak, Pratikool, Ghrina, Vairagya, Dvesha, Nakaratmak, Vibhavsu, Vairagya, Vairagya, Nirasha	Disgust is a feeling of revulsion or aversion towards something perceived as distasteful or offensive. In war and conflict, individuals may experience disgust in response to acts of cruelty, violence, and barbarity.
Empathy and Compassion	Sahaaanubhuti, hamderdari, Samajhdari, Daan Sahabhooti, Sensitivity, Supportiveness, Consideration, Kindness, Caring, Warmth, Tenderness, Daya, raham, Sahaaanubhuti, Sahyog, Dayalu, Samajh, Pyar, Daya, Narami, Paropkari	Empathy and compassion are vital for understanding and sharing the feelings of others. In war and conflict, empathy can be a powerful tool for connecting with others who have suffered similar experiences, fostering compassion, and promoting reconciliation. Compassion can motivate individuals to help those in need and contribute to healing and rebuilding efforts.
Hope and Gratitude	Umeed, aasha, Chah, Ichha, Sapna, Unnati, Ashvasan, Khushi, Utsaah, Ashirwad, Samvedansheelata, Vishwas, Bharosa, Shukraguzaar, eshaananand, Shukrana, Aabhaar, Namrata, Samaanya, Naman, Aasherewad, Badhai, Dhanyavaad, Abhivadn, Manobhav, Bhakti	Hope is a feeling of optimism and expectation that something good will happen despite the challenges and difficulties faced. Gratitude is a feeling of thankfulness and appreciation for the good things in one's life. In war and conflict, hope can be a source of strength and motivation, enabling individuals to persevere and work towards a better future. Gratitude can help individuals focus on the positive aspects of their lives, foster resilience, and cultivate a sense of peace amidst the turmoil.
Resilience	Jheelaanek, himmat, Sahasi, Sahas, Dhairya, Majbooti, Samvedansheel, Samarthya, Majbuti, Lachari	Resilience is the ability to adapt and cope with difficult or challenging situations. In war and conflict, resilience is essential for individuals and communities to survive and persevere in the face of adversity.

Table 2: Justification for selection of emotion. The table maps emotion groups to their Hinglish equivalents and explains their relevance in contexts of war and conflict.

However, it must be noted that the selection of these emotions is not only based on cluster analysis but also grounded in the theories of emotions, including the Cognitive Appraisal Theory (CAT), the Discrete Emotions Theory (DET), and the Process-Oriented Theory (POT) of Emotion⁴³.

Corpus details
The dataset consists of domain-specific (Wars, conflicts, and crises) and additional tweet datasets that contain a mix of Hindi and English tweets. Figure 3 is a snapshot of the tweet datasets available publicly being processed for this research work. The primary dataset is publicly available at https://data.mendeley.com/datasets/y63frd6pmf/7.

Figure 3: Corpus details. The availability of the dataset is explained here. Please click here to view a larger version of this figure.

Annotation of emotion labels
An initial corpus of 10,040 Hinglish tweets related to war and conflict was manually annotated with eight emotion labels by bilingual experts. To address the linguistic complexity of code-mixed text, a lexical rule-based framework was developed, incorporating several components. The framework includes Emotion-Specific Dictionaries that map Hindi/English terms to emotions, such as fear_words = {Fear, Dread, Bhayanak, , Terror, } and anger_words = {gussa, rage, , irritation, }. Cross-lingual equivalence rules linked the terms (e.g., if (Ae == Ah): anger = gussa | ). Lexical rules such as emotion dictionaries, tokenization for mixed languages, and subword decomposition are used. In order to annotate emotions, these rules preprocess text and extract features, which are then combined with machine learning embeddings⁴⁵.

For Tokenization, the framework utilized custom rules for language switching, punctuation, emojis, and subword tokenization. Hindi (Devanagari) text was tokenized at the character level, while English (Roman) used whitespace. Example: Mujhe frustration hai → [Mujhe, frustration, hai]. Special Characters like Hashtags (#) and mentions (@) were isolated as individual tokens (e.g., #WarCrimes → [#, WarCrimes]), while punctuation Marks like commas (,) and exclamation points (!) were split into separate tokens (e.g.,  → , !]).

Emojis were also treated as standalone tokens and mapped to emotions (e.g.,  → anger, → sadness). Subword tokenization done with Devanagari script in which compound words were split using regex patterns for Sandhi rules (e.g., ] [kingdom + world]) and Roman Scripts prefixes/suffixes were segmented (e.g., unbelievable → [un, believable]). For domain-specific expansion, tokens were replaced with emotion labels if matched in dictionaries. For example: Bhayanak → fear, Dahad" → fear, gussa → anger. Tweet Mujhe Bhayanak lag raha hai → Tokens [Mujhe, fear, lag, raha, hai].

After vectorization, the processed tokens (words, subwords, emojis) were converted to 300-dimensional embeddings using Word2Vec/fastText. The numerical representations in column vectors of tokens, a matrix of vectors corresponding to tokens. Each row corresponds to an embedding vector associated with a token in the text, representing the token's order. Columns in a row signify dimensions in the embedding space. The vectors contain real numbers computed using Word2Vec and FastText. Tokens with zero vectors, depicted by rows with all zero values, may denote spaces or special characters lacking meaningful information in this representation. Embeddings aim to capture contextual word relationships for improved annotation. Non-zero vectors indicate meaningful representations of words or symbols. Values in these vectors encode various semantic and syntactic features. Zero vectors typically represent padding, space, or unrecognized tokens. Variability in values reflects the richness of features captured by the embedding model. Different vector dimensions capture diverse aspects of a word's meaning, context, and usage. Figure 4shows how vectors are represented, and from Figure 5, the implications of using the vectorization process can be understood.

Figure 4: Custom tokenization. The figure shows how vectors are represented in the embedding space and depicts how each token is transferred into a numerical format Please click here to view a larger version of this figure.

Figure 5: Token vectorization process and its implications. The figure illustrates the implication of these processes, components, and highlights how these embeddings capture are semantics of emotions for accurate emotional classification. Please click here to view a larger version of this figure.

As per the research flow, the process starts with parsing the input text, expanding tokens using the rule-based dictionaries, and then breaking down these tokens into subwords. This approach provides a comprehension of the text’s emotional content and cultural context, and pseudo logic for the research is given below.

Initialize emotion dictionaries (e.g., fear_words = {"Fear", "Dread", "Bhayanak", ...}): Set Ae = Ah
SubwordRules(token, script): If Devanagari → split using regex (compound/Sandhi), If Roman → split prefixes/suffixes using regex, Return subwords
DomainSpecificExpansion(token): If token in emotional/linguistic dictionaries → return emotion
Else → return token
ProcessTweet(text): Define regex for Devanagari, Roman, others; Extract tokens using regex; Apply DomainSpecificExpansion and SubwordRules to tokens; Return processed subwords
Vectorise tokens into numerical embeddings
Apply Active Learning with human feedback

Results

The finding of this research suggests that integrating the lexical rules with machine learning and active learning techniques offers a viable pathway for enhancing the efficiency and accuracy of emotion annotation in code-mixed hinglish text. Through iterative refinement and expert suggestion, the proposed framework was able to achieve notable reductions in manual effort while sustaining high performance across evolution matrices. The outcomes indicate potential for broader applicability ...

Discussion

The dataset for this study was curated using a combination of manual annotation and active learning. Initially, 10,040 Hinglish tweets related to war and conflict were manually labeled with eight predefined emotions. The dataset was then expanded to 19,000 tweets using a semi-automated approach. Active learning enabled selective expert intervention, reducing manual effort by 40% while maintaining a high annotation accuracy of 81% with an F-score of 0.76. Lexical rules and emotion-specific dictionaries played a crucial ro...

Materials

List of materials used in this article

Name	Company	Catalog Number	Comments
fastText	Facebook AI	N/A	Word representation and classification
Google Colab	Google	N/A	Cloud-based Jupyter Notebook environment
Google Colab GPU/TPU	Google	N/A	Cloud-based hardware acceleration
Intel Core i5/i7 or AMD Ryzen 5/7	Intel / AMD	N/A	Processor for local execution (if required)
Matplotlib	Open-source	N/A	Data visualization library
NLTK	Open-source	N/A	Natural Language Toolkit for text processing
NumPy	Open-source	N/A	Numerical computing library
NVIDIA GTX 1650 or Higher (Optional)	NVIDIA	N/A	GPU for deep learning tasks
Pandas	Open-source	N/A	Data manipulation library
Python	Python Software Foundation	N/A	Programming language for ML and NLP
PyTorch	Meta AI	N/A	Deep Learning framework
RAM (8GB Minimum, 16GB Recommended)	Various	N/A	Memory requirement for ML tasks
Scikit-learn	Open-source	N/A	Machine Learning library
Seaborn	Open-source	N/A	Statistical data visualization
SpaCy	Explosion AI	N/A	Industrial-strength NLP library
SSD Storage (256GB Minimum, 512GB Recommended)	Various	N/A	Storage for dataset processing
TensorFlow	Google	N/A	Deep Learning framework