A Hybrid Model and Rule-Based Algorithm for Punctuation Prediction in Uzbek Texts

Abstract

Punctuation restoration is a routine task in NLP for well-resourced languages, but the picture changes for Uzbek. Annotated data is thin, the morphology is agglutinative, and correct punctuation directly affects how downstream systems read the text — from syntactic parsing to automatic processing pipelines. This paper describes a hybrid system built for the Uzbek case. The architecture has three parts: a BERT-based encoder (BERTbek) that provides contextual token representations, a  BiLSTM layer that models the sequence dependencies around each token, and a post-processing rule set that handles constructions where the neural component tends to be uncertain — those tied to Uzbek grammar and orthography rather than to context alone. The model was trained on a 122-million-word Uzbek corpus drawn from textbooks, fiction, official websites and technical texts, and evaluated on a balanced test set of 30,000 words covering 30 subject areas, each represented by roughly a thousand words. Punctuation-restoration accuracy across the 30 domains reached 76.2%, with token-level accuracy of 87.0% on the confusion matrix. Performance was highest on narrative and topic-specific texts — cinema (98.3%), agriculture (96.6%), stories (94.5%) and culture (94.2%) — and lowest on mathematics (74.6%), sport (76.6%) and mass media (76.7%), where sentence structure varies more sharply and rare punctuation marks are more common. The rule layer contributed most of the improvement on grammatically ambiguous and context-dependent cases. We see this design — a language-specific transformer with a light linguistic layer on top — as a practical route for languages that do not yet have the data volume needed to train purely neural systems to comparable levels. In our setting, the components most likely to benefit are automatic text editing, speech-to-text conversion, machine translation, information retrieval and text indexing.