COPA releases 9B Armenian model with full training data and recipe

Why it matters

COPA is making Armenian AI infrastructure inspectable at the data, training and evaluation layers, giving developers a foundation they can audit instead of another opaque set of weights.

Erik Arakelyan and the eight-person COPA team released a 9-billion-parameter Armenian base model alongside the datasets, code and training recipe used to build it, according to the team's Hugging Face release article. The project includes Khatun Avetisyan, Meri Davtyan, Heghine Grigoryan, Nane Khachatryan, Hayk Shahsuvaryan, Henrik Sergoyan and Vahan Martirosyan. The package includes 4.37 million Armenian news documents and 372,907 translated math and science problems.

COPA published its paper on September 3 and detailed the release in the Hugging Face community article on September 4. COPA says the resulting model, arm-gemma-e4b, recorded the highest mean score among the open Armenian models it evaluated. Its more consequential claim concerns inspectability: COPA says this is the first open Armenian LLM released with its complete training data and recipe.

Arakelyan has been working toward that problem from both inside Armenia's technical community and at larger AI organizations. His CV identifies him as a senior machine learning researcher at NVIDIA who earned a PhD in machine learning from the University of Copenhagen and previously held visiting research posts at Amazon Alexa AI and Cohere. Before that, he led model-tailoring work at Arm and completed a machine learning master's degree at University College London. In their SynDARin paper, Arakelyan and his coauthors proposed a method for generating and validating question-answering datasets in low-resource languages and tested it on a 1,200-sample Armenian dataset.

At COPA, Arakelyan works with co-founders Vahan Martirosyan, who leads strategy and delivery, and Henrik Sergoyan, who runs operations and partnerships. COPA builds decision-intelligence products for institutional and commercial clients. The Armenian release gives that work a public technical foundation while inviting outside researchers to inspect, reproduce and extend it.

COPA's contribution is unusually broad for a language-model release: it is publishing the raw ingredients, the filtering machinery and the evaluation setup rather than distributing model weights alone.

The data may outlast the model

ArmWeb contains 4.37 million documents and about 3.3 billion tokens under the Gemma-4 tokenizer. According to COPA's full paper, the team assembled it from a 15-year crawl of Armenian news sites spanning 2011 through 2026, then applied language identification, exact and near-duplicate removal, chronological test splits and contamination scans against Armenian benchmarks. The paper says the released records include per-document provenance metadata, including source and URL information.

That provenance matters because Armenian training text has typically appeared as one language slice inside larger multilingual web crawls. The paper's contamination analysis reports benchmark matches in 7.9% of CulturaX-hy documents, 10.9% of HPLT-v2-hy and 17.4% of FineWeb-2-hy. ArmWeb's post-split pool had a 3.3% contamination rate, representing 147,101 documents, before COPA removed the matches through 13-gram decontamination.

News remains a narrow foundation for a general-purpose model. It overrepresents formal writing, public events and the editorial choices of publishers. The news-heavy corpus leaves open questions about conversational Armenian and specialized-domain coverage beyond the accompanying STEM data.

ArmSTEM addresses a different shortage. It contains 372,907 parallel English-Armenian math and science problems, including 324,323 examples with step-by-step solutions. COPA's researchers machine-translated the material, masked numbers and mathematical notation to prevent corruption, and then asked an independent model to solve each Armenian version and reproduce the original answer.

Failures received a control test using the English problem. Items that defeated the solver in both languages were retained with a solver_limited label rather than treated as verified translations. In the paper's human-evaluation results, two native Armenian speakers separately assessed a 300-item sample for logical coherence and correctness. They agreed on every verdict, accepted 299 examples and recorded a Cohen's kappa of 1.0.

The process is still partly dependent on model-based judgment. COPA used automated systems for translation, repair, solving and portions of the audit. The human review covered 300 records from a dataset of nearly 373,000. ArmSTEM offers substantially more traceability than a translated corpus with no validation record, while leaving room for broader native-speaker evaluation.

Six percent of the mix changed the result

The arm-gemma-e4b model is an adaptation of Gemma-4-E4B trained on another 10 billion tokens. The paper's training configuration shows a final mixture of 69% ArmWeb, 6% ArmSTEM split between Armenian and English, 20% English web replay and 5% code.

The STEM share was small enough to look incidental. COPA's experiments indicate it was central to preserving capabilities during continued pretraining. In the ablation table, a news-heavy run at a learning rate of 1e-4 reduced Armenian Belebele accuracy from 0.619 for the unadapted base to 0.407, a loss of 21.2 percentage points. A lower 3e-5 learning rate improved the news-only result to 0.550, while adding the STEM material at that rate produced the released model's 0.716 score and pushed its six-task mean above the base.

On COPA's six-task Armenian likelihood suite, arm-gemma-e4b scored 0.500, versus 0.477 for the unadapted Gemma-4-E4B base. ArmenianGPT-1.0-3B scored 0.471 and HyGPT-10b scored 0.436. The adapted model led on three knowledge-focused tasks, although it slipped slightly behind its base on m-MMLU-hy and HellaSwag-hy. In the release article's generative ArmBench results, ArmenianGPT remained ahead on MMLU-Pro-Hy, scoring 0.281 against arm-gemma-e4b's 0.251 and the Gemma base's 0.154.

Those figures come from one checkpoint per model without seed averaging. COPA estimates a roughly 0.010 half-width for the 95% confidence interval around each model's six-task mean. Several individual test sets are small, and some tasks sit near chance or ceiling for most models. The aggregate supports COPA's claim within its chosen evaluation, but it does not establish uniform superiority across Armenian language tasks.

The released code provides a better basis for checking that claim than a leaderboard screenshot would. COPA published the corpus pipeline, translation and verification workflow, continued-pretraining recipes, evaluation configurations, learning rate, mixture weights and random seed. Researchers can rerun the study, challenge its benchmark choices or apply the process to another language.

A foundation, not an assistant

arm-gemma-e4b is a base model with no instruction tuning, chat template or safety tuning. Its model card describes text continuation, likelihood scoring and Armenian supervised fine-tuning as intended uses. It is not ready to drop into a customer-facing chat product without further work, evaluation and safeguards.

Licensing also travels with the components. COPA's release documentation says the ArmWeb compilation is available under ODC-BY 1.0 while the underlying article text remains the publishers' material. ArmSTEM inherits licenses from its source datasets. The model itself remains subject to Gemma's terms.

In its paper, COPA identifies an instruction-tuned arm-gemma-e4b and a harder competition-math addition to ArmSTEM as its next planned releases. The immediate release already fills a less visible gap: Armenian developers now have a documented stack for studying how data collection, translation and continued pretraining affect a model in their own language.

That work is arriving alongside better measurement. In August, RuntimeWire reported that Metric released an Armenian speech benchmark covering nearly 30 speech-recognition systems. COPA is tackling the training side for text models, while releasing enough of the machinery for other researchers to dispute the results constructively. For a language with limited openly curated training material, that is the useful form of competition.