mariya.fyi — research

$ cat research_statement.txt

My research focuses on agentic safety, multi-agent collaboration and communication, and long-horizon reasoning, with an emphasis on behavioral alignment in emergent model capabilities. I'm interested in understanding how intelligent systems solve complex problems under partial observability, share information and coordinate across specialized capabilities, and adapt their behavior to human preferences and intentions.

Across my projects, a core theme has been multimodal safety and alignment, building on my background in vision and language learning and large-scale multimodal systems. I study the behavior of frontier models through evaluation science, scalable oversight, and post-training methods. Much of my work includes benchmark design, large-scale evaluation across the full model lifecycle, automated grading calibrated against human judgments, and synthetic data generation to systematically probe model capabilities and characterize safety and alignment failures, particularly in long-horizon, long-tail, and agentic settings.

More recently, I've been exploring questions around AI-assisted scientific discovery: how AI can contribute to research from formulating hypotheses to designing experiments to interpreting results, what makes a result novel, and if language models are truly capable of generating original research ideas.

$ ls -lt ./projects/

If It Ain't Broke, Don't Fix It: Failures of Epistemic Control in Language-Model Agents

UNDER REVIEW

HYPOTHESIS TRACKINGEPISTEMIC CONTROLTHEORY OF MIND

▶

Mariya I. Vasileva

Before acting, a language-model agent must decide which explanation of the evidence to believe, how much of the system to change, and whether the evidence can be trusted. We introduce an executable testbed in which the correct answer is known exactly, and every revision is executed. With two hidden changes, each model keeps the true environment among its plausible hypotheses in 80.0–97.8% of tasks, yet the leading hypothesis is correct in only 5.8%, below the 27.8% expected by chance. Models also show weak epistemic control, acting on false-evidence reports without double-checking: relative to a truthful passing report, a fabricated failure report raises edits to correct programs by 84.5 percentage points.

GlobeBench: Evaluating Text-Based World Model Capabilities for Scalable Environment Simulations

IAEVAL 2026

BENCHMARKWORLD MODELS

▶

Nicholas Saban, Charvi Bannur, Mariya I. Vasileva, Darshan Deshpande

Text world models offer a way to reduce dependence on expensive external environments during agent training, but plausible observations do not necessarily preserve state, apply the correct transition, or enforce an environment’s rules. We introduce GlobeBench, a 1,632-item benchmark for text-based world simulation spanning five domains, four capabilities, and 20 fine-grained diagnostics. The best of 12 models achieves 28.13% task success, and neither simulation-tuned checkpoint outperforms the strongest general-purpose models. Success falls from 43.15% for local effects to 3.66% for the broadest state changes, and drops below 11% when the relevant state-setting event is more than 40 turns away.

OpenReview

Large Language Models Can Follow Instructions, But Not Many at Once: Phase Transitions in Compositional Constraint Satisfaction

UNDER REVIEW

EVALUATIONCOMPOSITIONAL LEARNING

▶

Mariya I. Vasileva

Introduces Constraint Saturation Evaluation (CSE), a procedurally generated benchmark that varies the number of simultaneous constraints a model must satisfy, with every constraint scored by a deterministic rule-based verifier and no LLM judge: 15 models, 36 constraint types, 369,753 checks at k=1–12. Per-constraint pass rate decays gradually while the chance of satisfying all k collapses — a model passing individual constraints at ~41% at k=8 succeeds on all eight just 5.7% of the time. Reliable instruction following breaks down beyond 5–6 simultaneous constraints.

arXiv

BBB: Bound—Beating Benchmark for Formal Math Theorem Proving

STAY TUNED

BENCHMARKRSIAUTORESEARCH

▶

Contributing to the development of a formal mathematical reasoning benchmark that evaluates LLMs on open quantitative problems across domains — covering optimization, number theory, combinatorics, physics, and beyond, with the goal of enabling incremental, verifiable improvements.

The Llama 4 Herd: The Beginning of a New Era of Natively Multimodal AI Innovation

META 2025

MODEL RELEASE

▶

Meta Superintelligence Labs

Llama 4 introduces the first open-weight natively multimodal models built on a mixture-of-experts architecture. The herd spans Scout (17B active params, 16 experts, 10M context window) and Maverick (17B active params, 128 experts), both distilled from Behemoth — a 288B active parameter teacher model surpassing GPT-4.5, Claude Sonnet 3.7, and Gemini 2.0 Pro on STEM benchmarks. Maverick achieves competitive results with DeepSeek v3 at less than half the active parameters, with a best-in-class performance-to-cost ratio.

models blog post

Polyvore Outfits Dataset

DATASET

BENCHMARKFASHION AI

▶

Mariya I. Vasileva et al.

A large-scale fashion dataset curated by real users on the Polyvore platform. Each outfit consists of manually constructed combinations that reflect authentic human styling choices — capturing implicit knowledge of color harmony, stylistic coherence, and garment compatibility. Widely adopted across computer vision, recommendation systems, and multimodal learning research.

dataset page

HandsOff: Labeled Dataset Generation with No Additional Human Annotations

CVPR 2023

GENERATIVE MODELSSYNTHETIC DATAHIGHLIGHT

▶

Austin Xu, Mariya I. Vasileva, Achal Dave, Arjun Seshadri

A framework capable of producing unlimited synthetic images and labels after training on fewer than 50 pre-existing labeled images. Unifies GAN inversion with dataset generation to produce rich pixel-wise labels across faces, cars, full-body poses, and urban driving scenes. Achieves state-of-the-art on semantic segmentation, keypoint detection, and depth estimation, and addresses long-tail challenges in model development.

project page code arXiv

Why do These Match? Explaining the Behavior of Image Similarity Models

ECCV 2020

EXPLAINABILITYVISION-LANGUAGE

▶

Bryan A. Plummer*, Mariya I. Vasileva*, Vitali Petsiuk, Kate Saenko, David A. Forsyth

Introduces SANE (Salient Attributes for Network Explanation) — a method to explain image similarity models where outputs are similarity scores rather than classification labels. SANE pairs saliency maps with attributes that best explain a match between two images, providing information beyond standard saliency alone, and improving attribute recognition performance.

code arXiv

Learning Similarity Conditions Without Explicit Supervision

ICCV 2019

WEAK SUPERVISIONEMBEDDING LEARNING

▶

Reuben Tan, Mariya I. Vasileva, Bryan A. Plummer, Kate Saenko, David A. Forsyth

An approach that jointly learns representations for multiple similarity conditions and their contributions as a latent variable — without explicit supervision at test time. Outperforms state-of-the-art on Polyvore-Outfits, Maryland-Polyvore and UT-Zappos50k across fill-in-the-blank, compatibility prediction, and triplet tasks, even against strongly supervised methods.

code arXiv

Learning Type-Aware Embeddings for Fashion Compatibility

ECCV 2018

CONTRASTIVE LEARNINGVISION-LANGUAGE

▶

Mariya I. Vasileva, Bryan A. Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, David A. Forsyth

Presents a contrastive learning vision-language embedding approach that respects item type and jointly learns notions of similarity and compatibility end-to-end. Evaluated on 68,306 outfits from Polyvore, achieving 3–5% improvement over state-of-the-art on compatibility prediction and fill-in-the-blank tasks, while supporting a variety of structured queries.

code arXiv

mariya@research:~$