Generating an image that retains the structure and semantics of a content reference while adopting the style of a separate style reference — known as style-content dual-reference generation — remains a challenging problem in AI research. A primary bottleneck, according to the authors of a new paper on arXiv, is the lack of large-scale triplet data with clean content-style separation and broad coverage of long-tail styles. To address this, the research team proposes FreeStyle, a scalable dual-reference generation framework based on community LoRA mining.
The FreeStyle Framework
FreeStyle treats community LoRAs (Low-Rank Adaptations) as compositional anchors for both style and content. The authors designed a rigorous generation and filtering pipeline to construct large-scale Style-Reference and Content-Reference triplets across multiple base models. This approach leverages the vast diversity of styles available in public LoRA repositories, enabling the framework to handle a wide range of artistic and photographic styles without requiring manual triplet creation.
Two-Stage Curriculum for Disentanglement
To address content leakage — where the style reference inadvertently influences the content or structure of the output — the framework adopts a two-stage curriculum with stage-specific disentanglement mechanisms. In the style-transfer stage, an attention-level enrichment constraint suppresses leakage from the style reference by modifying attention patterns to focus on content features. In the harder dual-reference stage, a frequency-aware RoPE (Rotary Position Embedding) modulation strategy targets positional-correspondence-based leakage. This combined approach allows FreeStyle to maintain content fidelity while accurately applying the desired style.
Benchmark and Evaluation
The researchers introduce a benchmark covering both style-reference and dual-reference generation tasks. Evaluations are conducted on style similarity, content preservation, aesthetics, instruction following, and leakage rejection. Two new metrics are proposed: a style-invariant Content Alignment Score (CAS) that measures how well the content is preserved regardless of style, and a calibrated VLM-based Rejection Score that evaluates generation reliability and identifies leakage. Experimental results, as reported in the paper, show that FreeStyle achieves a strong balance among style alignment, content preservation, and leakage suppression.
| Component | Description |
|---|---|
| Style-Reference Triplets | Generated from community LoRAs via filtering pipeline |
| Content-Reference Triplets | Generated alongside style pairs for dual-reference tasks |
| Attention-Level Enrichment | Constraint in style-transfer stage to reduce style leakage |
| Frequency-Aware RoPE Modulation | Mechanism in dual-reference stage to suppress positional leakage |
| Content Alignment Score (CAS) | Style-invariant metric for content preservation |
| VLM-Based Rejection Score | Calibrated metric for evaluating generation reliability |
The work, authored by Jinghong Lan, Wei Cheng, Yunuo Ye, Ziqi Xing, Peng Fang, Yixiao Wang, Rui Yang, Yufeng Zhang, Xuanyang Zeng, Xianfang Zou, Difan Gang, and Chi, is published on arXiv under a Creative Commons BY-NC-SA 4.0 license. The framework's reliance on community-contributed LoRAs makes it inherently scalable, as new styles can be incorporated without retraining from scratch. For enterprise technology leaders exploring generative AI for content creation, this approach offers a blueprint for building robust, style-adaptable generation systems that minimize leakage and maintain brand consistency.
