FlexAR-3B-T2I

Autoregressive Text-to-Image Model trained on FlexTok.

This model is the 3B parameter version of the FlexAR architecture, introduced in the paper ** (1D) Ordered Tokens Enable Efficient Test-Time Search**.

Project Website | GitHub | Paper

Model Details

• Model Type: Autoregressive Text-to-Image Generation
• Architecture: Transformer Decoder with Cross-Attention
• Embedding Dimension: 2304
• Number of Blocks: 36
• Number of Heads: 36
• Image Resolution: 256x256
• Tokenizer: EPFL-VILAB/flextok_d18_d28_dfn
• Text Encoder: google/flan-t5-xl

Installation

To use this model, install the flextok_ar package from the official repository:

git clone https://github.com/EPFL-VILAB/search-over-tokens.git
cd search-over-tokens
pip install -e ./flextok_ar

Usage

You can generate images from text prompts using the following code:

# Generate image from text prompt
from flextok_ar.utils.helpers import load_model

# Load model
model, tokenizer, cfg = load_model(
    model_id="ZhitongGao/FlexAR-3B-T2I",
    device="cuda"
)

# Generate image
images = model.generate(
    data_dict={"text": ["A serene lake at sunset"]},
    cfg_factor=3.0,
    temperature=1.0,
)

# Save result
images[0].save("output.png")

Citation

If you find this model helpful, please consider citing the following work:

@article{gao2026ordered,
  title={(1D) Ordered Tokens Enable Efficient Test-Time Search},
  author={Zhitong Gao and Parham Rezaei and Ali Cy and Mingqiao Ye and Nata{\v{s}}a Jovanovi{\'{c}} and Jesse Allardice and Afshin Dehghan and Amir Zamir and Roman Bachmann and O{\u{g}}uzhan Fatih Kar},
  journal={arXiv preprint arXiv:2604.15453},
  year={2026}
}

@article{flextok,
  title={{FlexTok}: Resampling Images into 1D Token Sequences of Flexible Length},
  author={Roman Bachmann and Jesse Allardice and David Mizrahi and Enrico Fini and O{\u{g}}uzhan Fatih Kar and Elmira Amirloo and Alaaeldin El-Nouby and Amir Zamir and Afshin Dehghan},
  journal={arXiv preprint arXiv:2502.01234},
  year={2025}
}