StyleT2F
Generating Human Faces from Textual Description Using StyleGAN2
Visual samples showing the consistent mapping between input textual keywords and generated facial features.
Overview
StyleT2F is a text-controlled face generation pipeline built around StyleGAN2. The method extracts facial attributes from an input description, converts those attributes into latent-space navigation targets, and generates a detailed human face whose visual features match the textual prompt.
Goal — The paper targets consistent mapping between language and generated facial features. It focuses on a wide set of facial attributes, including hair color, eye color, facial hair, age, gender, skin color, makeup, glasses, and face-shape properties.
Visual samples showing the consistent mapping between input textual keywords and generated facial features.
Method
The system consists of three stages: text processing, latent code generation, and StyleGAN2 synthesis. The paper describes this as an analysis of StyleGAN2 latent space combined with a text processing module for face generation from descriptions.
The system extracts facial attributes from text, uses them to manipulate StyleGAN2 latent space, and synthesizes the final face image.
Text Processing — The text module predicts values for 32 facial features. The authors formulate this as multi-label classification while using the actual logits as continuous feature values. DistilBERT is used because it is compact and performs well for the task.
Feature Directions — To manipulate StyleGAN2, the method samples generated faces, labels them with facial-attribute recognizers, and learns latent directions for each feature. Discrete attributes use logistic regression, while continuous attributes use linear regression.
A simple 2D latent-space example illustrating feature directions and navigation.
Latent Code Generation — Generation starts from a random StyleGAN2 latent vector. The current feature values are computed by projecting the latent vector onto feature directions, and sequential navigation adjusts the latent code toward the target values extracted from text. Unmentioned features are left unchanged.
Face Synthesis — The final latent vector is passed to the full StyleGAN2 synthesis network, which generates a 1024 by 1024 face image.
Training Setup
The paper uses two datasets: a synthetic text-description dataset for facial feature extraction and generated StyleGAN2 samples for learning latent feature directions.
Text Dataset — Because no directly suitable public dataset is identified, the authors create training descriptions manually and augment them through paraphrasing with cycle translation. The resulting dataset contains about 40,000 descriptions with labels for the 32 facial features.
Implementation — DistilBERT is trained with mean-squared error loss, Adam, a learning rate of 0.001, a batch size of 64, and 30 epochs. The paper reports roughly 12 hours of training on an Nvidia GTX 1080ti. About 3,000 StyleGAN2 samples are used to fit the feature directions in the extended W+ latent space.
Results
The paper reports that the generated faces capture requested attributes consistently and that many learned feature directions are close to orthogonal, which supports disentangled manipulation.
Consistency — Visual examples show generated faces matching prompts for attributes such as hair, eyes, facial hair, glasses, and makeup. The paper also reports that the full pipeline runs in about one second on an Nvidia GTX 1080ti.
Sequential navigation results show that moving along a feature direction can independently manipulate one facial attribute with little change to others.
Failure Cases — Failure cases occur when the prompt contains contradicting attributes, when feature combinations are uncommon, or when sequential navigation interacts with entangled directions.
Failure cases caused by contradicting features, uncommon feature combinations, sequential latent navigation, or entangled directions.
Conclusion
StyleT2F demonstrates a complete text-to-face pipeline based on StyleGAN2 latent manipulation. The paper concludes that the approach provides consistent text-image mapping for a broad set of facial features, while noting that future work can include more fine-grained attributes.
Citation
More Information
Open Positions
JoinInterested in pursuing a PhD in computer graphics?
Never miss an update
FollowJoin us on Twitter / X for the latest updates of our research group and more.