The outfit generation problem involves recommending a complete outfit to a user based on their interests. Existing approaches focus on recommending items based on anchor items or specific query styles but do not consider customer interests in famous characters from movie, social media, etc. In this paper, we define a new Character-based Outfit Generation (COG) problem, designed to accurately interpret character information and generate complete outfit sets according to customer specifications such as age and gender. To tackle this problem, we propose a novel framework LVA-COG that leverages Large Language Models (LLMs) to extract insights from customer interests (e.g., character information) and employ prompt engineering techniques for accurate understanding of customer preferences. Additionally, we incorporate text-to-image models to enhance the visual understanding and generation (factual or counterfactual) of cohesive outfits. Our framework integrates LLMs with text-to-image models and improves the customer's approach to fashion by generating personalized recommendations. With experiments and case studies, we demonstrate the effectiveness of our solution from multiple dimensions.
IIT Kharagpur（印度理工学院卡哈拉格普尔分校）；Flipkart Internet Pvt. Ltd.（Flipkart互联网私人有限公司）；Leibniz University Hannover（汉诺威莱布尼茨大学）
Recommendation in the fashion domain has seen a recent surge in research in various areas, for example, shop-the-look, context-aware outfit creation, personalizing outfit creation, etc. The majority of state of the art approaches in the domain of outfit recommendation pursue to improve compatibility among items so as to produce high quality outfits. Some recent works have realized that style is an important factor in fashion and have incorporated it in compatibility learning and outfit generation. These methods often depend on the availability of fine-grained product categories or the presence…
UT Austin（德克萨斯大学奥斯汀分校）；Facebook AI Research（Facebook人工智能研究）；Georgia Tech（佐治亚理工学院）；Cornell Tech（康奈尔科技学院）
Given an outfit, what small changes would most improve its fashionability? This question presents an intriguing new vision challenge. We introduce Fashion++, an approach that proposes minimal adjustments to a full-body clothing outfit that will have maximal impact on its fashionability. Our model consists of a deep image generation neural network that learns to synthesize clothing conditioned on learned per-garment encodings. The latent encodings are explicitly factorized according to shape and texture, thereby allowing direct edits for both fit/presentation and color/patterns/material, respec…
Flipkart Internet Pvt. Ltd.（Flipkart互联网私人有限公司）；Indian Institute of Technology Kharagpur（印度理工学院卡哈拉格普尔分校）
Fashion recommendation has witnessed a phenomenal growth of research, particularly in the domains of shop-the-look, contextaware outfit creation, personalizing outfit creation etc. Majority of the work in this area focuses on better understanding of the notion of complimentary relationship between lifestyle items. Quite recently, some works have realised that style plays a vital role in fashion, especially in the understanding of compatibility learning and outfit creation. In this paper, we would like to present the end-to-end design of a methodology in which we aim to generate outfits guided …
CFALR: Collaborative Filtering-Augmented Large Language Model for Personalized Fashion Outfit Recommendation
The Hong Kong Polytechnic University（香港理工大学）；University of Electronic Science and Technology of China（电子科技大学）；Singapore Management University（新加坡管理大学）；Tongji University（同济大学）；National University of Singapore（新加坡国立大学）
FashionStylist: An Expert Knowledge-enhanced Multimodal Dataset for Fashion Understanding
Yanshan University（燕山大学）；Central South University（中南大学）；Zhejiang University（浙江大学）；Southwest University（西南大学）；Singapore Management University（新加坡管理大学）；Singapore University of Technology and Design（新加坡科技设计大学）
National Chiao Tung University（国立交通大学）；National Chung Hsing University（国立中兴大学）；Peking University（北京大学）；Institut Teknologi Sepuluh Nopember（泗水理工学院）
Fashion is the way we present ourselves to the world and has become one of the world's largest industries. Fashion, mainly conveyed by vision, has thus attracted much attention from computer vision researchers in recent years. Given the rapid development, this paper provides a comprehensive survey of more than 200 major fashion-related works covering four main aspects for enabling intelligent fashion: (1) Fashion detection includes landmark detection, fashion parsing, and item retrieval, (2) Fashion analysis contains attribute recognition, style learning, and popularity prediction, (3) Fashion…
Ludwig-Maximilian University Munich（慕尼黑路德维希-马克西米利安大学）
Novel architectures have recently improved generative image synthesis leading to excellent visual quality in various tasks. Of particular note is the field of ``AI-Art'', which has seen unprecedented growth with the emergence of powerful multimodal models such as CLIP. By combining speech and image synthesis models, so-called ``prompt-engineering'' has become established, in which carefully selected and composed sentences are used to achieve a certain visual style in the synthesized image. In this note, we present an alternative approach based on retrieval-augmented diffusion models (RDMs). In…
Recent powerful vision classifiers are biased towards textures, while shape information is overlooked by the models. A simple attempt by augmenting training images using the artistic style transfer method, called Stylized ImageNet, can reduce the texture bias. However, Stylized ImageNet approach has two drawbacks in fidelity and diversity. First, the generated images show low image quality due to the significant semantic gap betweeen natural images and artistic paintings. Also, Stylized ImageNet training samples are pre-computed before training, resulting in showing the lack of diversity for e…
Investigating Pedagogical Teacher and Student LLM Agents: Genetic Adaptation Meets Retrieval Augmented Generation Across Learning Style
RespAI Lab, KIIT Bhubaneswar（KIIT布巴内斯瓦尔 RespAI 实验室）；BITS Pilani（皮拉尼比尔拉理工学院）；University of South Florida（南佛罗里达大学）；Pennsylvania State University（宾夕法尼亚州立大学）
University at Albany, SUNY（纽约州立大学奥尔巴尼分校）；JD Finance AI Lab（京东金融人工智能实验室）；GE Global Research（通用电气全球研究中心）
Video style transfer is a useful component for applications such as augmented reality, non-photorealistic rendering, and interactive games. Many existing methods use optical flow to preserve the temporal smoothness of the synthesized video. However, the estimation of optical flow is sensitive to occlusions and rapid motions. Thus, in this work, we introduce a novel evolve-sync loss computed by evolvements to replace optical flow. Using this evolve-sync loss, we build an adversarial learning framework, termed as Video Style Transfer Generative Adversarial Network (VST-GAN), which improves upon …
Hengyang Normal University（衡阳师范学院）；Cardiff University（卡迪夫大学）
This paper presents an automatic image synthesis method to transfer the style of an example image to a content image. When standard neural style transfer approaches are used, the textures and colours in different semantic regions of the style image are often applied inappropriately to the content image, ignoring its semantic layout, and ruining the transfer result. In order to reduce or avoid such effects, we propose a novel method based on automatically segmenting the objects and extracting their soft semantic masks from the style and content images, in order to preserve the structure of the …
Lionsgate Entertainment Corp.（狮门娱乐公司）；University of Southern California（南加州大学）
The ability to synthesize style and content of different images to form a visually coherent image holds great promise in various applications such as stylistic painting, design prototyping, image editing, and augmented reality. However, the majority of works in image style transfer have focused on transferring the style of an image to the entirety of another image, and only a very small number of works have experimented on methods to transfer style to an instance of another image. Researchers have proposed methods to circumvent the difficulty of transferring style to an instance in an arbitrar…
ByteDance Inc（字节跳动公司）；National University of Singapore（新加坡国立大学）
While substantial progresses have been made in automated 2D portrait stylization, admirable 3D portrait stylization from a single user photo remains to be an unresolved challenge. One primary obstacle here is the lack of high quality stylized 3D training data. In this paper, we propose a novel framework \emph{AgileGAN3D} that can produce 3D artistically appealing and personalized portraits with detailed geometry. New stylization can be obtained with just a few (around 20) unpaired 2D exemplars. We achieve this by first leveraging existing 2D stylization capabilities, \emph{style prior creation…
UC Santa Barbara（加州大学圣巴巴拉分校）；UC Santa Cruz（加州大学圣克鲁兹分校）；UC San Diego（加州大学圣地亚哥分校）；Google（谷歌）
One of the most challenging topics in Natural Language Processing (NLP) is visually-grounded language understanding and reasoning. Outdoor vision-and-language navigation (VLN) is such a task where an agent follows natural language instructions and navigates a real-life urban environment. Due to the lack of human-annotated instructions that illustrate intricate urban scenes, outdoor VLN remains a challenging task to solve. This paper introduces a Multimodal Text Style Transfer (MTST) learning approach and leverages external multimodal resources to mitigate data scarcity in outdoor navigation ta…
Style-Preserving Lip Sync via Audio-Aware Style Reference
School of Computer Science and Engineering, Sun Yat-sen University（中山大学计算机科学与工程学院）；Shandong Inspur Database Technology Co., Ltd.（山东 Inspur 数据库技术有限公司）；Peking University（北京大学）；Peng Cheng Laboratory（鹏城实验室）
This paper studies the task of full generative modelling of realistic images of humans, guided only by coarse sketch of the pose, while providing control over the specific instance or type of outfit worn by the user. This is a difficult problem because input and output domain are very different and direct image-to-image translation becomes infeasible. We propose an end-to-end trainable network under the generative adversarial framework, that provides detailed control over the final appearance while not requiring paired training data and hence allows us to forgo the challenging problem of fitti…
It is difficult to train classifiers on paintings collections due to model bias from domain gaps and data bias from the uneven distribution of artistic styles. Previous techniques like data distillation, traditional data augmentation and style transfer improve classifier training using task specific training datasets or domain adaptation. We propose a system to handle data bias in small paintings datasets like the Kaokore dataset while simultaneously accounting for domain adaptation in fine-tuning a model trained on real world images. Our system consists of two stages which are style transfer …
University of Chinese Academy of Sciences（中国科学院大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Jilin University（吉林大学）；Kuaishou Technology（快手科技）
Video style transfer is getting more attention in AI community for its numerous applications such as augmented reality and animation productions. Compared with traditional image style transfer, performing this task on video presents new challenges: how to effectively generate satisfactory stylized results for any specified style, and maintain temporal coherence across frames at the same time. Towards this end, we propose Multi-Channel Correction network (MCCNet), which can be trained to fuse the exemplar style features and input content features for efficient style transfer while naturally mai…
TaleForge: Interactive Multimodal System for Personalized Story Creation
1 University of Science, VNU-HCM, Ho Chi Minh City, Vietnam；2 Vietnam National University, Ho Chi Minh City, Vietnam；3 Department of Computer Science \ of Dayton Ohio, United States
NavRAG: Generating User Demand Instructions for Embodied Navigation through Retrieval-Augmented LLM
School of Artificial Intelligence, Beijing Normal University（北京师范大学人工智能学院）；School of Computing, National University of Singapore（新加坡国立大学计算机学院）；College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）
Shandong University（山东大学）；University of Amsterdam（阿姆斯特丹大学）
The task of fashion recommendation includes two main challenges: visual understanding and visual matching. Visual understanding aims to extract effective visual features. Visual matching aims to model a human notion of compatibility to compute a match between fashion items. Most previous studies rely on recommendation loss alone to guide visual understanding and matching. Although the features captured by these methods describe basic characteristics (e.g., color, texture, shape) of the input items, they are not directly related to the visual signals of the output items (to be recommended). Thi…
Cornell University（康奈尔大学）；Cornell Tech（康奈尔理工学院）；University of Copenhagen（哥本哈根大学）
In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an existing image. Most image generation methods, however, are ill-suited to this problem setting, as they make the assumption that one can assign arbitrary color to each pixel. In fact, naive application of existing methods fails even in simple domains such as MNIST digits, since one cannot create darker pixels by adding light. We know, however, that the human visual system can be fooled by optical illusions involving c…
ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents
The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；University College London（伦敦大学学院）；AI Lab, The Yangtze River Delta（长江三角洲人工智能实验室）
University of Houston（休斯顿大学）；University Hospital of Cologne（科隆大学医院）
The classification of Antibody Mediated Rejection (AMR) in kidney transplant remains challenging even for experienced nephropathologists; this is partly because histological tissue stain analysis is often characterized by low inter-observer agreement and poor reproducibility. One of the implicated causes for inter-observer disagreement is the variability of tissue stain quality between (and within) pathology labs, coupled with the gradual fading of archival sections. Variations in stain colors and intensities can make tissue evaluation difficult for pathologists, ultimately affecting their abi…
A large number of empirical studies on applying self-attention models in the domain of recommender systems are based on offline evaluation and metrics computed on standardized datasets, without insights on how these models perform in real life scenarios. Moreover, many of them do not consider information such as item and customer metadata, although deep-learning recommenders live up to their full potential only when numerous features of heterogeneous types are included. Also, typically recommendation models are designed to serve well only a single use case, which increases modeling complexity …
Fashion as characterized by its nature, is driven by style. In this paper, we propose a method that takes into account the style information to complete a given set of selected fashion items with a complementary fashion item. Complementary items are those items that can be worn along with the selected items according to the style. Addressing this problem facilitates in automatically generating stylish fashion ensembles leading to a richer shopping experience for users. Recently, there has been a surge of online social websites where fashion enthusiasts post the outfit of the day and other user…
