Progressive Domain Expansion Network for Single Domain Generalization
Total Page:16
File Type:pdf, Size:1020Kb
Progressive Domain Expansion Network for Single Domain Generalization Lei Li12, Ke Gao1∗, Juan Cao1∗, Ziyao Huang12, Yepeng Weng12, Xiaoyue Mi12, Zhengze Yu12, Xiaoya Li12, Boyang xia12 1Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China 2University of Chinese Academy of Sciences, Beijing, China flilei17b,caojuan,huangziyao19f,wengyepeng19s,[email protected] [email protected], fyuzhengze,lixiaoya18s,[email protected] domain space Feature space Abstract 푠Ƹ 푆 푆 푆 푆 푠Ƹ23 Single domain generalization is a challenging case of 1 2 3 푆푠1Ƹ model generalization, where the models are trained on a single domain and tested on other unseen domains. A promising solution is to learn cross-domain invariant rep- resentations by expanding the coverage of the training do- main. These methods have limited generalization perfor- (a) The traditional decision bound- (b) The new decision boundary mance gains in practical applications due to the lack of ap- ary learned with the original train- learned with our progressively ex- propriate safety and effectiveness constraints. In this pa- ing domain. panded domains. per, we propose a novel learning framework called pro- gressive domain expansion network (PDEN) for single do- Train/Source domain Test/Unseen target domains main generalization. The domain expansion subnetwork Model and representation learning subnetwork in PDEN mutu- MNIST[22] MNIST-M[10] SVHN[31] ally benefit from each other by joint learning. For the Task1: Object Recognition domain expansion subnetwork, multiple domains are pro- gressively generated in order to simulate various photo- Model metric and geometric transforms in unseen domains. A series of strategies are introduced to guarantee the safety SYNTHIA[35]-DAWN SYNTHIA-NIGHT SYNTHIA-WINTER Task2: Semantic segmentation and effectiveness of the expanded domains. For the do- main invariant representation learning subnetwork, con- (c) Domain generalization scenario trastive learning is introduced to learn the domain invari- Figure 1. The illustration of our PDEN for single domain general- ant representation in which each class is well clustered so ization. The tiny images in (a) and (b) with red border denote the that a better decision boundary can be learned to improve source domain and the one with green border denote the expanded arXiv:2103.16050v1 [cs.CV] 30 Mar 2021 it’s generalization. Extensive experiments on classification domains with our PDEN. and segmentation have shown that PDEN can achieve up to 15.28% improvement compared with the state-of-the-art trinsic attributes(such as color, texture, pose etc.), as shown single-domain generalization methods. Codes will be re- in Fig.1. The performance of a deep model usually drops leased soon at https://github.com/lileicv/PDEN when applied to unseen domains. For example, The accu- racy of the CNN model(trained on MNIST) on MNIST test set is 99%, while that on SVHN test set is only 30%. Model 1. Introduction generalization is important to machine learning. Two solutions have been proposed to deal with the above In this paper, we define domains as various distributions issue, namely, domain adaptation [10, 30, 36,9] and do- of objects appearance caused by different external condi- main generalization [29, 11, 12, 16]. Domain adaptation tions(such as weather, background, illumination etc.) or in- aims to generalize to a known target domain whose labels *Corresponding author are unknown. Distribution alignment(e.g., MMD) and style transfer(e.g., CycleGAN) are frequently used in these meth- be learned to improve it’s generalization. ods to learn domain-invariant features. However, it requires • Extensive experiments on classification and segmen- data from the target domain to train the model, which is dif- tation have shown the superior performance of our ficult to achieve in many tasks due to lack of data. method. The proposed method can achieve up to Domain generalization, which not requires access to any 15.28% improvement compared with other single- data from the unseen target domain, can solve these prob- domain generalization methods. lems. The idea of domain generalization is to learn a domain-agnostic model from one or multiple source do- mains. Particularity, in many fields we are usually faced 2. Related Work with the challenge of giving a single source domain, which Domain Adaptation. In recent years, many domain is defined as single domain generalization [33]. Recently, adaptation methods [10, 30, 36,9] have been proposed to studies have made progress on this task [34, 45, 38, 40, solve the problem of domain drift between source and target 33, 46]. All of these methods, which are essentially data domain, including feature-based adaptation[10], instance- augmentation, improve the robustness of the model to the based adaptation [6] and model parameter based adapta- unseen domain by extending the distribution of the source tion [14]. The domain adaptation method in deep learn- domain. Specifically, additional samples are generated by ing is mainly to align the distribution of source domain manually selecting the augmentation type[45, 34] or by and target domain, including two kinds of methods: MMD learning the augmentation through neural networks[33, 46]. based adaptation method[39, 25] and adversarial based Data augmentation has proved to be an important means method[10]. DDC[39] is first proposed to solve the do- for improving model generalization [44]. However, such main adaptation problems in deep networks. DDC fixes methods require the selection of an augmentation type and the weights of the first 7 layers in AlexNet, and MMD is magnitude based on the target domain, which is difficult to used on the 8th layer to reduce the distribution difference achieve in other tasks. They cannot guarantee the safety between the source domain and target domain. DAN[25] and effectiveness of synthetic data or even reduce accuracy. increased the number of adaptive layers (three in front of [42, 20]. the classifier head) and introduced MK-MMD instead of In this paper, we propose the progressive domain expan- MMD. AdaBN[24] proposed to measure the distribution of sion network (PDEN) to solve the single domain general- the source domain and target domain in BN layer. With ization problem. Task models and generators in PDEN mu- the emergence of GAN, a lot of domain adaptation meth- tually benefit from each other through joint learning. Safe ods based on adversarial learning have been developed. and effective domains are generated by the generator under DANN[10] is the first research work to reduce the dis- the precise guidance of the task model. The generated do- tribution difference between the source domain and tar- mains are progressively expanded to increase the coverage get domain by adversarial learning. DSN[1] assumes that and improve the completeness. Contrastive learning is in- each domain includes a domain-shared distribution and a troduced to learn the cross-domain invariant representation domain-specific distribution. Based on this assumption, with all the generated domains. It is noteworthy that we can DSN learned the shared feature and the domain-specific fea- flexibly replace the generator in PDEN to achieve different ture respectively. DAAN[43] measures the marginal distri- types of domain expansion. bution and conditional distribution with a learnable weight. Our main contributions are as follows: Domain Generalization. Domain generalization is • We propose a novel framework called progressive do- more challenging than domain adaptation.Domain general- main expansion network (PDEN) for single domain ization aims to learn the model with data from the source generalization. The PDEN contains domain expansion domain and the model can be generalized to unseen do- subnetwork and domain invariant representation learn- mains. ing subnetwork, which mutually benefit from each Domain generalization can be categorized as such sev- other by joint learning. eral research interests: Domain alignment[29, 28, 23,8] • For the domain expansion subnetwork, multiple do- and domain ensemble[26]. Domain alignment methods as- mains are progressively generated to simulate various sume that there is a distribution shared by different domains. photometric and geometric transforms in unseen do- These methods map the distribution from different domains mains. A series of strategies are introduced to guaran- to the shared one. CCSA[28] propose the contrastive se- tee the safety and effectiveness of these domains. mantic alignment loss to minimize the distance between • For the domain invariant representation learning sub- data with the same label but from different domains and network, contrastive learning is introduced to learn the maximize the distance between the data with different class domain invariant representation in which each class is labels. In MMD-AAE[23], the feature distribution of source well clustered so that a better decision boundary can domain and target domain are aligned by MMD, and then init 1푡ℎ generator 2푡ℎ generator 퐾푡ℎ generator init 1 generator 2 generator 퐾 generator 푡ℎ 퐺1 푡ℎ 퐺2 푡ℎ return 퐺1 퐺2 return 푀 푀1 푆መ1 푀 푀2 푆መ2 푀 푀 푀 푀1 푆መ1 푀 푀2 푆መ2 푀 푀 푆 푆푎푙푙 푆푎푙푙 … 푆푎푙푙 푆 푆푎푙푙 푆푎푙푙 … 푆푎푙푙 (a)(a) Progressive Train K unseen domain expansion domain one generators by one 1 by 1 (a) Train K unseen domain generators 1 by 1 푀 퐺1 Classifier 푀 퐿푐푒 퐺1 Classifier Head Feature 퐿푐푒 푆 퐺2 푆푎푙푙 Feature Head 퐿푠푟푐 푆 퐺 푆 Extractor Projection 퐿푠푟푐 2 푎푙푙 Extractor Projection 퐿 … Head 퐿 푁퐶퐸 … Head 푁퐶퐸 퐺푘−1 퐺푘−1 Weight sharing 퐿 Weight sharing 퐿푎푑푣 푎푑푣 푀 퐺푘 퐺푘 푀k k 푁푁(0(,01,)1) 푛 푛 ProjectionProjection Head Head FeatureFeature EncoderEncoder DecoderDecoder AdaIN ExtractorExtractor ClassifierClassifier AdaIN 퐿푐푙푠 퐿 Head Head 푐푙푠 퐺cyc 퐿푐푦푐 퐺cyc 퐿푐푦푐 (b) Expand the kth unseen domain Figure 2. Illustration of the proposed method PDEN. (a) We show how the domain is progressively extended. We trained the task model M and unseen domain generator G alternately. G is trained to synthesize the unseen domain S^ under the guidance of M.