Learning to Learn Single Domain Generalization Fengchun Qiao Long Zhao Xi Peng University of Delaware Rutgers University University of Delaware [email protected] [email protected] [email protected] Abstract : Source domain(s) <latexit sha1_base64="glUSn7xz2m1yKGYjqzzX12DA3tk=">AAAB8nicjVDLSsNAFL3xWeur6tLNYBFclaQKdllw47KifUAaymQ6aYdOJmHmRiihn+HGhSJu/Rp3/o2TtgsVBQ8MHM65l3vmhKkUBl33w1lZXVvf2Cxtlbd3dvf2KweHHZNkmvE2S2SieyE1XArF2yhQ8l6qOY1Dybvh5Krwu/dcG5GoO5ymPIjpSIlIMIpW8vsxxTGjMr+dDSpVr+bOQf4mVViiNai894cJy2KukElqjO+5KQY51SiY5LNyPzM8pWxCR9y3VNGYmyCfR56RU6sMSZRo+xSSufp1I6exMdM4tJNFRPPTK8TfPD/DqBHkQqUZcsUWh6JMEkxI8X8yFJozlFNLKNPCZiVsTDVlaFsq/6+ETr3mndfqNxfVZmNZRwmO4QTOwINLaMI1tKANDBJ4gCd4dtB5dF6c18XoirPcOYJvcN4+AY5ZkWY=</latexit> <latexit sha1_base64="9X8JvFzvWSXuFK0x/Pe60//G3E4=">AAACD3icbVDLSsNAFJ3UV62vqks3g0Wpm5LW4mtVcOOyUvuANpTJZNIOnUzCzI1YQv/Ajb/ixoUibt26829M2iBqPTBwOOfeO/ceOxBcg2l+GpmFxaXllexqbm19Y3Mrv73T0n6oKGtSX/iqYxPNBJesCRwE6wSKEc8WrG2PLhO/fcuU5r68gXHALI8MJHc5JRBL/fxhzyMwpEREjQnuAbuD6AI3ptOx43uEy6I+muT6+YJZMqfA86SckgJKUe/nP3qOT0OPSaCCaN0tmwFYEVHAqWCTXC/ULCB0RAasG1NJPKataHrPBB/EioNdX8VPAp6qPzsi4mk99uy4Mtle//US8T+vG4J7ZkVcBiEwSWcfuaHA4OMkHOxwxSiIcUwIVTzeFdMhUYRCHOEshPMEJ98nz5NWpVQ+LlWvq4VaJY0ji/bQPiqiMjpFNXSF6qiJKLpHj+gZvRgPxpPxarzNSjNG2rOLfsF4/wJA4Zw6</latexit> S S : Target domain(s) <latexit sha1_base64="ssITTP/Vrn2uchq9aDxvcfruPQc=">AAACD3icbVDLSgNBEJz1bXxFPXoZDEq8hI2Kr5PgxWOEJAaSEHonnWTI7Owy0yuGJX/gxV/x4kERr169+TfuJkF8FTQUVd10d3mhkpZc98OZmp6ZnZtfWMwsLa+srmXXN6o2iIzAighUYGoeWFRSY4UkKayFBsH3FF57/YvUv75BY2WgyzQIselDV8uOFECJ1MruNnygngAVl4e8QXhL8Rkvg+ki8Xbgg9R5uzfMtLI5t+COwP+S4oTk2ASlVva90Q5E5KMmocDaetENqRmDISkUDjONyGIIog9drCdUg4+2GY/+GfKdRGnzTmCS0sRH6veJGHxrB76XdKbX299eKv7n1SPqnDRjqcOIUIvxok6kOAU8DYe3pUFBapAQEEYmt3LRAwOCkgjHIZymOPp6+S+p7heKB4XDq8Pc+f4kjgW2xbZZnhXZMTtnl6zEKkywO/bAntizc+88Oi/O67h1ypnMbLIfcN4+ATKRnDE=</latexit> T <latexit sha1_base64="jg7q+7X3OPw0DaaGhRyCxwQsmlY=">AAAB9HicjVDLSgNBEOz1GeMr6tHLYBA8hd0omGPAi8eI5gHJEnons8mQ2dl1ZjYQlnyHFw+KePVjvPk3ziY5qChY0FBUddNFBYng2rjuh7Oyura+sVnYKm7v7O7tlw4OWzpOFWVNGotYdQLUTHDJmoYbwTqJYhgFgrWD8VXutydMaR7LOzNNmB/hUPKQUzRW8nsRmhFFkd3O+tV+qexV3DnI36QMSzT6pffeIKZpxKShArXuem5i/AyV4VSwWbGXapYgHeOQdS2VGDHtZ/PQM3JqlQEJY2VHGjJXv15kGGk9jQK7mYfUP71c/M3rpias+RmXSWqYpItHYSqIiUneABlwxagRU0uQKm6zEjpChdTYnor/K6FVrXjnlerNRbleW9ZRgGM4gTPw4BLqcA0NaAKFe3iAJ3h2Js6j8+K8LlZXnOXNEXyD8/YJvX+SCw==</latexit> S2 … We are concerned with a worst-case scenario in model … <latexit sha1_base64="JCBb0CnHIdl/Wiv44m3dVy/qfYM=">AAAB9HicjVDLSgNBEOz1GeMr6tHLYBA8hd0omGPAi8cIeUGyhNlJbzJkdnadmQ2EJd/hxYMiXv0Yb/6Ns0kOKgoWNBRV3XRRQSK4Nq774aytb2xubRd2irt7+weHpaPjto5TxbDFYhGrbkA1Ci6xZbgR2E0U0igQ2AkmN7nfmaLSPJZNM0vQj+hI8pAzaqzk9yNqxoyKrDkfeINS2au4C5C/SRlWaAxK7/1hzNIIpWGCat3z3MT4GVWGM4HzYj/VmFA2oSPsWSpphNrPFqHn5NwqQxLGyo40ZKF+vchopPUsCuxmHlL/9HLxN6+XmrDmZ1wmqUHJlo/CVBATk7wBMuQKmREzSyhT3GYlbEwVZcb2VPxfCe1qxbusVO+uyvXaqo4CnMIZXIAH11CHW2hACxjcwwM8wbMzdR6dF+d1ubrmrG5O4Buct0+9gpIL</latexit> 1 <latexit sha1_base64="LsxaGz69WnMvF7RaFKyb/i+Pha0=">AAAB9HicjVDLSgNBEOz1GeMr6tHLYBA8hd0omGPAi8eI5gHJEmYns8mQ2dl1pjcQlnyHFw+KePVjvPk3ziY5qChY0FBUddNFBYkUBl33w1lZXVvf2CxsFbd3dvf2SweHLROnmvEmi2WsOwE1XArFmyhQ8k6iOY0CydvB+Cr32xOujYjVHU4T7kd0qEQoGEUr+b2I4ohRmd3O+l6/VPYq7hzkb1KGJRr90ntvELM04gqZpMZ0PTdBP6MaBZN8VuylhieUjemQdy1VNOLGz+ahZ+TUKgMSxtqOQjJXv15kNDJmGgV2Mw9pfnq5+JvXTTGs+ZlQSYpcscWjMJUEY5I3QAZCc4ZyagllWtishI2opgxtT8X/ldCqVrzzSvXmolyvLesowDGcwBl4cAl1uIYGNIHBPTzAEzw7E+fReXFeF6srzvLmCL7BefsEu/uSCg==</latexit> S1 T <latexit sha1_base64="DncTg5PltQRUlIV2vRtg0s8T0Nc=">AAAB9HicjVDLSgNBEOz1GeMr6tHLYBA8hd0omGPAi8cIeUGyhNnJbDJkdnad6Q2EJd/hxYMiXv0Yb/6Ns0kOKgoWNBRV3XRRQSKFQdf9cNbWNza3tgs7xd29/YPD0tFx28SpZrzFYhnrbkANl0LxFgqUvJtoTqNA8k4wucn9zpRrI2LVxFnC/YiOlAgFo2glvx9RHDMqs+Z8UB2Uyl7FXYD8TcqwQmNQeu8PY5ZGXCGT1Jie5yboZ1SjYJLPi/3U8ISyCR3xnqWKRtz42SL0nJxbZUjCWNtRSBbq14uMRsbMosBu5iHNTy8Xf/N6KYY1PxMqSZErtnwUppJgTPIGyFBozlDOLKFMC5uVsDHVlKHtqfi/EtrVindZqd5dleu1VR0FOIUzuAAPrqEOt9CAFjC4hwd4gmdn6jw6L87rcnXNWd2cwDc4b5+/BpIM</latexit> T2 <latexit sha1_base64="glUSn7xz2m1yKGYjqzzX12DA3tk=">AAAB8nicjVDLSsNAFL3xWeur6tLNYBFclaQKdllw47KifUAaymQ6aYdOJmHmRiihn+HGhSJu/Rp3/o2TtgsVBQ8MHM65l3vmhKkUBl33w1lZXVvf2Cxtlbd3dvf2KweHHZNkmvE2S2SieyE1XArF2yhQ8l6qOY1Dybvh5Krwu/dcG5GoO5ymPIjpSIlIMIpW8vsxxTGjMr+dDSpVr+bOQf4mVViiNai894cJy2KukElqjO+5KQY51SiY5LNyPzM8pWxCR9y3VNGYmyCfR56RU6sMSZRo+xSSufp1I6exMdM4tJNFRPPTK8TfPD/DqBHkQqUZcsUWh6JMEkxI8X8yFJozlFNLKNPCZiVsTDVlaFsq/6+ETr3mndfqNxfVZmNZRwmO4QTOwINLaMI1tKANDBJ4gCd4dtB5dF6c18XoirPcOYJvcN4+AY5ZkWY=</latexit> generalization, in the sense that a model aims to perform <latexit sha1_base64="orM/DHwO1fdvPqBY0pRjCsNYNKs=">AAAB8nicjVDLSsNAFL3xWeur6tLNYBFclaQKdllw47JCX5CGMplO2qGTTJi5EUroZ7hxoYhbv8adf+Ok7UJFwQMDh3Pu5Z45YSqFQdf9cNbWNza3tks75d29/YPDytFx16hMM95hSirdD6nhUiS8gwIl76ea0ziUvBdObwq/d8+1ESpp4yzlQUzHiYgEo2glfxBTnDAq8/Z8WKl6NXcB8jepwgqtYeV9MFIsi3mCTFJjfM9NMcipRsEkn5cHmeEpZVM65r6lCY25CfJF5Dk5t8qIRErblyBZqF83chobM4tDO1lEND+9QvzN8zOMGkEukjRDnrDloSiTBBUp/k9GQnOGcmYJZVrYrIRNqKYMbUvl/5XQrde8y1r97qrabKzqKMEpnMEFeHANTbiFFnSAgYIHeIJnB51H58V5XY6uOaudE/gG5+0Tj96RZw==</latexit> <latexit sha1_base64="orM/DHwO1fdvPqBY0pRjCsNYNKs=">AAAB8nicjVDLSsNAFL3xWeur6tLNYBFclaQKdllw47JCX5CGMplO2qGTTJi5EUroZ7hxoYhbv8adf+Ok7UJFwQMDh3Pu5Z45YSqFQdf9cNbWNza3tks75d29/YPDytFx16hMM95hSirdD6nhUiS8gwIl76ea0ziUvBdObwq/d8+1ESpp4yzlQUzHiYgEo2glfxBTnDAq8/Z8WKl6NXcB8jepwgqtYeV9MFIsi3mCTFJjfM9NMcipRsEkn5cHmeEpZVM65r6lCY25CfJF5Dk5t8qIRErblyBZqF83chobM4tDO1lEND+9QvzN8zOMGkEukjRDnrDloSiTBBUp/k9GQnOGcmYJZVrYrIRNqKYMbUvl/5XQrde8y1r97qrabKzqKMEpnMEFeHANTbiFFnSAgYIHeIJnB51H58V5XY6uOaudE/gG5+0Tj96RZw==</latexit> well on many unseen domains while there is only one single S T T <latexit sha1_base64="LSxFJVXUHR05Oiam/FPFRqm/rDc=">AAACF3icbVDLSsNAFJ3UV62vqEs3g0VwFZJafICLghuXFfuCJoTJdNIOnTyYmQgl5C/c+CtuXCjiVnf+jZM2FGs9MHDmnHu59x4vZlRI0/zWSiura+sb5c3K1vbO7p6+f9ARUcIxaeOIRbznIUEYDUlbUslIL+YEBR4jXW98k/vdB8IFjcKWnMTECdAwpD7FSCrJ1Q07QHKEEUvvM2hjFMO50MpcCq8X/65eNQ1zCrhMrIJUQYGmq3/ZgwgnAQklZkiIvmXG0kkRlxQzklXsRJAY4TEakr6iIQqIcNLpXRk8UcoA+hFXL5Rwqv7uSFEgxCTwVGW+pPjr5eJ/Xj+R/qWT0jBOJAnxbJCfMCgjmIcEB5QTLNlEEYQ5VbtCPEIcYamirExDuMpxPj95mXRqhnVm1O/q1UatiKMMjsAxOAUWuAANcAuaoA0weATP4BW8aU/ai/aufcxKS1rRcwgWoH3+ACP7oAA=</latexit> < S \ Ti Ti (a) <latexit sha1_base64="VfNyC2imCxD1BgaUZyVUbJ54L1M=">AAACE3icbVDLSsNAFL2pr1pfUZduBosgLkpSBV24KLhxWbEvaEKZTKft0MmDmYlQQv7Bjb/ixoUibt2482+ctEFq64GBM+fcy733eBFnUlnWt1FYWV1b3yhulra2d3b3zP2DlgxjQWiThDwUHQ9LyllAm4opTjuRoNj3OG1745vMbz9QIVkYNNQkoq6PhwEbMIKVlnrmmeNjNSKYJ/cpcgiO0K/QSNH1/K9nlq2KNQVaJnZOypCj3jO/nH5IYp8GinAsZde2IuUmWChGOE1LTixphMkYD2lX0wD7VLrJ9KYUnWiljwah0C9QaKrOdyTYl3Lie7oyW1Euepn4n9eN1eDKTVgQxYoGZDZoEHOkQpQFhPpMUKL4RBNMBNO7IjLCAhOlYyzpEOzFk5dJq1qxzyvVu4tyrZrHUYQjOIZTsOESanALdWgCgUd4hld4M56MF+Pd+JiVFoy85xD+wPj8AdXNnhQ=</latexit> < (b) i j > i/j (c) > <latexit sha1_base64="o8CWxFnysqvXJrfmnzd1HWVYr5Q=">AAACHXicbVDLSsNAFJ3UV62vqEs3g0VwVdNafGyk4MZlRfuAJoTJdNJOO3kwMxFKyI+48VfcuFDEhRvxb5ykQVrrgYEz59zLvfc4IaNCGsa3VlhaXlldK66XNja3tnf03b22CCKOSQsHLOBdBwnCqE9akkpGuiEnyHMY6Tjj69TvPBAuaODfy0lILA8NfOpSjKSSbL1uekgOMWLxXWJTaOIohLPSCF7N/WN6MkpsvWxUjAxwkVRzUgY5mrb+afYDHHnEl5ghIXpVI5RWjLikmJGkZEaChAiP0YD0FPWRR4QVZ9cl8EgpfegGXD1fwkyd7YiRJ8TEc1Rluqj466Xif14vku6FFVM/jCTx8XSQGzEoA5hGBfuUEyzZRBGEOVW7QjxEHGGpAi1lIVymOPs9eZG0a5XqaaV+Wy83ankcRXAADsExqIJz0AA3oAlaAINH8AxewZv2pL1o79rHtLSg5T37YA7a1w8lyKKq</latexit> i j i/j domain available for training. We propose a new method S \ T T S [ S S T<latexit sha1_base64="buX0a3t6uP+MD+u5UA7rCJWfN/g=">AAACHXicbVDLSsNAFJ3UV62vqEs3g0VwVZNafGyk4MZlhb6gCWEynbTTTiZhZiKUkB9x46+4caGICzfi35i0QVrrgYEz59zLvfe4IaNSGca3VlhZXVvfKG6WtrZ3dvf0/YO2DCKBSQsHLBBdF0nCKCctRRUj3VAQ5LuMdNzxbeZ3HoiQNOBNNQmJ7aMBpx7FSKWSo9csH6khRixuJg6FFo5COC+N4M3CP6Zno8TRy0bFmAIuEzMnZZCj4eifVj/AkU+4wgxJ2TONUNkxEopiRpKSFUkSIjxGA9JLKUc+kXY8vS6BJ6nSh14g0scVnKrzHTHypZz4blqZLSr/epn4n9eLlHdlx5SHkSIczwZ5EYMqgFlUsE8FwYpNUoKwoOmuEA+RQFilgZamIVxnuPg9eZm0qxXzvFK7r5Xr1TyOIjgCx+AUmOAS1MEdaIAWwOARPINX8KY9aS/au/YxKy1oec8hWID29QMqnKKt</latexit> [ T T named adversarial domain augmentation to solve this Out- Figure 1. The domain discrepancy: (a) domain adaptation, (b) do- of-Distribution (OOD) generalization problem. The key main generalization, and (c) single domain generalization. idea is to leverage adversarial training to create “fictitious” yet “challenging” populations, from which a model can learn to generalize with theoretical guarantees. To facilitate various success in tackling ordinary domain discrepancy is- fast and desirable domain augmentation, we cast the model sue, however, we argue that existing methods can hardly training in a meta-learning scheme and use a Wasserstein succeed in the aforementioned single domain generalization Auto-Encoder (WAE) to relax the widely used worst-case problem. As illustrated in Fig. 1, the former usually expects constraint. Detailed theoretical analysis is provided to tes- the availability of target domain data (either labeled or unla- tify our formulation, while extensive experiments on multi- beled); While the latter, on the other hand, always assumes ple benchmark datasets indicate its superior performance multiple (rather than one) domains are available for train- in tackling single domain generalization. ing. This fact emphasizes the necessity to develop a new learning paradigm for single domain generalization. In this paper, we propose adversarial domain augmenta- 1. Introduction tion (Sec. 3.1) to solve this challenging task. Inspired by the recent success of adversarial training [35, 50, 49, 36, 24], Recent years have witnessed rapid deployment of ma- we cast the single domain generalization problem in a chine learning models for broad applications [17, 42,