{"607617":{"#nid":"607617","#data":{"type":"news","title":"Georgia Tech Solves \u0027Texture Fill\u0027 Problem with Machine Learning","body":[{"value":"\u003Cp\u003EA new machine learning technique developed at Georgia Tech may soon give budding fashionistas and other designers the freedom to create realistic, high-resolution visual content without relying on complicated 3-D rendering programs.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Ca href=\u0022http:\/\/texturegan.eye.gatech.edu\/\u0022\u003ETextureGAN\u003C\/a\u003E is the first deep image synthesis method that can realistically spread multiple textures across an object. With this new approach, users drag one or more texture patches onto a sketch \u0026mdash; say of a handbag or a skirt \u0026mdash;\u0026nbsp;and the network texturizes the sketch to accurately account for 3-D surfaces and lighting.\u003C\/p\u003E\r\n\r\n\u003Ch5\u003E\u003Ca href=\u0022https:\/\/youtu.be\/bCBDPfWdpDc\u0022 target=\u0022_blank\u0022\u003E[VIDEO: See\u0026nbsp;TextureGAN\u0026nbsp;in action]\u003C\/a\u003E\u003C\/h5\u003E\r\n\r\n\u003Cp\u003EPrior to this work, producing realistic images of this kind could be tedious and time-consuming, particularly for those with limited experience. And, according to the researchers, existing machine learning-based methods are not particularly good at generating high-resolution texture details.\u003C\/p\u003E\r\n\r\n\u003Ch4\u003E\u003Cstrong\u003EUsing a neural network to improve results\u003C\/strong\u003E\u003C\/h4\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;The \u0026lsquo;texture fill\u0026rsquo; operation is difficult for a deep network to learn because it not only has to propagate the color, but also has to learn how to synthesize the structure of texture across 3-D shapes,\u0026rdquo; said \u003Cstrong\u003EWenqi Xian\u003C\/strong\u003E, computer science (CS) major and co-lead developer.\u003C\/p\u003E\r\n\r\n\u003Ch5\u003E\u003Ca href=\u0022https:\/\/youtu.be\/XWr0Fg5XbPs?t=1h32m44s\u0022 target=\u0022_blank\u0022\u003E[VIDEO:\u0026nbsp;Wenqi\u0026nbsp;Xian presents TextureGAN at CVPR\u0026nbsp;2018]\u003C\/a\u003E\u003C\/h5\u003E\r\n\r\n\u003Cp\u003EThe researchers initially trained a type of neural network called a conditional generative adversarial network (GAN) on sketches and textures extracted from thousands of ground-truth photographs. In this approach,\u0026nbsp;a generator neural network creates images that a discriminator neural network then evaluates for accuracy. The goal is for both to get increasingly better at their respective tasks, which leads to more realistic outputs.\u003C\/p\u003E\r\n\r\n\u003Cp\u003ETo ensure that the results look as realistic as possible, researchers fine-tuned the new system to minimize pixel-to-pixel style differences between generated images and training data. But the results were not quite what the team had expected.\u003C\/p\u003E\r\n\r\n\u003Ch4\u003E\u003Cstrong\u003EProducing more realistic images\u003C\/strong\u003E\u003C\/h4\u003E\r\n\r\n\u003Cp\u003E\u0026ldquo;We realized that we needed a stronger constraint to preserve high-level texture in our outputs,\u0026rdquo; said Georgia Tech CS Ph.D. student \u003Cstrong\u003EPatsorn Sangkloy\u003C\/strong\u003E. \u0026ldquo;That\u0026rsquo;s when we developed an additional discriminator network that we trained on a separate texture dataset. Its only job is to be presented with two samples and ask \u0026lsquo;are these the same or not?\u0026rsquo;\u0026rdquo;\u003C\/p\u003E\r\n\r\n\u003Cp\u003EWith its sole focus on a single question, this type of discriminator is much harder to fool. This, in turn, leads the generator to produce images that are not only realistic, but also true to the texture patch the user placed onto the sketch.\u003C\/p\u003E\r\n\r\n\u003Cp\u003EThe work was presented in June at the conference on\u0026nbsp;\u003Ca href=\u0022http:\/\/cvpr2018.thecvf.com\/\u0022 target=\u0022_blank\u0022\u003EComputer Vision and Pattern Recognition (CVPR) 2018\u003C\/a\u003E held in Salt Lake City and is funded through National Science Foundation award 1561968. \u003Ca href=\u0022https:\/\/www.ic.gatech.edu\/\u0022\u003ESchool of Interactive Computing\u003C\/a\u003E Associate Professor \u003Cstrong\u003EJames Hays\u003C\/strong\u003E advises Xian and Sangkloy. Georgia Tech is collaborating on this research with Adobe Research, University of California at Berkeley, and Argo AI.\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":"","field_summary_sentence":[{"value":"A new technique allows users to spread textures across sketches of objects to create high resolution images."}],"uid":"32045","created_gmt":"2018-07-10 17:52:19","changed_gmt":"2018-07-12 14:29:39","author":"Ben Snedeker","boilerplate_text":"","field_publication":"","field_article_url":"","dateline":{"date":"2018-07-10T00:00:00-04:00","iso_date":"2018-07-10T00:00:00-04:00","tz":"America\/New_York"},"extras":[],"hg_media":{"607631":{"id":"607631","type":"image","title":"Georgia Tech Using Machine Learning to Solve Texture Fill Problem","body":null,"created":"1531254179","gmt_created":"2018-07-10 20:22:59","changed":"1531254179","gmt_changed":"2018-07-10 20:22:59","alt":"Georgia Tech Using Machine Learning to Solve Texture Fill Problem","file":{"fid":"231786","name":"zebra-texture-11297063007KgE.jpg","image_path":"\/sites\/default\/files\/images\/zebra-texture-11297063007KgE.jpg","image_full_path":"http:\/\/tlwarc.hg.gatech.edu\/\/sites\/default\/files\/images\/zebra-texture-11297063007KgE.jpg","mime":"image\/jpeg","size":462426,"path_740":"http:\/\/tlwarc.hg.gatech.edu\/sites\/default\/files\/styles\/740xx_scale\/public\/images\/zebra-texture-11297063007KgE.jpg?itok=zZkrw4ag"}}},"media_ids":["607631"],"groups":[{"id":"47223","name":"College of Computing"},{"id":"576481","name":"ML@GT"},{"id":"50876","name":"School of Interactive Computing"},{"id":"1299","name":"GVU Center"}],"categories":[],"keywords":[{"id":"9167","name":"machine learning"},{"id":"178516","name":"texture GAN"},{"id":"178517","name":"neural network"},{"id":"109581","name":"deep learning"}],"core_research_areas":[{"id":"39501","name":"People and Technology"}],"news_room_topics":[{"id":"71881","name":"Science and Technology"}],"event_categories":[],"invited_audience":[],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EAlbert Snedeker, Communications Manager\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Ca href=\u0022mailto:albert.snedeker@cc.gatech.edu?subject=Texture%20Patch\u0022\u003Ealbert.snedeker@cc.gatech.edu\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"email":["albert.snedeker@cc.gatech.edu"],"slides":[],"orientation":[],"userdata":""}}}