{"618695":{"#nid":"618695","#data":{"type":"event","title":"Modern Statistical Theory Inspired by Deep Learning","body":[{"value":"\u003Cp\u003E\u003Cstrong\u003ETitle: \u003C\/strong\u003EModern Statistical Theory Inspired by Deep Learning\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EAbstract:\u0026nbsp;\u003C\/strong\u003EModern learning algorithms, such as deep learning, have gained great successes in real applications. However, some\u0026nbsp;of their empirical\u0026nbsp;behaviors\u0026nbsp;may not\u0026nbsp;be interpreted within the\u0026nbsp;classical statistical learning framework. For example, deep learning algorithms achieve small testing error even when the training error is zero, i.e., over-fitting. Another phenomenon is observed in\u0026nbsp;image recognition applications\u0026nbsp;where\u0026nbsp;a hardly noticeable change of data may lead to a dramatic increase\u0026nbsp;in misclassification rates. Inspired by these observations,\u0026nbsp;we attempt\u0026nbsp;to illustrate\u0026nbsp;new theoretical\u0026nbsp;insights for data-interpolation and adversarial testing using the very simple\u0026nbsp;nearest neighbor algorithms. In particular,\u0026nbsp;we prove statistical optimality\u0026nbsp;of interpolated nearest neighbor algorithms. More surprisingly, it is discovered that the classification performance, under a proper interpolation, is even\u0026nbsp;better than the best kNN in terms of multiplicative constant. As for adversarial testing, we demonstrate that different adversarial mechanisms lead to different\u0026nbsp;phase\u0026nbsp;transition phenomena of\u0026nbsp;the misclassification rate in terms of its upper bound. Additionally, our technical\u0026nbsp;analysis invented to deal with adversarial samples\u0026nbsp;can also be applied to other variants\u0026nbsp;of kNN, e.g. pre-processed 1NN and distributed-NN.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EBio: \u003C\/strong\u003EGuang Cheng is a Professor of Statistics at Purdue University. \u0026nbsp;He received his Ph.D. in Statistics from the University of Wisconsin-Madison in 2006. \u0026nbsp;His research interests include Big Data and High Dimensional Statistical Inferences, and more recently turned to Deep Learning and Reinforcement Learning. \u0026nbsp;Cheng is the recipient of\u0026nbsp;the NSF CAREER award, Noether Young Scholar Award and Simons Fellowship in Mathematics. Please visit his big data theory research group at\u0026nbsp;\u003Ca href=\u0022http:\/\/www.science.purdue.edu\/bigdata\/\u0022\u003Ehttp:\/\/www.science.purdue.edu\/bigdata\/\u003C\/a\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n","summary":null,"format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003E\u003Cstrong\u003EAbstract:\u0026nbsp;\u003C\/strong\u003EModern learning algorithms, such as deep learning, have gained great successes in real applications. However, some\u0026nbsp;of their empirical\u0026nbsp;behaviors\u0026nbsp;may not\u0026nbsp;be interpreted within the\u0026nbsp;classical statistical learning framework. For example, deep learning algorithms achieve small testing error even when the training error is zero, i.e., over-fitting. Another phenomenon is observed in\u0026nbsp;image recognition applications\u0026nbsp;where\u0026nbsp;a hardly noticeable change of data may lead to dramatic increase\u0026nbsp;of mis-classification rates. Inspired by these observations,\u0026nbsp;we attempt\u0026nbsp;to illustrate\u0026nbsp;new theoretical\u0026nbsp;insights for data-interpolation and adversarial testing using the very simple\u0026nbsp;nearest neighbor algorithms. In particular,\u0026nbsp;we prove statistical optimality\u0026nbsp;of interpolated nearest neighbor algorithms. More surprisingly, it is discovered that the classification performance, under a proper interpolation, is even\u0026nbsp;better that the best kNN in terms of multiplicative constant. As for adversarial testing, we demonstrate that different adversarial mechanisms lead to different\u0026nbsp;phase\u0026nbsp;transition phenomena of\u0026nbsp;mis-classification rate in terms of its upper bound. Additionally, our technical\u0026nbsp;analysis invented to deal with adversarial samples\u0026nbsp;can also be applied to other variants\u0026nbsp;of kNN, e.g. pre-processed 1NN and distributed-NN.\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u0026nbsp;\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EBio: \u003C\/strong\u003EGuang Cheng is a Professor of Statistics at Purdue University. \u0026nbsp;He received his PhD in Statistics from University of Wisconsin-Madison in 2006. \u0026nbsp;His research interests include Big Data and High Dimensional Statistical Inferences, and more recently turn to Deep Learning and Reinforcement Learning. \u0026nbsp;Cheng is the recipient of\u0026nbsp;the NSF CAREER award, Noether Young Scholar Award and Simons Fellowship in Mathematics. Please visit his big data theory research group at\u0026nbsp;\u003Ca href=\u0022http:\/\/www.science.purdue.edu\/bigdata\/\u0022\u003Ehttp:\/\/www.science.purdue.edu\/bigdata\/\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"field_summary_sentence":[{"value":"Modern Statistical Theory Inspired by Deep Learning"}],"uid":"34963","created_gmt":"2019-03-04 01:52:56","changed_gmt":"2019-03-04 01:58:09","author":"Xiaoming Huo","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2019-03-06T13:30:00-05:00","event_time_end":"2019-03-06T14:30:00-05:00","event_time_end_last":"2019-03-06T14:30:00-05:00","gmt_time_start":"2019-03-06 18:30:00","gmt_time_end":"2019-03-06 19:30:00","gmt_time_end_last":"2019-03-06 19:30:00","rrule":null,"timezone":"America\/New_York"},"extras":[],"groups":[{"id":"602673","name":"TRIAD "}],"categories":[],"keywords":[{"id":"109581","name":"deep learning"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"177814","name":"Postdoc"},{"id":"78771","name":"Public"},{"id":"174045","name":"Graduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[],"email":[],"slides":[],"orientation":[],"userdata":""}}}