{"669260":{"#nid":"669260","#data":{"type":"event","title":"CSIP Seminar: Attaining Sparsity in Large Language Models: Is It Easy or Hard?","body":[{"value":"\u003Ch3\u003E\u003Cstrong\u003ECenter for Signals and Information Processing (CSIP)\u0026nbsp;Seminar\u003C\/strong\u003E\u003C\/h3\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EDate:\u003C\/strong\u003E\u0026nbsp;Friday, September 8,\u0026nbsp;2023\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ETime:\u003C\/strong\u003E\u0026nbsp;2:30 p.m. - 3:30 p.m. EST\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ELocation:\u0026nbsp;\u003C\/strong\u003ECentergy Building 5126.\u0026nbsp;The associated zoom link is:\u0026nbsp;\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Ca href=\u0022https:\/\/gatech.zoom.us\/j\/91428447481\u0022\u003Ehttps:\/\/gatech.zoom.us\/j\/91428447481\u003C\/a\u003E.\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ESpeaker:\u0026nbsp;\u003C\/strong\u003EZhangyang \u201cAtlas\u201d Wang\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ESpeakers\u0027 Title:\u003C\/strong\u003E\u0026nbsp;Associate Professor in the VITA Research Group at the University of Texas at Austin\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ESeminar Title:\u0026nbsp;\u003C\/strong\u003EAttaining Sparsity in Large Language Models: Is It Easy or Hard?\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003EAbstract:\u0026nbsp;\u003C\/strong\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003E\u003Cspan\u003EIn the realm of contemporary deep learning, large pre-trained transformers have seized the spotlight. Understanding the underlying frugal structures within these burgeoning models has become imperative. Although the tools of sparsity, like pruning, the lottery ticket hypothesis, and sparse training, have enjoyed popularity and success in traditional deep networks, their efficacy in the new era of colossal pre-trained models, such as Large Language Models (LLMs), remains uncertain. This presentation aims to elucidate two contradictory perspectives. On one hand, we explore the notion that compressing LLMs is \u0022easier\u0022 compared to earlier deep models; but on the other hand, we delve into the aspects that make this endeavor \u0022harder\u0022 in its own unique way. My goal is to convince you that I am indeed not contradicting myself :)\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/span\u003E\u003C\/p\u003E\r\n\r\n\u003Cp\u003E\u003Cstrong\u003ESpeaker Bio:\u003C\/strong\u003E\u0026nbsp;Atlas Wang (\u003Ca href=\u0022https:\/\/vita-group.github.io\/\u0022\u003Ehttps:\/\/vita-group.github.io\/\u003C\/a\u003E) teaches and researches at UT Austin ECE (primary), CS, and Oden CSEM. He usually declares his research interest as machine learning but is never too sure what that means concretely. He has won some awards, but is mainly proud of three things: (1) he has done some (hopefully) thought-invoking and practically meaningful work on sparsity, from inverse problems to deep learning; his recent favorites include essential sparsity (\u003Ca href=\u0022https:\/\/arxiv.org\/abs\/2306.03805\u0022\u003Ehttps:\/\/arxiv.org\/abs\/2306.03805\u003C\/a\u003E), heavy-hitter oracle (\u003Ca href=\u0022https:\/\/arxiv.org\/abs\/2306.14048\u0022\u003Ehttps:\/\/arxiv.org\/abs\/\/2306.14048\u003C\/a\u003E), and sparsity-may-cry (\u003Ca href=\u0022https:\/\/openreview.net\/forum?id=J6F3lLg4Kdp\u0022\u003Ehttps:\/\/openreview.net\/forum?id=J6F3lLg4Kdp\u003C\/a\u003E); (2) he co-founded the Conference on Parsimony and Learning (CPAL), known as the new \u0022 conference for sparsity\u0022 to its community, and serves as its inaugural program chair (\u003Ca href=\u0022https:\/\/cpal.cc\/\u0022\u003Ehttps:\/\/cpal.cc\/\u003C\/a\u003E); (3) he is fortunate enough to work with a sizable group of world-class students, all smarter than himself. He has so far graduated 13 Ph.D. students and postdocs that are well placed, including three (assistant) professors; and his students have altogether won seven prestigious PhD fellowships (NSF GRFP, IBM, Apple, Adobe, Amazon, Qualcomm, and Snap), among many other honors.\u003C\/p\u003E\r\n","summary":"","format":"limited_html"}],"field_subtitle":"","field_summary":[{"value":"\u003Cp\u003EZhangyang \u201cAtlas\u201d Wang, an associate professor in the VITA Research Group at the University of Texas at Austin will present the CSIP Seminar, \u0022Attaining Sparsity in Large Language Models: Is It Easy or Hard?\u0022 on September 8, 2023.\u003C\/p\u003E\r\n","format":"basic_html"}],"field_summary_sentence":[{"value":"Featuring Zhangyang \u201cAtlas\u201d Wang, Associate Professor from the University of Texas at Austin"}],"uid":"36172","created_gmt":"2023-08-30 17:25:56","changed_gmt":"2023-09-01 20:13:15","author":"dwatson71","boilerplate_text":"","field_publication":"","field_article_url":"","field_event_time":{"event_time_start":"2023-09-08T14:30:00-04:00","event_time_end":"2023-09-08T15:30:00-04:00","event_time_end_last":"2023-09-08T15:30:00-04:00","gmt_time_start":"2023-09-08 18:30:00","gmt_time_end":"2023-09-08 19:30:00","gmt_time_end_last":"2023-09-08 19:30:00","rrule":null,"timezone":"America\/New_York"},"location":"Centergy Building 5126","extras":[],"groups":[{"id":"1255","name":"School of Electrical and Computer Engineering"}],"categories":[],"keywords":[{"id":"192224","name":"CSIP Seminar"}],"core_research_areas":[],"news_room_topics":[],"event_categories":[{"id":"1795","name":"Seminar\/Lecture\/Colloquium"}],"invited_audience":[{"id":"78761","name":"Faculty\/Staff"},{"id":"78771","name":"Public"},{"id":"78751","name":"Undergraduate students"}],"affiliations":[],"classification":[],"areas_of_expertise":[],"news_and_recent_appearances":[],"phone":[],"contact":[{"value":"\u003Cp\u003EGhazal Kaviani\u003Cbr \/\u003E\r\n\u003Ca href=\u0022gkaviani3@gatech.edu\u0022\u003Egkaviani3@gatech.edu\u003C\/a\u003E\u003C\/p\u003E\r\n","format":"limited_html"}],"email":[],"slides":[],"orientation":[],"userdata":""}}}