{"id":996,"date":"2023-09-29T15:22:23","date_gmt":"2023-09-29T08:22:23","guid":{"rendered":"https:\/\/vinuni.edu.vn\/research\/?post_type=event&p=996"},"modified":"2023-09-29T15:22:38","modified_gmt":"2023-09-29T08:22:38","slug":"research-seminar","status":"publish","type":"event","link":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/","title":{"rendered":"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity – Dr. Tan Nguyen, National University of Singapore"},"content":{"rendered":"

\"\"<\/p>\n\n\n\n\n\n\n
Seminar Title:<\/td>\nPrincipled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity<\/strong><\/td>\n<\/tr>\n
Speaker:<\/td>\nDr. Tan Nguyen – Assistant Professor of Mathematics at the National University of Singapore (NUS)<\/strong><\/td>\n<\/tr>\n
Time:<\/td>\nWednesday, July 12, 2023 | 3:00 \u2013 4:00 PM<\/strong><\/td>\n<\/tr>\n
Venue:<\/td>\nC204, VinUniversity<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

\u00a0<\/strong><\/p>\n

About the Seminar
\n<\/strong>Designing deep learning models for practical applications, including those in computer vision, natural language processing, and mathematical modeling, is an art that often involves an expensive search over candidate architectures. In this talk, I present novel frameworks to facilitate the process of designing efficient and robust deep learning models with better expressivity via three principled approaches: optimization, differential equation, and statistical modeling.<\/p>\n

From an optimization viewpoint, I leverage the continuous limit of the classical momentum accelerated gradient descent to improve Neural ODEs training and inference. The resulting Momentum Neural ODEs accelerate both forward and backward ODE solvers, as well as alleviate the vanishing gradient problem (Efficiency).<\/p>\n

From a differential equation approach, I present a random walk interpretation of graph neural networks (GNNs), revealing a potentially inevitable over-smoothing phenomenon. Based on this random walk viewpoint of GNNs, I then propose the graph neural diffusion with a source term (GRAND++) that overcomes the over-smoothing issue and achieves better accuracy in low-labeling rate regimes (Robustness).<\/p>\n

Using statistical modeling as a tool, I show that the attention in transformer models can be derived from solving a nonparametric kernel regression problem. I then propose the FourierFormer, a new class of transformers in which the softmax kernels are replaced by the novel generalized Fourier integral kernels. The generalized Fourier integral kernels can automatically capture the dependency of the features of data and remove the need to tune the covariance matrix (Expressivity).<\/p>\n

About the Speaker
\n<\/strong>Dr. Tan Nguyen is currently an Assistant Professor of Mathematics at the National University of Singapore (NUS). Before joining NUS, he was a postdoctoral scholar in the Department of Mathematics at the University of California, Los Angeles, working with Dr. Stanley J. Osher. He obtained his Ph.D. in Machine Learning from Rice University, where he was advised by Dr. Richard G. Baraniuk. Dr. Nguyen is an organizer of the 1st Workshop on Integration of Deep Neural Models and Differential Equations at ICLR 2020. He also had two awesome long internships with Amazon AI and NVIDIA Research. He is the recipient of the prestigious Computing Innovation Postdoctoral Fellowship (CIFellows) from the Computing Research Association (CRA), the NSF Graduate Research Fellowship, and the IGERT Neuroengineering Traineeship. He received his M.S. and B.S. in Electrical and Computer Engineering from Rice University in May 2018 and May 2014, respectively.<\/p>\n","protected":false},"excerpt":{"rendered":"

Seminar Title: Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity Speaker: Dr. Tan Nguyen – Assistant Professor of Mathematics at the National University of Singapore (NUS) Time: Wednesday, July 12, 2023 | 3:00 \u2013 4:00 PM Venue: C204, VinUniversity \u00a0 About the Seminar Designing deep learning models for practical applications, including those […]<\/p>\n","protected":false},"featured_media":997,"template":"","event_category":[19],"acf":[],"yoast_head":"\n[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website\" \/>\n<meta property=\"og:description\" content=\"Seminar Title: Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity Speaker: Dr. Tan Nguyen – Assistant Professor of Mathematics at the National University of Singapore (NUS) Time: Wednesday, July 12, 2023 | 3:00 \u2013 4:00 PM Venue: C204, VinUniversity \u00a0 About the Seminar Designing deep learning models for practical applications, including those […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/\" \/>\n<meta property=\"og:site_name\" content=\"Vinuni Research Website\" \/>\n<meta property=\"article:modified_time\" content=\"2023-09-29T08:22:38+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/vinuni.edu.vn\/research\/wp-content\/uploads\/2023\/09\/Research-Website-920-x-360-px.png\" \/>\n\t<meta property=\"og:image:width\" content=\"920\" \/>\n\t<meta property=\"og:image:height\" content=\"360\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/\",\"url\":\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/\",\"name\":\"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website\",\"isPartOf\":{\"@id\":\"https:\/\/vinuni.edu.vn\/research\/#website\"},\"datePublished\":\"2023-09-29T08:22:23+00:00\",\"dateModified\":\"2023-09-29T08:22:38+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity – Dr. Tan Nguyen, National University of Singapore\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/vinuni.edu.vn\/research\/#website\",\"url\":\"https:\/\/vinuni.edu.vn\/research\/\",\"name\":\"Vinuni Research Website\",\"description\":\"\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/vinuni.edu.vn\/research\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/","og_locale":"en_US","og_type":"article","og_title":"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website","og_description":"Seminar Title: Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity Speaker: Dr. Tan Nguyen – Assistant Professor of Mathematics at the National University of Singapore (NUS) Time: Wednesday, July 12, 2023 | 3:00 \u2013 4:00 PM Venue: C204, VinUniversity \u00a0 About the Seminar Designing deep learning models for practical applications, including those […]","og_url":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/","og_site_name":"Vinuni Research Website","article_modified_time":"2023-09-29T08:22:38+00:00","og_image":[{"width":920,"height":360,"url":"https:\/\/vinuni.edu.vn\/research\/wp-content\/uploads\/2023\/09\/Research-Website-920-x-360-px.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/","url":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/","name":"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity - Dr. Tan Nguyen, National University of Singapore - Vinuni Research Website","isPartOf":{"@id":"https:\/\/vinuni.edu.vn\/research\/#website"},"datePublished":"2023-09-29T08:22:23+00:00","dateModified":"2023-09-29T08:22:38+00:00","breadcrumb":{"@id":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/vinuni.edu.vn\/research\/event\/research-seminar\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"[Research Seminar] Principled Frameworks for Designing Deep Learning Models: Efficiency, Robustness, and Expressivity – Dr. Tan Nguyen, National University of Singapore"}]},{"@type":"WebSite","@id":"https:\/\/vinuni.edu.vn\/research\/#website","url":"https:\/\/vinuni.edu.vn\/research\/","name":"Vinuni Research Website","description":"","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/vinuni.edu.vn\/research\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/event\/996"}],"collection":[{"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/event"}],"about":[{"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/types\/event"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/media\/997"}],"wp:attachment":[{"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/media?parent=996"}],"wp:term":[{"taxonomy":"event_category","embeddable":true,"href":"https:\/\/vinuni.edu.vn\/research\/wp-json\/wp\/v2\/event_category?post=996"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}