Error correction code is a major part of the physical communication layer, ensuring the reliable transfer of data over noisy channels. Recently, neural decoders were shown to outperform classical decoding techniques. In this talk, we will present recent neural decoders based on Transformers and Diffusion models able to outperform existing neural decoders by very large margins.

Yoni Choukroun is a Ph.D. candidate at TAU under the supervision of Prof. Lior Wolf. He holds a B.Sc. degree in Computer Engineering (magna cum laude) and an M.Sc. degree in Computer Science from the Technion - Israel Institute of Technology. His research focuses on deep learning, computer vision, and numerical optimization.

Foundational work in machine learning theory has long established the relationship between and generalization and the (in)capacity to memorize randomly assigned labels. The link between the two remains even in the deep learning era as we confront models capable (given sufficient training time) of memorizing arbitrary labels. In this talk I’ll introduce several works centered on the theme of memorization and label noise, including (i) a work that suggests that if at some point along a training run we have high accuracy on clean training data but haven’t memorized the (incorrect) noisy labels, then it must be the case that we have already generalized to the underlying population. I’ll go on to discuss some empirical work aimed at empirically characterizing the phenomenon of memorization deep learning, including efforts to identify mislabeled points and to identify the neural network structures implicated in memorization.

Zachary Chase Lipton is an Assistant Professor of Machine Learning and Operations Research at Carnegie Mellon University and a Visiting Scientist at Amazon AI. He directs the Approximately Correct Machine Intelligence (ACMI) lab, whose research focuses including the theoretical and engineering foundations of robust and adaptive machine learning algorithms, applications to both prediction and decision-making problems in clinical medicine, natural language processing, and the impact of machine learning systems on society. A key theme in his current work is to take advantage of causal structure underlying the observed data while dealing with the messy high-dimensional data that typifies deep learning settings. He is the founder of the Approximately Correct blog (approximatelycorrect.com) and a co-author of Dive Into Deep Learning, an interactive open-source book drafted entirely through Jupyter notebooks. He can be found on Twitter (@zacharylipton), GitHub (@zackchase), or his lab's website (acmilab.org). 

AI is paving the way in academic research and innovation breakthroughs in the industry. Yet, much of the workload is cumbersome or scattered over siloed environments, making it hard to utilize and scale compute resources for Machine Learning development.

In this talk we'll discuss best practices for architecting, managing, and maintaining computing infrastructure for AI research in the next years to come.

Run:ai co-founder and CTO Ronen Dar was previously a research scientist at Bell Labs and has worked at Apple and in Intel in multiple R&D roles. As CTO, Ronen manages research and product roadmap for Run:ai, a startup he co-founded in 2018. Ronen is the co-author of many patents in the fields of storage, coding, and compression. Ronen received his B.S., M.S. and Ph.D. degrees from Tel Aviv University.
 
 

Dynamic difficulty adjustment (DDA) is a process of automatically changing a game difficulty for the optimization of user experience. It is a vital part of almost any modern game. Most existing DDA approaches concentrate on the experience of a player without looking at the rest of the players. We propose a method that automatically optimizes user experience while taking into consideration other players and macro constraints imposed by the game. The method is based on deep neural network architecture that involves a count loss constraint that has zero gradients in most of its support. We suggest a method to optimize this loss function and provide theoretical analysis for its performance. Finally, we provide empirical results of an internal experiment that was done on 200, 000 players and was found to outperform the corresponding manual heuristics crafted by game design experts

A scientist, inventor, and technology visionary, Michael is passionate about making life better with technology and particularly AI. He has almost twenty years of experience in developing and leading successful projects in diverse technological areas. He co-founded several companies in the past and holds a Ph.D. in EE from Technion - Israel Institue of Technology. Currently, Michael is leading the AI research activities in Playtika - a world leader in mobile games. Playtika made AI the focus of the company's growth strategy and AI touches every aspect of its games.

We present a method to overcome a common challenge in deep learning: efficiently handling dynamically changing inputs. Consider for instance an AI writing assistant making suggestions as the document is edited, or an agent evaluating which of the next moves will lead to the most favorable board game position. Conventional neural networks must fully process each input, even if they differ by a small amount. We propose a novel deep learning architecture that supports incremental computation, allowing it to continuously update predictions as inputs change while only partially recalculating its computational graph. We demonstrate our method through its application to one of the most challenging problems in quantum physics: finding the ground state of quantum many-body systems. The critical step of neural network-based approaches to such problems is calculating physical quantities known as “local energies”, which involves repeated evaluations of the spin particles under small changes. By leveraging our incrementally computable neural network, we observe a linear acceleration (growing with the number of simulated particles), achieving >10X reduction in computational resources.

Or graduated from the Hebrew University of Jerusalem with a B.Sc. in physics, mathematics, and computer science, before completing his Ph.D. in machine learning under the supervision of Prof. Amnon Shashua. His work ranges from fundamental theoretical questions in machine learning, to practical applications in various domains, including computer vision and natural language understanding. In 2021 he joined Caltech, working jointly with Prof. Anima Anandkumar and Prof. Garnet Chan on methods at the intersection of machine learning and quantum many-body problems.

ML models are not trustworthy often because they focus too much on "what" than "how". That is, they care only about whether they are solving the task at hand ("what") but not so much about solving it right ("how"). Having recognised this issue, the ML field has been shifting its focus from "what" to "how" for the last five years. Arguably, the most common approach to address "how" is to extend the familiar *benchmarking approach* that used to work well for the "what" phase: build a benchmark dataset and perform "fair" comparisons by fixing the allowed ingredients. This encourages more and more complex tricks that are likely to simply overfit to the given benchmark (e.g. ImageNet). However, for the "how" problem, I believe it is more important to transfer the way humans recognise the world (human explanations) into the computational domain and eventually utilise it for model training. I will give an overview of my previous search for such ingredients that make models more explainable and more robust to distribution shifts. I will then discuss exciting future sources of such ingredients.

Seong Joon Oh has been an independent group leader at the University of Tübingen leading the group on Scalable Trustworthy AI (STAI) since 2022. He is interested in training trustworthy models (e.g. explainable, robust, and probabilistic models) and obtaining the necessary human supervision and guidance in a cost-effective way. He has been a research scientist at Naver AI Lab (2018-2022). He did his PhD in computer vision and machine learning at MPI-INF with Bernt Schiele and Mario Fritz (2014-2018). He did his masters and bachelors studies in Maths at the University of Cambridge (2010-2014).

Graph neural networks (GNNs) are widely used for modeling complex interactions between entities represented as vertices of a graph.  Despite recent efforts to theoretically analyze the expressive power of GNNs, a formal characterization of their ability to model interactions is lacking.  My talk will present a recent work aiming to address this gap.  Formalizing strength of interactions through an established measure known as separation rank, I will quantify the ability of certain GNNs to model interaction between sides of a given partition of vertices.  The result will reveal that the ability to model interaction is primarily determined by the partition’s walk index — a graph-theoretical characteristic defined by the number of walks originating from the boundary of the partition.  As a practical application of the theory, I will present an edge sparsification algorithm named Walk Index Sparsification (WIS), which preserves the ability of a GNN to model interactions when edges are removed.  WIS is simple, computationally efficient, and markedly outperforms alternative methods in terms of induced prediction accuracy. More broadly, it showcases the potential of improving GNNs by theoretically analyzing the interactions they model.

The presented work was in collaboration with my students Noam Razin and Tom Verbin.
 

Nadav Cohen is an Asst. Professor of Computer Science at Tel Aviv University, and Chief Scientist at Imubit. His academic research revolves around the theoretical and algorithmic foundations of deep learning, while at Imubit he leads the development of deep reinforcement learning systems controlling industrial manufacturing lines. Nadav earned a BSc in electrical engineering and a BSc in mathematics (both summa cum laude) at the Technion Excellence Program for Distinguished Undergraduates. He obtained his PhD (direct track, summa cum laude) at the Hebrew University, and was subsequently a postdoctoral scholar at the Institute for Advanced Study in Princeton. For his contributions to deep learning, Nadav won a number of awards, including the Google Research Scholar Award, the Google Doctoral Fellowship in Machine Learning, the Final Prize for Machine Learning Research, the Rothschild Postdoctoral Fellowship, the Zuckerman Postdoctoral Fellowship, and TheMarker's 40 under 40 list.

Consider a finite sample from an unknown distribution over a countable alphabet. Unobserved events are alphabet symbols which do not appear in the sample. Estimating the probabilities of unobserved events is a basic problem in statistics and related fields, which was extensively studied in the context of point estimation. In this work we introduce a novel interval estimation scheme for unobserved events. Our proposed framework applies selective inference, as we construct confidence intervals (CIs) for the desired set of parameters. Interestingly, we show that obtained CIs are dimension-free, as they do not grow with the alphabet size. Further, we show that our CIs are (almost) tight, in the sense that they cannot be further improved without violating the prescribed coverage rate. We demonstrate the performance of our proposed scheme in synthetic and real-world experiments, showing a significant improvement over alternatives.

Dr. Amichai Painsky is an Assistant Professor at the Industrial Engineering Department of Tel Aviv University. Amichai received his B.Sc. in Electrical Engineering from Tel Aviv University (2007), his Masters in Electrical Engineering from Princeton University (2009) and his Ph.D. in Statistics from Tel Aviv University (2016). Following his graduation, Amichai joined Massachusetts Institute of Technology (MIT) as a postdoctoral fellow (2019). Amichai’s research focuses on statistical inference and learning, and their connection to information theory.