optimization

Ahmed Youssef Ragab Radwan

Visiting Student (former), Information Science Lab
Visiting student at the Information Science Lab, in the Electrical Engineering Department, CEMSE, King Abdullah University of Science and Technology (KAUST)

Estimating the expected information gain in Bayesian optimal experimental design

-

B9 L2 H1

Optimal experimental design for parameter estimation is a fast-growing area of research. Let us consider the experimental goal to be the inference of some attributes of a complex system using measurement data of some chosen system responses, and the optimal designs are those that maximize the value of measurement data. The value of data is quantified by the expected information gain utility, which measures the informativeness of an experiment. Often, a mathematical model is used that approximates the relationship between the system responses and the model parameters acting as proxies for the attributes of interest.

Langevin Monte Carlo as an optimization algorithm

-

B9 L2 H1 R2322

Adil Salim is mainly interested in stochastic approximation, optimization, and machine learning. He is currently a Postdoctoral Research Fellow working with Professor Peter Richtarik at the Visual Computing Center (VCC) at King Abdullah University of Science and Technology (KAUST).

Dynamics and Convergence of Weight Normalization for Training Neural Networks

Guido Montufar, Assistant Professor, Departments of Mathematics and Statistics, University of California, Los Angeles (UCLA)

-

B1 L3 R3119

We present a result on the convergence of weight normalized training of artificial neural networks. In the analysis, we consider over-parameterized 2-layer networks with rectified linear units (ReLUs) initialized at random and trained with batch gradient descent and a fixed step size. The proof builds on recent theoretical works that bound the trajectory of parameters from their initialization and monitor the network predictions via the evolution of a ''neural tangent kernel'' (Jacot et al. 2018). We discover that training with weight normalization decomposes such a kernel via the so called ''length-direction decoupling''. This in turn leads to two convergence regimes. From the modified convergence we make a few curious observations including a natural form of ''lazy training'' where the direction of each weight vector remains stationary.

Quantitative understanding of transcription factor regulation at network- and molecular-levels through optimization and deep learning

-

KAUST

Transcription factors are an important family of proteins that control the transcription rate from DNAs to messenger RNAs through the binding to specific DNA sequences. Transcription factor regulation is thus fundamental to understanding not only the system-level behaviors of gene regulatory networks, but also the molecular mechanisms underpinning endogenous gene regulation. In this talk, I will introduce our efforts on developing novel optimization and deep learning methods to quantitatively understanding transcription factor regulation at network- and molecular-levels. Specifically, I will talk about how we estimate the kinetic parameters from sparse time-series readout of gene circuit models, and how we model the relationship between the transcription factor binding sites and their binding affinities.

Trajectory Planning for Autonomous Underwater Vehicles: A Stochastic Optimization Approach

-

KAUST

In this work, we develop a new framework of trajectory planning for AUVs in realistic ocean scenarios. We divide this work into three parts. In the first part, we provide a new approach for deterministic trajectory planning in steady current, described using Ocean General Circulation Model (OGCM) data. The latter are used to specify both the ocean current and the bathymetry. We apply a NLP to the optimal-time trajectory planning problem. To demonstrate the effectivity of the resulting model, we consider the optimal time trajectory planning of an AUV operating in the Red Sea and the Gulf of Aden. In the second part, we generalize our 3D trajectory planning framework to time-dependent ocean currents. We also extend the framework to accommodate multi-objective criteria, focusing specifically on the Pareto front curve between time and energy. The scheme is demonstrated for time-energy trajectory planning problems in the Gulf of Aden. In the last part, we address uncertainty in the ocean current field. The uncertainty in the current is described in terms of a finite ensemble of flow realizations. The proposed approach is based on a non-linear stochastic programming methodology that uses a risk-aware objective function, accounting for the full variability of the flow ensemble. Advanced visualization tools are used to amplify simulation results.

Powering the bottom line

1 min read ·
A risk-based optimization scheme boosts confidence and profitability for future mixed-technology power plants.

Accelerating the grapevine effect

1 min read ·
By looking at classical gossip algorithms from a novel perspective, KAUST Professor Peter Richtarik has found a way to significantly speed up gossip-based information sharing, and in the process, he discovered new applications for this efficient mathematical approach. Gossip involves the sharing of information between individuals in a network and can be applied mathematically in both human social networks and data networks, such as distributed sensors. “A network is a collection of nodes, each connected to other nodes via links,” explains Richtarik. “In social networks, for instance

KAUST Professor Peter Richtárik wins Distinguished Speaker Award

1 min read ·
Peter Richtárik, KAUST professor of computer science, recently received a Distinguished Speaker Award at the Sixth International Conference on Continuous Optimization (ICCOPT 2019) held in Berlin from August 3 to 8. ICCOPT 2019 was organized by the Mathematical Optimization Society and was hosted this year by the Weierstrass Institute for Applied Analysis and Stochastics.

Meet KAUST Prospective Student: Igor Sokolov

1 min read ·
Igor Sokolov, 23, is an applied mathematics and physics graduate who will join KAUST from the Moscow Institute of Physics and Technology, Russia. Igor first came to KAUST as a visiting student in 2019 and was highly impressed by the University's living and working facilities. It was this first impression that prompted him to pursue an M.S. at KAUST. Sokolov will join KAUST in the Fall of 2020 as an M.S. candidate under the supervision of Professor Peter Richtarik.

Meet KAUST student: Grigory Malinovsky

1 min read ·
Grigory Malinovsky joined KAUST this fall as a M.S./Ph.D. candidate and member of Professor Peter Richtarik's Optimization and Machine Learning Lab. Grigory holds a bachelor's degree in applied mathematics and physics from the Moscow Institute of Physics and Technology, Russia. Malinovsky first came to KAUST as a visiting student and was instantly impressed by the research facilities and working atmosphere. His outstanding internship experience was a catalyst in choosing KAUST as the next step in his academic career.

Yongliang Yang

Research Scientist, Computer Science
Yongliang Yang was a Research Scientist at the Visual Computing Center in King Abdullah University of Science and Technology (KAUST). Currently, he’s an Assistant Professor in Department of Computer Science at University of Bath in United Kingdom. Education and Early Career Yongliang received his Bachelor degree in Computer Science and Technology in 2004, from Tsinghua University. He continued to pursue his Master and Ph.D. degree in Graphics and Geometric Computing Group in Tsinghua, under the supervision of Prof. Shi-Min Hu. In 2009, he joined KAUST as a Postdoctoral Fellow working with Prof

Rolf Krause

Professor Rolf Krause is an experienced researcher and educator specializing in designing and analyzing algorithms to solve complex problems in scientific computing and machine learning.

Rafal Szlendak

Rafał Szlendak is an MS/PhD student in the AMCS program under the supervision of Prof. Peter Richtarik. His biggest passions are research, weightlifting and travelling. His career path is motivated by his endless passion for self-development and discovery. Education and Early Career Rafał Szlendak has obtained his Bachelor's degree in Mathematics and Statistics from University of Warwick Research Interest In his research Rafał focuses on stochastic gradient descent algorithms for large-scale machine learning problems. His primary goal is developing state-of-the-art methods with sound

Peter Richtarik

Professor, Computer Science
Professor Richtárik specializes in big data optimization and machine learning. He is known for his work on randomized coordinate descent algorithms, stochastic gradient descent and federated learning.

Mher Safaryan

Mher Safaryan is a Postdoctoral Research Fellow working with Professor Peter Richtarik at the Visual Computing Center (VCC) at King Abdullah University of Science and Technology (KAUST). Education and Early Career Mher received his Ph.D. in Mathematics from Yerevan State University, Armenia under the supervision of Prof. Grigori Karagulyan. Before joining KAUST, he was a Junior Researcher at the Institute of Mathematics of National Academy of Sciences, Real Analysis Department in Armenia. After that, he joined the Computer, Electrical and Mathematical Sciences and Engineering Division (CEMSE)

Laurent Condat

I am a senior researcher working in optimization, in the group of Prof. Peter Richtárik.

Igor Sokolov

Igor Sokolov is an M.S. candidate in the KAUST Peter Richtarik Research Group under the supervision of Professor Peter Richtarik. Before joining KAUST, Igor obtained a bachelor's degree in applied mathematics and physics from the Moscow Institute of Physics and Technology, Russia. Research Interests Igor's research interests lie in the intersection of optimization and machine learning. He is interested in both the theoretical analysis of optimization algorithms as well as their practically efficient applications.

Grigory Malinovsky

Grigory Malinovsky is a M.S./Ph.D. candidate at the KAUST Optimization and Machine Learning Lab under the supervision of Professor Peter Richtarik. Before joining KAUST, Grigory obtained a bachelor's degree in applied mathematics and physics from the Moscow Institute of Physics and Technology, Russia. Research Interests Grigory's research interests include machine learning, optimization, and applied statistics. Specifically, he is interested in the theoretical aspects of optimization algorithms that can be applied to real machine-learning problems. Education Profile B.A., Major in Applied

Ganesh Sundaramoorthi

Ganesh Sundaramoorthi was an Assistant Professor of Electrical Engineering and jointly Assistant Professor of Applied Mathematics and Computational Science at KAUST starting in 2011. He is currently Principal Research Scientist at United Technologies Research Center in East Hartford, CT, USA, formulating computer vision algorithms for robotic inspection applications. Education and Early Career His PhD is in Electrical and Computer Engineering from the Georgia Institute of Technology in Atlanta, GA, USA in 2008. His PhD developed fundamental shape optimization methods for computer vision that

Filip Hanzely

Filip Hanzely is a recent PhD graduate from the group of Peter Richtarik. His research focuses mostly on various aspects of optimization for machine learning, designing provably efficient algorithms for solving big data problems. Filip is joining Toyota Technological Institute in Chicago (TTIC) as a research assistant professor in December 2020. Education and Early Career Filip Hanzely graduated with honor for his bachelor degree in Economics and Financial Mathematics from Comenius University in Bratislava, Slovakia. Later on in 2017, he received his master in science degree in Mathematics and

Elnur Gasanov

Elnur Gasanov is a master student at Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), studying under the supervision of Professor Peter Richtárik in his research group. Education and Early Career Elnur Gasanov obtained his bachelor degree in Applied Mathematics and Physics from Moscow Institute of Physics and Technology (MIPT) in Moscow, Russia in 2018. Research Interest His research interests are in optimization and machine learning. Education Profile ​B.Sc., Applied Mathematics and Physics, Moscow Institute of Physics and Technology (State

David Ketcheson

Professor Ketcheson is an expert in numerical analysis and computational science, specializing in the study of numerical methods for differential equations and the modeling and analysis of time-dependent nonlinear wave phenomena.

Alibek Sailanbayev

Alibek Sailanbayev is a Ph.D. candidate in Applied Mathematics and Computational Science at Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), studying under the supervision of Professor Peter Richtarik in his research group. Education and Early Career Alibek obtained his bachelor degree Applied Mathematics from Nazarbayev University in Kazakhstan. He joined KAUST in Fall 2017 and received his M.S. degree in Applied Mathematics and Computational Sciences in December 2018. He is now pursuing Ph.D. studies. Alibek took a Summer Internship at Loria/Inria

Alexandra Aguiar Gomes

Alexandra Aguiar Gomes has been a university educator since 1996. Her area of expertise is optimization, with current interests in the mathematics of decision-making for sustainable development and the mathematics of generative artificial intelligence. Alexandra finds joy in simplifying mathematical concepts, helping her students discover the practicality of mathematics in their fields.