algorithms

From the Ball-Proximal (Broximal) Point Method to Efficient Training of LLMs

-

B1 L3 R3119

This talk introduces the Ball-Proximal Point Method, a new foundational algorithm for non-smooth optimization with surprisingly fast convergence, and Gluon, a new theoretical framework that closes the gap between theory and practice for modern LMO-based deep learning optimizers.

From the Ball-Proximal (Broximal) Point Method to Efficient Training of LLMs

-

B9 L2 R2325

This talk introduces the Ball-Proximal Point Method, a new foundational algorithm for non-smooth optimization with surprisingly fast convergence, and Gluon, a new theoretical framework that closes the gap between theory and practice for modern LMO-based deep learning optimizers.

Applied Mathematics School 2024

-
The KAUST Applied Mathematics School is a 1-week visiting program aiming to introduce undergraduate and master's students to special research topics focusing on key areas in applied mathematics and computational science.

PCCFD - Predictive Complex Computational Fluid Dynamics

-

B9 L2 H1

The PCCFD workshop will focus on cutting-edge research in the field of algorithmic development for CFD and multi-scale complex flow simulations.

Estimation Problems for Partial and Fractional Differential Equations - Autumn 2019

-

B9 L2 H1 R2322

I will present an overview of our activities around estimation problems for partial and fractional differential equations. I will present the methods and the algorithms we develop for the state, source and parameters estimation and illustrate the results with some simulations and real applications.

On the resolution of a theoretical question related to the nature of local training in federated learning

-

B1 L3 R3119

In this talk, I will explain the problem, its solution, and some subsequent work generalizing, extending and improving the ProxSkip method in various ways. We study distributed optimization methods based on the local training (LT) paradigm - achieving improved communication efficiency by performing richer local gradient-based training on the clients before parameter averaging - which is of key importance in federated learning. Looking back at the progress of the field in the last decade, we identify 5 generations of LT methods: 1) heuristic, 2) homogeneous, 3) sublinear, 4) linear, and 5) accelerated. The 5th generation, initiated by the ProxSkip method of Mishchenko et al (2022) and its analysis, is characterized by the first theoretical confirmation that LT is a communication acceleration mechanism.

Proximal Algorithms for Large-Scale Convex Non-smooth Optimization

-

B2 L5 R5220

Convex nonsmooth optimization problems, whose solutions live in very high dimensional spaces, have become ubiquitous. To solve them, the class of iterative fixed-point algorithms known as proximal splitting algorithms is particularly adequate: they consist of simple operations, handling the terms in the objective function separately. I will present a selection of recent primal-dual algorithms within a unified framework, which consists in solving monotone inclusions with well-chosen spaces and metrics.

Machine learning models gather momentum

1 min read ·
New methods for training machine learning models are quicker and more accurate than current approaches, previously considered state-of-the-art.

Ferrofluid surface simulations go more than skin deep

1 min read ·
Computer models efficiently and accurately simulate the magnetic responses of ferrofluids by considering only the fluid’s surface.

Peter Richtarik

Professor, Computer Science
Professor Richtárik specializes in big data optimization and machine learning. He is known for his work on randomized coordinate descent algorithms, stochastic gradient descent and federated learning.

Alibek Sailanbayev

Alibek Sailanbayev is a Ph.D. candidate in Applied Mathematics and Computational Science at Visual Computing Center (VCC) in King Abdullah University of Science and Technology (KAUST), studying under the supervision of Professor Peter Richtarik in his research group. Education and Early Career Alibek obtained his bachelor degree Applied Mathematics from Nazarbayev University in Kazakhstan. He joined KAUST in Fall 2017 and received his M.S. degree in Applied Mathematics and Computational Sciences in December 2018. He is now pursuing Ph.D. studies. Alibek took a Summer Internship at Loria/Inria