Computer-assisted Algorithm Design for Optimization and Machine Learning

CT Score

1500

#0 of 0 in CMOR

Difficulty

—/10

Workload

—h/wk

Median grade

—

from grade distribution

Would take again

—

n = 0 verified

Grade distribution

No distribution published for this course.

By professor: same course, different game

PROFESSORSCOREGRADEHRS/WKTERMS

Professor ratings →

Sections

No section data for this term yet.

In the tree

CMOR 531 or CMOR 532 or CMOR 533 or CMOR 536 or INDE 517 or INDE 577 Description: In a traditional first course in continuous optimization, students learn to use and analyze existing algorithms, such as applying Stochastic Gradient Descent to train neural networks, using Accelerated Gradient Descent to solve image processing problems, or study their convergence analysis. In contrast, this course methodically addresses the fundamental questions: Where do these algorithms come from? How do we know they are any good? And, most importantly, could we design something provably better? The core of this course is a transition from merely using and analyzing optimization algorithms to actively designing them, treating the algorithm design process itself as an optimization problem. The goal is to construct algorithms that are the provably fastest methods for specific classes of problems in consideration. In particular, we will focus on designing the optimal first-order methods (FOMs), which are methods that rely solely on gradient or subgradient information. These are the most commonly used optimization algorithms today, given their efficacy in solving high-dimensional problems, compatibility with large-scale datasets, and applicability to machine learning. The curriculum is structured in two main parts. First, we will learn to build a mathematical model (an optimization problem in itself) that calculates the absolute worst-case performance for a given algorithm. This provides a rigorous framework for computing tight convergence guarantees for a known algorithm. Subsequently, we will formulate the search for the provably fastest algorithm as a new, higher-level optimization problem. The solution to this problem is, in fact, the optimal algorithm we seek. This modern, computer-assisted approach to algorithm design is known as Performance Estimation Problem (PEP). This is a hands-on course with a strong emphasis on implementation, where students will learn to build the tools that discover these algorithms from scratch using PEP. By the end, students will not only understand the theory behind cutting-edge algorithm design but will also have the skills to discover and analyze new, high-performance algorithms for a variety of challenges in optimization and data science→CMOR 537→unlocks 0 courses

No mapped courses require this one yet. See on the map →

Study groups

0 active

🔒 Verified-only writes. You post as “sophomore, lecture section”: real enough to trust, anonymous enough to be honest.