Class lectures: Tuesdays 12:35-2:25pm in Eng253, Mudd

Instructor: Sanjiv Kumar (Office Hours: Tuesdays 2:25pm - 3:25pm)
TA: Jun Wang / Junfeng He (Office Hours: Tuesdays 1:25-2:25pm/4:25-5:25pm)

Driven by rapid advances in many fields such as Biology, Finance and Web Services, applications involving millions or even billions of data items such as documents, user records, reviews, images or videos are not that uncommon. Can we develop methods that can learn efficiently from these massive amounts of potentially noisy data? There is an urgent need to revisit the traditional machine learning methods and tools to bridge the wide gap between large scale practical requirements and traditional learning approaches.

The goal of this course is to introduce fundamental concepts of large-scale machine learning. Both theoretical and practical aspects will be discussed. The primary focus of the course will be on analyzing basic tools of large-scale learning including the relevant theory and algorithms rather than focusing on specific machine learning techniques. It will also provide running examples from real-world settings from various fields including Vision and Information Retrieval. The course will prepare students to evolve a new dimension while developing models and optimization techniques to solve a practical problem - scalability.

We will analyze tools for large-scale learning that can be applied to a variety of commonly used machine learning techniques for classification, regression, ranking, clustering, density estimation and semi-supervised learning. Example applications of these tools to specific learning methods will also be provided. A tentative list of tools we plan to discuss is given below:

  1. Randomized Algorithms
  2. Matrix Approximations I (low-rank approximation, decomposition)
  3. Matrix Approximations II (sparse matrices, matrix completion)
  4. Approximate Nearest Neighbor Search I (trees)
  5. Approximate Nearest Neighbor Search II (hashes)
  6. Fast Optimization (first-order methods)
  7. Kernel Methods I (fast training)
  8. Kernel Methods II (fast testing)
  9. Dimensionality Reduction (linear and nonlinear methods)
  10. Sparse Methods/Streaming (sparse coding...)


Please check this section frequently for new announcements.

Course Prerequisites