List of
Publications
Machine Learning and Computer Vision
- A. K. Menon, S. Jayasumana, A. S. Rawat, S. Kim, S. J. Reddi, and S. Kumar
In Defense of Dual-Encoders for Neural Ranking
International Conference on Machine Learning (ICML), 2022.
[pdf]
- Z. Li, S. Bhojanapalli, M. Zaheer, S. J. Reddi and S. Kumar
Robust Training of Neural Networks Using Scale Invariant Architectures
International Conference on Machine Learning (ICML), 2022.
[pdf]
- E. M. Lindgren, S. J, Reddi, R. Guo and S. Kumar
Efficient Training of Retrieval Models Using Negative Cache
Neural Information Processing Systems (NeurIPS), 2021.
[pdf]
- G. Citovsky, G. DeSalvo, C. Gentile, L. Karydas,
A. Rajagopalan, A. Rostamizadeh and S. Kumar
Batch Active Learning at Scale
Neural Information Processing Systems (NeurIPS), 2021.
[pdf]
- A. K. Menon, A. S. Rawat, S. J. Reddi, S. Kim, and S. Kumar
A Statistical Perspective on Distillation
International Conference on Machine Learning (ICML), 2021.
[pdf]
- A. S. Rawat, A. K. Menon, W. Jitkrittum, S. Jayasumana, F. X. Yu, S. J. Reddi, and S. Kumar
Disentangling Labeling and Sampling Bias for Learning in Large-output Spaces
International Conference on Machine Learning (ICML), 2021.
[pdf]
- S. J. Reddi, R. K. Pasumarthi, A. K. Menon, A. S. Rawat, F. Yu, S. Kim, A. Veit, and S. Kumar
RankDistil: Knowledge Distillation for Ranking
International Conference on Artificial Intelligence and Statistics (AISTATS) 2021.
[pdf]
- A. K. Menon, A. S. Rawat, and S. Kumar
Overparameterisation and Worst-case Generalisation: Friend or Foe?
International Conference on Learning Representations (ICLR), 2021.
[pdf]
- S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konecný, S. Kumar, and H. B. McMahan
Adaptive Federated Optimization
International Conference on Learning Representations (ICLR), 2021.
[pdf]
- A. K. Menon, S. Jayasumana, A. S. Rawat, H. Jain
A. Veit and S. Kumar
Long-tail Learning via Logit Adjustment
International Conference on Learning Representations (ICLR), 2021.
[pdf]
- C.-Y. Hsieh, C.-K. Yeh, X. Liu, P. Ravikumar,
S. Kim, S. Kumar, and C.-J. Hsieh
Evaluations and Methods for Explanation Through Robustness Analysis
International Conference on Learning Representations (ICLR), 2021.
[pdf]
- J. Zhang, A. K. Menon, A. Veit, S. Bhojanapalli, S. Kumar, and S. Sra
Coping With Label Shift via Distributionally Robust Optimisation
International Conference on Learning Representations (ICLR), 2021.
[pdf]
- C. Yun, Y.-W. Chang, S. Bhojanapalli, A. S. Rawat, S. Reddi, and S. Kumar
O(n) Connections are Expressive Enough:
Universal Approximability of Sparse Transformers
Neural Information Processing Systems (NeurIPS), 2020.
[pdf]
- J. Zhang, S. P. Karimireddy, A. Veit, S. Kim, S. Reddi, and S. Kumar
Why are Adaptive Methods Good
for Attention Models?
Neural Information Processing Systems (NeurIPS), 2020.
[pdf]
- M. Weber, M. Zaheer, A. S. Rawat, A. Menon, and S. Kumar
Robust Large-Margin Learning in Hyperbolic Space
Neural Information Processing Systems (NeurIPS), 2020.
[pdf]
- Y. Liu, A. T. Suresh, F. Yu, S. Kumar, and M. Riley
Learning Discrete Distributions: User vs Item-Level
Privacy
Neural Information Processing Systems (NeurIPS), 2020.
[pdf]
- H. Chen, S. Si, Y. Li, C. Chelba,
S. Kumar, D. Boning, and C.-J. Hsieh
Multi-Stage Influence Function
Neural Information Processing Systems (NeurIPS), 2020.
[pdf]
- S. Bhojanapalli, C. Yun, A. S. Rawat, S. Reddi, and S. Kumar
Low-Rank Bottleneck in Multi-head Attention Models
International Conference on Machine Learning (ICML), 2020.
[pdf]
- M. Lukasik, S. Bhojanapalli, A. K. Menon, and S. Kumar
Does Label Smoothing Mitigate Label Noise?
International Conference on Machine Learning (ICML), 2020.
[pdf]
- R. Guo, P. Sun, E. Lindgren, Q. Geng, D. Simcha, F. Chern and S. Kumar
Accelerating Large-Scale Inference with Anisotropic Vector Quantization
International Conference on Machine Learning (ICML), 2020.
[pdf]
- F. X. Yu, A. S. Rawat, A. K. Menon, and S. Kumar
Federated Learning with Only Positive Labels
International Conference on Machine Learning (ICML), 2020.
[pdf]
- Y. You, J. Li, S. Reddi, J. Hseu, S. Kumar, S. Bhojanapalli, X. Song, J. Demmel, K. Keutzer, and C.-J. Hsieh
Large Batch Optimization for Deep Learning:
Training BERT in 76 Minutes
International Conference on Learning Representations (ICLR), 2020.
[pdf]
- C. Yun, S. Bhojanapalli, A. S. Rawat, S. J. Reddi, and S. Kumar
Are Transformers Universal Approximators
of Sequence-to-Sequence Functions?
International Conference on Learning Representations (ICLR), 2020.
[pdf]
- A. K. Menon, A. S. Rawat, S. J. Reddi, and S. Kumar
Can Gradient Clipping Mitigate Label Noise?
International Conference on Learning Representations (ICLR), 2020.
[pdf]
- W.-C. Chang, F. Yu, Y.-W. Chang, and S. Kumar
Pre-training Tasks for Embedding-based Large-scale Retrieval
International Conference on Learning Representations (ICLR), 2020.
[pdf]
- Y. Ruan, Y. Xiong, S. Reddi, S. Kumar,
C.-J. Hsieh
Learning to Learn by Zeroth-Order Oracle
International Conference on Learning Representations (ICLR), 2020.
[pdf]
- C.-J. Hsieh, Q. Cao, S. Kumar, S. Si, T. Xiao, and X. Liu
How Does Noise Help Robustness? Explanation and Exploration under the Neural SDE Framework
International Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
[pdf]
- A. K. Menon, A. S. Rawat, S. J. Reddi, and S. Kumar
Multilabel reductions: what is my loss optimising?
Neural Information Processing Systems (NeurIPS), 2019.
[pdf]
- C. Guo, A. Mousavi, X. Wu, D. Holtmann-Rice, S. Kale, S. Reddi and S. Kumar
Breaking the Glass Ceiling for Embedding-Based Classifiers for Large Output Spaces
Neural Information Processing Systems (NeurIPS), 2019.
[pdf]
- A. S. Rawat, J. Chen, F. Yu, A. T. Suresh, and S. Kumar
Sampled Softmax with Random Fourier Features
Neural Information Processing Systems (NeurIPS), 2019.
[pdf]
- M. Staib, S. Reddi, S. Kale, S. Kumar, and S. Sra
Escaping Saddle Points with Adaptive Gradient Methods
International Conference on Machine Learning (ICML), 2019.
[pdf]
- S. Wu, A. G. Dimakis, S. Sanghavi, F. X. Yu, D. Holtmann-Rice, D. Storcheus, A. Rostamizadeh, and S. Kumar
Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling
International Conference on Machine Learning (ICML), 2019.
[pdf]
- P.-H. (Patrick) Chen, S. Si, S. Kumar, Y. Li, and C.-J. Hsieh
Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks
International Conference on Learning Representations (ICLR), 2019.
[pdf]
- S. Reddi, S. Kale, F. X. Yu, D. Holtmann-Rice, J. chen and S. Kumar
Stochastic Negative Mining for Learning with Large Output Spaces
International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.
[pdf]
- Q. Geng, W. Ding, R. Guo, and S. Kumar
Optimal Noise-Adding Mechanism in Additive Differential Privacy
International Conference on Artificial Intelligence and Statistics (AISTATS), 2019.
[pdf]
- S. J. Reddi, M. Zaheer, D. Sachan, S. Kale, and S. Kumar
Adaptive Methods for Nonconvex Optimization
Neural Information Processing Systems (NIPS), 2018.
[pdf]
- N. Agarwal, A. T. Suresh, F. X. Yu, S. Kumar, and H. B. McMahan
cpSGD: Communication-efficient and differentially-private distributed SGD
Neural Information Processing Systems (NIPS), 2018.
[pdf]
- Ian E. H. Yen, S. Kale, F. X. Yu, D. Holtmann-Rice, S. Kumar, P. Ravikumar
Loss Decomposition for Fast Learning in Large Output Spaces
International Conference on Machine Learning (ICML), 2018.
[pdf]
- S. Reddi, S. Kale, S. Kumar [best paper award]
On the Convergence of Adam and Beyond
International Conference on Learning Representations (ICLR), 2018.
[pdf]
- X. Wu, R. Guo, A. T. Suresh, S. Kumar, D. Holtmann-Rice, D. Simcha, F. X. Yu
Multiscale Quantization for Fast Similarity Search
Neural Information Processing Systems (NIPS), 2017.
[pdf]
- B. Dai, R. Guo, S. Kumar, N. He, L. Song
Stochastic Generative Hashing
International Conference on Machine Learning (ICML), 2017.
[pdf]
- A. T. Suresh, F. X. Yu, S. Kumar, H. B. McMahan
Distributed Mean Estimation with Limited Communication
International Conference on Machine Learning (ICML), 2017.
[pdf]
- X. Zhang, F. X. Yu, S. Kumar, S. F. Chang
Learning Spread-out Local Feature Descriptors
International Conference on Computer Vision (ICCV), 2017.
[pdf]
- K. Zhong, R. Guo, S. Kumar, B. Yan, D. Simcha, I. S. Dhillon
Fast Classification with Binary Prototypes
International Conference on Artificial Intelligence and Statistics (AISTATS), 2017.
[pdf]
- F. X. Yu, A. T. Suresh, K. Choromanski, D. Holtmann-Rice, S. Kumar
Orthogonal Random Features
Neural Information Processing Systems (NIPS), 2016.
[pdf]
- A. Choromanska, K. Choromanski, M. Bojarski, T. Jebara, S. Kumar, Y. LeCun
Binary Embeddings with Structured Hash Projections
International Conference on Machine Learning (ICML), 2016.
[pdf]
- R. Guo, S. Kumar, K. Choromanski, and D. Simcha
Quantization based Fast Inner Product Search
International Conference on Artificial Intelligence and Statistics (AISTATS), 2016.
[pdf]
- J. Pennington, F. X. Yu, S. Kumar
Spherical Random Features for Polynomial Kernels
Neural Information Processing Systems (NIPS), 2015.
[pdf]
- V. Sindhwani, T. Sainath, S. Kumar
Structured Transforms for Small-Footprint Deep Learning
Neural Information Processing Systems (NIPS), 2015.
[pdf]
- X. Zhang, F. X. Yu, Ruiqi Guo, S. Kumar, S. Wang, S.-F. Chang
Fast Orthogonal Projection Based on Kronecker Product
International Conference on Computer Vision (ICCV), 2015.
[pdf]
- Y. Cheng, F. X. Yu, R. S. Feris, S. Kumar,
A. Choudhary, and S. F. Chang
An Exploration of Parameter Redundancy in Deep Networks with
Circulant Projections
International Conference on Computer Vision (ICCV), 2015.
[pdf]
- R. Guo, S. Kumar, K. Choromanski, and D. Simcha
Quantization based Fast Inner Product Search
arXiv:1509.01469, 2015.
[pdf]
.
W. Liu, C. Mu, S. Kumar,
and S. F. Chang
Discrete Graph Hashing
Neural Information Processing Systems (NIPS), 2014.
[pdf]
Supplementary material can be found here.
.
F. X. Yu, S. Kumar, Y. Gong, and S. F. Chang
Circulant Binary Embedding
International Conference on Machine Learning (ICML), 2014.
[pdf]
Matlab code can be found here.
.
F. X. Yu, D. Liu, S. Kumar, T. Jebara,
and S. F. Chang
pSVM for Learning with Label Proportions
International Conference on Machine Learning (ICML), 2013.
[pdf]
The supplementary file with additional proofs and experiments is here.
.
Y. Gong, S. Kumar, H. Rowley,
and S. Lazebnik
Learning Binary Codes for High-Dimensional Data Using Bilinear Projections
IEEE Computer Vision and Pattern Recognition (CVPR), 2013.
[pdf]
.
A. Talwalkar, S. Kumar, M. Mohri
and H. Rowley
Large-scale SVD and Manifold Learning
Journal of Machine Learning Research (JMLR), 2013.
[pdf]
.
Y. Gong, S. Kumar, V. Verma,
and S. Lazebnik
Angular Quantization-based Binary Codes for Fast Similarity Search
Advances in Neural Information Processing Systems (NIPS), 2012.
[pdf]
.
J. He, S. Kumar,
and S. F. Chang
On the Difficulty of Nearest Neighbor Search
International Conference on Machine Learning (ICML), 2012.
[pdf]
The supplementary file containing all the proofs is here.
NOTE: This is a slightly edited version of what is in the ICML proceedings.
.
W. Liu, J. Wang, Y. Mu, S. Kumar,
and S. F. Chang
Compact Hyperplane Hashing with Bilinear Functions
International Conference on Machine Learning (ICML), 2012.
[pdf]
The supplementary file containing extended proofs and results is here.
.
J. Wang, S. Kumar,
and S. F. Chang
Semi-Supervised Hashing for Large Scale Search
IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2012.
[pdf]
.
S. Kumar, M. Mohri
and A. Talwalkar
Sampling Methods for the Nystrom Method
Journal of Machine Learning Research (JMLR), 2012.
[pdf]
.
W. Liu, J. Wang, S. Kumar, and S. F. Chang
Hashing With Graphs
International Conference on
Machine Learning (ICML), 2011.
[pdf]
.
A. Talwalkar, S. Kumar, M. Mohri
and H. Rowley
Large-Scale Manifold Learning
Book chapter in Manifold Learning Theory and Applications. Editors: Y. Ma and Y. Fu. CRC Press, 2011.
[pdf]
.
S. Kumar,
M. Mohri and A. Talwalkar
Ensemble Nystrom
Book chapter in Ensemble Machine Learning: Theory and Applications, Springer, 2011.
[pdf]
Modified to correct an error in the computational complexity analysis. April 2011.
.
A. Makadia, V. Pavlovic and S. Kumar
Baselines for Image Annotation
International Journal on Computer Vision (IJCV), 2010.
[pdf]
.
J. Wang, S. Kumar,
and S. F. Chang
Sequential Projection Learning for Hashing with Compact Codes
International Conference on
Machine Learning (ICML), 2010.
[pdf]
.
Z. Wang, M. Zhao, Y. Song, S. Kumar
and B. Li
YouTubeCat: Learning to Categorize Wild Web Videos
IEEE Computer Vision and Pattern Recognition (CVPR), 2010.
[pdf]
.
J. Wang, S. Kumar,
and S. F. Chang
Semi-Supervised Hashing for Scalable Image Retrieval
IEEE Computer Vision and Pattern Recognition (CVPR), 2010.
[pdf]
.
S. Kumar,
M. Mohri and A. Talwalkar
Ensemble Nystrom
Method
Neural Information Processing Systems (NIPS), 2009.
[pdf]
Modified to correct an error in the computational complexity analysis. April 2011.
.
S. Kumar,
M. Mohri and A. Talwalkar
On Sampling-based
Approximate Spectral Decomposition
International Conference on
Machine Learning (ICML), 2009.
[pdf]
.
S. Kumar,
M. Mohri and A. Talwalkar
Sampling Techniques for the Nystrom Method
Twelfth International Conference on
Artificial Intelligence and Statistics (AISTATS), 2009.
[pdf]
.
A. Makadia,
V. Pavlovic and S. Kumar
A New Baseline for Image Annotation
European Conference on Computer Vision (ECCV), 2008.
[pdf]
.
A. Talwalkar, S. Kumar and
H. A. Rowley
Large-Scale Manifold Learning
IEEE Computer Vision and Pattern Recognition (CVPR), 2008.
[pdf]
.
M. Kim,
S. Kumar, V. Pavlovic and
H. A. Rowley
Face Tracking and Recognition with
Visual Constraints in Real-World Videos
IEEE Computer Vision and Pattern Recognition (CVPR), 2008.
[pdf]
·
S. Kumar and
H. A. Rowley
Classification of Weakly-Labeled Data
with Partial Equivalence Relations
IEEE International Conference on Computer
Vision (ICCV), 2007.
[pdf]
Some additional results and parts of the
video and retrieval datasets used in this work can be seen here.
·
S. Kumar and M. Hebert
Discriminative Random Fields
International Journal of Computer Vision
(IJCV), 68(2), 179-201, 2006.
[pdf]
·
S. Kumar,
J. August and M. Hebert
Exploiting Inference for Approximate
Parameter Learning in Discriminative Fields: An Empirical Study
Energy Minimization Methods in Computer Vision and Pattern Recognition
(EMMCVPR), 2005.
[pdf]
This paper is an extended and revised version of the earlier
work presented in Snowbird Learning Workshop, 2004.
·
S. Kumar
Models for Learning Spatial Interactions
in Natural Images for Context-Based Classification
PhD Thesis, The Robotics Institute, School of Computer Science,
Carnegie Mellon University, September 2005.
[pdf] [ps]
Revised October 2005.
·
S.
Kumar and M. Hebert
A Hierarchical Field Framework for Unified
Context-Based Classification
IEEE International Conference on Computer
Vision (ICCV), 2005.
[pdf] [ps]
Revised October 2005.
·
C. Rother, S. Kumar, V.
Kolmogorov and A. Blake
Digital Tapestry
IEEE
International Conference on Computer Vision and Pattern Recognition (CVPR),
June, 2005.
[pdf]
·
S. Kumar and M. Hebert
Discriminative Random Fields
In review, International Journal of
Computer Vision (IJCV), Submitted, 2004.
·
S. Kumar and M. Hebert
Approximate Parameter Learning in
Discriminative Fields
Snowbird Learning
Workshop, Utah, 2004.
[pdf]
[ps]
The synthetic dataset used for
learning and inference experiments can be obtained from here.
·
S. Kumar and M. Hebert
Multiclass Discriminative Fields for
Parts-Based Object Detection
Snowbird Learning
Workshop, Utah, 2004.
[pdf]
Discriminative
Fields for Modeling Spatial Dependencies in Natural Images
Advances in Neural Information Processing Systems,
NIPS 16, 2004.
[pdf] [ps]
The binary denoising synthetic
dataset used for training and testing can be obtained from here.
·
S.
Kumar and M. Hebert
Discriminative Random Fields: A
Discriminative Framework for Contextual Interaction in Classification
IEEE International
Conference on Computer Vision (ICCV), 2003.
[pdf] [ps]
Man-Made Structure Detection in
Natural Images using a Causal Multiscale Random Field
IEEE International
Conference on Computer Vision and Pattern Recognition (CVPR), 2003.
[pdf]
Some more example
results and comparisons.
The structure detection database
used for training and testing can be obtained from here.
- S. Kumar, A. C. Loui, and M. Hebert
An Observation-Constrained Generative
Approach for Probabilistic Classification of Image Regions
Image
and Vision Computing, 21, pp. 87-97, 2003.
[pdf]
A shorter
version of this paper appeared in the following workshop:
- S. Kumar, A. C. Loui, and M. Hebert
Probabilistic Classification of
Image Regions using an Observation-Constrained Generative Approach
ECCV
Workshop on Generative Models based Vision (GMBV), 2002.
[pdf]
Medical Imaging and Robotics
- B. Nabbe, S. Kumar, and M. Hebert
Path Planning with Hallucinated Worlds
In Proc. IEEE/RSJ International
Conference on Intelligent Robots and Systems (IROS), October 2004.
[pdf]
- S. Kumar, M. I. Kassim, and K. V. Asari
Design of a Vision-guided Microrobotic Colonoscopy System
International Journal of Advanced Robotics, vol. 14, no.
2, pp. 87-104, 2000.
[link]
- K. V. Asari, S. Kumar, and M. I. Kassim
A Fully Automatic Microrobotic Endoscopy System
Journal
of Intelligent and Robotic Systems, vol. 28, pp. 325-341, 2000.
[link]
- K. V. Asari, S. Kumar, and D. Radhakrishnan
A New Approach for Nonlinear
Distortion Correction in Endoscopic Images Based on Least Squares Estimation
IEEE Transactions
on Medical Imaging, vol. 18, no. 4, pp. 345-354,1999.
[link]
- S. Kumar, K. V. Asari, and D. Radhakrishnan
Real-Time Automatic Extraction of Lumen Region and
Boundary from Endoscopic Images
IEE Journal of Medical & Biological Engineering
& Computing, vol. 37, pp. 600-604, 1999.
[link]
- K. V. Asari, S. Kumar, and D. Radhakrishnan
Technique of Distortion Correction in Endoscopic
Images using a Polynomial Expansion
IEE Journal of Medical & Biological Engineering
& Computing, vol. 37, no. 1, pp. 8-12, 1999.
[link]
- K. V. Asari, T. Srikanthan, S. Kumar, and D.
Radhakrishnan
A Pipelined Architecture for Image Segmentation by
Adaptive Progressive Thresholding
Journal of Microprocessors and Microsystems, vol. 23,
no. 8-9, pp. 493-499, 1999.
[link]
- S. Kumar, K. Vijayan Asari, and D. Radhakrishnan
Online Extraction of Lumen Region and Boundary from
Endoscopic Images Using a Quad Structure
IEE Conference on Image Processing and its
Applications (IPA), pp. 818-822, 1999.
- S. Kumar, K. Vijayan Asari, and D. Radhakrishnan
A New Technique for the Segmentation of Lumen from
Endoscopic Images by Differential Region Growing
42nd Midwest Symposium on Circuits and Systems, New
Mexico, 1999.
- S. M. Krishnan, S. Kumar, C. J. Yap, M. I.
Kassim, and P. M. Y. Goh
Development of a Microrobotic System for Intelligent Endoscopy
2nd Scientific Meet. of Biomed. Eng. Soc., Singapore,
January 1999.
- S. M. Krishnan, S. Kumar, C. J. Yap, M. I.
Kassim, and P.M. Y Goh
A Computer-Based Endoscopic Image Segmentation
Technique for Lumen Extraction
13th
Int. Congress and Exhibition on Comp. Asst. Radio. Surgery, Paris, June, 1999.
- S. M. Krishnan, S. Kumar, C. J. Yap, M. I.
Kassim, and P.M. Y Goh
Computer-Assisted Intelligent Endoscopy
13th
Int. Congress and Exhibition on Comp. Asst. Radio. Surgery, Paris, June, 1999.
- S. Kumar, M. Singaperumal and Y. G. Srinivasa
Design of a Self Guided Vehicle (SGV) with Laser
Based Navigation System
National Seminar on Mechatronics, India, Madras,
1997.