skaramcheti@cs.stanford.edu Stanford, CA · · · ·

I am a PhD Student in computer science at Stanford University where I'm grateful to be co-advised by Percy Liang and Dorsa Sadigh. I am honored to be supported by the Open Philanthropy Project AI Fellowship. Research: I work on robot learning and natural language processing. My research focuses on two axes: Large-scale pretraining and representation learning (developing better "raw materials" for robotics & language) Language & interaction for human-robot collaboration (e.g., via shared autonomy, or online language corrections) Timeline: I recently completed a year-long part-time research internship at Hugging Face 🤗 working on multimodal pretraining with an incredible team of collaborators. Before Stanford, I was a resident at Facebook AI Research in New York, where I was lucky to work with Rob Fergus, Douwe Kiela, Jason Weston, and Arthur Szlam on grounded language understanding. Here's a short Q&A I did about my residency. Prior to that, I was lucky to do two summer internships at Bloomberg Research with wonderful mentors Gideon Mann and David Rosenberg. I completed my undergraduate degrees in computer science and literary arts at Brown University. While there, I did research in NLP & human-robot interaction advised by Stefanie Tellex and Eugene Charniak.

News

• [May 2023]: We’re organizing the 2nd Workshop on Learning from Diverse, Offline Data (L-DOD) at ICRA 2023. Happy to keep this awesome workshop going for another year in a row! See y’all in London!

• [February 2023]: Introducing Voltron! Check out our paper on “Language-Driven Representation Learning for Robotics”; we show that we can use language supervision to learn general visual representations for multiple robotics tasks (e.g., grasp segmentation, intent inference, learning for control, amongst others). Our models and pretraining code are open-source, as well as our diverse evaluation suite.

• [January 2023]: We’ll be presenting our work “No, to the Right” – Online Language Corrections for Robotic Manipulation via Shared Autonomy at the ACM/IEEE International Conference on Human Robot Interaction (HRI). More work on building viable systems with language and shared autonomy, co-led by Yuchen Cui, with our amazing mentees Raj Palleti, Nidhya Shivakumar, and my advisors Percy and Dorsa. See y’all in Stockholm!

• [December 2022]: Dorsa and I gave a joint talk on “Bridging Language and Robotics for Interaction, Learning, and Teaching” at the Language and Reinforcement Learning Workshop @ NeurIPS 2022 (LaReL @ NeurIPS). This was a really fun and great experience, covering our work on Language-Informed Latent Actions, Exploration through Learned Language Abstraction as well as some upcoming work!

• [October 2022]: Our paper “Eliciting Compatible Demonstrations for Multi-Human Imitation Learning” with Kanishk Gandhi, Madeline Liao, and Dorsa Sadigh was accepted to the Conference on Robot Learning (CoRL) 2022. Really excited about what this work means for scaling robotics data collection; it was great chatting with y’all in New Zealand!

Publications

2023

Language-Driven Representation Learning for Robotics
Siddharth Karamcheti, Suraj Nair, Annie S. Chen, Thomas Kollar, Chelsea Finn, Dorsa Sadigh, Percy Liang.
Preprint, February 2023.
[pdf] [code - models] [code - evaluation] [project page]

“No, to the Right” – Online Language Corrections for Robotic Manipulation via Shared Autonomy
Yuchen Cui*, Siddharth Karamcheti*, Raj Palleti, Nidhya Shivakumar, Percy Liang, Dorsa Sadigh.
ACM/IEEE International Conference on Human Robot Interaction (HRI), March 2023.
[pdf] [homepage] [code]

2022

Eliciting Compatible Demonstrations for Multi-Human Imitation Learning
Kanishk Gandhi, Siddharth Karamcheti, Madeline Liao, Dorsa Sadigh.
Conference on Robot Learning (CoRL), December 2022.
[pdf] [homepage]

What Makes Representation Learning from Videos Hard for Control?
Tony Z. Zhao, Siddharth Karamcheti, Thomas Kollar, Chelsea Finn, Percy Liang.
2nd Workshop on Scaling Robot Learning @ RSS 2022, June 2022.
Best Paper Finalist
[pdf]

Shared Autonomy for Robotic Manipulation with Language Corrections
Siddharth Karamcheti*, Raj Palleti*, Yuchen Cui, Percy Liang, Dorsa Sadigh.
Workshop on Learning with Natural Language Supervision (NL-Supervision) @ ACL 2022, May 2022.
[pdf]

2021

ELLA: Exploration through Learned Language Abstraction
Suvir Mirchandani, Siddharth Karamcheti, Dorsa Sadigh.
Conference on Neural Information Processing Systems (NeurIPS), December 2021.
[pdf] [talk] [slides] [code]

LILA: Language-Informed Latent Actions
Siddharth Karamcheti*, Megha Srivastava*, Percy Liang, Dorsa Sadigh.
Conference on Robot Learning (CoRL), November 2021.
[pdf] [homepage] [code] [poster]

On the Opportunities and Risks of Foundation Models
Center for Research on Foundation Models (CRFM) – 100+ authors, directed by Percy Liang.
       - Robotics (§2.3): Siddharth Karamcheti (Lead), Annie Chen, Suvir Mirchandani, Suraj Nair, Krishnan Srinivasan, Kyle Hsu, Jeannette Bohg, Dorsa Sadigh, and Chelsea Finn.
       - Interaction (§2.5): Joon Sung Park, Chris Donahue, Mina Lee, Siddharth Karamcheti, Dorsa Sadigh, Michael Bernstein.
[pdf] [homepage] [workshop] [press]

Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answering
Siddharth Karamcheti, Ranjay Krishna, Li Fei-Fei, Christopher D. Manning.
Annual Meeting of the Association for Computational Linguistics (ACL-IJCNLP), August 2021.
Outstanding Paper Award
[pdf] [talk] [slides] [code] [other coverage]

Targeted Data Acquisition for Evolving Negotiation Agents
Minae Kwon, Siddharth Karamcheti, Mariano-Florentino Cuéllar, Dorsa Sadigh.
International Conference on Machine Learning (ICML), July 2021.
[pdf] [talk] [slides]

Learning Visually Guided Latent Actions for Assistive Teleoperation
Siddharth Karamcheti, Albert J. Zhai, Dylan P. Losey, Dorsa Sadigh.
Learning for Dynamics and Control (L4DC), June 2021.
[pdf] [talk] [slides] [code] [poster]

2020

Learning Adaptive Language Interfaces through Decomposition
Siddharth Karamcheti, Dorsa Sadigh, Percy Liang.
Workshop for Interactive and Executable Semantic Parsing (IntEx-SemPar) @ EMNLP 2020, November 2020.
[pdf] [slides]

Generating Interactive Worlds with Text
Angela Fan*, Jack Urbanek*, Pratik Ringshia, Emily Dinan, Emma Qian, Siddharth Karamcheti, Shrimai Prabhumoye, Douwe Kiela, Tim Rocktäschel, Arthur Szlam, and Jason Weston.
Association for the Advancement of Artificial Intelligence (AAAI), February 2020
[pdf] [dataset]

2019

Finding Generalizable Evidence by Learning to Convince Q&A Models
Ethan Perez, Siddharth Karamcheti, Rob Fergus, Jason Weston, Douwe Kiela, and Kyunghyun Cho.
Empirical Methods in Natural Language Processing (EMNLP), November 2019
[pdf] [blog post] [code]

Learning to Speak and Act in a Fantasy Text Adventure Game
Jack Urbanek, Angela Fan, Siddharth Karamcheti, Saachi Jain, Samuel Humeau, Emily Dinan, Tim Rocktäschel, Douwe Kiela, Arthur Szlam, and Jason Weston.
Empirical Methods in Natural Language Processing (EMNLP), November 2019
[pdf] [dataset]

Improving Grey-Box Fuzzing by Modeling Program Control Flow
Siddharth Karamcheti, Gideon Mann, and David Rosenberg
Workshop on Machine Learning for Software Engineering (ML4SE), June 2019
[pdf] [slides]

Grounding Natural Language Instructions to Semantic Goal Representations for Abstraction and Generalization
Dilip Arumugam*, Siddharth Karamcheti*, Nakul Gopalan, Edward C. Williams, Mina Rhee, Lawson L.S. Wong, and Stefanie Tellex
Autonomous Robots (AuRO), February 2019
[free-to-read pdf]

2018

Adaptive Grey-Box Fuzz Testing with Thompson Sampling
Siddharth Karamcheti, Gideon Mann, and David Rosenberg
11th ACM Workshop on Artificial Intelligence and Security (AISEC), October 2018
[pdf] [slides]

2017

Modeling Latent Attention within Neural Networks
Christopher Grimm, Dilip Arumugam, Siddharth Karamcheti, David Abel, Lawson L.S. Wong, and Michael Littman
Preprint
[pdf]

A Tale of Two DRAGGNs: A Hybrid Approach for Interpreting Action-Oriented and Goal-Oriented Instructions
Siddharth Karamcheti, Edward C. Williams, Dilip Arumugam, Mina Rhee, Nakul Gopalan, Lawson L.S. Wong, and Stefanie Tellex
1st Workshop in Language Grounding for Robotics (RoboNLP) @ ACL, August 2017
Best Paper Award
[pdf]

Accurately and Efficiently Interpreting Human-Robot Instructions of Varying Granularities
Dilip Arumugam*, Siddharth Karamcheti*, Nakul Gopalan, Lawson L.S. Wong, and Stefanie Tellex
Robotics: Science and Systems (RSS), June 2017
[pdf]

Blog Posts

The Annotated S4 – Efficiently Modeling Long Sequences with Structured State Spaces
Sasha Rush and Siddharth Karamcheti – January, 2022.
[ICLR blog track] [code] [original paper]

Mistral – A Journey towards Reproducible Language Model Training
Siddharth Karamcheti* and Laurel Orr* – August, 2021.
Team: Jason Bolton, Tianyi Zhang, Karan Goel, Avanika Narayan, Rishi Bommasani, Deepak Narayanan
Advisors: Tatsunori Hashimoto, Dan Jurafsky, Christopher D. Manning, Christopher Potts, Christopher Ré, Percy Liang
[code] [checkpoints] [talk]