[Interests] [Experiences] [Publications] [Professional Activities]

Interests

My core research interest is in machine learning for interactive systems that maximizes a utility function by taking actions, which is in contrast to prediction-oriented machine learning like supervised learning. My area of focus is reinforcement learning, including the important subclass known as contextual bandits; I am also interested in related areas such as large-scale online learning with big data, active learning, and planning. In the past, I have applied my work to recommendation, Web search, advertising, and conversational systems.

Most of my work can be grouped into several clusters:

More information can be found in Google Scholar, DBLP, LinkedIn.

Somewhat up-to-date CV, and short bio.

I am also on Twitter as @LihongLi20.

Experiences

2020 - present: Senior Principal Scientist, Amazon
2017 - 2020: Research Scientist, Google Research
2012 - 2017: Senior, Principal, Sr Principal Researcher, Microsoft Research
2009 - 2012: Research Scientist, Yahoo! Research
2009: PhD in Computer Science, Rutgers University
2004: MSc in Computing Science, University of Alberta
2002: BE in Computer Science and Technology, Tsinghua University

Professional Activities

Recent and upcoiming events
- [2023/05] Seoul Test of Time Award by IW3C2, for our WWW'2010 paper on contextual bandits for personalized recommendation. See the article by Amazon Science.
- [2022/01] Article by Amazon Science on how we use RL to delight customers in Amazon, and more. [link]
- [2021/08] Invited talk at KDD Workshop on Multi-Armed Bandits and Reinforcement Learning (with Yi Liu). [slides]
- [2021/07] Co-organizer of ICML Workshop on RL for Real Life.
- [2021/07] Minitutorial on Optimization in Policy Gradient (with Lin Xiao) at the SIAM Conference on Optimization.
- [2021/04] Amazon Advertising opens the Early-career Scientist program with full-time research positions. Application deadline: May 14. Please consider applying and share broadly!
- [2020/12/15] Invited talk at AI for Economics Seminar.
- [2020/12/12] Co-organizer of NeurIPS Workshop on Offline Reinforcement Learning.
- [2020/10] Invited talk at Department of Computer Science, University of California, Los Angeles, CA, USA.
- [2020/09] Co-organizer of Simons Institute's Deep Reinforcement Learning workshop, as part of the Theory of Reinforcement Learning program.
- [2020/09] Invited talk at Department of Statistics and Data Science, Yale University, New Haven, CT, USA.
- [2020/04 (cancelled)] Invited talk at Reinforcement Learning workshop at the DALI (Data, Learning and Inference) meeting, Balearic Islands, Spain.
- [2019/03 (cancelled)] Invited talk at CISS, Princeton, NJ, USA.
- [2019/12] Co-organizer of NeurIPS Workshop on the Optimization Foundations of Reinforcement Learning, Vancouver, BC, Canada.
- [2019/10] Machine Learning Journal Special Issue on "Reinforcement Learning for Real Life". Paper deadline: March 15, 2020. Please consider submitting to it! (CFP)
- [2019/10] Invited talk at University of Alberta, Edmonton, AB, Canada
- [2019/10] Invited talk at DeepMind, Edmonton, AB, Canada
- [2019/08] Invited talk at Simons Institute's Emerging Challenges in Deep Learning workshop.
- [2019/06] Co-organizer of ICML Workshop on RL for Real Life, Long Beach, CA, USA.
- [2019/04] Invited talk at GSO Spring Seminar, Department of Statistics, Purdue University, West Lafayette, IN, USA.
- [2018/12] Invited talk at NIPS Workshop on Wordplay: Reinforcement and Language Learning in Text-based Games, Montreal, QC, Canada.
- [2018/10] Invited talk at MADDD, University of California, Davis, CA, USA.
- [2018/09] Invited talk at University of Texas, Austin, TX, USA.
- [2018/07] Tutorial on ``Neural Approaches to Conversational AI'' (with Jianfeng Gao and Michel Galley) at ACL, Melbourne, Australia. [PPT , PDF]
- [2018/07] Tutorial on ``Neural Approaches to Conversational AI'' (with Jianfeng Gao and Michel Galley) at SIGIR, Ann Arbor, MI, USA.
- [2018/07] Invited talk at ISMP, Bordeaux, France.
- [2018/06] Invited talk at INFORMS, Taipei.
- [2018/06] Invited talk at Machine Learning Theory Workshop, Peking University, Beijing, China.
- [2018/04] Guest lecture at Tsinghua University, Beijing, China.
- [2018/03] Invited talk at CISS, Princeton, NJ, USA.
- [2018/02] Invited talk at Stanford University, Palo Alto, CA, USA.
- [2017/12] Co-organizer of NIPS Workshop on From "What If" to "What Next", Long Beach, CA, USA.
- [2017/11] Co-organizer of the AI Frontiers conference, San Jose, CA, USA.
- [2017/09] Invited talk at Google Brain, Montreal, Canada.
- [2017/08] Invited talk at ICML Workshop on Interactive Machine Learning and Semantic Information Retrieval, Sydney, Australia.
- [2017/06] Invited talk at Korea Advanced Institute of Science and Technology, Daejeon, Korea.
- [2017/06] Invited talk at Sungkyunkwan University, Suwon, Korea.
- [2017/06] Invited talk at RLDM, Ann Arbor, MI, USA.
- [2017/05] Co-organizer of Algorithms for Reinforcement Learning Minisymposium at SIAM Optimization Conference, Vancouver, Canada.
- [2017/05] Invited talk at Stanford University, Palo Alto, CA, USA.
- [2017/05] Invited talk at New York University, New York, NY, USA.
- [2017/02] Invited talk at Simons Institute, Berkeley, CA, USA.
Area chair and/or senior program committee member for AAAI (2017-2019), AISTATS (2017, 2019), ICLR (2019, 2022), ICML (2012-2017, 2019-2021), IJCAI (2011, 2016, 2017), NIPS/NeurIPS (2014, 2017-2021).
Editorial board of IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI).