Avatar

Somak Aditya

Postdoc Researcher

Microsoft Research

Biography

I have joined Microsoft Research India as a Postdoctoral Researcher. At MSRI, with Dr. Monojit Choudhury, I am exploring evaluation and enhancement of Neural language models using the combined theories of Language and Logic. Our exploration leads us to consider both traditional neural models and neuro-symbolic variants. Parallel to exploring neuro(symbolic) methods in the context of NLI, I am also exploring multi-hop reasoning capabilities of neural methods in symbolic domains with Dr. Navin Goyal.

Overall, my central goal in research has been devloping models that learns from data, yet enriched by knowledge and capable of reasoning with such knowledge. Developing such models require three central considerations: knowledge acquisition, representation and reasoning. At MSRI and during my PhD, a fundamental observation was that, often an equal measure of concentration is required for each application.

During my doctoral studies, my efforts revolved around Image Understanding under the supervision of Prof. Chitta Baral (Past President of KR.inc, 2016-2018). I was also co-advised by Dr. Yezhou Yang (Assistant Professor, ASU). Through a combination of Deep Learning, Knowledge Representation and Probabilistic Logical Reasoning; I demonstrated the benefits of using reasoning and knowledge in Visual Question-Answering (IJCAI, AAAI), Captioning (CVIU), image puzzle solving (UAI) and visual reasoning (WACV). I completed my Ph.D. in Computer Science from CIDSE, Arizona State University on June 27, 2018.

I completed my Masters (ME) in Computer Science from the Indian Institute of Science, Bangalore in 2011 with a concentration in Machine Learning under the supervision of Prof. M Narasimha Murty (Dean, Faculty of Engineering, IISc). I have done my Bachelors in CS from Jadavpur University in 2009. Prior to joining PhD, I worked as a developer in the Next-Generation Sequencing (NGS) team at Strand Life Sciences for nearly 2.5 years.

Interests

  • Knowledge Representation and Reasoning
  • Probabilistic Logical Reasoning
  • Natural Language Processing

Education

  • PhD in Computer Science, 2018

    Arizona State University

  • MEng in Computer Science, 2011

    Indian Institute of Science, Bangalore

  • BE in Computer Science, 2009

    Jadavpur University, Kolkata

News

[Dec 2020] ACL-IJCNLP-2021: Invited as PC member.
[Nov 2020] CoNLL 2020: Our pre-recorded talk on TaxiNLI is online. Hope you enjoy (modulo my lack of expressions :))!
[Oct 2020] NAACL-2021: Invited as PC Member.
[Oct 2020] CoNLL-2020: TaxiNLI dataset released publicly!
[Sep 2020] CoNLL-2020: TaxiNLI: Taking a ride up the NLU Hill accepted at CoNLL.
[Aug 2020] EACL-2021: Invited as PC Member.
[Mar 2020] EMNLP-20: Invited as PC Member.
[Mar 2020] LANTERN@COLING-20: Invited as PC Member.
[Feb 2020] Joined MSR India as a Postdoc Researcher.
[Jan 2020] ACL-20: Invited as PC Member.
[Dec 2019] USPTO: "Knowledge-sharing between cross-domain Agents" work approved internally to be filed..
[Dec 2019] AAAI-20: "Uncovering Relations for Marketing Knowledge Representation" accepted in StarAI workshop full paper.
[Dec 2019] IJCAI-20: Invited as PC Member.
[Oct 2019] AAAI-20: Exploratory Navigation and Selective Readoing got accepted as AAAI demo.
[Sep 2019] Adobe News: Knowledge Graph Research appears in Adobe News !
[Aug 2019] AAAI-20: Invited as an AAAI 2020 PC Member!
[Aug 2019] IJCAI-19: Invited as an IJCAI DC Career Panelist!
[Aug 2019] IJCAI-19: Our work on Integrating Knowledge and Reasoning in Image Understanding" got accepted in IJCAI 2019 Survey!
[May 2019] USPTO: First patent on Marketing Knowledge Graph Creation approved!
[Nov 2018] WACV-19: Our work on "Spatial Knowledge Distillation on Visual Reasoning" got accepted in WACV 2019!
[Oct 2018] KR-19: We had a successful workshop filled with intersting talks from researchers from University of Leeds, IBM Research US, University of Adelaide and Army Research Lab. Their talks are avaibale at: .
[Aug 2018] My doctoral dissertation is now publicly available for download in the ASU Digital Repostory.
[Aug 2018] UAI-18: We presented this poster for "Combining Knowledge and Reasoning through Probabilistic Soft Logic for Image Puzzle Solving" in UAI 2018.
[June 2018] Successfully defended my thesis. My slides are available on dropbox.
[June 2018] Published the Image Riddles code for public use that was used in UAI-18 work. Visit Github.
[June 2018] Published the PSL engine code for public use that was used in AAAI-18 work. Visit Github.
[May 2018] UAI-18: Our paper on Image Riddles is accepted in UAI 2018 (30% acceptance rate).
[May 2018] Invited as a reviewer in Robotics and Autonomus Systems journal.
[May 2018] Our Website for KR-2018 workshop is live! Please consider submitting to the workshop.
[Mar 2018] We are organizing the first workshop on "Induce and Deduce: Integrating learning of representations and models with deductive, explainable reasoning that leverages knowledge" in KR 2018 (Phoenix, 27-29 Oct). Website coming soon!
[Mar 2018] I am awarded the University Graduate Fellowship for Spring 2018 for the third time. Thank you ASU, CIDSE!
[Feb 2018] AAAI-18: Presented our work on "Explicit Reasoning over End-to-End Neural Architectures" in AAAI 2018.
[Jan 2018] ASU SW Robotics Symposium-18: Presented our work on "Explicit Reasoning over End-to-End Neural Architectures" in ASU SouthWest Robotics Symposium. The work was nominated for Best Abstract award.
[Dec 2017] CVIU-17: Our accepted manuscript in CVIU is now available online: "Image Understanding using Vision and Reasoning through Scene Description Graph"
[Dec 2017] CVIU-17: Our work "Image Understanding using Vision and Reasoning through Scene Description Graph" has been accepted in the reputed Computer Vision and Image Understanding (CVIU) journal.
[Nov 2017] Our Work on Explicit Reasoning over End-To-End Neural Architectures has been accepted in AAAI 2018 (Acceptance Rate: 24.5%, 933 accepted out of 3.8k).
[Nov 2017] Vision and Reasoning Website is live.
[Oct 2017] Invited as a reviewer for The Visual Computer journal.
[July 2017] Work on Image Riddles accepted as Extended Abstract on Vision Meets Cognition Workshop, CVPR 2017.
[May 2017] Thesis proposal accepted, and officially advanced to candidacy.
[May 2017] Joined JDE, Verisk Analytics as Cognitive Analytics and Machine Learning Research Intern, under Dr. Maneesh Singh, Director, JDE, Cognitive Analytics.
[May 2017] Unofficially a Ph.D. Candidate, after successfully defending my proposal on "Knowledge and Reasoning in Image Understanding".
[May 2017] Invited as a review-assistant for IJCAI 2017.
[March 2017] Awarded University Graduate Fellowship for Spring 2017 from ASU, for the third time.
[Feb 2017] Attended AAAI-2017 (DC and the main Conference). Great to know that people are interested in Vision and Reasoning approaches. Here is the poster I presented at the main conference.

Recent Publications

Quickly discover relevant content by filtering publications.

TaxiNLI: Taking a Ride up the NLU Hill

Pre-trained Transformer-based neural architectures have consistently achieved state-of-the-art …
In CoNLL 2020.

Uncovering Relations for Marketing Knowledge Representation

Online behaviors of consumers and marketers generate massive marketing data, which ever more …
In AAAI 2020, StarAI Workshp.

Integrating Knowledge and Reasoning in Image Understanding

Deep learning based data-driven approaches have been successfully applied in various image under- …
In IJCAI 2019.

Spatial Knowledge Distillation to aid Visual Reasoning

For tasks involving language and vision, the current state-of-the-art methods do not leverage any …
In WACV 2019.

Explicit Reasoning over End-to-End Neural Architectures

Many vision and language tasks require commonsense reasoning beyond data-driven image and natural …
In AAAI 2018.

Projects

Solving NLI using Reasoning and Learning

We investigate how to evaluate, explain and enhance neural models under the lens of reasoning external_link.

Spatial Knowledge Distillation to aid Visual Reasoning

In this work, we propose a way to integrate spatial commonsense knowledge to aid visual reasoning external_link.

Solving Visual QA using Reasoning and Deep Learning

In this work, we propose an exlicit reasoning layer over Deep Neural Architectures to solve VQA. external_link.

Solving Image Puzzles

In this work, we propose image puzzles that require background knowledge to solve. This project is also aimed to advocates logically …

From Images to Sentences

In this project, we aim to achieve a deeper understanding of images and videos with the help of background or common-sense knowledge. …

K-Parser: A Knowledge Parser

We build a semantic parser to extract semantic knowledge from natural language sentences. external_link.

Softwares

  • TaxiNLI dataset released for CoNLL 2020 work (Github Link)
  • PSL engine used for AAAI 2018 work (Gihub Link)
  • Image Riddles code for UAI 2018 paper (Gihub Link)
  • K-Parser (First Author- Arpit Sharma), used in IJCAI-15 work (Online Demo)

Experience

 
 
 
 
 

Postdoctoral Researcher

Microsoft Research India

Feb 2020 – Present Bangalore
I am continuing my explorations on different aspects of knowledge integration approaches to natural language inferencing with Dr. Monojit Choudhury. I will be also exploring few fundamental problems such as structure learning in probabilistic logic.
 
 
 
 
 

Research scientist

Adobe Research

Sep 2018 – Feb 2020 Bangalore
As a Research Scientist, I extensively collaborated with other researchers and product managers to understand product needs, formulate research problems that revolves around business use-cases. Interesting projects include marketing knowledge graph creation, diversity in sequence recommendation, and storytelling from data. My other responsibilities include interviewing interns and incoming researchers, running summer internships and mentoring research associates.
 
 
 
 
 

Specialist Software III

Strand Life Sciences

Jun 2014 – Dec 2011 Bangalore
 
 
 
 
 

Senior Software Engineer

Yahoo!

Jun 2011 – Nov 2011 Bangalore

Teaching and Mentoring

I have served as a teaching assistant for the following courses at Arizona State University:

  • CSE-576 Natural Language Processing, Fall 2015 and Fall 2016
    • Responsibilities included: Creating Homework assignments, proposing class projects, and mentoring multiple groups.
  • CSE-471 Introduction To Artificial Intelligence, Spring 2016
  • CSE-310 Data Structures and Algorithms, Spring 2015

I collaborated with an amazing bunch of bright students (during their Post-graduate or under-graduate study):

Internship Mentoring/Collaborating:

  • (2019) Pranil Joshi (IIT-B), Abhinav Mishra (IIT-G), Bhavy Khatri (IIT-K): Knowledge-sharing between Cross-domain Agents
    • Mentored with Kushal Chawla (USC), Sharmila Nangi Reddy
    • (Feb 2020) Patent accepted and to be filed in USPTO.
  • (2020) Pratik Joshi, Aalok Sathe: TaxiNLI: Taking a ride up the NLU Hill.

Affiliations

Research & Development groups

Industrial Organizations

Universities

Contact