(04) 2018 - invited talk, Human-AI Collaboration for Sustainable Market Design, at the Ethics and Governance of AI event by Media Lab and Berkman Klein Center, Harvard Univeristy
(03) 2018 - program committee of the 27th International Joint Conference on Artificial Intelligence (IJCAI)
(01) 2018 - represented Daemo at a European Dialogue on the Platform Economy event by the European Trade Union Institute and partners, Brussels
(01) 2018 - Adobe Research PhD Fellowship Finalist
(12) 2017 - awarded the MIT Graduate Student Life Grant for the MIT SP Graduate Student Dinner Seminar Series initiative
(11) 2017 - paper got accepted at the AAAI-2018, New Orleans, USA.
(11) 2017 - TEDx talk "the Future of Markets in the Era of AI" at TEDxBeaconStreet, Boston.
(10) 2017 - invited to the Industry Panel at the AAAI HCOMP conference, Canada― decided to step down to increase the panel's gender diversity.
(09) 2017 - invited at the HPI-Stanford Design Thinking Research Program, Potsdam Germany.
(08) 2017 - awarded best paper honorable mention for the ACM UIST research paper, Quebec City, Canada.
(06) 2017 - awarded MIT's Kaufman Teaching Certificate (KTCP), evidence of a strong commitment to the teaching enterprise.
(01) 2017 - guided inventors across India to solve pressing societal challenges in agriculture and healthcare using AI [MIT, Innovating for Billions].
(10) 2016 - presented the paper at the ACM UIST, Tokyo, Japan.
(09) 2016 - paper got accepted at the ACM CSCW, Portland, USA.
(06) 2016 - EteRNA featured in Werner Herzog's film ``Lo And Behold: Reveries of the Connected World’’ (Trailer)
(09) 2015 - Daemo, crowdsourcing marketplace completed its first project with Microsoft Research.
(10) 2014 - EteRNA got listed as one of the 25 great ideas invented at the Carnegie Mellon SCS.
(02) 2014 - paper got accepted to PNAS.
(01) 2011 - co-launched EteRNA, crowd-computing game that helps invent medicine.
I conduct research at the interface of human-AI collaboration, human-centered design, and algorithmic economics (computational social choice)― with an emphasis on design, computational modeling, and engineering of sustainable markets, prosocial behavior, and socio-technical systems for development. My research draws on MIT's Mens et Manus motto and is motivated to solve pressing engineering and public policy challenges related to the future of creative work, crisis response, sharing economies, agriculture markets, and financial inclusion. This research has published in top-tier computer science conferences (AAAI, ACM UIST, ACM CSCW) and a scientific journal (PNAS), and featured in the New York Times, Bloomberg, WIRED, and the Wall Street Journal. I've recently gave a TEDx talk about the Future of Markets in the Era of Artificial Intelligence.
I am one of the principal creators of Daemo, a self-governed open sourced crowdsourcing market. I led its technical architecture and invented Boomerang, an incentive-compatible reputation system. My contributions draw on fundamentals from algorithmic game theory (incentive design for pro-social behavior), structured finance (guilds organization as tranches), and human-centered design (prototype tasks), with aspirations to design sharing economies and the future of work. I was also one of the principal creators and founding members EteRNA, crowdcomputation game that harnesses human creativity with machine learning to help invent medicine. EteRNA is an example of human-machine collaboration that fuses human and artificial intelligence to solve problems that neither can solve alone. It has reached over 100,000 citizen scientists across the world and was listed as one of 25 great ideas emerged from the CMU School of Computer Science. EteRNA was also featured in Werner Herzog's film "Lo And Behold: Reveries of the Connected World" (Trailer).
I am a PhD student in the Space Enabled group led by Danielle Wood at the MIT Media Lab. I have been fortunate to have work with Iyad Rahwan at MIT. In the past I have collaborated with Michael Bernstein from the Computer Science department at Stanford University. I have earned my M.S. from the School of Computer Science at Carnegie Mellon where I worked with Adrien Treuille at the Graphics Group in the Robotics Institutes and Anita Woolley at the OBT Group. I will be spending this summer at the Santa Fe Institute.
I come from the Western Ghats of India. One of the major missions of my life is to open up the STEM (Science, Technology, Engineering, and Mathematics) research and educational opportunities for people across the globe. In this pursuit, I moved back to academia from the mathematical finance industry (Wall Street). Over the years, I have mentored dozens of students who went on pursuing undergraduate and graduate studies at Carnegie Mellon, UC Berkeley, UCLA, and Cornell University. I'm was one of the core members of the Stanford Crowd Research Initiative and a part of MIT's Innovating for Billions in Emerging Worlds Leadership Council.
ॐ असतो मा सद्गमय ।
तमसो मा ज्योतिर्गमय ।
मृत्योर्मा अमृतं गमय ।।
A Voting-Based System for Ethical Decision Making
AAAI 2018: Association for the Advancement of Artificial Intelligence.
We present a general approach to automating ethical decisions, drawing on machine learning and computational social choice. In a nutshell, we propose to learn a model of societal preferences, and, when faced with a specific ethical dilemma at runtime, efficiently aggregate those preferences to identify a desirable choice. We provide a concrete algorithm that instantiates our approach; some of its crucial steps are informed by a new theory of swap-dominance efficient voting rules. Finally, we implement and evaluate a system for ethical decision making in the autonomous vehicle domain, using preference data collected from 1.3 million people through the Moral Machine website.
PRESS: Social Media Has Failed Its Self-Driving Test (Bloomberg) | Researchers go after the biggest problem with self-driving cars (Axios)
Belongie, Serge., Goel, Sharad., Davis, James., and Bernstein, Michael.
UIST 2017: ACM Symposium on User Interface Software and Technology
Research experiences today are limited to a privileged few at select universities. Providing open access to research experiences would enable global upward mobility and increased diversity in the scientific workforce. But, how do we coordinate a crowd of diverse volunteers on open-ended research? How could a PI have enough visibility into each person's contributions to recommend them for further study? We present Crowd Research, a crowdsourcing technique that coordinates open-ended research through an iterative cycle of open contribution, synchronous collaboration, and peer assessment. To aid upward mobility and recognize contributions in publications, we introduce a decentralized credit system: participants allocate credits to each other, which a graph centrality algorithm translates into a collectively-created author order. Over 1,500 people from 62 countries have participated, 74% from institutions with low access to research. Over two years and three projects, this crowd has produced articles at top-tier Computer Science venues, and participants have gone on to leading graduate programs.
PRESS: A Stanford-led Platform for Crowdsourced Research Gives Experience to Global Participants (Stanford News)
PDF Web [ACM UIST Best Paper Honorable Mention]
UIST 2016: ACM Symposium on User Interface Software and Technology
Paid crowdsourcing platforms suffer from low-quality work and unfair rejections, but paradoxically, most workers and requesters have high reputation scores. These inflated scores, which make high-quality work and workers difficult to find, stem from social pressure to avoid giving negative feedback. We introduce Boomerang, a reputation system for crowdsourcing that elicits more accurate feedback by rebounding the consequences of feedback directly back onto the person who gave it. With Boomerang, requesters find that their highly-rated workers gain earliest access to their future tasks, and workers find tasks from their highly-rated requesters at the top of their task feed. Field experiments verify that Boomerang causes both workers and requesters to provide feedback that is more closely aligned with their private opinions. Inspired by a game-theoretic notion of incentive-compatibility, Boomerang opens opportunities for interaction design to incentivize honest reporting over strategic dishonesty.
* In the readings of advanced Human Computation classes at KAIST, Cornell University, University of Washington
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work and Social Computing
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other’s quality through double-blind peer assessment. A two week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current platforms, and more accurate than in the traditional model.
RNA Design Rules From a Massive Open Laboratory
Gaikwad, S., Yoon, Sungroh., Treuille, Adrien., Das, Rhiju., and EteRNA Participants
PNAS 2014: Proceedings of the National Academy of Sciences of the United States of America
Self-assembling RNA molecules present compelling substrates for the rational interrogation and control of living systems. However, imperfect in silico models—even at the secondary structure level—hinder the design of new RNAs that function properly when synthesized. Here, we present a unique and potentially general approach to such empirical problems: the Massive Open Laboratory. The EteRNA project connects 37,000 enthusiasts to RNA design puzzles through an online interface. Uniquely, EteRNA participants not only manipulate simulated molecules but also control a remote experimental pipeline for high-throughput RNA synthesis and structure mapping. We show herein that the EteRNA community leveraged dozens of cycles of continuous wet laboratory feedback to learn strategies for solving in vitro RNA design problems on which automated methods fail. The top strategies—including several previously unrecognized negative design rules-were distilled by machine learning into an algorithm, EteRNABot. Over a rigorous 1-y testing phase, both the EteRNA community and EteRNABot significantly outperformed prior algorithms in a dozen RNA secondary structure design tests, including the creation of dendrimer-like structures and scaffolds for small molecule sensors. These results show that an online community can carry out large-scale experiments, hypothesis generation, and algorithm design to create practical advances in empirical science.
PRESS: Videogamers are recruited to fight Tuberculosis and other ills (Wall Street Journal) | RNA game lets players help find a biological prize (New York Times) | How turning science into a game rouses more public interest (WIRED) | Game lets citizen scientists participate in creating large-scale library of synthetic RNA designs (Scientific American) | Rebooting science outreach (ASBMB Today) | Why video games are key to modern science (CNN) | Online game helps predict how RNA folds (New Scientist)
Crowdcomputing and Citizen Science for Large-Scale Experiments
IC2S2 2017: International Conference on Computational Social Science
Historically, scientific experiments have been conducted at a small scale either with artificial environments or with the expertise of limited number of scientists. While social science literature investigates very deep questions to understand human behavior, many experiments are usually limited by the number of participants and duration of a study. On the contrary, computer science literature exploits advanced computational techniques to crunch voluminous datasets, but research designs are generally not experimental, which limits the opportunity to generate causal inferences. In this tutorial we demonstrate how crowdcomputing can enable computational social scientists to engage with millions of users on the Internet and study human behavior at scale for a longer time. We showcase pitfalls and lessons learned from various crowdcomputing and citizen science projects. Furthermore, we provide insights about how to build a sustainable citizen science community to scale science beyond the traditional laboratories. We envisage this tutorial will help computational social scientists effectively use crowdcomputing to investigate deep research questions and longitudinally validate their hypotheses in large scale experiments.
Prototype Tasks: Improving Crowdsourcing Results through Rapid, Iterative Task Design
AAAI HCOMP 2017: AAAI Conference on Human Computation
Low-quality results have been a long-standing problem on microtask crowdsourcing platforms, driving away requesters and justifying low wages for workers. To date, workers have been blamed for low-quality results: they are said to make as little effort as possible, do not pay attention to detail, and lack expertise. In this paper, we hypothesize that requesters may also be responsible for low-quality work: they launch unclear task designs that confuse even earnest workers, under-specify edge cases, and neglect to include examples. We introduce prototype tasks, a crowdsourcing strategy requiring all new task designs to launch a small number of sample tasks. Workers attempt these tasks and leave feedback, enabling the requester to iterate on the design before publishing it. We report a field experiment in which tasks that underwent prototype task iteration produced higher-quality work results than the original task designs. With this research, we suggest that a simple and rapid iteration cycle can improve crowd work, and we provide empirical evidence that requester “quality” directly impacts result quality.
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work
The success of crowdsourcing markets is dependent on a strong foundation of trust between workers and requesters. In current marketplaces, workers and requesters are often unable to trust each other’s quality, and their mental models of tasks are misaligned due to ambiguous instructions or confusing edge cases. This breakdown of trust typically arises from (1) flawed reputation systems which do not accurately reflect worker and requester quality, and from (2) poorly designed tasks. In this demo, we present how Boomerang and Prototype Tasks, the fundamental building blocks of the Daemo crowdsourcing marketplace, help restore trust between workers and requesters. Daemo’s Boomerang reputation system incentivizes alignment between opinion and ratings by determining the likelihood that workers and requesters will work together in the future based on how they rate each other. Daemo’s Prototype tasks require that new tasks go through a feedback iteration phase with a small number of workers so that requesters can revise their instructions and task designs before launch.
CI 2017: Collective Intelligence Conference
A key principle of Daemo is to provide crowd workers and requesters with a means of governing the development and evolution of the platform into the future. To facilitate this, we introduced the Daemo constitution, outlining the goals of the platform, relationship between members of the community and its standards, methods for seeking ideas, amending the constitution and resolving conflicts. The Daemo constitution has been a collaborative effort by the Stanford Crowd Research Collective. However, we have also endeavored to solicit feedback from the MTurk worker and requester communities via TurkerNation, Reddit and other channels.
PDF AVAILABLE UPON REQUEST
UIST 2015: ACM Symposium on User Interface Software and Technology
Crowdsourcing marketplaces provide opportunities for autonomous and collaborative professional work as well as social engagement. However, in these marketplaces, workers feel disrespected due to unreasonable rejections and low payments, whereas requesters do not trust the results they receive. The lack of trust and uneven distribution of power among workers and requesters have raised serious concerns about sustainability of these marketplaces. To address the challenges of trust and power, this paper introduces Daemo, a self-governed crowdsourcing marketplace. We propose a prototype task to improve the work quality and open-governance model to achieve equitable representation. We envisage Daemo will enable workers to build sustainable careers and provide requesters with timely, quality labor for their businesses.
PRESS: The future of work the peoples uber (Pacific Standard)
PDF Poster System
Simoiu, Camelia., Veit, Andreas., Wilber, Michael., Zhou, Sharon., Belongie, Serge., Goel, Sharad., Davis, James., Bernstein, Michael.
HCI+Design Open House 2016: Stanford University
Scientific research is becoming increasingly collaborative, yet primarily limited to professional researchers in labs and universities. Can we scale traditional research approach and invite anyone from around the world to participate, and crowdsource large-scale, open-ended research problems? We're scaling the research process to solve open-ended novel problems, by providing access and connecting hundreds of people with top researchers.
Click to see my detailed analysis of PageRank & Credit Distribution
BioX 2011: Interdisciplinary Initiatives Symposium Poster Session, Stanford University
The Effects of Team Strategic Orientation On Team Process and Information Search
Organizational Behavior and Human Decision Processes, 2013
We tested the effects of team strategic orientation on team member perceptions, work strategy and information search. In Experiment 1, 80 teams worked on a hidden profile decision-making task. A defensive team strategic orientation increased members’ perceptions of the problem’s scope, leading to a more process-focused work strategy and broader information search compared to an offensive team strategic orientation. When teams needed critical information from the environment, defensive teams outperformed offensive teams; offensive teams performed better when critical information resided within the team. In Experiment 2, these findings were replicated with 92 teams performing a different decision task. When making a second decision, half of the teams were led to change their strategic orientation; teams shifting from offense to defense altered their information search behavior more readily than did teams shifting in the opposite direction, suggesting an asymmetric adaptation effect.
Psychologists have repeatedly shown that a single statistical factor—often called “general intelligence”—emerges from the correlations among people’s performance on a wide variety of cognitive tasks. But no one has systematically examined whether a similar kind of “collective intelligence” exists for groups of people. In two studies with 699 people, working in groups of two to five, we find converging evidence of a general collective intelligence factor that explains a group’s performance on a wide variety of tasks. This “c factor” is not strongly correlated with the average or maximum individual intelligence of group members but is correlated with the average social sensitivity of group members, the equality in distribution of conversational turn-taking, and the proportion of females in the group.
PRESS: What Google Learned From Its Quest to Build the Perfect Team (New York Times) | Forget IQ: The Emerging Science of Collective Intelligence (Time) | Why some teams are smarter than others (New York Times) | Social savvy boosts the collective intelligence of groups (Science) | A better way to pay workers (The New Yorker)
Students Supervised & Mentored
Sourav Das (MIT EECS), Project: Computational Sustainability, Spring 2018
Alice Jin (MIT EECS), Project: Superforecasters IARPA, Co-supervised, Fall 2017
Some of the students with positions/scholarships they acquired after the projects:
Aditi Mithal (Awarded Google Venkat Panchapakesan Memorial Scholarship, CS grad program at the University of California, Los Angeles)
Aditi Nath (CS grad program at Arizona State University)
Ankita Sastry (Info Security grad program Carnegie Mellon)
Karan Rajpal (CS grad program at Cornell University)
Nalin Chhibber (Mathematics grad program at the University of Waterloo)
Prastut Kumar (The Google Summer of Code Berkman Klein Center Harvard University)
Prithvi Raj (Awarded Dr. Shanker Dayal Sharma Medal at the IIT Kanpur)
Rahul Sheth (Undergraduate program at the University of California, Los Angeles)
Radhika Bhanu K (CS grad program at Cornell University)
William Dai (Undergraduate program at the University of California, Berkeley)
Teaching and mentoring have been an integral part of my quest― to help grow and inspire a new generation of creative thinkers, scientists, inventors, and engineers who will collectively engage in resolving some of the biggest societal-challenges of our time. I get a great satisfaction when my students succeed, and their accomplishments give me an immense sense of gratification that cannot be measured. I continue to supervise, mentor, and empower students to do impactful research. Similar to Dr. A.P.J. Abdul Kalam, I would like to be remembered as good teacher and mentor (listen to my interview at MIT's WMBR 88.1 FM Post-It Wall). Feel free to get in touch if I can help you in any ways.
I believe entrepreneurship can be taught and students can be trained to innovate technology ventures with a passion for making the world a better place. As the teaching assistant for S. Thomas Emerson, a charismatic teacher at Carnegie Mellon, I had an opportunity to guide various student projects. These projects focused on market research, product design, pricing, financial forecasting, and other factors needed to develop a strong business plan for technology-based ventures.
Outreach & Service
Program Committee 27th International Joint Conference on Artificial Intelligence and the 23rd European Conference on Artificial Intelligence IJCAI-ECAI 2018.
Reviewer 26th International World Wide Web Conference WWW 2017.
Reviewer iConference 2010, University of Illinois Urbana-Champaign.
Chair of the Graduate Student Seminar Series, Committee on Scholarly Interactions (CoSI), a graduate organization dedicated to bringing top thinkers to the MIT campus and the Sidney Pacific graduate residence, MIT's largest graduate community.
Started an initiative to address educational and environmental challenges in a village in Ahmednagar, Maharashtra state. As a first step, participated in the government's 20 million tree plantation program that aims to enhance the state forest density from 20% to 33%. Due to the worst drought of 2015, many farmers committed suicides and around 330 million people were affected in India.
Volunteered to teach kids in the Big Brothers Big Sisters program to build robots. This initiative was a part of 100 Robots for 100 Kids, a grassroots project by graduate students at the CMU Robotics Institute.
I'm also an artist and a progressive rock guitarist. My aerial photograph `Glacial Place' has featured on National Geographic. Through the lens of my camera, I spread awareness about cultures and burining societal issues across the globe (e.g., climate change, farmer's suicide, education, etc). You can follow my portfolio at `Explore the Planet Earth'
The Game of Cricket
My research draws a lot of inspiration from the sport and team dynamics. I was a former cricket player— opening batter and leg spinner. In the US leagues I also kept wickets (similar to being a catcher in Baseball) for Jermaine Lawson, former international cricketer and Harshal Patel, Royal Challengers Bangalore's fast-bowler in the IPL.
Some of the archived man of the match scorecards: [146 not out], [71 not out]
Sports (Life) Quote: "and so you touch this limit, something happens and you suddenly can go a little bit further. With your mind power, your determination, your instinct, and the experience as well, you can fly very high" -- Ayrton Senna, one of the greatest Formula One drivers of all time.
Networks Analysis of MOOR (Massive Open Online Research)
Where The Mind is Without Fear
Where knowledge is free
Where the world has not been broken up into fragments
By narrow domestic walls
Where words come out from the depth of truth
Where tireless striving stretches its arms towards perfection
Where the clear stream of reason has not lost its way
Into the dreary desert sand of dead habit
Where the mind is led forward by thee
Into ever-widening thought and action
Into that heaven of freedom, my Father, let my country awake.
Nobel Laureate Rabindranath Tagor
The Impossible Dream (The Quest)
To fight the unbeatable foe
To bear with unbearable sorrow
To run where the brave dare not go
To right the unrightable wrong | To love pure and chaste from afar
To try when your arms are too weary | To reach the unreachable star
This is my quest | To follow that star
No matter how hopeless | No matter how far
To fight for the right | Without question or pause
To be willing to march into Hell | For a heavenly cause
And I know if I'll only be true | To this glorious quest
That my heart will lie peaceful and calm | When I'm laid to my rest
And the world will be better for this
That one man, scorned and covered with scars
Still strove with his last ounce of courage
To reach the unreachable star!
Joe Darion (From The Man of La Mancha)