(05) 2018 - received the Graduate Teaching Award, presented annually to one MIT professor or teaching assistant from each school, for excellence in teaching a graduate level course.
(04) 2018 - invited talk, "Human-AI Collaboration for Sustainable Market Design", at the Ethics of AI event by the Media Lab and Berkman Klein Center, Harvard University.
(03) 2018 - program committee of the 27th International Joint Conference on Artificial Intelligence (IJCAI).
(01) 2018 - represented Daemo at a European Dialog on the Platform Economy event by the European Trade Union Institute and partners, Brussels.
(12) 2017 - awarded the MIT Graduate Student Life Grant for the MIT SP Graduate Student Dinner Seminar Series initiative.
(11) 2017 - our paper got accepted at the AAAI-2018, New Orleans, USA.
(11) 2017 - TEDx talk "the Future of Markets in the Era of AI", Boston.
(10) 2017 - invited to the Industry Panel at the AAAI HCOMP conference, Canada― decided to step down to increase the panel's gender diversity.
(09) 2017 - invited at the Hasso Plattner Institute and Stanford Design Thinking Research Program, Potsdam Germany.
(01) 2017 - guided inventors across India to solve pressing societal challenges in agriculture and healthcare using AI [MIT, Innovating for Billions].
(06) 2016 - EteRNA featured in Werner Herzog's film ``Lo And Behold: Reveries of the Connected World’’ (Trailer).
(09) 2015 - Daemo, crowdsourcing marketplace completed its first project with Microsoft Research.
(02) 2014 - our paper got accepted to PNAS.
(01) 2011 - co-launched EteRNA, crowd-computing game that helps design RNA molecules and invent medicine.
I conduct research at the intersection of interactive machine learning, human-centered design, and algorithmic economics (computational social choice). My research envisions theoretical and practical innovations to design and engineer human-AI powered socio-technical systems, such as markets and institutions, for sustainable development. For centuries, these socio-technical systems have been instrumental to civilization, but over the last few decades people's confidence has begun to erode. The financial crises of 2008, black swan events such as the flash crash of 2:45, and unfair practices in gig-economies (e.g, Amazon Mechanical Turk, Uber) have threatened the livelihood of many. The sustainable innovations to examine and address these societal challenges can only emerge from the convergence of informed research at the boundaries of design, policy, and computational social sciences. My TEDx talk, the Future of Markets in the Era of Artificial Intelligence, highlights this innovation philosophy.
My research has been published in top-tier artificial intelligence and human-computer interaction conferences (AAAI, ACM UIST, ACM CSCW) and a scientific journal (PNAS), and featured in the New York Times, Bloomberg, WIRED, and the Wall Street Journal. It has a wide range of applications in human-centered AI, gig economy design, the future of creative work, crisis response, urban planning, and financial inclusion. Some of the examples of socio-technical system research include Daemo, a self-governed crowdsourcing market, and EteRNA, a crowdcomputing game that harnesses human-machine collaboration to help design RNA molecules, the dark matter of biology. I am one of the principal creators of Daemo. I led its technical architecture and invented Boomerang, an incentive-compatible reputation system. My contributions draw on fundamentals from game theory (incentive design for pro-social behavior), structured finance (guilds organization as tranches), and human-centered design (tasks composition) with aspirations to engineer the foundations of the future of work. I was also one of the principal creators and founding members EteRNA. EteRNA is an example of how human-machine collaboration can help solve computational science problems that neither humans nor machines can solve alone. It has reached over 100,000 citizen scientists across the world, and was featured in Werner Herzog's film "Lo And Behold: Reveries of the Connected World" (the trailer).
I am a PhD student in the Space Enabled group led by Danielle Wood at the MIT Media Lab. I earned my M.S. from the school of computer science at Carnegie Mellon where I worked with Adrien Treuille at the Graphics Group at the Robotics Institute and Anita Woolley at the organizational behavior and theory group. I have collaborated with Iyad Rahwan at MIT, Ariel Procaccia and Pradeep Ravikumar from Carnegie Mellon, and Michael Bernstein from Stanford University.
I come from the Western Ghats of India and one of the major missions of my life is to open up the STEM research and educational opportunities for people across the globe. In this pursuit, I transitioned from the quantitative finance industry (Wall Street) to academia. I was one of the core members of the Stanford Crowd Research Initiative and a part of MIT's Innovating for Billions in Emerging Worlds Leadership Council. In 2018, I received the Graduate Teaching Award, presented annually to one MIT professor or teaching assistant from each school, for excellence in teaching a graduate level course.
Neil S. Gaikwad
MIT Media Lab
75 Amherst St
Cambridge MA, 02139
firstname.lastname@example.org | @neilsgaikwad
Recent awards and honors
Graduate Teaching Award, MIT
Adobe Research PhD Fellowship Finalist
Best Paper Honorable Mention, ACM UIST
Talks and travel
|Summer||Santa Fe Institute|
|April||AI-Ethics, Berkman Harvard|
|February||Wadhwani AI, Mumbai|
|November||AI and Markets, TEDx|
A Voting-Based System for Ethical Decision Making
AAAI 2018: Association for the Advancement of Artificial Intelligence. (Acceptance rate of 24.6%)
We present a general approach to automating ethical decisions, drawing on machine learning and computational social choice. In a nutshell, we propose to learn a model of societal preferences, and, when faced with a specific ethical dilemma at runtime, efficiently aggregate those preferences to identify a desirable choice. We provide a concrete algorithm that instantiates our approach; some of its crucial steps are informed by a new theory of swap-dominance efficient voting rules. Finally, we implement and evaluate a system for ethical decision making in the autonomous vehicle domain, using preference data collected from 1.3 million people through the Moral Machine website.
PRESS: Social Media Has Failed Its Self-Driving Test (Bloomberg) | Researchers go after the biggest problem with self-driving cars (Axios)
Belongie, Serge., Goel, Sharad., Davis, James., and Bernstein, Michael.
UIST 2017: ACM Symposium on User Interface Software and Technology, (Acceptance rate of 22.5%)
Research experiences today are limited to a privileged few at select universities. Providing open access to research experiences would enable global upward mobility and increased diversity in the scientific workforce. But, how do we coordinate a crowd of diverse volunteers on open-ended research? How could a PI have enough visibility into each person's contributions to recommend them for further study? We present Crowd Research, a crowdsourcing technique that coordinates open-ended research through an iterative cycle of open contribution, synchronous collaboration, and peer assessment. To aid upward mobility and recognize contributions in publications, we introduce a decentralized credit system: participants allocate credits to each other, which a graph centrality algorithm translates into a collectively-created author order. Over 1,500 people from 62 countries have participated, 74% from institutions with low access to research. Over two years and three projects, this crowd has produced articles at top-tier Computer Science venues, and participants have gone on to leading graduate programs.
PRESS: A Stanford-led Platform for Crowdsourced Research Gives Experience to Global Participants (Stanford News)
PDF Web [ACM UIST Best Paper Honorable Mention, top 2.5% of the technical paper submissions]
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work and Social Computing (Acceptance rate of 25%)
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other’s quality through double-blind peer assessment. A two week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current platforms, and more accurate than in the traditional model.
UIST 2016: ACM Symposium on User Interface Software and Technology, (Acceptance rate of 20.6%)
Paid crowdsourcing platforms suffer from low-quality work and unfair rejections, but paradoxically, most workers and requesters have high reputation scores. These inflated scores, which make high-quality work and workers difficult to find, stem from social pressure to avoid giving negative feedback. We introduce Boomerang, a reputation system for crowdsourcing that elicits more accurate feedback by rebounding the consequences of feedback directly back onto the person who gave it. With Boomerang, requesters find that their highly-rated workers gain earliest access to their future tasks, and workers find tasks from their highly-rated requesters at the top of their task feed. Field experiments verify that Boomerang causes both workers and requesters to provide feedback that is more closely aligned with their private opinions. Inspired by a game-theoretic notion of incentive-compatibility, Boomerang opens opportunities for interaction design to incentivize honest reporting over strategic dishonesty.
* In the readings of advanced Human Computation classes at KAIST, Cornell University, University of Washington
RNA Design Rules From a Massive Open Laboratory
Gaikwad, S., Yoon, Sungroh., Treuille, Adrien., Das, Rhiju., and EteRNA Participants
PNAS 2014: Proceedings of the National Academy of Sciences of the United States of America
Self-assembling RNA molecules present compelling substrates for the rational interrogation and control of living systems. However, imperfect in silico models—even at the secondary structure level—hinder the design of new RNAs that function properly when synthesized. Here, we present a unique and potentially general approach to such empirical problems: the Massive Open Laboratory. The EteRNA project connects 37,000 enthusiasts to RNA design puzzles through an online interface. Uniquely, EteRNA participants not only manipulate simulated molecules but also control a remote experimental pipeline for high-throughput RNA synthesis and structure mapping. We show herein that the EteRNA community leveraged dozens of cycles of continuous wet laboratory feedback to learn strategies for solving in vitro RNA design problems on which automated methods fail. The top strategies—including several previously unrecognized negative design rules-were distilled by machine learning into an algorithm, EteRNABot. Over a rigorous 1-y testing phase, both the EteRNA community and EteRNABot significantly outperformed prior algorithms in a dozen RNA secondary structure design tests, including the creation of dendrimer-like structures and scaffolds for small molecule sensors. These results show that an online community can carry out large-scale experiments, hypothesis generation, and algorithm design to create practical advances in empirical science.
PRESS: Videogamers are recruited to fight Tuberculosis and other ills (Wall Street Journal) | RNA game lets players help find a biological prize (New York Times) | How turning science into a game rouses more public interest (WIRED) | Game lets citizen scientists participate in creating large-scale library of synthetic RNA designs (Scientific American) | Rebooting science outreach (ASBMB Today) | Why video games are key to modern science (CNN) | Online game helps predict how RNA folds (New Scientist)
Crowdcomputing and Citizen Science for Large-Scale Experiments
IC2S2 2017: International Conference on Computational Social Science
Historically, scientific experiments have been conducted at a small scale either with artificial environments or with the expertise of limited number of scientists. While social science literature investigates very deep questions to understand human behavior, many experiments are usually limited by the number of participants and duration of a study. On the contrary, computer science literature exploits advanced computational techniques to crunch voluminous datasets, but research designs are generally not experimental, which limits the opportunity to generate causal inferences. In this tutorial we demonstrate how crowdcomputing can enable computational social scientists to engage with millions of users on the Internet and study human behavior at scale for a longer time. We showcase pitfalls and lessons learned from various crowdcomputing and citizen science projects. Furthermore, we provide insights about how to build a sustainable citizen science community to scale science beyond the traditional laboratories. We envisage this tutorial will help computational social scientists effectively use crowdcomputing to investigate deep research questions and longitudinally validate their hypotheses in large scale experiments.
Prototype Tasks: Improving Crowdsourcing Results through Rapid, Iterative Task Design
AAAI HCOMP 2017: AAAI Conference on Human Computation
Low-quality results have been a long-standing problem on microtask crowdsourcing platforms, driving away requesters and justifying low wages for workers. To date, workers have been blamed for low-quality results: they are said to make as little effort as possible, do not pay attention to detail, and lack expertise. In this paper, we hypothesize that requesters may also be responsible for low-quality work: they launch unclear task designs that confuse even earnest workers, under-specify edge cases, and neglect to include examples. We introduce prototype tasks, a crowdsourcing strategy requiring all new task designs to launch a small number of sample tasks. Workers attempt these tasks and leave feedback, enabling the requester to iterate on the design before publishing it. We report a field experiment in which tasks that underwent prototype task iteration produced higher-quality work results than the original task designs. With this research, we suggest that a simple and rapid iteration cycle can improve crowd work, and we provide empirical evidence that requester “quality” directly impacts result quality.
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work
The success of crowdsourcing markets is dependent on a strong foundation of trust between workers and requesters. In current marketplaces, workers and requesters are often unable to trust each other’s quality, and their mental models of tasks are misaligned due to ambiguous instructions or confusing edge cases. This breakdown of trust typically arises from (1) flawed reputation systems which do not accurately reflect worker and requester quality, and from (2) poorly designed tasks. In this demo, we present how Boomerang and Prototype Tasks, the fundamental building blocks of the Daemo crowdsourcing marketplace, help restore trust between workers and requesters. Daemo’s Boomerang reputation system incentivizes alignment between opinion and ratings by determining the likelihood that workers and requesters will work together in the future based on how they rate each other. Daemo’s Prototype tasks require that new tasks go through a feedback iteration phase with a small number of workers so that requesters can revise their instructions and task designs before launch.
CI 2017: Collective Intelligence Conference
A key principle of Daemo is to provide crowd workers and requesters with a means of governing the development and evolution of the platform into the future. To facilitate this, we introduced the Daemo constitution, outlining the goals of the platform, relationship between members of the community and its standards, methods for seeking ideas, amending the constitution and resolving conflicts. The Daemo constitution has been a collaborative effort by the Stanford Crowd Research Collective. However, we have also endeavored to solicit feedback from the MTurk worker and requester communities via TurkerNation, Reddit and other channels.
PDF AVAILABLE UPON REQUEST
UIST 2015: ACM Symposium on User Interface Software and Technology
Crowdsourcing marketplaces provide opportunities for autonomous and collaborative professional work as well as social engagement. However, in these marketplaces, workers feel disrespected due to unreasonable rejections and low payments, whereas requesters do not trust the results they receive. The lack of trust and uneven distribution of power among workers and requesters have raised serious concerns about sustainability of these marketplaces. To address the challenges of trust and power, this paper introduces Daemo, a self-governed crowdsourcing marketplace. We propose a prototype task to improve the work quality and open-governance model to achieve equitable representation. We envisage Daemo will enable workers to build sustainable careers and provide requesters with timely, quality labor for their businesses.
PRESS: The future of work the peoples uber (Pacific Standard)
PDF Poster System
Simoiu, Camelia., Veit, Andreas., Wilber, Michael., Zhou, Sharon., Belongie, Serge., Goel, Sharad., Davis, James., Bernstein, Michael.
HCI+Design Open House 2016: Stanford University
Scientific research is becoming increasingly collaborative, yet primarily limited to professional researchers in labs and universities. Can we scale traditional research approach and invite anyone from around the world to participate, and crowdsource large-scale, open-ended research problems? We're scaling the research process to solve open-ended novel problems, by providing access and connecting hundreds of people with top researchers.
Click to see my detailed analysis of PageRank & Credit Distribution
BioX 2011: Interdisciplinary Initiatives Symposium Poster Session, Stanford University
Mentored as a teaching assistant
Conference on Fairness, Accountability, and Transparency (FAT), 2018
ACM CHI Conference on Human Factors in Computing Systems, 2017
Organizational Behavior and Human Decision Processes, 2013
Students Supervised & Mentored
Sourav Das (MIT EECS), Project: Computational Sustainability, Spring 2018
Alice Jin (MIT EECS), Project: Superforecasters IARPA, Co-supervised, Fall 2017
Some of the students with positions/scholarships they acquired after the projects:
Aditi Mithal (Awarded Google Venkat Panchapakesan Memorial Scholarship, CS grad program at the University of California, Los Angeles)
Aditi Nath (CS grad program at Arizona State University)
Ankita Sastry (Info Security grad program Carnegie Mellon)
Karan Rajpal (CS grad program at Cornell University)
Nalin Chhibber (Mathematics grad program at the University of Waterloo)
Prastut Kumar (The Google Summer of Code Berkman Klein Center Harvard University)
Prithvi Raj (Awarded Dr. Shanker Dayal Sharma Medal at the IIT Kanpur)
Rahul Sheth (Undergraduate program at the University of California, Los Angeles)
Radhika Bhanu K (CS grad program at Cornell University)
William Dai (Undergraduate program at the University of California, Berkeley)
Teaching and research mentoring have been an integral part of my quest― to help grow and inspire a new generation of creative thinkers, scientists, inventors, and engineers who will collectively engage in resolving some of the biggest societal-challenges of our time. I get a great satisfaction when my students succeed, and their accomplishments give me an immense sense of gratification that cannot be measured. I continue to supervise, mentor, and empower students to do impactful research. Similar to Dr. A.P.J. Abdul Kalam, I would like to be remembered as good teacher and research mentor (listen to my interview at MIT's WMBR 88.1 FM Post-It Wall). Feel free to get in touch if I can help you in any ways.
I believe entrepreneurship can be taught and students can be trained to innovate technology ventures with a passion for making the world a better place. As the teaching assistant for S. Thomas Emerson, a charismatic teacher at Carnegie Mellon, I had an opportunity to guide various student projects. These projects focused on market research, product design, pricing, financial forecasting, and other factors needed to develop a strong business plan for technology-based ventures.
Outreach & Service
AI and HCI Conferences
Program committee, 27th International Joint Conference on Artificial Intelligence Conference on Artificial Intelligence IJCAI-ECAI 2018
Reviewer, 21st ACM CSCW 2018
Reviewer, 26th Int. World Wide Web Conference WWW 2017
Reviewer, iConference, University of Illinois Urbana-Champaign, 2010
Service at MIT
Chair, MIT SP Committee on Scholarly Interactions (CoSI), a graduate organization dedicated to bringing top scholars to the MIT campus and the Sidney Pacific graduate residence
Lead and co-founder, MIT SP Graduate Student Dinner Seminar Series, (visit)
Leadership Council, MIT's Innovating for Billions in Emerging Worlds Media Lab Design Rep for the, MIT GradRat Ring
Service at Carnegie Mellon Robotics Institute
Volunteer Robotics Teacher, taught s kids in the Big Brothers Big Sisters program to build robots. This initiative was a part of 100 Robots for 100 Kids, a grassroots project by graduate students at the CMU Robotics Institute.
Co-organizer, EteRNA sessions for Leap@CMU, summer enrichment program for high school students.
Students recruitment from Carnegie Mellon
I'm a photographer and progressive rock guitarist. My aerial photograph `Glacial Place' has been featured on National Geographic. I think photography is an art of inquiry that helps us better understand nature, landscapes, cultural evolution, and burning societal issues such as climate change. I maintain my photography portfolio at `Explore the Planet Earth'.
The Game of Cricket
My research draws a lot of inspiration from the sport and team dynamics. I was a former cricket player— opening batter and leg spinner. In the US leagues I also kept wickets (similar to being a catcher in Baseball) for Jermaine Lawson, former international cricketer and Harshal Patel, Royal Challengers Bangalore's fast-bowler in the IPL.
Some of the archived man of the match scorecards: [146 not out], [71 not out]
Sports (Life) Quote: "and so you touch this limit, something happens and you suddenly can go a little bit further. With your mind power, your determination, your instinct, and the experience as well, you can fly very high" -- Ayrton Senna, one of the greatest Formula One drivers of all time.
Networks in the Massive Open Online Research
Where The Mind is Without Fear
Where knowledge is free
Where the world has not been broken up into fragments
By narrow domestic walls
Where words come out from the depth of truth
Where tireless striving stretches its arms towards perfection
Where the clear stream of reason has not lost its way
Into the dreary desert sand of dead habit
Where the mind is led forward by thee
Into ever-widening thought and action
Into that heaven of freedom, my Father, let my country awake.
Nobel Laureate Rabindranath Tagor
The Impossible Dream (The Quest)
To fight the unbeatable foe
To bear with unbearable sorrow
To run where the brave dare not go
To right the unrightable wrong | To love pure and chaste from afar
To try when your arms are too weary | To reach the unreachable star
This is my quest | To follow that star
No matter how hopeless | No matter how far
To fight for the right | Without question or pause
To be willing to march into Hell | For a heavenly cause
And I know if I'll only be true | To this glorious quest
That my heart will lie peaceful and calm | When I'm laid to my rest
And the world will be better for this
That one man, scorned and covered with scars
Still strove with his last ounce of courage
To reach the unreachable star!
Joe Darion (From The Man of La Mancha)