ॐ असतो मा सद्गमय । तमसो मा ज्योतिर्गमय । मृत्योर्मा अमृतं गमय ।।
(02) 2017 - invited at the HPI-Stanford Design Thinking Research Program.
(01) 2017 - hosted MIT Emerging Worlds workshops and guided inventors across India to solve pressing societal challenges in agriculture and healthcare.
(10) 2016 - presented the paper at the ACM UIST, Tokyo, Japan.
(09) 2016 - paper got accepted at the ACM CSCW, Portland, USA.
(06) 2016 - Daemo, crowdsourcing marketplace helped build the Stanford Question Answering Dataset.
(01) 2016 - one of the finalists in the Knights News Challenge (top 20 of 1,000+). Led the efforts and co-authored the grant.
(09) 2015 - Daemo, crowdsourcing marketplace completed its first project with Microsoft Research.
(10) 2014 - EteRNA got listed as one of the 25 great ideas invented at the Carnegie Mellon SCS.
(02) 2014 - paper got accepted to PNAS.
(01) 2011 - co-launched EteRNA, crowd computing game that helps invent medicine.
I'm a graduate student in the Scalable Cooperation research group at MIT Media Lab. I'm very fortunate to be advised by Iyad Rahwan. My research interests fall at the interface of Human Computer Interaction, Cognitive Artificial Intelligence, Interactive Machine Learning, and Algorithmic Game Theory. I design, engineer, and empirically analyze self-organizing socio-technical systems that fuse human and artificial intelligence to enable massive-scale collaboration. My research emphasizes harnessing computational and design thinking to solve pressing societal challenges with application to market design, AI ethics, open-governance, crisis response, and democratization of education. At MIT Media Lab, I'm a part of Space Exploration and Innovating for Billions in Emerging Worlds initiatives.
I'm one of the principal creators of Daemo, a Self Governed Crowdsourcing Marketplace. I led its technical architecture. My contributions draw on fundamentals from algorithmic economics (mechanism design) and structured finance (guilds organization as tranches), with aspiration to augment trust in sharing economies. I was also one of the principal creators, inventors and founding members of EteRNA, crowdcomputing game that harnesses human creativity with machine intelligence to help invent medicine. EteRNA has reached over 100,000 citizen scientists across the world and was listed as one of 25 great ideas emerged from the CMU School of Computer Science.
I had (and continue to have) the great fortune to work with Michael S. Bernstein at the Computer Science department at Stanford University. Previously, I earned my M.S. from the School of Computer Science at Carnegie Mellon where I spent wonderful years working with Adrien Treuille at the Robotics Institute Graphics Lab as well as Anita W. Woolley at the OBT Group. I received my B.E. from the University of Pune and completed my undergraduate research at the Machine Learning Group at TATA Research (TRDDC) SRL.
I come from Nagar, a small town near the Western Ghats (Sahyadri in Sanskrit) of India. One of the major missions of my life is to open up the research and educational opportunities for students across the globe. In this pursuit, I moved back to academia from Wall Street (mathematical finance industry). Along with Rajan Vaish and Michael S. Bernstein at Stanford University, I'm working on democratizing the research process though Massive Open Online Research.
ॐ असतो मा सद्गमय ।
तमसो मा ज्योतिर्गमय ।
मृत्योर्मा अमृतं गमय ।।
Boomerang: Rebounding the Consequences of Reputation Feedback on Crowdsourcing Platforms
UIST 2016: ACM Symposium on User Interface Software and Technology
Paid crowdsourcing platforms suffer from low-quality work and unfair rejections, but paradoxically, most workers and requesters have high reputation scores. These inflated scores, which make high-quality work and workers difficult to find, stem from social pressure to avoid giving negative feedback. We introduce Boomerang, a reputation system for crowdsourcing that elicits more accurate feedback by rebounding the consequences of feedback directly back onto the person who gave it. With Boomerang, requesters find that their highly-rated workers gain earliest access to their future tasks, and workers find tasks from their highly-rated requesters at the top of their task feed. Field experiments verify that Boomerang causes both workers and requesters to provide feedback that is more closely aligned with their private opinions. Inspired by a game-theoretic notion of incentive-compatibility, Boomerang opens opportunities for interaction design to incentivize honest reporting over strategic dishonesty.
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work
Crowd workers are distributed and decentralized. While decentralization is designed to utilize independent judgment to promote high-quality results, it paradoxically undercuts behaviors and institutions that are critical to high-quality work. Reputation is one central example: crowdsourcing systems depend on reputation scores from decentralized workers and requesters, but these scores are notoriously inflated and uninformative. In this paper, we draw inspiration from historical worker guilds (e.g., in the silk trade) to design and implement crowd guilds: centralized groups of crowd workers who collectively certify each other’s quality through double-blind peer assessment. A two week field experiment compared crowd guilds to a traditional decentralized crowd work model. Crowd guilds produced reputation signals more strongly correlated with ground-truth worker quality than signals available on current platforms, and more accurate than in the traditional model.
RNA Design Rules From a Massive Open Laboratory
Gaikwad, S., Yoon, S., Treuille, A., Das, R., and EteRNA Participants
PNAS 2014: Proceedings of the National Academy of Sciences of the United States of America
Self-assembling RNA molecules present compelling substrates for the rational interrogation and control of living systems. However, imperfect in silico models—even at the secondary structure level—hinder the design of new RNAs that function properly when synthesized. Here, we present a unique and potentially general approach to such empirical problems: the Massive Open Laboratory. The EteRNA project connects 37,000 enthusiasts to RNA design puzzles through an online interface. Uniquely, EteRNA participants not only manipulate simulated molecules but also control a remote experimental pipeline for high-throughput RNA synthesis and structure mapping. We show herein that the EteRNA community leveraged dozens of cycles of continuous wet laboratory feedback to learn strategies for solving in vitro RNA design problems on which automated methods fail. The top strategies—including several previously unrecognized negative design rules-were distilled by machine learning into an algorithm, EteRNABot. Over a rigorous 1-y testing phase, both the EteRNA community and EteRNABot significantly outperformed prior algorithms in a dozen RNA secondary structure design tests, including the creation of dendrimer-like structures and scaffolds for small molecule sensors. These results show that an online community can carry out large-scale experiments, hypothesis generation, and algorithm design to create practical advances in empirical science.
Crowdcomputing and Citizen Science for Large-Scale Experiments
IC2S2 2017: International Conference on Computational Social Science
Historically, scientific experiments have been conducted at a small scale either with artificial environments or with the expertise of limited number of scientists. While social science literature investigates very deep questions to understand human behavior, many experiments are usually limited by the number of participants and duration of a study. On the contrary, computer science literature exploits advanced computational techniques to crunch voluminous datasets, but research designs are generally not experimental, which limits the opportunity to generate causal inferences. In this tutorial we demonstrate how crowdcomputing can enable computational social scientists to engage with millions of users on the Internet and study human behavior at scale for a longer time. We showcase pitfalls and lessons learned from various crowdcomputing and citizen science projects. Furthermore, we provide insights about how to build a sustainable citizen science community to scale science beyond the traditional laboratories. We envisage this tutorial will help computational social scientists effectively use crowdcomputing to investigate deep research questions and longitudinally validate their hypotheses in large scale experiments.
The Daemo Crowdsourcing Marketplace
CSCW 2017: ACM Conference on Computer-Supported Cooperative Work
The success of crowdsourcing markets is dependent on a strong foundation of trust between workers and requesters. In current marketplaces, workers and requesters are often unable to trust each other’s quality, and their mental models of tasks are misaligned due to ambiguous instructions or confusing edge cases. This breakdown of trust typically arises from (1) flawed reputation systems which do not accurately reflect worker and requester quality, and from (2) poorly designed tasks. In this demo, we present how Boomerang and Prototype Tasks, the fundamental building blocks of the Daemo crowdsourcing marketplace, help restore trust between workers and requesters. Daemo’s Boomerang reputation system incentivizes alignment between opinion and ratings by determining the likelihood that workers and requesters will work together in the future based on how they rate each other. Daemo’s Prototype tasks require that new tasks go through a feedback iteration phase with a small number of workers so that requesters can revise their instructions and task designs before launch.
CI 2017: Collective Intelligence Conference
A key principle of Daemo is to provide crowd workers and requesters with a means of governing the development and evolution of the platform into the future. To facilitate this, we introduced the Daemo constitution, outlining the goals of the platform, relationship between members of the community and its standards, methods for seeking ideas, amending the constitution and resolving conflicts. The Daemo constitution has been a collaborative effort by the Stanford Crowd Research Collective. However, we have also endeavored to solicit feedback from the MTurk worker and requester communities via TurkerNation, Reddit and other channels.
PDF AVAILABLE UPON REQUEST
UIST 2015: ACM Symposium on User Interface Software and Technology
Crowdsourcing marketplaces provide opportunities for autonomous and collaborative professional work as well as social engagement. However, in these marketplaces, workers feel disrespected due to unreasonable rejections and low payments, whereas requesters do not trust the results they receive. The lack of trust and uneven distribution of power among workers and requesters have raised serious concerns about sustainability of these marketplaces. To address the challenges of trust and power, this paper introduces Daemo, a self-governed crowdsourcing marketplace. We propose a prototype task to improve the work quality and open-governance model to achieve equitable representation. We envisage Daemo will enable workers to build sustainable careers and provide requesters with timely, quality labor for their businesses.
Simoiu, C., Veit, A., Wilber, M., Zhou, S., Belongie, S., Goel, S., Davis, J., Bernstein, M.
HCI+Design Open House 2016: Stanford University
Scientific research is becoming increasingly collaborative, yet primarily limited to professional researchers in labs and universities. Can we scale traditional research approach and invite anyone from around the world to participate, and crowdsource large-scale, open-ended research problems? We're scaling the research process to solve open-ended novel problems, by providing access and connecting hundreds of people with top researchers.
Poster Click to see my detailed analysis of PageRank & Credit Distribution
BioX 2011: Interdisciplinary Initiatives Symposium Poster Session, Stanford University
The Effects of Team Strategic Orientation On Team Process and Information Search
Organizational Behavior and Human Decision Processes, 2013
We tested the effects of team strategic orientation on team member perceptions, work strategy and information search. In Experiment 1, 80 teams worked on a hidden profile decision-making task. A defensive team strategic orientation increased members’ perceptions of the problem’s scope, leading to a more process-focused work strategy and broader information search compared to an offensive team strategic orientation. When teams needed critical information from the environment, defensive teams outperformed offensive teams; offensive teams performed better when critical information resided within the team. In Experiment 2, these findings were replicated with 92 teams performing a different decision task. When making a second decision, half of the teams were led to change their strategic orientation; teams shifting from offense to defense altered their information search behavior more readily than did teams shifting in the opposite direction, suggesting an asymmetric adaptation effect.
Psychologists have repeatedly shown that a single statistical factor—often called “general intelligence”—emerges from the correlations among people’s performance on a wide variety of cognitive tasks. But no one has systematically examined whether a similar kind of “collective intelligence” exists for groups of people. In two studies with 699 people, working in groups of two to five, we find converging evidence of a general collective intelligence factor that explains a group’s performance on a wide variety of tasks. This “c factor” is not strongly correlated with the average or maximum individual intelligence of group members but is correlated with the average social sensitivity of group members, the equality in distribution of conversational turn-taking, and the proportion of females in the group.
Students Supervised & Mentored
Some of the students with positions/scholarships they acquired after the projects:
Aditi Mithal (Awarded Google Venkat Panchapakesan Memorial Scholarship)
Aditi Nath (CS grad program at Arizona State University)
Ankita Sastry (Info Security grad program Carnegie Mellon)
Karan Rajpal (CS grad program at Cornell University)
Nalin Chhibber (Mathematics grad program at the University of Waterloo)
Prastut Kumar (The Google Summer of Code Berkman Klein Center Harvard University)
Prithvi Raj (Awarded Dr. Shanker Dayal Sharma Medal at the IIT Kanpur)
Rahul Sheth (Undergraduate program at the University of California, Los Angeles)
Radhika Bhanu K (CS grad program at Cornell University)
William Dai (Undergraduate program at the University of California, Berkeley)
I find supervising, mentoring, and empowering students to do amazing research work very rewarding. Similar to Dr. A.P.J. Abdul Kalam, I would like to be remembered as good teacher and mentor. I believe entrepreneurship can be taught and students can be trained to innovate technology ventures with a passion for making the world a better place. As the teaching assistant for S. Thomas Emerson, a charismatic teacher at Carnegie Mellon, I had an opportunity to guide various student projects. These projects focused on market research, product design, pricing, financial forecasting, and other factors needed to develop a strong business plan for technology-based ventures.
Outreach & Service
Reviewer 26th International World Wide Web Conference WWW 2017.
Reviewer iConference 2010, University of Illinois Urbana-Champaign.
Co-chair of the Committee on Scholarly Interactions (CoSI), a graduate organization dedicated to bringing top thinkers to the MIT campus and the Sidney Pacific graduate residence, MIT's largest graduate community.
Started an initiative to address educational and environmental challenges in a village in Ahmednagar, Maharashtra state. As a first step, participated in the government's 20 million tree plantation program that aims to enhance the state forest density from 20% to 33%. Due to the worst drought of 2015, many farmers committed suicides and around 330 million people were affected in India. I'm looking forward to redefine the ways in which agriculture is being done.
Volunteered to teach kids in the Big Brothers Big Sisters program to build robots. This initiative was a part of 100 Robots for 100 Kids, a grassroots project by graduate students at the CMU Robotics Institute.
The Game of Cricket
My research draws a lot of inspiration from the sport and team dynamics. I was a former cricket player— opening batter and leg spinner. In the US leagues I also kept wickets (similar to being a catcher in Baseball) for Jermaine Lawson, former international cricketer and Harshal Patel, Royal Challengers Bangalore's fast-bowler in the IPL.
Some of the archived man of the match scorecards: [146 not out], [71 not out]
Sports (Life) Quote: "and so you touch this limit, something happens and you suddenly can go a little bit further. With your mind power, your determination, your instinct, and the experience as well, you can fly very high" -- Ayrton Senna, one of the greatest Formula One drivers of all time.
Networks Analysis of MOOR (Massive Open Online Research)
Play with the RNA
Where The Mind is Without Fear
Where knowledge is free
Where the world has not been broken up into fragments
By narrow domestic walls
Where words come out from the depth of truth
Where tireless striving stretches its arms towards perfection
Where the clear stream of reason has not lost its way
Into the dreary desert sand of dead habit
Where the mind is led forward by thee
Into ever-widening thought and action
Into that heaven of freedom, my Father, let my country awake.
Nobel Laureate Rabindranath Tagor
The Impossible Dream (The Quest)
To fight the unbeatable foe
To bear with unbearable sorrow
To run where the brave dare not go
To right the unrightable wrong | To love pure and chaste from afar
To try when your arms are too weary | To reach the unreachable star
This is my quest | To follow that star
No matter how hopeless | No matter how far
To fight for the right | Without question or pause
To be willing to march into Hell | For a heavenly cause
And I know if I'll only be true | To this glorious quest
That my heart will lie peaceful and calm | When I'm laid to my rest
And the world will be better for this
That one man, scorned and covered with scars
Still strove with his last ounce of courage
To reach the unreachable star!
Joe Darion (From The Man of La Mancha)