CURRENT FUNDING
National Science Foundation (NSF)
EAGER: Hybrid human-AI technologies for forecasting trends
Funding: $299,549 (2025-2027)
Role: PI
This project is the first comprehensive effort to build and test hybrid prediction markets for integrated human-AI forecasting, and thus is a high risk-high reward EAGER effort. The approach will fold in human input as other machine learning algorithms cannot, i.e., directly within the algorithm's deployment, showcasing an opportunity for the integration of first-person perspectives and machine learning for complex tasks where participatory design is desirable. Markets will be studied theoretically and in practice, and thoroughly compared against a suite of computational, crowdsourced and hybrid benchmarks, including supervised and semi-supervised algorithms, survey elicitation, and ensembling. The project considers community experiences of social media as a proving ground for design and development. In doing so, it is laying theoretical and empirical groundwork for future technologies supporting user governance in shared spaces online.
National Science Foundation (NSF)
SaTC: CORE: Small: Toward Privacy Equity through Contextual Understanding of Self-Disclosure
Funding: $599,851 (2023-2026)
Role: PI
The project addresses concerns about the privacy risks of personal information sharing in social media. Low socioeconomic and underrepresented populations report feeling especially concerned about their digital privacy in social media. The project team is studying how socioeconomic status and group identity influence the challenges of privacy self-management. Research is being undertaken to understand differences in self-disclosure attitudes and behaviors across socioeconomic groups. The research team is: conducting small- and big-data empirical studies of personal disclosure, developing data-informed descriptions of normative information flow and metrics for measurement of contextual integrity, and is experimenting with prototype technologies to bring equity to privacy management. The project is based on core principles of participatory design and governance. These principles are operationalized through broad community outreach and embedded processes for mutual learning, including field studies, community workshops, and cooperative prototyping.
Open Philanthropy & Center for Open Science (COS)
Benchmark: Whether LLM agents can evaluate, replicate, and independently conduct
research processes
Funding: $1,700,000 (2024-2026)
Role: Co-PI
The project is developing systematic benchmarks assessing how effectively large language models (LLMs) can evaluate, replicate, and conduct scientific research. The project looks at three LLM-as-researcher personas: “peer reviewer” persona, LLM agents will be assessed for their ability to evaluate the quality of research that has been completed; “replicator” persona, LLM agents will be assessed for their ability to design, conduct, and evaluate research that is trying to repeat prior research; and, “discovery scientist” persona, LLM agents will be assessed for their ability to generate and conduct novel research. Such abilities may be well-beyond current LLM agent capabilities, but a purpose of this framework is to specify targets that present and future innovations could aim to achieve.
A key activity of this grant will be to elaborate the framework and define relevant benchmarks within components for the broader research community to engage.
National Science Foundation (NSF)
HNDS-I: Digitally Accountable Public Representation
Funding: $649,380 (2019-2023)
Role: Co-PI
The project is assembling a database about public online discussions with prominent public figures in the U.S. across a variety of online platforms. These data are used to create measures of public figures' positions relevant to their prominent roles in society. Transparency, responsiveness, and accessibility are some of the aspects of online behavior that can be evaluated with these measures. The investigators use an experimental approach to determine the relationship between online communications and elites' responses. Some questions to be asked include: Are public figures more likely to respond quickly to comments when elections are close? When health and/or security threats emerge in the areas they represent, how quickly do public figures work to inform their followers? This project provides data, tools, and illustrative research to help many different types of users understand how prominent public figures engage online. Software for data analysis is also made available so others can analyze similar data.
COMPLETED FUNDING
Air Force Office of Scientific Research (AFOSR)
Toward Cognitive Realism in Game Theoretic Models of Social Behavior
Funding: $905,185 (2021-2024)
Role: Co-PI
The project is developing a unifying mathematical foundation by which to represent psychological elements in behavioral game theory. Proposed models will allow artificial agents engaged in strategic social planning to use representations of people, places and things that vary in their level of abstraction. This process is posited by Construal Level Theory and thought to support memory consolidation and planning. This work offers a new perspective on experience-based reasoning and complements current efforts in reinforcement learning. By allowing agents to reason over a dynamic environment including other agents and their construal processes, this approach can be used to simulate higher order social phenomena such as in-group/out-group categorization and emergence of norms. Initial evaluation and test embeds agents in a customized Minecraft environment to perform scavenger hunt tasks relying on varying degree of sociality.
Defense Advanced Research Projects Agency (DARPA)
Synthetic Prediction Markets with Algorithm Traders for Determining Experimental Reproducibility
Funding: $2,930,995 (2019-2023)
Role: Co-PI
The project developed artificial prediction markets to evaluate the reproducibility of published research claims in the social and behavioral science literatures. Markets are populated by artificial agents, trained and updated within human-expert prediction markets, but deployable offline. Artificial agents represent atomic properties of relevant signals, including full text of scientific papers, metadata for specific papers, and metadata about the community and the field. Agents (“trader bots”) learn trading patterns from subject matter experts engaged in prediction markets, but unlike their human counterparts, have comprehensive, unbiased view on the existing literature and metadata. The project team includes collaborators at Penn State ARL, Smeal College of Business, Texas A&M, Old Dominion University, and Microsoft Research.
National Science Foundation (NSF)
Visualizing and Enlarging the Statistics of Publication Information under Dynamics (VESPID)
Funding: $507,492 (2020-2022)
Role: Co-PI
The project, through the National Science Foundation's National Center for Science and Engineering Statistics (NCSES) studies trends in research and publishing, integrating data from the NSF's Survey of Doctorate Recipients, patents, and leading publication indicators with publication and bibliometrics analyses. The broader aim of the project is to inform data-driven policies on scientific processes and funding, with specific focus on underrepresented PhD recipients. The work is led by Quantitative Scientific Solutions, LLC.
National Science Foundation (NSF)
RAPID: Social Un-distancing: Understanding self-privacy violations in online communities during the Coronavirus pandemic
Funding: $200,000 (2020-2022)
Role: Co-PI
Perceptions of privacy evolve over time as concerns about security and privacy are tied to the day’s events, and the longer arc of shifting norms. Little is known more specifically about the unique evolution of privacy attitudes during crises, but understanding this is consequential. We posit that self-privacy violations during acute events can leave individuals susceptible to deviant actors at times when, critically, these individuals are most vulnerable. We know that victims of Hurricane Harvey sought assistance through social media, in some cases revealing their full names and addresses online. But what we are witnessing in the case of the Coronavirus pandemic is inherently distinct from previous crises in important ways. COVID-19 is a global, relatively protracted acute threat. Unlike natural disasters or military engagements, the pandemic has left communications infrastructure intact. Digital outlets have become lifelines. We posit that self-privacy violations during the Coronavirus pandemic can help individuals feel more socially connected during a time of anxiety and physical distance. Yet, this benefit may leave them susceptible to risk when, critically, these individuals are most vulnerable. In this project, we propose to study online self-disclosure during the Coronavirus epidemic. This research will help us detect acts of self-disclosure in online settings and learn how oversharing is expedited or even encouraged during the coronavirus crisis.
Office of Naval Research (ONR)
Sociolinguistic Information Filtering Tool (SIFT)
Funding: $150,000 (2020-2021)
Role: Co-PI
The Sociolinguistic Information Filtering Tool (SIFT) project developed a desktop extension and associated mobile app or account filtering and user-initiated bulk action toward automated, malicious, or otherwise unwanted accounts on Twitter. The work was led by Quantitative Scientific Solutions, LLC through the STTR program.