Presentation 1 
7 - Bridge2AI Goals, Roles, and Watering holes: A playbook for the leading edge of team science
Presented by: Tursynay Issabekova, University of Colorado
Authored by: Julie Bletz, Sage Bionetworks
James A. Eddy, University of Colorado
Kaitlin Flynn, University of Colorado
Sarah Gehrke, University of Colorado
Melissa Haendel, University of Colorado
Tursynay Issabekova, University of Colorado
Sruthi Magesh, University of Colorado
Julie A. McMurry, University of Colorado
Monica Munoz-Torres, University of Colorado
Anh Nguyen, University of Colorado
Shawn O’Neil, University of Colorado
Anne Thessen, University of Colorado
Anita Walden, University of Colorado

The most pressing biomedical challenges of our time require collaboration across disciplinary and institutional boundaries. Over the last two decades it has become clearer how to more successfully approach this; however, there are often few resources and infrastructure available to apply known team science best practices to data-intensive research. Further, methods for evaluating collaboration and the multivariate effects of individual and team characteristics on collaboration efficacy are active areas of research. In our experience on dozens of such projects (and based on years of team science research literature), the most successful programs have clear governance, a shared understanding about goals, and incentives aligned to those goals. Additionally, this healthy triad (Goals, Roles and Incentives) is supported by sound operational infrastructure. We share our experiences and resources in creating and supporting successful transdisciplinary collaborations, from strategies to building healthy collaborative communities to technological support for knowledge exchange and resource sharing. Our playbook includes collaborative agreements to foster sound governance, training and guidance around team science, and evaluation approaches to team health; together these will support the vital work of transdisciplinary science.

Presentation 2
10 - CFHealthHub Learning Health System: A national multi-disciplinary community of practice supporting people with cystic fibrosis to live as normal a life as possible 
Presented by: Lana Lai
Authored by: Lana Lai

Cystic fibrosis (CF) is a life-limiting inherited condition, characterised by a build-up of thick mucus causing progressive damage to multiple organ systems. Median life expectancy for people with CF in the UK is 53 years. The commonest cause of death is progressive deterioration in lung function due to chronic infection and inflammation. Daily inhaled therapy can control lung infection and slow lung function decline, but real-world adherence to treatment regimens is around 30-40%. CFHealthHub was conceived by the multi-disciplinary team in an adult UK CF Centre with the clear aim of enabling people with CF to live as normal a life as possible. A collaborative exercise involving people with CF and multi-disciplinary clinicians identified supporting adherence to inhaled therapy as a way of meeting this aim. 

A theory-based multi-component behaviour change intervention including habit formation as means of supporting adherence was developed with experienced psychologists at a local university. A bespoke digital platform, co-designed by people with CF and clinicians was developed by collaborators in a university-based digital health software team. This platform presents real-time, objective adherence data, allowing people with CF and their clinicians to see how well the habit formation intervention is going.

The intervention and digital platform underwent a mixed-methods process evaluation in two centres as a pilot study, before demonstrating sustained improvements in adherence in a 19-centre, 607-participant randomised controlled trial. CFHealthHub is now being implemented into routine care across 14 of the 23 adult CF centres in England. CFHealthHub provides a large national platform for research into the effects of co-adherence to inhaled therapy for people with who are taking a novel oral medicine (elexacaftor/texacaftor/ivacaftor), which has revolutionised CF care since it became available in 2020. Adherence data from CFHealthHub are also used to optimise medicines supply by reducing wastage/stockpiling, with the potential to deliver significant cost-savings to CF centres.

Central to the success of CFHealthHub is the multi-disciplinary community of practice representing each centre, who meet at a fixed time weekly to share learning, experience, and ideas whilst co-ordinating formal activities in research and quality improvement. The community includes physicians, nurses, physiotherapists, dieticians, pharmacists, psychologists and social workers. The community are provided with training and support to undertake national and local improvement work and are given a space to learn and share. All programme work is underpinned by elements from established theory, such as the COM-B (Capability, Opportunity, Motivation - Behaviour) model for behaviour change, the Dartmouth Microsystems approach to quality improvement and the learning health system approach to combining science, informatics, data and culture to deliver continuous improvement. CFHealthHub, which was recently recognised as the only condition-specific full learning health system in the UK by the Health Foundation, is a rich case study demonstrating how a multi-disciplinary team, united by a clear shared aim can optimise care and deliver high-quality research and improvement work.

Presentation 3
18 - Complexity Leadership in Action: A Team Science Case Study
Presented by: Gemma Jiang, Colorado State University
Authored by: Diane Boghrat, Imageomics Institute
Jennifer Cross, Colorado State University
Jenny Grabmeier, The Ohio State University
Gemma Jiang, Colorado State University

We describe a successful team science case study in which complexity leadership theory is applied to redesigning a cross-disciplinary science institute’s weekly research coordination meeting. 
CLT outlines three types of leadership: entrepreneurial, operational and enabling. Entrepreneurial leadership is responsible for experimentation, innovation and novelty; operational leadership is responsible for exploitation, productivity and results; enabling leadership operates in the interface between operational and entrepreneurial leadership and enables the adaptive process until a new adaptive order is established. The three types of leadership work together to pivot a system away from an order response and towards an adaptive response that capitalizes on pressure to change.

The narrative arc of this paper follows the four stages of the adaptive process in complex adaptive systems: disequilibrium, amplification, emergence, and new order.

1. Disequilibrium: pressures and tension
The adaptive process begins with disequilibrium when a system feels the pressure to change. These pressures are often referred to as adaptive challenges that require new ways of thinking and behaving. During the summer of the first year of this institute’s funding, a new program director introduced disequilibrium to the weekly research coordination meetings. 
 
2. Amplification: entrepreneurial and operational systems
In response to the adaptive challenges, entrepreneurial leadership begins an ideation process that is characterized by adaptive tension and task related conflicts. Innovative team members seek to identify and experiment with different pathways. Their ideas conflict, combine, and recombine until potential adaptive responses are identified. The program director and team scientist, the two main entrepreneurial leaders of the change initiative, organized a town hall to gather feedback and redesigned the meeting series. But they were met with resistance from the operational leaders (i.e. a few senior principal investigators).

3. Emergence: enabling leadership and adaptive space
Enabling leadership creates the conditions for an adaptive solution to be integrated in the operational system. In this stage, enabling and entrepreneurial leaders work together to morph the ideas until they can be more broadly applied and adopted by the operational system. The team scientist, now primarily an enabling leader, continued to engage with the junior scientists as entrepreneurial leaders to refine the ideas, and gain support. They co-created an adaptive solution. 

4. Stabilizing feedback: new order
Stabilizing feedback is the last stage in the adaptive process. The entrepreneurial and/or enabling leaders link up with operational leaders and get the adaptive solution incorporated in the operational system. The system operates in the form of a new adaptive order with new processes, procedures or products. The team scientist presented the adaptive solution to the full institute leadership team and gained approval. The new design has gone through five monthly cycles, with 20 weekly research coordination meetings in total. 

Building on insights from this novel application of complexity leadership theory, we share important implications for team science practice pertaining to generating momentum for change, re-examining power dynamics, defining critical roles, building multiple pathways towards team capacity development, and holding adaptive spaces. We also explore promising areas for further exploration.

Presentation 4
33 - Interdisciplinary Dialogue Fosters Team Science Skills in Undergraduate Researchers
Presented by: Valerie Imbruce, Washington College
Authored by: Jaime Garcia Vila, Michigan State University
Jessica Hua, University of Wisconsin, Madison
Valerie Imbruce, Washington College
Michael O’Rourke, Michigan State University
Marisa Rinkus, Michigan State University
Kyra Ricci, University of Wisconsin, Madison

Team science that unites specialists of different, discipline-based knowledge domains is critical for solving complex societal problems. Professionals who can merge, contrast, and identify complementary or divergent insights, and who can empathize, be self-reflexive, and seek to understand others’ perspectives, are needed to facilitate successful team approaches. In higher education, interdisciplinary teaching and learning can foster these team skills at the undergraduate level, but is challenged by the dominant practice of individual assessment to establish GPAs and the separation of disciplinary training through traditional majors and minors. We present a novel and easy to implement workshop to enable students to identify complementary or divergent approaches and insights among academic specializations—a skill built from raising interdisciplinary consciousness. We describe a workshop designed to enhance undergraduates’ interdisciplinary consciousness that can be easily deployed within courses or co-curricular programs to foster team science skills. We focused on implementing the workshop with research-intensive students in a summer program who are not being assessed for academic credit. Our central question is: How do we facilitate interdisciplinary consciousness and assess its impact on our students? We designed a set of prompts for undergraduate researchers to discuss key aspects of the research process, inclusive of novel prompts on how students’ identities influence their positionality in research settings and their choice of research topics. We propose that this dialogue-based intervention can be easily replicated to the benefit of summer research programs as well as other academic programs that require the interaction and integration of multiple discipline-based norms. We found that our dialogue intervention opens students' perspectives on the nature of research, who research is for, epistemological differences among researchers, and the importance of practicing the research process as a unique educational experience. Students found these new perspectives through self-description, assimilation, differentiation, and asking for classification or elaboration of each others’ ideas. Our findings indicate that such dialogue demonstrates important gains for students navigating an increasingly competitive and demanding professional world that requires team approaches to problem solving. To realize how the practice of a research process is valuable not only across disciplines but as a professional skill is powerful. And that undergraduates could become attuned to hierarchies of disciplinary power within higher education could bring us one step closer to embracing pluralities of knowing and becoming effective team players. Even one small dialogue, during one short summer program, could catalyze lasting change in participants who likely have not had the opportunity to engage in such reflexive work with a diverse group of budding experts and may not have such an opportunity again.

Presentation 1
1 - A Framework for Developing Team Science Expertise Using a Reflective-Reflexive Design Method (R2DM)
Presented by: Gaetano Lotrecchiano
Authored by: L. Michelle Bennett, Roger Schwarz and Associates
Gaetano Lotrecchiano, George Washington University
Yianna Vovides, Georgetown University

Effective integration and implementation of knowledge in research are dependent on team science expertise grounded in collaboration principles and techniques that advance individual and group scientific agendas. The Science of Team Science (SciTS) provides evidence-based research and best practices that strive to develop scientists’ collaborative skills so that they can work across disciplinary boundaries while developing strong and diverse teaming relationships. Identifying the motivations of those involved in collaborative teaming can contribute to maximizing team effectiveness and applying the knowledge emerging from understanding team members’ motivations has the potential to shape teams’ adaptation of a shared mutual learning mindset as a core tenet of scientific teamwork. In addition, surfacing motivations has the potential of helping team members examine their own needs about their scientific and career goals. 

In this paper we draw from the domains of the Motivation Assessment for Team, Integration, and Collaboration (MATRICx) framework, Maslow’s Hierarchy of Needs motivational theory, and The Team Effectiveness Model for Science (TEMS) to develop an integrative reflective-reflexive design that focuses on the development of intrapersonal attributes within the context of a team. Approaching expertise development from this integrative design invites individual reflection in the context of group reflexivity to serve as the cornerstone of deep team science competency. We used a design thinking approach to identify a framework that merges individual reflection with group reflexivity. The core questions we asked are: (i) What constitutes expertise to participate and succeed in science teams? And (ii) How might we approach the design of learning engagements that enable the development of the needed expertise?

Presentation 2
3 - Applying the UX-Informed Community of Practice Learning Model in Transdisciplinary Research
Presented by: Peng Warweg, George Mason University
Authored by: Brenda Bannan, George Mason University
Dawn Hathaway, George Mason University
Peng Warweg, George Mason University

Transdisciplinary research transcends traditional disciplinary boundaries to address societal problems through knowledge integration and co-production (Cockburn & Cundill, 2018). Research institutions have been exploring effective ways to structure and stimulate transdisciplinary practice, many of which focus on team dynamics, scientific culture, and policy (Hall et al., 2019). Recently, there is a growing interest in applying learning frameworks to facilitate transdisciplinary research. Nembhard pointed out (2010) that deliberate learning activities foster interdisciplinary collaboration. Keen et al. (2005) argued for a focus on adult learning in groups for sustainability science. 
One of the learning frameworks is Community of Practice (CoP) (Lave & Wenger, 1991), which refers to informal and collaborative learning occurring inside a group of practitioners who share the same practice, interest and concerns. As an organizational learning model, CoP encompasses many key features intrinsic to transdisciplinary practice, a community of research practitioners, engagements, collaborations, and knowledge co-production. Researchers conclude that CoP holds many advantages in promoting member participation, collaboration, and knowledge co-production (e.g., Cundill et al., 2015, Degn et al., 2018, Zhao et al., 2018). 

This paper highlights our experience of applying the CoP framework to initiate and expand transdisciplinary practice at a newly established research center in a R1 university. Applications at the center were informed by prior studies on research CoPs (e.g., Cundill et al., 2015, Degn et al., 2018, Zhao et al., 2018). The center readily facilitates member participation and responds to common CoP challenges, and is more methodological in its approaches, including from a human-centered, user-experience (UX) perspective. Hartson and Pyla (2018) defined UX as the totality of the effects felt by the user before, during, and after interaction with a product or system in an ecology. We find a UX-inspired research CoP model helps enhance understandings of member participation, instigate meaningful collaborations and drive sustainable productivity. More specifically,

• Previous studies find CoPs combat traditional departmental silos and provide a platform for researchers to intersect across different disciplines. We find researchers who join our research CoP already self-identify as transdisciplinary. While our CoP does not dissolve departmental boundaries, it does provide a safe space where these researchers reimagine both self and group identities. 
• Prior studies indicate that researchers are motivated to join a CoP which is marked by the bottom-up approach, and equal power distribution. While our experience concurs with these findings, we also recognize individual professional goals significantly impact the level of participation. 
• Prior studies also applaud CoPs’ informal and fluid nature for mobilizing members to shift roles according to research needs and encouraging new member integration. While purposefully keeping our membership informal and inclusive, we constantly juggle between maintaining a relatively focused group identity and growing new members of diverse backgrounds.

We find an explicit and shared vision of building a research CoP among the leadership and members early on can greatly promote members participation and mitigate common challenges. We also find it especially beneficial to adopt UX-informed design thinking in developing intentional facilitations to really engage members and catalyze research.
 

Presentation 3
22 - Design Sprints: A Method for Making Informed Team Decisions Rapidly
Presented by: Maureen Brudzinski, University of Michigan
Authored by: Maureen Brudzinski, University of Michigan
Beth LaPensee, Michigan Institute for Clinical & Health Research

Design Sprints were developed at Google Ventures as a way to move rapidly from developing an idea to testing prototypes. The process uses human-centered design principles to allow a group to understand multiple perspectives on a given problem, create viable solution options, and move forward to evaluate a first prototype. At the Michigan Institute for Clinical & Health Research, we have used Design Sprints in different scenarios, including creating a COVID caregivers' toolkit and designing a medical deterioration alert system. These co-creation sessions have used various methods, including empathy interviews, assumptions mapping, crazy 8’s, solutions sketching, and storyboards to advance teams rapidly through the decision making process while simultaneosly making space for individual ideas and opinions to be shared. We find the Design Sprint process and corresponding tools and methods to be endlessly adaptable across disciplines and uniquely suited to bringing together diverse perspectives to solve problems. We posit that Design Sprint methods can have broad utility in the service of team science by fostering community building and team culture; ambitious ideation and risk-taking; solutions-focused mindsets; and productivity, among others. Additionally, they should be well-suited to both emergent and established teams as well as a multitude of desired goals and outcomes. In this session, the panelists will share their experience implementing Design Sprints, discuss how the myriad methods could be repurposed for other team-based efforts, and engage the audience in visioning how these methods can be further adapted for novel use in team-based scenarios.

Presentation 4
58 - Toward a Translational Team Science Hierarchy of Needs: Exploring the Information Management Challenges of Team Science
Presented by: Betsy Rolland, University of Wisconsin-Madison
Authored by: Jason Chladek, University of Wisconsin-Madison
Patrick Kelly, University of Wisconsin-Madison
Betsy Rolland, University of Wisconsin-Madison

Clinical and Translational Research (CTR) requires a team-based approach, with successful teams engaging in skilled management and use of information. To conduct high-quality, rigorous research and advance scientific knowledge, Translational Teams (TTs) engage in information behaviors, including seeking, using, creating, sharing, storing, and retrieving information, in ways specific to the translational context. Yet we know little about the ways these TTs engage with information across the lifecycle of CTR projects. This qualitative pilot study explored the sociotechnical challenges that information management imposes on the conduct of team-based CTR through interviews of 10 TT members at the University of Wisconsin-Madison.

We found that TT members did not recognize the centrality of information or information behaviors to their scientific work. Furthermore, TT members did not receive support or guidance from their institution in managing information, leading to individualized choices that conflicted at the team level, causing confusion and increasing the potential for data and information loss. Furthermore, we found that TTs’ piecemeal and reactive approaches to information management created conflict within the team and slowed scientific progress. The lack of cohesive information management strategies made it more difficult for teams to develop strong team processes like communication, scientific coordination, and project management. While TTs’ research was hindered by the siloed university that provided little support or guidance, TTs who had developed shared approaches to information management that foregrounded transparency, accountability, and trust, described substantial benefits to their teamwork.

While more research is needed to develop approaches to mitigate to the challenges we identified, one lightweight solution is to implement a team-based intervention such as UW-ICTR’s Collaboration Planning. In delivering Collaboration Planning to more than 40 teams at UW and beyond, we have seen that teams benefit from the opportunity to discuss a team-wide strategy to information management.

Finally, we propose a new model for the SciTS field—a Translational Team Science Hierarchy of Needs. Based on Maslow’s famous Hierarchy of Needs, our model suggests new considerations for the design, development, and evaluation of interventions, with a focus on targeting the appropriate stage of team development. Such a focus has the potential for helping TTs create a strong base that supports team processes that to maximize a team’s scientific potential.

Presentation 5
61 - Using self-evaluation to build transdisciplinary capacity in team-based approaches to urban sustainability: student reflections on their own learning
Presented by: Mathieu Feagan, University of Waterloo
Authored by: Marta Berbes, University of Waterloo
Mercy Borbor Cordova, Escuela Superior Politécnica del Litoral (ESPOL)
Elizabeth Cook, Barnard College
Maria del Pilar Cornejo de Grunauer, ESPOL - Escuela Superior Politécnica del Litoral
Mathieu Feagan, University of Waterloo
Nancy Grimm, Arizona State University
Shruti Jadala, University of Florida
Addison Martin, Utah State University
Margot Mattson, San Diego State University
Isabella Pacenza, Barnard College
Carlos Romero, Columbia University
Chloé St. Amand, University of Waterloo

This project investigates the use of self-evaluation in building student capacity for transdisciplinary teams-based research in the context of an international collaboration in Latin America on urban sustainability, nature-based solutions, and informal settlements. After completing a two-month preparatory course, a team of US graduate and undergraduate students from different disciplines developed a shared rubric for evaluating their own success implementing a transdisciplinary project working closely with local university research partners, the municipality, and community leaders in the informal settlements. The student team revisited the rubric criteria at the beginning, middle, and end of their ten-week fieldwork experience, to reflect on the evolution of their criteria and finally score themselves. Students were then individually interviewed to help understand their perceptions of the value of self-evaluation in developing and implementing a transdisciplinary approach to their project. The results show that this experience of using self-evaluation and working in a transdisciplinary team was new for students, and it posed challenges that no prior academic training had quite prepared them for. Still, with strong on-the-ground support from local and international faculty, the student team developed communication and group process skills to integrate their different disciplinary expertise in a way that was highly valued by the communities and local research partners. The student team found that using the self-evaluation rubric helped build confidence in their ability to identify criteria and implement a project that in retrospect lived up to their own expectations of success. All five participants said they would continue engaging with transdisciplinary research approaches and that they saw self-evaluation as a critical tool for helping them navigate the process and enhance learning. Suggested modifications and insights from student experiences are now informing another iteration of the process of using self-evaluation to build student capacity for a transdisciplinary team-based approach with a new cohort of students.

Presentation 1 
20 - Cross-cutting themes and opportunities for global collaborative research and networks
Presented by: Jane Payumo, Michigan State University
Authored by: David DeYoung, Michigan State University
Joseph Huesing, Michigan State University; Purdue University
Jane Payumo, Michigan State University

In development work, cross-cutting themes are issue areas, such as gender inclusion, which are identified by the project sponsor as important and that impact not only the specific project but also the broader development goals of the sponsor. These cross-cutting themes are seen as key in attaining a broader development impact across programs by acting as a system-wide ‘force multiplier’ extending the reach of any single investment. The U.S. Agency for International Development (USAID), an independent agency of the U.S. federal government, is responsible for administering global aid and development assistance. USAID uses cross-cutting themes across all its programing to create a lasting and greater impact on development. Experience has shown, however, that despite the commonality of cross-cutting themes across programs and projects the level of successful implementation is less than ideal. First, the idea of multiple goals loosely tied to a project’s central focus is antithetical to good project management practices. Second, because each project necessarily has varying interpretations and approaches to implementing and measuring the progress of cross-cutting goals, the goals become secondary for the project management team. 

This paper will highlight the experience of the USAID Feed the Future Innovation Lab for Legume Systems Research (Legume Systems Innovation Lab for short) and its research partners, in mainstreaming cross-cutting themes for global research collaborations and networks and addressing opportunities and issues for this objective. The Legume Systems Innovation Lab, managed by Michigan State University, is one of the 21 Feed the Future Innovation Labs (https://www.feedthefuture.gov/feed-the-future-innovation-labs/), which draw on the expertise of U.S. universities and developing country research institutions to design and implement solutions to a wide array of agricultural and food security issues. Specifically, we will highlight the Legume System Innovation Lab’s experience in integrating overarching concepts and topics, including gender, capacity development, youth, nutrition, and resilience, across its projects. We will also present the Legume Systems Innovation Lab’s approach to address knowledge gaps and implementation and offer insight into pivotal elements for designing, implementing, monitoring, and evaluating cross-cutting themes. Finally, we will provide the results of a qualitative study using document reviews, surveys, and group interviews that highlight where cross-cutting themes need to reside in a program’s theory of change design, what are the formative processes (inputs and outputs) required, and the emerging lessons and insights for integrating cross-cutting themes for collaborative research and networks in an international development context. While qualitative studies have limitations, this study illustrates a new and shared understanding of managing cross-cutting themes as the basis for other organizations and other development practitioners working in a similar environment.

Presentation 2
53 - Team Satisfaction and Network Integration: A Case Study of Two NSF Funded Science Teams with Varying Levels of Success
Presented by: Anneloes Mook, Colorado State University
Authored by: Jennifer Cross, Colorado State University
Verena Knerich, Colorado State University
Anneloes Mook, Colorado State University

As the world becomes more complex, interdisciplinary team-based research is required to understand wicked problems such as climate change. Yet despite the extensive focus on the essential scientific competencies to undertake these ambitious research projects, educational institutions overlook the importance of collaboration skills. In this paper, we use social network, quantitative, and qualitative data to evaluate team satisfaction, network integration, and productivity of two National Science Foundation funded teams over a two-year time period. One team performed well and continued to grow and integrate, while also reporting high levels of satisfaction and productivity. However, the other team remained similar in size and level of network integration. Though the leaders of this team who are situated in the core of the network reported high levels of satisfaction, the team members in the periphery were dissatisfied. Overall, the team was less productive than the first. As both teams were led by established scientists, the difference in team satisfaction, network integration, and productivity can be explained by the varying levels of collaboration skills. We argue that effective team science training as well as collaborative leadership structures explain the trajectories of the two teams. Given the importance of collaboration skills in scientific activities, more team science training is necessary to prepare the next generation scholars for interdisciplinary careers.

Presentation 3
60 - Using Matched Controls to Measure the Impact of Small Research Pilot Grants
Presented by: Griffin Weber - Harvard Medical School
Authored by: Griffin Weber - Harvard Medical School

Introduction
Each year, the NIH Clinical and Translational Science Award (CTSA) program spends millions of dollars to fund hundreds of small pilot grant projects at academic medical centers across the country. The NIH tracks how many projects were funded and the percent of projects with at least one publication. However, the impact of pilot grant program on Team Science is unknown. It is unclear whether the teams that assembled for pilot grant projects and the publications they wrote would have happened anyway without funding. Furthermore, it is critical to realize that CTSA pilot grant funding might have had an effect on the thousands of investigators who came together to collaborate on proposals that were not funded.

Methods
To better understand the impact of pilot grant funding, we developed a novel method called Random Teams, which matches both funded and unfunded teams to appropriate controls, in order to separate the effects of the pilot grant program from baseline scientific collaboration and productivity. The controls are thousands of “virtual teams” consisting of randomly selected investigators who were eligible to collaborate on a proposal but did not. They were matched to actual teams in various ways. Characteristics of the virtual collaborations provide null distributions which can be used to statistically test hypotheses about pilot grant funding.

Results
We applied our Random Teams approach retrospectively to one year of CTSA pilot grants at Harvard University, with a five-year follow-up window. Out of 37,266 faculty who were eligible to apply, a total of 1,469 investigators assembled into 458 teams that submitted proposals, of which 65 were funded. We found: (1) Teams that applied for funding were more open to new collaborations. On average 50% of the authors on publications at Harvard were prior co-authors on other publications, while only 20% of the investigators on pilot grant teams were prior co-authors. (2) Funded teams published 2.6 articles citing CTSA funding. However, randomly matched teams published 0.9 articles citing the CTSA program for other reasons, such as using its consultation services. Thus, we need to consider this baseline when estimating the added publication impact of pilot grant funding. (3) After five years, funded pilot grant teams, on average, created 2.5 new lasting collaborations (people who co-authored papers together for the first time after receiving funding). However, on average, 1.7 people from the un-funded teams also later published together for the first time, perhaps though other funding mechanisms--a previously unrecognized Team Science benefit of the program. Surprisingly, since there were many more un-unfunded than funded teams, most of total new collaborations came from the un-unfunded teams.

Discussion
Having comparison teams is critical to understanding the impact of pilot grant funding, both on scientific productivity and collaboration. A limitation of this study was that it was performed at a single institution through one funding mechanism. However, our Random Teams method is generalizable and a useful tool for evaluating Team Science.

Presentation 4
62 - What Factors Facilitate Multidisciplinary Collaboration After NSF Awards?
Presented by: Chelsea Basore, Penn State University
Authored by: Chelsea Basore, Penn State University

Objectives 
Granting agencies such as the National Science Foundation (NSF) often require multidisciplinary research teams to be competitive for funding. However, to what extent do funded research teams have multidisciplinary authorships after they win an award, given that awards are not contracts? Would, for example, two computer scientists choose to continue working with a psychologist after winning an NSF award? Furthermore, what factors facilitate post-award cross-disciplinary collaboration?

This study examined the extent to which NSF-funded multidisciplinary teams collaborated with team members from different disciplines following their NSF award and the factors that facilitated continued collaboration.

Archival and Interview Research Methods
Our sample was 150 PIs and Co-PIs (67% male) from 58 NSF-funded projects receiving EAGER (EArly-concept Grants for Exploratory Research) grants between 2013 and 2019. Using publicly available data, we collected the number of conference papers, publications, and grants the PIs and co-PIs published with each other. We also catalogued whether PIs and Co-PIs represented unidisciplinary or multidisciplinary collaboration after their NSF award. Multidisciplinary collaboration consisted of multidivisional (Ph.D. disciplines across NSF divisions) and/or multidirectorate (Ph.D. disciplines across NSF directorates) authorship. We also conducted 30-minute to 1-hour interviews with 33 PIs/co-PIs in our sample, which were transcribed and coded.

Results
Of the 58 original EAGER PI and Co-PI teams, 74% continued to collaborate after the EAGER grant. Of the 74%, 87% were multidisciplinary, 79% of which were multidirectorate (e.g., sociology and computer science) and 8% were multidivisional (e.g., political science and cognitive psychology). Of the 327 collaboration outputs (conferences, journals, grants) produced after EAGER awards in our sample, 82% had multidisciplinary authorship, 67% of which were multidirectorate. 
To address why some PI/co-PI teams collaborated post-EAGER and others did not, we analyzed pre-EAGER award collaborations, gender diversity, and PI/co-PI university distribution (single versus multiple university) as possible contributing factors from the archival and interview data. Although gender diversity and university distribution did not exhibit clear patterns, pre-EAGER award collaboration was predictive of authorship after successfully garnering the grant. Of the teams who collaborated prior to their EAGER award, 92% collaborated before EAGER. Among post-EAGER award collaboration teams, 47% authored together pre-EAGER.

Conclusion
Of the approximately 74% of PI and Co-PI teams who continued to collaborate after the EAGER grant, almost 9 out of 10 chose to work with a collaborator from a different discipline. NSF EAGER grants serve as a catalyst for continued multidisciplinary (especially multidirectorate) collaboration. Pre-EAGER award collaboration facilitated post-EAGER award collaboration.

Presentation 5
63 - What Knowledge Must Cross-Disciplinary Team Members Share for Effective Collaboration? A Mixed Methods Study
Presented by: Susan Mohammed, Penn State University
Authored by: Chelsea Basore, Penn State University
Susan Mohammed, Penn State University

Objective
Tackling “grand challenges” demands broad representation across multiple disciplines, but teams must also build substantial integration to make the best use of diverse expertise. Efficiently harnessing expertise in teams requires both knowledge divergence and convergence. What we do not know, however, is what knowledge should be shared and what knowledge should remain unique to individual members for interdisciplinary teams to be effective. Addressing these questions is crucial because building shared knowledge in teams demands high coordination and communication costs, which can tax members’ limited attentional and temporal resources. Therefore, converging when diverging is needed or diverging when converging is needed mishandles team resources, impairing team processes and performance. Regrettably, however, the team cognition and interdisciplinary team literatures provide few answers and little guidance regarding the complex process of knowledge exchange in interdisciplinary collaborations or how knowledge convergence and divergence should be balanced. 

Seeking to address these deficiencies regarding this foundational issue for both theory and practice, the objective of this research was to explore the effects of simultaneously held knowledge divergence and convergence in cross-disciplinary teams. Our research questions include: 1) What knowledge should be shared versus uniquely held by interdisciplinary team members? 2) How does the extent to which interdisciplinary team members report more of a unique or shared understanding of relevant team knowledge affect a) perceived collaboration outcomes and b) archivally measured productivity outcomes? 

Mixed-Method Approach 
We first conducted 33 semi-structured interviews with PIs and co-PIs funded by the National Science Foundation, who reflected on what knowledge should be shared and what should be unique among interdisciplinary team members. Based on the coding of the interviews (using NVivo), we designed and administered a quantitative survey completed by 38 grant-funded PIs and co-PIs. We then examined the relationship between survey data and archival measures of research productivity (number of PI/co-PI grants, publications, and conference presentations).

Results
Results revealed a consistent pattern across interview, survey, and archival methodologies. Interview and survey data were skewed toward knowledge convergence rather than divergence. Participants especially favored knowledge similarity over uniqueness for what they planned to accomplish, what they produced, who did what, and when work should be done. However, research outcomes (where to publish) and research content (theory, methodology, analysis) evidenced less convergence than vision or teamwork items. Moreover, more actual knowledge convergence was associated with higher collaboration satisfaction, trust, respect, and research impact and trended in the direction of more grants, publications, and conference presentations.

Presentation 1
4 - A Team Science Community Toolkit: A Co-Developed Approach to Open Science Up to Community Organizations
Presented by: Madison Hartstein, Northwestern University
Authored by: Stephanie Schmitz Bechteler, Chicago Urban League
Kareem Butler, Chicago Appleseed Center for Fair Courts
Elyse Daly, Northwestern University
Ontisar Freelain, Health Research and Awareness NFP
Joanne Glenn, W.O.T. Foundation
Arielle Guzmán, Chicago Medical Organization for Latino Advancement
Madison Hartstein, Northwestern University
Tessneem S. Hasan, Northwestern University
Candace Henley, The Blue Hat Foundation
TaLana Hughes, Sickle Cell Disease Association of Illinois
Angela E. Jordan, University of South Alabama
Rana K. Mazzetta, Northwestern University
David A. Moskowitz, University of Chicago
Megha A. Patel, Northwestern University
Heather J. Risser, Northwestern University
Sheila Sanders. Northwestern University
Bonnie Spring, Northwestern University
Héctor Torres, Colibri Counseling
Kimberly M. Williams, Erie Family Health Centers

Background
To redress health disparities, there is a need to extend the scope and application of Team Science beyond the academic biomedical sciences to include other key domain experts: Community Organizations (COs).

Problem
Although the importance of Community-Engaged Research (CER) has become well-recognized in theory, in practice, successful engagement in CER continues to be thwarted by an absence of established bridges to practical partnerships. Imbalances in power, information, and resources in CO and academic partnerships constrain the contribution of CO's voice and expertise; thereby impeding development of a shared mental model that incorporates both scientific and real-world knowledge. 

Activities/Methods
To facilitate more insightful productive community-academic research partnerships, we co-created the first, public-facing Team Science Community Toolkit. We co-designed the toolkit with community partners to level the playing field and reduce the hindrance of unfamiliarity with scientific jargon, grant finances, and research methodology. It is intended to create a bridge that invites citizen scientists from the community into the biomedical research endeavor to better address persistent health disparities. During needs assessment qualitative interviews, CO staff suggested that learning more about how the research process works and having tools to support them throughout a scientific project could restore balance and provide greater agency for the CO.

We assembled a diverse project team of community and academic partners to co-develop all content and tools. We utilized a community based participatory research (CBPR) approach, grounded in the principles of the Science of Team Science (SciTS) and User-Centered Design (UCD). The resulting toolkit includes templates, checklists, and interactive tools to support collaborative decision-making and communication. A real-world simulation takes users through the five stages of the research process, introduces all tools, and provides context for their application.

To validate the concept, content, and usability of the toolkit, we conducted focus groups and usability testing with community experts outside of the project team. Participants expressed pride and enthusiasm to contribute, “we can actually influence it to be what we need” and “I’ve seen a lot of educational content that tries to be helpful, useful, and easy to use and this is the first one that I actually think is.”

To address community citizen scientists as full partners in the research effort, we integrated the toolkit as a module in the existing open-access COALESCE (teamscience.net) platform. Created to facilitate communication and research collaborations between scientists from different disciplines, COALESCE’s modules have been used by 50,000+ individuals worldwide. Inclusion of the community toolkit in the platform invites COs into the research discussion and exposes academic and community partners to each others’ mental models. This presentation will demonstrate how to use the new toolkit, including some of its interactive components. Use-case applications will be described, and collaboration will be invited to evaluate the toolkit’s effectiveness.

Conclusion
The Team Science Community Toolkit is designed to empower self-advocacy and increase equity for Community Organizations engaging in research with academic partners. Many of the tools can be downloaded, customized, and deployed to foster productive communication in community-academic partnerships.

Presentation 2
23 - Developing Integration Skills in Convergent Engineering Teams
Presented by: Theresa Lant, Pace University
Authored by: Susan Day, University of Louisville
Theresa Lant, Pace University

Our research explores how engineering investigators learn and apply integrative skills in larger, multi-university convergent engineering research programs including the non-technical components of collaboration. One of the challenges for engineering investigators, with their high-consensus research disciplinarity, is to understand standards of practice for low-consensus research domains like the social sciences(1). Engineering research has generally thrived in the context of multidisciplinarity, in a supply chain fashion, across the fine fields of engineering with some lesser measure of system integration. However, high impact social science applications require a collaborative and comprehensive design process for conceptualizing integrative strategies across subsystems. In exceptional STEM education ecosystems in non-EPSCoR (2) states (e.g., Silicon Valley/ Stanford, Berkeley), through standards of practice developed to some degree through NSF funding, (e.g., IGERT/ NRT/ IUSE 3) a collaborative process between social scientists and STEM investigators occurs. With successful NRT programming, for example, investigators have opportunities to engage in research that tests a theory of change in human behavior across the non-technical components of research programs. In the low consensus social sciences, conceptualization of numerous program components (i.e., DEIA across the innovation ecosystem, designs for knowledge management, curriculum, and professional development, etc.) allow for numerous configurations for tailoring evidenced-based models and practices to the policy subsystem in which the research is embedded. STEM investigators who learn integrative skills are more likely to consistently apply theories of change across essential programmatic components, learn cognitive flexibility and maximizing impact.

Our research question centers on how engineers learn and apply integrative skills in convergent research programs. We have initial data using the NSF funded BRIDGES (4) survey from multi-university engineering research teams through an on-going collaboration with a collaborative robotics research institute funded in part through NSF EPSCoR funds. The BRIDGES survey measures the cognitive and social precursors to developing an integrative capacity (5). Through application of this survey, meeting transcripts, and interviews over time during the evolution of this project team, we will conduct a longitudinal investigation of how engineer investigators learn and apply integrative skills in convergent research programs. The work we propose to present at INSCiTS 2023 will describe the initial conditions of the teams engaged in this multi-institution collaborative effort, as demonstrated through survey responses, interviews, and meeting transcripts.

1. Borrego, M & Newswander, L, (2008) Characteristics of successful cross-disciplinary engineering education collaborations, Journal of Engineering Education, DOI: 10.1002/j.2168-9830.2008.tb00962.x
2. Established Program to Stimulate Competitive Research (EPSCOR) https://new.nsf.gov/funding/initiatives/epscor
3. National Science Foundation Research Traineeship Programs (NRT, IGERT); https://new.nsf.gov/funding/opportunities/national-science-foundation-research-traineeship https://new.nsf.gov/funding/opportunities/improving-undergraduate-stem-education-directorate (IUSE).
4. BRIDGES: Building Resources through Integrating Disciplines for Group Effectiveness in Science, NSF SciSIP, Award #1262754, PIs: Lant, T & Salazar, M.
5. Salazar, M., Lant, T., Fiore, S., & Salas, E., (2012) Integrative Capacity: A New Perspective for Understanding Interdisciplinary Team Processes and Outcomes," Small Group Research. October 2012 vol. 43, no. 5; 527-558

Presentation 3
27 - Evidence-based Longitudinal Assessment of Interprofessional Teamwork: Assessing team experience in an educational context
Presented by: Tony Lingham, Interaction Science
Authored by: Tony Lingham, Interaction Science
Tyler Reimschisel, Case Western Reserve University

We describe an interprofessional education (IPE) course in which 435 students in 72 teams work on community-based projects while learning teamwork content and skills. Students were from the following programs: Dental Medicine (76), Genetic Counseling (8), Medicine (215), Nursing (311), Nutrition (1), Physician Assistant (34), Psychology (4), Social Work (58), and Speech-language pathology (8). A total of 40 sponsoring organizations with 83 community leaders (“champions”), 40 classroom instructors, and 28 faculty. The student teams collaborated with the champions on authentic projects designed by the champions. We employed a longitudinal, mixed-methods design to assess the experience of team interaction (i.e., pre-post design). Since the teams were embedded in a larger system, we chose a 360-degree assessment for the student teams. The team-level assessments focus on the lived experience of interprofessional team members (internal to the team), the team’s assessment of their outcomes based on their perception of working together (internal to the team), and the assessment from the champions (external assessment). We used the Team Learning Inventory or TLI (a 360 team-level assessment) together with a structured team coaching process to provide evidence-based team development from Time 1 and Time 2. Team coaching was provided to each team immediately after Time 1. The TLI was selected as the assessment tool because for almost two decades the TLI combined with structured team coaching has shown strong validity, reliability, and robustness with teams across all levels of diverse organizations in different countries and cultures. Four major dimensions of the experience of teamwork are assessed using the TLI:

1. Diverging Dimension (5 aspects),
2. Converging Dimension (3 aspects);
3. Power and Influence Dimension; and
4. Openness Dimension.

By forming partnerships among the education, practice, and research communities, this unique design for an IPE experience ensured that the IPE course focused on evidence-based team development. Based on exploratory analysis from the first year of the course, the teams that indicated in the goals and action steps of their coaching session that they wanted to improve in specific dimensions and/or aspects of team interaction showed improvements across all dimensions and/or aspects that were indicated in the teams’ learning plans with a range from 3.1% - 36.9%, including 33.8 % improving in the Power and Influence Dimension, 36.9% improving in the Planning aspect of the Converging Dimension, and 30.8% improving in the Relationality aspect of the Diverging Dimension. Champions’ ratings from 10 questions (1-7 response rating) of their experience with the team had an average of 5.8 – 6.2 for each question (Time 1) and 6.0 – 6.4 (Time 2) showing an improvement from the champions’ assessments of their experience working with the teams post the team coaching. The evidence of improvement in team interaction is promising as our focus is to develop High-Impact Teams (HITs) demonstrated by the team’s internal dynamics and functionality as well as the benefit the student team project has on the champion’s organization.

Presentation 4
34 - Interdisciplinary Graduate Student Reflections on Transdisciplinary Team Science Training
Presented by: Deborah DiazGranados, Virginia Commonwealth University
Authored by: Deborah DiazGranados, Virginia Commonwealth University
Stephen Fiore, University of Central Florida
Troy Hartley, William & Mary

Solving today's most challenging societal problems requires innovative breakthroughs and novel solutions that transcend disciplines, reaching a deeper level of knowledge integration. However, achieving such integration through team science is challenging due to the lack of adequate training. To address this, methods from allied disciplines need to be adapted for use in training future transdisciplinary researchers. This presentation discusses team science training focused on coastal resilience. It brings together a multidisciplinary team of scholars focused on improving problem solving and teamwork in science.

First, a team of faculty coaches was recruited to guide a class of diverse doctoral and master's students from the natural and physical coastal, marine and environmental sciences, engineering, design, and social and economic sciences. Second, to develop and test different types of transdisciplinary pedagogies, a series of workshops was developed to train students on the fundamentals of team science as well as collaborative knowledge building on complex transdisciplinary problems. We emphasized the development of conceptual models that are capable of capturing system level problems as well as integrating diverse disciplinary perspectives. Third, to foster individual and team learning, an intervention focusing on reflection in teamwork processes was used to ensure students monitor both the task of transdisciplinary problem solving, as well as the teamwork processes engaged while collaborating.

Graduate students were introduced to the principles of team science, collaborative problem solving, and effective self-reflective tools and strategies. Additionally, students gained experience working with coastal community partners (e.g., municipalities, NGOs). As such, this community-based climate-resilience project enabled students to practice team science research and use reflective practices to improve their competencies with various stakeholders. Assessment of team processes, along with reflections on teamwork and taskwork, were used to highlight areas of collaboration needing improvement.

For this presentation we first provide an overview of the project and follow this with results from the first cohort of graduate student teams. In addition to attitudinal and knowledge based surveys, team members also provided diary type reflections. We focus on these team reflections where they describe their experiences in collaborative knowledge building and problem solving sessions. These reflections were based on prompts designed to helps students consider their experience in teamwork and in taskwork and what they found challenging. Additionally, they reflected on what they learned about interdisciplinarity teamwork and taskwork. Analyses of reflections following their initial phases of collaboration showed that the students were initially challenged by the lack of awareness of team members expertise and knowledge from other disciplines. They were similarly challenged by the scope of the work and temporal dynamics of the different research methods. Analyses of reflections following intense problem solving sessions found that teams were gaining awareness of how to utilize the diversity of disciplines while better understanding team roles. Additionally, they reported gaining an awareness of location-based taskwork and variations in timing of the work when tasks differed in scale. We discuss these and related findings and how the project evolved to better address the team science training needs of the graduate students.