The digital revolution is producing vast quantities of social, psychological, and organizational data that could be harnessed to hasten the production of knowledge needed to tackle society’s most vexing problems. Social welfare organizations have not been left out of this massive data collection effort. Computerized social service, education, juvenile and criminal justice, and health records, open data platforms, social media posts, web searches, all contain potentially useful data for illuminating the dimensions of social problems and propelling effective solutions. These data are amassed as part of the normal course of agency operations and individuals’ daily lives. Technological advances are making it possible for them to be managed and analyzed in real time. A grand challenge for social work is to deploy these vast data resources to craft and evaluate promising innovations, improve frontline practice, and support continuous improvement and accountability.
Big data, by definition, overwhelm standard computing and analysis methods. Yet the technical feasibility and extraordinary value proposition resulting from the application of big data in social work and social welfare is already becoming evident. Randomized controlled trials of social interventions are completed quickly and at significantly reduced cost when they rely on cross-sector integrated data to measure outcomes rather than wholly on primary data collection. Agencies from multiple sectors coordinate their work and demonstrate their collective impact on entire populations by using big data. Promising social work practices are uncovered through mining text data that appears in millions of digitized progress notes and assessments, as well as audio and video recordings of important social work settings. Emerging social problems in communities are detected early though the analysis of millions of social media posts and on line searches. New modes of financing policy innovations, such as social impact bonds, are made possible through mining of financial and service records integrated across service systems.
To benefit from the promise of big data, the social and human services sector must advance in the application of data and computational sciences and evolve to an organizational and professional culture that incorporates a continuous flow of data analytics to inform social policy and practice. Data and computational science is inherently interdisciplinary, and it is essential that social work develop a cadre of professionals that have basic competence in big data management and analytics so that they can interface with specialists in these fields and take leadership in the creation of data driven knowledge that will advance the field. Moreover, social and human service organizations must overcome administrative barriers to break down data silos, both within systems and across sectors, to exploit the potential of integrated data to drive strategic investments in programs and practices that work and demonstrate long term cost benefit. Finally, the exploitation of big data for social good requires a commensurate emphasis on ethical and equity concerns. As social data become more ubiquitous, greater consideration will need to be given to balancing privacy protection, social justice and the public interest.