What Does the Practice Analysis Have To Do With Questions on the ASWB Exam?

by Nancy Sidell, PhD, LSW

(Editor’s Note: This is one of a series of articles on the social work licensing process. Thank you to the Association of Social Work Boards [ASWB] for collaborating with The New Social Worker on providing this information to our readers.)

     Years ago, as I sat down for my licensing exam, I remember feeling anxious and pressured to perform well, common reactions to such a high-stakes test. I reasoned that I had studied everything possible and believed I could prepare no more. I remember taking a deep breath and diving in. As I moved through the questions, I recall thinking, “Where in the world do these test questions come from? Why is this being asked and not something else?” At the time, I knew nothing about the process used to develop exam questions and how content is selected for inclusion in the test. I did not know that what I was wondering about is the exam’s blueprint, a well-developed, thoughtful tool that involves an exacting analysis of current social work practice. It is how ASWB knows what a social worker needs to know the first day on the job.

From the Beginning: Exam Blueprints

    Merriam-Webster (n.d.) defines a blueprint as a detailed plan that serves as a model. The first known use of the word in 1857 referred to the white and blue print still used when copying mechanical drawings and architectural plans. You would never build a house without following a blueprint, and ASWB does not build an exam without one, either. Blueprints provide the road map and structure for each exam, no matter the practice category. Each exam taken by a candidate follows the blueprint established for that practice category (Associate, Bachelor’s, Master’s, Advanced Generalist, and Clinical).

    Here are the pieces of the exam blueprint. Each exam consists of 170 multiple-choice test questions. Each question, or item, is associated with a knowledge, skills, and abilities (KSA) statement. KSA statements are easily understood concepts that item writers use when they develop test questions. They represent discrete knowledge that test-takers must know in order to answer the item correctly. KSA statements are organized into competencies, which describe meaningful sets of KSA statements. Content areas, in turn, are groups of several related competencies. The total number of items appearing on an exam must match the percentage required by the blueprint for the content area. An illustration from the Clinical exam is below.

Content Area:    

I. Human Development, Diversity, and Behavior in the Environment (24 percent)

Competency:    

IB. Human Behavior in the Social Environment

KSA:    

The effects of addiction and substance abuse on individuals, families, groups, organizations, and communities

    The content area Human Development, Diversity, and Behavior in the Environment accounts for 24 percent of items on the Clinical exam. You can view the assigned percentages for all content areas at https://www.aswb.org/exam-candidates/about-the-exams/exam-content-outlines/

    How are KSA statements, competencies, content areas, and content area percentages developed for each exam level? How are the blueprints created? It all happens through a practice analysis.

Practice Analysis Overview

    A practice analysis is a comprehensive, multistage process that aims to ensure that a licensing test measures what a candidate should know to enter practice. Providing the basis for a valid exam, it is “built on a rationale that clearly shows how the content of the exam reflects that knowledge” (ASWB, 2017, p. 3). At the heart of any practice analysis is a survey of practitioners across jurisdictions, practice settings, and diverse backgrounds. The goal of a practice analysis is to ensure that exam content is relevant to current professional practice and to reflect the expertise of the widest, most diverse group of stakeholders possible. ASWB completed its first practice analysis in 1981 and has engaged in this process every five to seven years since. Its sixth analysis was completed in 2016.

Critical Phases of the Practice Analysis

    The 2016–2017 practice analysis consisted of four phases that occurred over two years. In addition to engaging ASWB staff members, the process involved the services of a psychometric consulting firm, a 5-member oversight panel, several task forces consisting of a range of diverse subject matter experts, and the participation of more than 23,000 social workers from across the United States and Canada who completed a web-based occupation survey. This survey response represents the largest number of useable responses in any ASWB practice analysis. The survey’s aim was to obtain a snapshot of the tasks performed by social workers at entry to practice and the necessary knowledge to complete those tasks successfully.

    Like any valid and reliable research project, the first phase of the analysis involved planning. Survey development, pilot testing, and forming of the oversight panel and task forces were some of the activities in this phase. The survey consisted of two sections: background questions to collect demographic information and the survey itself.

    The second phase, survey administration, sought to ensure representation across jurisdictions and settings. All respondents answered several background questions and were then assigned to complete either the task or the knowledge survey based on their years of social work experience. The task survey attempted to understand what tasks were necessary to perform the respondent’s job. The knowledge survey focused on the knowledge necessary to perform those tasks. Because the survey was administered in Canadian provinces where French is an official language, translation of the survey to French was necessary during this phase. ASWB took several steps to ensure the large sample size represented the profession and its rich diversity.

    Consulting psychometricians working closely with ASWB staff, the oversight panel, and subject matter experts led the third phase—data analysis. Overriding considerations included making meaning of the findings and determining whether there were differences between social work practice in the United States and Canada. Linking the data obtained from both the task and knowledge portions of the survey was an important component of the analysis.

    The fourth and final phase was updating the exam blueprints based on the survey findings, with an emphasis on knowledge statements that were easy to understand and meaningful to those involved in exam development. A diverse group of subject matter experts reviewed the results, unexpected ratings or potential data anomalies, emerging trends, and knowledge requirements gleaned from the data and then established the content weights through this process. These weights determined the percentages of questions in each content area for the five exam categories (Associate, Bachelor’s, Master’s, Advanced Generalist, and Clinical).

    Ultimately, the results of the 2016 practice analysis study determined little difference between social work practice in the United States and Canada. It also resulted in the creation of several new KSA statements reflecting changes in practice that had emerged since the previous practice analysis. For example, trauma-informed theory and substance use issues emerged as two areas of greater importance to practice than had been seen in the previous analysis.

On the Horizon

    The thing about social work, like the society in which it is practiced, is that it keeps evolving. For example, in 1922, rural social work pioneer Josephine Brown advised against keeping any notes at all on social work services being delivered (Brown, 1922). Nearly a hundred years later, no one would agree that this is good advice; our thinking about documentation has evolved since then. Other changes happen more rapidly. Who could have foreseen a year ago, for example, the impact of the COVID-19 pandemic on both society and the profession? The meaning of cultural competence, the influences of systemic racism, and the impacts of intersectionality and microaggressions have also come into greater focus, both societally and professionally.

    These examples demonstrate that the environment in which social work is practiced morphs constantly. To remain relevant, social work must adapt to a changing environment. That’s why ASWB undertakes a new practice analysis on a regularly scheduled basis. Even though its last practice analysis process was completed in 2017, plans are already under way for the next practice analysis, which will commence in 2022.

How You Can Get Involved

    Social workers can get involved in this important survey of the profession in several ways.

If you have the opportunity to complete the practice analysis survey, please make time in your busy schedule to respond. Your voice is important and will contribute to informing future exam content.

When you hear of opportunities to serve as a subject matter expert for the Practice Analysis Task Force, submit an application.

When you learn about opportunities to become an item writer, seek out more information.

    ASWB wants to engage as many social workers as possible in the upcoming practice analysis. You will benefit from getting involved and so will our profession. By lending your expertise and unique perspective to the practice analysis process, you will be contributing to forming the blueprints that direct the content and the amount of that content that will appear on future social work exams. You will know that you are helping to move our profession forward in keeping both current and relevant in our ever-changing world.

References

Association of Social Work Boards. (2017). 2017 Analysis of the profession of social work: Final report.

Association of Social Work Boards. (2021). Exam content outlines. https://www.aswb.org/exam-candidates/about-the-exams/exam-content-outlines/

Brown, J. C. (1922). A city case worker in the country. Family, 8(3), 187b–193.

Merriam Webster Inc. (n.d.). Blueprint. In Merriam-Webster Dictionary. https://www.merriam-webster.com/dictionary/blueprint#h1

Nancy Sidell, PhD, LSW (she/her), is Examination Development Consultant for the Association of Social Work Boards. She has been involved with ASWB since 2002, first as an item writer and then as an item development consultant. She is an instructor for Capella University’s DSW program and professor emeritus at Mansfield University of Pennsylvania. She received her doctorate from The Ohio State University.