How Does ASWB Guard Against Bias on the Social Work Licensing Exams?

by Stacey Owens, MSW, LCSW-C

(Editor’s Note: This is the third and final article in a series of articles on the social work licensing process. Please see the sidebar to this article: Demographic Pass-Rate Data Release Under Discussion.)

     I recall preparing to take the social work licensing exam during the last semester of my MSW program. I had some trepidation because I’d heard stories of social workers who passed with flying colors on their first try and stories of social workers who struggled to achieve a passing score. If I’m being honest, I also had some unease because from the time that I began taking important exams in high school, I’d heard that these types of tests sometimes assume background knowledge that is not always held by diverse groups of test-takers.

    I wondered about the process that went into building this important component of entry into the social work profession, and I wondered who was involved in that process. Fast forward five years: I was reading an NASW newsletter during some down time at my emergency room job, and I saw an advertisement from ASWB calling for social workers to apply to become item writers. I realized that I could become involved in the process of building the exam. During the time that I’ve been involved with ASWB’s exam development program, I’ve seen how social work values are infused into the exam-building process and the steps taken to ensure that the social work exams are fair and equitable for everyone who’s testing.

Monitoring for Bias

    ASWB works to ensure fairness by monitoring for bias in two ways. The first tool ASWB uses is a testing industry statistical measurement called Differential Item Functioning (DIF). DIF indicates whether an exam question shows tendencies to advantage or disadvantage one group of test-takers over another (ASWB, 2020). DIF is identified by statistically analyzing responses to the exam questions—called items—during pretesting. Scored items are continually monitored for DIF.  On an annual basis, less than 5% of all items released show DIF. Items flagged for DIF are removed from the bank of potential exam questions.

    The second way that ASWB monitors for bias is through use of a diverse group of social work subject matter experts who bring their lived experiences as social work practitioners and educators to every stage of exam development. In addition to ensuring that an item is only testing current social work knowledge, they focus on excluding any extra information that could cause test-takers to arrive at varying conclusions based on having diverse backgrounds.  

Who Writes the Questions?

    Currently practicing licensed social workers from across the United States and Canada, not ASWB staff members, write all the questions. ASWB recruits for new writers every year or every other year through a comprehensive recruitment process. Part of making the selections for item writers includes balancing for diversity in demographic, geographic, and practice areas (ASWB, 2021c). Once selected, writers attend training and work with an assigned item development consultant to formulate exam questions. Writers are trained to recognize potential microaggressions and bias and to avoid words, assumptions, and stereotyping that can make an item unfair for some groups. In August 2021, ASWB trained a class of 24 writers. The new writers joined 71 active item writers creating questions for the exams.

    Writers contract with ASWB to write 30 questions approved by their consultant (ASWB, 2021a). Successful item writers can continue renewing their contracts and can also become involved in other areas of exam development. For example, I was selected as an item writer in 2009, appointed to the Exam Committee in 2011, and appointed as Exam Committee co-chair in 2013. After completing my term as co-chair, I returned to item writing, served as a form reviewer, and became an item development consultant in 2020. I explain these roles in the next section.

How Are Diverse Perspectives Reflected?

    Writing questions for the exams is only one step in the exam development process. Diverse voices are sought at each stage.

    Practice analysis. To keep the exam content current and relevant to practice, ASWB conducts a practice analysis, or survey of the profession, every five to seven years. The most recent survey was completed in 2016 and had the largest participation in ASWB history. More than 23,000 licensed social workers responded from every U.S. state, multiple U.S. territories, and all 10 Canadian provinces (ASWB, 2017a; Sidell, 2021).

    Of U.S. participants, 79% identified as white/Caucasian, 13% identified as African American/Black, and 10% identified as other ethnicities (ASWB, 2017b, p. C6). Of all participants, 88% identified as female and 12% as male. Less than 1% identified as transgender, nonbinary, queer, or differently gendered. In type of practice, 53% identified as a direct service provider, and the type of practice/work setting was reported as mental and/or behavioral health (54%); medical, hospital, or health services (28%); family and children’s services (27%); and advocacy (24%) (ASWB, 2017b, p. 11).

    Management of the practice analysis process includes social workers selected to serve on a 20-member Practice Analysis Task Force and a 5-member Oversight Panel (ASWB, 2017a). The task force is charged with developing the survey content and updating the exam blueprint based on survey results. Of U.S. social workers on the task force and oversight panel, 14 reported their race as Caucasian American/white; four as African American/Black; three as Hispanic or Latino; one as Asian (non-Hispanic); and one as American Indian/Alaskan Native. The two Canadian social workers identified themselves as Chinese. Practice backgrounds included social work education, clinical practice, and direct or macro practice as Bachelors, Masters, and Advanced Generalist social workers (ASWB, 2017b, pp. A2–A3).  

    Standard setting. In May 2017, ASWB convened the Passing Score Study Panel of 54 licensed social workers, a group also balanced for demographic, geographic, and practice diversity. Their job was to take the exams multiple times and rate each test question for probability that it would be answered correctly. They also evaluated whether the content of the exams measured minimum competence. Throughout this process, panelists discussed and averaged their judgments. These averages were used to compute the cut score, that is, the point on which pass-fail determinations are made (ASWB, 2017a).

    Item development. After item writers create an item, they work closely with item development consultants, who advise on fine-tuning structure, format, subject matter, and language used in the question. ASWB employs six consultants, two each for Bachelors, Masters, and Clinical exam categories. Five are women, three are African American/Black, four are in direct practice or agency management, and two are educators/retired educators (ASWB, 2021d).

    Questions approved by the consultants are next reviewed by the Examination Committee. The ASWB Board of Directors appoints Exam Committee members, who are former item writers, to serve up to three 1-year terms. ASWB staff recommend potential committee members based on item writing proficiency and the need to balance the committee for demographic, geographic, and practice diversity. Committee members evaluate individual items and modify them as needed to eliminate potential bias, including word choice. The committee must reach consensus for an item to be approved for pretesting on the exams. The committee also ensures that the values of diversity, equity, and inclusion are upheld (ASWB, 2021b).

    Form development. Exams, called forms, are constructed from approved items and are rotated online for use each quarter. Form reviewers, who are former Exam Committee members, are charged with reviewing ASWB examination forms before the forms go online. They also have responsibility for replacing items that may show potential bias or microaggressions.

    ASWB publishes an exam program yearbook identifying and highlighting the social workers involved in the exam development program. Yearbooks include demographic, geographic, and practice areas of these social workers. The ASWB web page Measuring Competence Fairly (see https://aswb.org/fairtesting) includes yearbooks from 2015 through 2020.

Measuring Competence Fairly

    After the Exam Committee approves an item, the next vital step is pretesting. When test-takers sit for the exam, they are presented with 170 items. Only 150 count toward the final score. The additional 20 pretest items are scattered throughout the exam so that test-takers are approaching them in the same manner as scored questions. This allows ASWB to ensure that items are pretested in an inclusive way.

    Quality assurance and statistical measurement continue after an item has been pretested and moves to the live scored exam. Items are continually monitored for reliability, validity, potential bias, and Differential Item Functioning. ASWB works with testing experts to statistically analyze the exams. As explained earlier, if any exam items are flagged for DIF, they are deleted and never used again. The Examination Committee reviews all questions that are flagged for DIF as a learning opportunity and to continue process improvement. To learn more about DIF, please watch ASWB’s exam development video (https://vimeo.com/468137793).

Get Involved

    In my 12 years of involvement with the ASWB exam program, I’ve learned that members of the social work community can help guard against bias on the exam by becoming knowledgeable, getting involved, and spreading the word. You can become more familiar with the process by reviewing the references and websites in this article. I also invite you to participate in the next practice analysis. Apply to be on the Practice Analysis Task Force forming in 2022 or complete the survey scheduled for launch March 1, 2023. Watch for announcements on social media and via email. If you’re interested in writing exam questions, check the ASWB website for updates and submit an application during the next recruitment cycle, which typically opens in September. Being part of the ASWB exam development community has been a rich and rewarding experience in my life.

    Please consider adding your diverse perspective to ASWB’s exam development program. Every voice is important to the process.

References

ASWB. (2017a). Analysis of the practice of social work, 2017 [PDF]. https://www.aswb.org/wp-content/uploads/2017/08/Summary-Document-FINAL.pdf

ASWB (2017b). 2017 analysis of the practice of social work: Final report [PDF].  https://www.aswb.org/wp-content/uploads/2020/11/2017-Tech-Report.pdf

ASWB. (2020, October 14). Ensuring item fairness in the ASWB exams [Video]. Vimeo. https://vimeo.com/468137793

ASWB. (2021a, February 23). Item writer program. https://www.aswb.org/exam/measuring-social-work-competence/item-writer-program/

ASWB. (2021b, February 23). Measuring competence fairly. https://aswb.org/fairtesting

ASWB. (2021c). Noteworthy [item writer graphic]. Association News, 31(3). https://www.aswb.org/noteworthy-june2021/

ASWB. (2021d). Examination program 2020 yearbook [PDF]. https://www.aswb.org/wp-content/uploads/2020/11/2020-Yearbook-ASWB-exam-program.pdf

Sidell, Nancy. (2021). What does the practice analysis have to do with questions on the ASWB exams? The New Social Worker, 28(2), 28–29. https://www.mediafire.com/file/ietmha7cojk0vls/springsummer2021.pdf/file

Stacey Owens, MSW, LCSW-C, is an item development consultant with the ASWB exam development program. She has been involved with exam development since 2009. She became an item development consultant for the Bachelors exam in 2020. Stacey has worked in the federal government, serving the military and veteran community. She also works as a PRN clinician with people who are involved in the justice system. She received her BSW from Prairie View A&M University and her MSW from the University of Texas and currently works in Maryland.