Methodology for Developing Questions

Research

The ARK team spent months combing through every major Catholic religion textbook in the country to identify the topics taught in 2nd through 8th grades in Catholic schools and parish faith formation programs. Once all of the textbooks were reviewed, those topics covered in all textbooks were included on the ARK test at their appropriate grade level.

At the high school level, topics were drawn from the USCCB’s curriculum framework, which all major textbooks adhere to. 

Drafting

Once the topics were identified, a team of Catholic teachers with 45 years combined experience in the classroom prepared the first draft of each multiple-choice question.

Professional, Nationwide Review

These draft questions then went through a nationwide review process wherein multiple teachers at each grade level rated the questions based on grade-level appropriateness. Their involvement helped us to finalize the questions and include among them a mix of easy, medium, and challenging questions. Those questions that were rated as too easy or too hard were removed from the question pool.

Each question was carefully reviewed by our team of psychometricians who ensured that all questions were written in a straightforward and clear way. Any question that was confusingly worded or could possibly lead to invalid results was stricken. 

Pilot Testing

Finally, hundreds of students are part of a pilot test that provides statistical validation of both the test and the scoring system.

You may view the sample questions for each grade by visiting this page.

Scroll to Top