TrueLearn Data Lab: Creating the Most Effective Test Modes
In the last Data Lab, Which Test Modes Do Higher Performers Use?, we discussed the different types of test types (Timed, Untimed, Tutor). Now, we will look at the most effective mode for creating exams within the TrueLearn application.
TRUELEARN’S TEST MODES
- Quick Create – Simulate the Exam: Test using blueprint categories in balance with the actual exam.
- Custom Test Creation: This mode provides the most flexibility in testing and allows you to customize your test type, question type, marked items, and lets you decide which categories to include.
- Quick Create – Improve Weaknesses: Test using questions in your weakest categories, questions you previously answered incorrectly and questions you mark along the way.
- Quick Create – Diagnose Strengths and Weaknesses: Create an exam from categories where you have used the fewest number of questions.
- Retake: Repeat a previously completed test.
Since the implementation of different modes of test creation in February 2016, over one million tests have been completed by TrueLearn users. Here are some numbers of test creation by question and by tests:
OVER-PERFORMERS AGAINST UNDER-PERFORMERS
In the last Data Lab, we compared the study patterns of over-performing and under-performing users based on the exam type (Timed, Untimed, Tutor). This analysis is continued for test creation modes.
The same dataset is being used and that includes over 2,500 users who finished at least 800 questions in our COMLEX Level 1 2016 QBank. Since the effect of number of questions taken on performance has already been established in previous Data Labs, we decided to focus on the difference brought by test creation modes.
The first step, which is identical to the previous Data Lab, is to identify top/bottom performers based on the improvement and the score for the last 30 percent of questions taken, as shown in the figure below. After identifying both groups of users, the next step is to compare their proportions of test creation modes. From the summary table below, over-performers take significantly more questions under tests created by Simulation Mode, and they take less questions under tests that are custom created by themselves.
Considering all three “quick create” modes together (Simulate, Improve, Diagnose), the over-performing students attempted more questions than the lower performing group. Upon applying the Wilcoxon test, we can see the group population averages for the over-performers and under-performers for the percentage of questions taken under Simulation Mode and under Custom Mode are significantly different with a confidence level of 95 percent.
BECOME AN OVER-PERFORMER
It should go without saying that “Simulating your Exam” before you take the real one is recommended. You want to become familiar with taking questions randomly from all categories, and you will want to become comfortable with taking tests under a realistic time constraint.
However, after analyzing the latest COMLEX Level 1 2016 users’ data and the groups of better performing and lower performing students, we now have data to back this up. We see that our highest performing students take more Simulate Mode questions and less custom created tests.
As our previous Data Lab shows, the more practice questions you take, the better result you can expect. Now, with this conclusion, we encourage you to try out the Simulate Mode as well as the other Quick Create practice modes.