Before we jump into a different type of question review process that's available in OASIS, let's discuss one of the biggest weakness within the category-based review. This diagram is a highly stylized mechanics behind the category-based review. As you can see, I have a red category for questions, purple, and then green. But in this idealized example, category red is a much, much bigger category that has way more questions when compared to the purple and the green category. And we are using category-based review with three levels, question writer, section leader, and chairperson. So imagine a question writer has written all these questions and questions are going through the review process. Section leaders for red is going to have far more questions to review compared with purple category and the green category. As a matter of fact, if you assign one section leader for red, one for purple, and one for green, the section leader for red is going to have three times as much workload. So they are definitely going to be overwhelmed. A way to overcome this is by assigning, say, three section leader to red category so that they will have the equal amount of workload compared to the purple and green, even though number may not always work out that way. Or obviously, you can assign a section leader who is just extremely knowledgeable and fast with their review process. But as you can see here, because the review is scoped based on which section leader is responsible for which category, you could create a workload imbalance. And if you assign multiple section leaders for one topic or multiple chairperson for one topic, you might also have the problem where there is a little bit of confusion over which section leader is supposed to review which question, because all the section leaders for topic will be able to review all the questions. Obviously, you can use email back and forth to keep everyone synchronized, but that will once again defeat the purpose of having a web-based system. To overcome this weakness, OASIS has a different question review mechanic called question-based review. So we are taking the previous example, but only use the purple questions to show. You could assign, say, question one to faculty four, two to faculty three. And then faculty number two has two questions. Faculty number one has two questions. So you could set up this relationship. And then you can say faculty three is in charge of reviewing question six, when it has been submitted for review. So this is round one reviewer, which is equivalent to the section leader. And then you can also have round two reviewer, which is equivalent to the section leader in the previous example. And even though it is possible in this case to have more than one in each round of review, typically, our client who use question-based review will not have more than one person at each cell here, if you consider this as Excel file. So typically, they will have essentially a mapped out matrix. If they have, say, ten questions, actually, let's say they have six questions, they will have 18 relationship because there are three columns times six. Now, if you have 100 questions, you will have 300 relationship because you are manually specifying to the system who is writing the first question all the way to 100, who is doing the first round of review from question one to question 100, and who is doing the second round of review from question one to 100. So this is definitely a far more manual intensive way of setting up question review because you are specifying every step of the review. But it is the most sort of a precise way of doing question review. Next set of tutorials will go through how to mechanically implement this review process in OASIS.

weakness

category-based question review process

OASIS

question-based review mechanic

review process implementation