In validating the RAWA (i.e., a rater-mediated measure), the study aims to investigate how the examinees’ responses relate to examinee-ability, rater-severity, scale-difficulty, and score-step by scale via a mixed method that includes qualitative post-rating interview data to supplement quantitative score data. The rating scales were applied to 60 responses of 30 master’s and 30 doctoral students. Five raters with varying degrees of expertise were invited. A total of 600 data-points are planned for the Multi-faceted Rasch Model (MFRM) analysis via CONQUEST program. The MFRM results addressed the study purpose by testing 4 hypotheses. First, the doctoral students outperform the master’s. Second, the expert raters demonstrate higher severity than the developing. Third, the global-move scale entails higher difficulty than the local-pattern. Fourth, there is an interaction effect between score-step difficulty and scale (i.e., Scores 0 to 1 in the local-pattern of lower probability, Scores 4 to 5 in the global-move so). The rater-interview results revealed major rating strategies and decision-making process. These findings showed how effective the RAWA score is in manifesting the L2 RA-abstract writing competence, supporting the structural aspect of construct validity of RAWA. The RAWA validation may support the major constructs of the competence that may be decomposed into the global move and local pattern sub-competences. This validation has general implications for: (a) theory advancement in L2 RA-writing or L2-writing, (b) measure development in L2 RA-writing, (c) methodology framework for validating rater-mediated measure by considering rater-effect alongside other key factors, and (d) pedagogical insights into how to effectively develop L2 RA-writing by instruction or material compilation.
