Maomi UENO (植野 真臣)

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Challenging Research (Exploratory), Category: Grant-in-Aid for Challenging Research (Exploratory), Fund Type: -, Overall Grant Amount: - (direct: 4900000, indirect: 1470000)
In automated essay scoring (AES), essays are automatically graded without human raters. Many AES models based on various manually designed features or various architectures of deep neural networks have been proposed over the past few decades. Each AES model has unique advantages and characteristics. Therefore, rather than using a single AES model, appropriate integration of predictions from various AES models is expected to achieve higher scoring accuracy. In the present paper, we develops 1) a new item response theory model that can estimate scores while considering characteristics of individual human-raters and rubric-items, and 2) a method that uses the item response theory model to integrate prediction scores from various AES models while taking into account differences in the characteristics of scoring behavior.

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (S), Category: Grant-in-Aid for Scientific Research (S), Fund Type: -, Overall Grant Amount: - (direct: 123900000, indirect: 37170000)
本研究は，近年ニーズが高まっている筆記式試験や実技試験などのパフォーマンステストを含んで，高精度の測定誤差が持続するeテスティングプラットフォームを開発し，実運用によりその有効性を示すことを目指している．令和2年度は，提案プラットフォームを構成する以下の基礎技術について，令和元年度に引き続き研究を進めた．1）最大クリーク・アルゴリズムと整数計画法を用いて，テスト生成数をより向上させるアルゴリズムの開発，2）項目露出を一様とする等質テスト自動構成アルゴリズムの開発，3）項目露出を制御する等質適応型テストの開発，４）異質評価者の同定と継続的なトレーニング手法の開発，5）自然言語処理を用いた筆記試験における自動採点手法の開発．1）と2）については実装と評価実験が完了し，研究成果は電子情報通信学会に掲載された．3）についても順調に開発が進行しており，その成果は複数の国内学会で発表を行ない，Advances in Artificial Intelligenceに論文が掲載された．4）については，異質評価者の特性を表現できる新たな項目反応モデルを開発し，関連する成果が国際論文誌のBehaviormetrikaとBehavior Research Methods，および電子情報通信学会論文誌に掲載された．5）については，深層学習モデルと項目反応理論を組み込んだ新たな方法論を提案し，教育分野における人工知能活用に関する主要国際会議であるArtificial Intelligence in Educationに２件，自然言語処理分野の主要国際会議の一つであるInternational Conference on Computational Linguisticsに1件の論文が採択された．
また，本研究テーマの主要課題の一つである「パフォーマンステストの継続運用を想定した運用デザインの設計とその実施支援システムの開発」に関しては，東京医科歯科大学での実証実験を想定して令和元年度に構築したシステムについて予備実験を進めた．

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (C), Category: Grant-in-Aid for Scientific Research (C), Fund Type: -, Overall Grant Amount: - (direct: 3600000, indirect: 1080000)
Summary of research: Critical thinking is considered important. However, this area is wide and measuring the ability takes time. On the other hand, test theory measures ability efficiently using computer adaptive test (CAT) based on item response theory (IRT). Therefore, in this research, we aimed at the development of CAT based on IRT for the examination of the critical thinking test, and addressed the following issues.
Research Results: Scale development: IRT promoted further scale development by optimal estimation of parameters. (1) Analytical thinking ability scale (2) Logic and reasoning ability scale (3) Reading and understanding ability scale．In addition, data were collected from the participants in the experiment to verify their reliability and validity.

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Challenging Exploratory Research, Category: Grant-in-Aid for Challenging Exploratory Research, Fund Type: -, Overall Grant Amount: - (direct: 2900000, indirect: 870000)
This study developed a scaffolding system that provides adaptive hints to adjust the predictive probability of the learner's successful performance to the previously determined certain value, using the item response theory (IRT). Furthermore, using the scaffolding system, we compared learning performances by changing the predictive probability. Results show that scaffolding to achieve 0.5 learner success probability provides the best performance. Additionally, results demonstrate that a scaffolding system providing 0.5 probability decreases the number of hints automatically as a fading function according to the learner's growth capability.

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (A), Category: Grant-in-Aid for Scientific Research (A), Fund Type: -, Overall Grant Amount: - (direct: 30900000, indirect: 9270000)
This study developed a state-of-the-art e-Testing system, which realizes to construct the most number of uniform tests, and its operational guidelines based on actual practice of e-testing in several testing organizations. The results were published in the top international journals (IEEE transactions) and top conferences (AIED). The developed system and guidelines were evaluated in actual test organizations, such as National information technology engineer examination in INFORMATION-TECHNOLOGY PROMOTION AGENCY(IPA), Common achievement tests organization(CATE), Benesse corporation, Eiken, the synthetic personality inventory (SPI) examination, which are popular tests in Japan. We reported the details of each test organizations operation model of the e-testing system in the Japan Association for practice on testing (JART2017).

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Challenging Exploratory Research, Category: Grant-in-Aid for Challenging Exploratory Research, Fund Type: -, Overall Grant Amount: - (direct: 2900000, indirect: 870000)
Educational assessments occasionally require uniform test forms for which each test form comprises a different set of items, but the forms meet equivalent test specifications (i.e., qualities indicated by test information functions based on item response theory). We propose two maximum clique algorithms for uniform test form assembly. The proposed methods can assemble uniform test forms with allowance of overlapping items among uniform test forms. First, we propose an exact method that maximizes the number of uniform test forms from an item pool. However, the exact method presents computational cost problems. To relax those problems, we propose an approximate method that maximizes the number of uniform test forms asymptotically. Accordingly, the proposed methods can use the item pool more efficiently than traditional methods can. We demonstrate the efficiency of the proposed methods using simulated and actual data.

Offer Organization: Japan Society for the Promotion of Science, System Name: Grants-in-Aid for Scientific Research Grant-in-Aid for Scientific Research (B), Category: Grant-in-Aid for Scientific Research (B), Fund Type: -, Overall Grant Amount: - (direct: 14800000, indirect: 4440000)
There are four characteristics that a learning community must have: (1) diversity of expertise among its members, (2) a shared objective of continually advancing the collective knowledge and skills, (3) an emphasis on learning how to learn, and (4) mechanisms for sharing what is learned. To enhance the development of learning communities, we developed an ePortfolio recommendation system. The unique features of this system are as follows: 1. The system recommends excellent other students who have similar learning histories with the user, 2. The system searches diverse others as much as possible. Namely, the system recommends excellent other students with similar learning histories to the target user but dissimilar each other. Actual trial use of the system demonstrates that the system does indeed promote learning from others，and supports sustainability of learning and deeper robust acquisition of knowledge; not superficial learning based on memorization.