で紹介した深層学習技術やそれらをRaNNC[12]で更に大規模にしたより高品質な言語モデルを構築し、また、これまでに構築した言語リソース等も活用して、より興味深く、かつ、社会で役立つシステムの研究開発に取り組んでいきたいと考えている。参考文献】【1A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” CoRR, vol.abs/1706.03762, 2017.2J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” Pro-ceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech-nologies (NAACL-HLT 2019), pp.4171–4186, 2019.3大竹 清敬, “社会知コミュニケーション技術の概要,” 情報通信研究機構研究報告，本特集号，3-1, 2022.4T.B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D.M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei, “Language models are few-shot learners,” CoRR, vol.abs/2005.14165, 2020.5B. Kim, H. Kim, S. Lee, G. Lee, D. Kwak, D.H. Jeon, S. Park, S. Kim, S. Kim, D. Seo, H. Lee, M. Jeong, S. Lee, M. Kim, S. Ko, S. Kim, T. Park, J. Kim, S. Kang, N. Ryu, K.M. Yoo, M. Chang, S. Suh, S. In, J. Park, K. Kim, H. Kim, J. Jeong, Y.G. Yeo, D. Ham, D. Park, M.Y. Lee, J. Kang, I. Kang, J. Ha, W. Park, and N. Sung, “What changes can large-scale language models bring? intensive study on HyperCLOVA: Billions-scale korean generative pretrained transformers,” CoRR, vol.abs/2109.04650, 2021.6V.N. Vapnik, The Nature of Statistical Learning Theory, Springer, 1995.7水野 淳太，クロエツェー ジュリアン，田仲 正弘，飯田 龍，呉 鍾勲，石田 諒，淺尾 仁彦，福原 裕一，藤原 一毅，大西 可奈子，阿部 憲幸，大竹 清敬，鳥澤 健太郎， “WEKDA：web40億ページを知識源とする質問応答システムを用いた博学対話システム,” 第84回言語・音声理解と対話処理研究会資料， pp.135–142，2018．8Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “RoBERTa: A robustly optimized BERT pretraining approach,” CoRR, vol.abs/1907.11692, 2019.9Z. Lan, M. Chen, S. Goodman, K. Gimpel, P. Sharma, and R. Soricut, “ALBERT: A lite BERT for self-supervised learning of language repre-sentations,” Proceedings of the 8th International Conference on Learn-ing Representations (ICLR 2020), 2020.10K. Clark, M. Luong, Q.V. Le, and C.D. Manning, “ELECTRA: pre-training text encoders as discriminators rather than generators,” Proceedings of the 8th International Conference on Learning Representations (ICLR 2020), 2020.11P. He, X. Liu, J. Gao, and W. Chen, “DeBERTa: decoding-enhanced BERT with disentangled attention,” Proceedings of the 9th International Conference on Learning Representations (ICLR 2021), 2021.12M. Tanaka, K. Taura, T. Hanawa, and K. Torisawa, “Automatic graph partitioning for very large-scale deep learning,” Proceedings of 35st IEEE International Parallel and Distributed Processing Symposium (IP-DPS 2021), pp.1004–1013, 2021.13水野 淳太，淺尾 仁彦， “高齢者介護支援用マルチモーダル音声対話システムMICSUS,” 情報通信研究機構研究報告, 本特集号, 3-5, 2022.14T. Luong, H. Pham, and C.D. Manning, “Effective approaches to atten-tion-based neural machine translation,” Proceedings of the 2015 Con-ference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp.1412–1421, 2015.15L. Dong, N. Yang, W. Wang, F. Wei, X. Liu, Y. Wang, J. Gao, M. Zhou, and H.-W. Hon, “Unified language model pre-training for natural lan-guage understanding and generation,” Proceedings of the 33rd Confer-ence on Neural Information Processing Systems (NeurIPS 2019), 2019.16S. Rothe, S. Narayan, and A. Severyn, “Leveraging pre-trained check-points for sequence generation tasks,” Transactions of the Association for Computational Linguistics, vol.8, pp.264–280, 2020.17J.-H. Oh, K. Torisawa, C. Hashimoto, M. Sano, S. De Saeger, and K. Ohtake, “Why-question answering using intra- and inter-sentential causal relations,” Proceedings of the 51st Annual Meeting of the Asso-ciation for Computational Linguistics (ACL 2013), pp.1733–1743, 2013.18K. Kadowaki, R. Iida, K. Torisawa, J.-H. Oh, and J. Kloetzer, “Event causality recognition exploiting multiple annotators’ judgments and background knowledge,” Proceedings of the 2019 Conference on Em-pirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP 2019), pp.5820–5826, 2019.19関 直哉，水野 淳太，門脇 一真，飯田 龍，鳥澤 健太郎， “ファクトイド質問応答におけるBERTのpre-trainedモデルの影響の分析,” 言語処理学会第26回年次大会 発表予稿集 (NLP2020)， pp.105–108，2020．20C. Hashimoto, K. Torisawa, J. Kloetzer, M. Sano, I. Varga, J.-H. Oh, and Y. Kidawara, “Toward future scenario generation: Extracting event causality exploiting semantic relation, context, and association fea-tures,” Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014), pp.987–997, 2014.21C. Kruengkrai, K. Torisawa, C. Hashimoto, J. Kloetzer, J.-H. Oh, and M. Tanaka, “Improving event causality recognition with multiple back-ground knowledge sources using multi-column convolutional neural networks,” Proceedings of the 31st AAAI Conference on Artificial Intel-ligence (AAAI 2017), pp.3466–3473, 2017.22J. Kloetzer, K. Torisawa, C. Hashimoto, and J.-H. Oh, “Large-scale acquisition of entailment pattern pairs by exploiting transitivity,” Pro-ceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP 2015), pp.1649–1655, 2015.23二宮 大空，門脇 一真，飯田 龍，鳥澤 健太郎，J. Kloetzer， “BERTを用いたバイナリパタン間の含意関係認識,” 言語処理学会第26回年次大会 発表予稿集 (NLP2020)， pp.1423–1426，2020．24鈴木 正敏，松田 耕史，岡崎 直観，乾 健太郎， “読解による解答可能性を付与した質問応答データセットの構築,” 言語処理学会第24回年次大会 発表予稿集 (NLP2018)， pp.702–705，2018．25呉 鍾勲, クロエツェー ジュリアン, “大規模Web情報分析システムWISDOM X深層学習版,” 情報通信研究機構研究報告, 本特集号, 3-3, 2022.26J.-H. Oh, K. Kadowaki, J. Kloetzer, R. Iida, and K. Torisawa, “Open–domain why-question answering with adversarial learning to encode answer texts,” Proceedings of the 57th Annual Meeting of the Associa-tion for Computational Linguistics (ACL 2019), pp.4227–4237, 2019.27J.-H. Oh, R. Iida, J. Kloetzer, and K. Torisawa, “BERTAC: Enhancing transformer-based language models with adversarially pretrained con-volutional neural networks,” Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro-cessing (ACL-IJCNLP 2021), pp.2103–2115, 2021.28R. Ishida, K. Torisawa, J.-H. Oh, R. Iida, C. Kruengkrai, and J. Kloetzer, “Semi-distantly supervised neural model for generating compact an-swers to open-domain why questions,” Proceedings of 32nd AAAI Conference on Artificial Intelligence (AAAI 2018), pp.5803–5811, 2018.29R. Iida, C. Kruengkrai, R. Ishida, K. Torisawa, J.-H. Oh, and J. Kloetzer, “Exploiting background knowledge in compact answer generation for why-questions,” Proceedings of the 33rd AAAI Conference on Artificial Intelligence (AAAI 2019), pp.142–151, 2019.30A. See, P.J. Liu, and C.D. Manning, “Get to the point: Summarization with pointer-generator networks,” Proceedings of the 55th Annual Meet-ing of the Association for Computational Linguistics (ACL 2017), pp.1073–1083, 2017.31C.-Y. Lin, “ROUGE: A package for automatic evaluation of summaries,” Proceedings of the ACL-04 Workshop on Text Summarization Branches Out, pp.74–81, 2004.32C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P.J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, vol.21, no.140, pp.1–67, 2020.33N. Reimers and I. Gurevych, “Sentence-BERT: Sentence embeddings using Siamese BERT-networks,” Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMN-LP-IJCNLP 2019), pp.3982–3992, 2019.138 情報通信研究機構研究報告 Vol.68 No.2 （2022）3 社会知コミュニケーション技術
元のページ ../index.html#144