Can automated machine translation evaluation metrics be used to assess students’ interpretation in the language learning classroom?

计算机科学 机器翻译 NIST公司 自然语言处理 人工智能 操作化 人气 机器翻译评价 语言习得 分级(工程) 公制(单位) 能力(人力资源) 数学教育 心理学 机器翻译软件可用性 认识论 工程类 社会心理学 哲学 土木工程 经济 基于实例的机器翻译 运营管理
作者
Chao Han,Xiaolei Lu
出处
期刊:Computer Assisted Language Learning [Informa]
卷期号:36 (5-6): 1064-1087 被引量:12
标识
DOI:10.1080/09588221.2021.1968915
摘要

The use of translation and interpreting (T&I) in the language learning classroom is commonplace, serving various pedagogical and assessment purposes. Previous utilization of T&I exercises is driven largely by their potential to enhance language learning, whereas the latest trend has begun to underscore T&I as a crucial skill to be acquired as part of transcultural competence for language learners and future language users. Despite their growing popularity and utility in the language learning classroom, assessing T&I is time-consuming, labor-intensive and cognitively taxing for human raters (e.g., language teachers), primarily because T&I assessment entails meticulous evaluation of informational equivalence between the source-language message and target-language renditions. One possible solution is to rely on automated quality metrics that are originally developed to evaluate machine translation (MT). In the current study, we investigated the viability of using four automated MT evaluation metrics, BLEU, NIST, METEOR and TER, to assess human interpretation. Essentially, we correlated the automated metric scores with the human-assigned scores (i.e., the criterion measure) from multiple assessment scenarios to examine the degree of machine-human parity. Overall, we observed fairly strong metric-human correlations for BLEU (Pearson's r = 0.670), NIST (r = 0.673) and METEOR (r = 0.882), especially when the metric computation was conducted on the sentence level rather than the text level. We discussed these emerging findings and others in relation to the feasibility of operationalizing MT metrics to evaluate students' interpretation in the language learning classroom.Supplemental data for this article is available online at https://doi.org/10.1080/09588221.2021.1968915 .
最长约 10秒,即可获得该文献文件

科研通智能强力驱动
Strongly Powered by AbleSci AI
更新
大幅提高文件上传限制,最高150M (2024-4-1)

科研通是完全免费的文献互助平台,具备全网最快的应助速度,最高的求助完成率。 对每一个文献求助,科研通都将尽心尽力,给求助人一个满意的交代。
实时播报
刚刚
顾矜应助林子采纳,获得10
1秒前
1秒前
2秒前
ycccccc完成签到,获得积分10
2秒前
202483067发布了新的文献求助10
2秒前
WHY发布了新的文献求助30
3秒前
我爱学习完成签到 ,获得积分10
3秒前
Hehhhh发布了新的文献求助10
4秒前
耍酷的素关注了科研通微信公众号
5秒前
Aprilapple发布了新的文献求助10
5秒前
ding应助萌妹采纳,获得10
5秒前
无情代亦发布了新的文献求助10
6秒前
song发布了新的文献求助10
6秒前
7秒前
涛涛tt完成签到,获得积分10
7秒前
7秒前
01发布了新的文献求助10
8秒前
9秒前
fane发布了新的文献求助30
10秒前
见录完成签到,获得积分10
11秒前
微风低回完成签到,获得积分10
11秒前
11秒前
hdc12138发布了新的文献求助10
11秒前
12秒前
zztmyhbk完成签到,获得积分10
13秒前
科研通AI2S应助fifteen采纳,获得10
14秒前
枯木逢春完成签到,获得积分20
14秒前
大模型应助楼萌黑采纳,获得10
15秒前
木鱼完成签到,获得积分10
15秒前
偷喝汽水发布了新的文献求助10
16秒前
紧张的世德完成签到,获得积分10
17秒前
脑洞疼应助Cloud采纳,获得10
17秒前
17秒前
17秒前
18秒前
CodeCraft应助ezreal采纳,获得10
18秒前
薰硝壤应助01采纳,获得10
18秒前
脑洞疼应助吴雨涛采纳,获得10
18秒前
涛涛tt发布了新的文献求助10
20秒前
高分求助中
The ACS Guide to Scholarly Communication 2500
Sustainability in Tides Chemistry 2000
Pharmacogenomics: Applications to Patient Care, Third Edition 1000
Studien zur Ideengeschichte der Gesetzgebung 1000
TM 5-855-1(Fundamentals of protective design for conventional weapons) 1000
Genera Insectorum: Mantodea, Fam. Mantidæ, Subfam. Hymenopodinæ (Classic Reprint) 800
Ethnicities: Media, Health, and Coping 700
热门求助领域 (近24小时)
化学 医学 生物 材料科学 工程类 有机化学 生物化学 物理 内科学 纳米技术 计算机科学 化学工程 复合材料 基因 遗传学 催化作用 物理化学 免疫学 量子力学 细胞生物学
热门帖子
关注 科研通微信公众号,转发送积分 3086960
求助须知:如何正确求助?哪些是违规求助? 2739744
关于积分的说明 7555906
捐赠科研通 2389490
什么是DOI,文献DOI怎么找? 1267216
科研通“疑难数据库(出版商)”最低求助积分说明 613651
版权声明 598611