检索结果-内蒙古大学图书馆

Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

Visual Grounding Helps Learn Word Meanings in Low-Data Regim...

2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024

作者： Zhuang, Chengxu Fedorenko, Evelina Andreas, Jacob McGovern Institute for Brain Research MIT United States Department of Brain and Cognitive Sciences MIT United States The Program in Speech and Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory MIT United States

ISBN: (纸本)9798891761148

Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with representations of language in the human brain. But to achieve these results, LMs must be trained in distinctly un-human-like ways-requiring orders of magnitude more language data than children receive during development, and without perceptual or social context. Do models trained more naturalistically-with grounded supervision-exhibit more human-like language learning? We investigate this question in the context of word learning, a key sub-task in language acquisition. We train a diverse set of LM architectures, with and without auxiliary visual supervision, on datasets of varying scales. We then evaluate these models’ learning of syntactic categories, lexical relations, semantic features, word similarity, and alignment with human neural representations. We find that visual supervision can indeed improve the efficiency of word learning. However, these improvements are limited: they are present almost exclusively in the low-data regime, and sometimes canceled out by the inclusion of rich distributional signals from text. The information conveyed by text and images is not redundant-models mainly driven by visual information yield qualitatively different from those mainly driven by word co-occurrences. However, our results suggest that current multimodal modeling approaches fail to effectively leverage visual information to build human-like word representations from human-scale data. © 2024 Association for Computational Linguistics.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Hering's opponent-colors theory fails a key test in a non-Western culture 44

Hering's opponent-colors theory fails a key test in a non-We...

引用

44th Annual Meeting of the Cognitive Science Society: Cognitive Diversity, CogSci 2022

作者： Conway, Bevil R. Malik-Moraleda, Saima Gibson, Edward Laboratory of Sensorimotor Research National Eye Institute National Institute of Mental Health Program in Speech and Hearing Bioscience and Technology Harvard University United States Department of Brain and Cognitive Sciences MIT

Opponent Colors Theory advances that four colors have special status and are yoked in opponent fashion (yellow-versus-blue, and red-versus-green). Classic hue cancelation studies provide evidence for this theory: people readily pick out colors that are neither red nor green, usually yellow. Here we conducted a version of a hue-cancelation experiment with the Tsimane' people, a nonindustrialized culture in the Amazon. Tsimane' speakers readily identified reddish and greenish color chips, but they showed idiosyncratic choices when asked to identify a color that is neither reddish nor greenish, unlike English speakers who consistently select focal yellow. The Tsimane' participants who also spoke Spanish and had a consistent label for English "yellow" ("amarillo"), performed similarly to the Tsimane' monolinguals, suggesting that simply having a label for "yellow" is not sufficient to explain the consistency of English speakers. The results add to a growing body of evidence that does not support Opponent Colors Theory. © 2022 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY)

关键词： Color

来源：评论

学校读者我要写书评

暂无评论

Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling

arXiv

引用

arXiv 2024年

Today’s most accurate language models are trained on orders of magnitude more language data than human language learners receive—but with no supervision from other sensory modalities that play a crucial role in human learning. Can we make LMs’ representations and predictions more accurate (and more human-like) with more ecologically plausible supervision? This paper describes LexiContrastive Grounding (LCG), a grounded language learning procedure that leverages visual supervision to improve textual representations. LexiContrastive Grounding combines a next-token prediction strategy with a contrastive visual grounding objective, focusing on early-layer representations that encode lexical information. Across multiple word-learning and sentence-understanding benchmarks, LexiContrastive Grounding not only outperforms standard language-only models in learning efficiency, but also improves upon vision-and-language learning procedures including CLIP, GIT, Flamingo, and Vokenization. Moreover, LexiContrastive Grounding improves perplexity by around 5% on multiple language modeling tasks. This work underscores the potential of incorporating visual grounding into language models, aligning more closely with the multimodal nature of human language acquisition. © 2024, CC BY.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

arXiv

引用

arXiv 2023年

Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with representations of language in the human brain. But to achieve these results, LMs must be trained in distinctly un-human-like ways—requiring orders of magnitude more language data than children receive during development, and without perceptual or social context. Do models trained more naturalistically—with grounded supervision—exhibit more human-like language learning? We investigate this question in the context of word learning, a key sub-task in language acquisition. We train a diverse set of LM architectures, with and without auxiliary visual supervision, on datasets of varying scales. We then evaluate these models’ learning of syntactic categories, lexical relations, semantic features, word similarity, and alignment with human neural representations. We find that visual supervision can indeed improve the efficiency of word learning. However, these improvements are limited: they are present almost exclusively in the low-data regime, and sometimes canceled out by the inclusion of rich distributional signals from text. The information conveyed by text and images is not redundant—models mainly driven by visual information yield qualitatively different from those mainly driven by word co-occurrences. However, our results suggest that current multimodal modeling approaches fail to effectively leverage visual information to build human-like word representations from human-scale data. © 2023, CC BY.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Preventing autosomal-dominant hearing loss in Bth mice with CRISPR/CasRx-based RNA editing

引用

Signal Transduction and Targeted Therapy 2022年第4期7卷 1258-1270页

作者： Ziwen Zheng Guo Li Chong Cui Fang Wang Xiaohan Wang Zhijiao Xu Huiping Guo Yuxin Chen Honghai Tang Daqi Wang Mingqian Huang Zheng-Yi Chen Xingxu Huang Huawei Li Geng-Lin Li Xiaoxiang Hu Yilai Shu ENT Institute and Department of Otorhinolaryngology Eye&ENT HospitalState Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain ScienceFudan UniversityShanghai 200031China Institutes of Biomedical Sciences Fudan UniversityShanghai 200032China NHC Key Laboratory of Hearing Medicine(Fudan University) Shanghai 200032China State Key Laboratory of Agrobiotechnology China Agricultural UniversityBeijing 100193China College of Biological Sciences China Agricultural UniversityBeijing 100193China School of Life Science and Technology Southeast UniversityNanjing 210096China Department of Otolaryngology-Head and Neck Surgery Graduate Program in Speech and Hearing Bioscience and Technology and Program in NeuroscienceHarvard Medical SchoolBostonMA 02115USA Eaton-Peabody Laboratory Massachusetts Eye and Ear InfirmaryBostonMA 02114USA School of Life Science and Technology ShanghaiTech UniversityShanghai 200031China CAS Center for Excellence in Molecular Cell Science Shanghai Institute of Biochemistry and Cell BiologyChinese Academy of SciencesUniversity of Chinese Academy of SciencesShanghai 200031China Institutes of Brain Science and the Collaborative Innovation Center for Brain Science Fudan UniversityShanghai 200031China

CRISPR/RfxCas13d(CasRx)editing system can specifically and precisely cleave single-strand RNAs,which is a promising treatment for various disorders by downregulation of related gene ***,we tested this RNA-editing approach on Beethoven(Bth)mice,an animal model for human DFNA36 due to a point mutation in *** first screened 30 sgRNAs in cell cultures and found that CasRx with sgRNA3 reduced the Tmc1^(Bth)transcript by 90.8%,and the Tmc1 wild type transcript(Tmc1^(+))by 44.3%.We then injected a newly developed AAV vector(***)based CasRx into the inner ears of neonatal Bth mice,and we found that Tmc1^(Bth)was reduced by 70.2%in 2 weeks with few off-target effects in the whole ***,we found improved hair cell survival,rescued hair bundle degeneration,and reduced mechanoelectrical transduction ***,the hearing performance,measured in both ABR and DPOAE thresholds,was improved significantly in all ages over 8 ***,therefore,have validated the CRISPR/CasRx-based RNA editing strategy in treating autosomal-dominant hearing loss,paving way for its further application in many other hereditary diseases in hearing and beyond.

关键词： CRISPR/Cas editing hearing

来源：评论

学校读者我要写书评

暂无评论

JOSA: Joint surface-based registration and atlas construction of brain geometry and function

arXiv

引用

arXiv 2023年

作者： Li, Jian Tuckute, Greta Fedorenko, Evelina Edlow, Brian L. Dalca, Adrian V. Fischl, Bruce A. A. Martinos Center for Biomedical Imaging MGH & HMS United States Center for Neurotechnology and Neurorecovery MGH & HMS United States Department of Brain and Cognitive Sciences MIT United States McGovern Institute for Brain Research MIT United States Program in Speech Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory MIT United States

Surface-based cortical registration is an important topic in medical image analysis and facilitates many downstream applications. Current approaches for cortical registration are mainly driven by geometric features, such as sulcal depth and curvature, and often assume that registration of folding patterns leads to alignment of brain function. However, functional variability of anatomically corresponding areas across subjects has been widely reported, particularly in higher-order cognitive areas. In this work, we present JOSA, a novel cortical registration framework that jointly models the mismatch between geometry and function while simultaneously learning an unbiased population-specific atlas. Using a semi-supervised training strategy, JOSA achieves superior registration performance in both geometry and function to the state-of-the-art methods but without requiring functional data at inference. This learning framework can be extended to any auxiliary data to guide spherical registration that is available during training but is difficult or impossible to obtain during inference, such as parcellations, architectonic identity, transcriptomic information, and molecular profiles. By recognizing the mismatch between geometry and function, JOSA provides new insights into the future development of registration methods using joint analysis of the brain structure and function. © 2023, CC BY.

关键词： Geometry

来源：评论

学校读者我要写书评

暂无评论

Joint cortical registration of geometry and function using semi-supervised learning

arXiv

引用

arXiv 2023年

作者： Li, Jian Tuckute, Greta Fedorenko, Evelina Edlow, Brian L. Fischl, Bruce Dalca, Adrian V. A. A. Martinos Center for Biomedical Imaging Department of Radiology Massachusetts General Hospital Harvard Medical School United States Center for Neurotechnology and Neurorecovery Department of Neurology Massachusetts General Hospital Harvard Medical School United States Department of Brain and Cognitive Sciences Massachusetts Institute of Technology United States McGovern Institute for Brain Research Massachusetts Institute of Technology United States Program in Speech Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology United States Harvard-MIT Program in Health Sciences and Technology United States

Brain surface-based image registration, an important component of brain image analysis, establishes spatial correspondence between cortical surfaces. Existing iterative and learning-based approaches focus on accurate registration of folding patterns of the cerebral cortex, and assume that geometry predicts function and thus functional areas will also be well aligned. However, structure/functional variability of anatomically corresponding areas across subjects has been widely reported. In this work, we introduce a learning-based cortical registration framework, JOSA, which jointly aligns folding patterns and functional maps while simultaneously learning an optimal atlas. We demonstrate that JOSA can substantially improve registration performance in both anatomical and functional domains over existing methods. By employing a semi-supervised training strategy, the proposed framework obviates the need for functional data during inference, enabling its use in broad neuroscientific domains where functional data may not be observed. The source code of JOSA will be released to the public at https://***. © 2023, CC BY.

关键词： Iterative methods

来源：评论

学校读者我要写书评

暂无评论

Differential transverse motion of individual outer hair cells measured in gerbil high-frequency region

引用

AIP Conference Proceedings 2024年第1期3062卷

作者： Sunil Puria Nam Hyun Cho John Guinan Eaton Peabody Laboratories Massachusetts Eye and Ear Boston USA Harvard Medical School Boston MA USA Speech and Hearing & Bioscience and Technology Graduate Program at Harvard University Boston MA USA

The great sensitivity and frequency selectivity of mammalian hearing originates in the mechanical properties of the cochlea. Cochlear motions in response to sound are amplified using metabolic energy. The motor element of this cochlear amplification is presumed to be the outer hair cell (OHC), that expands and contracts lengthwise in response to a change in receptor potential at audio frequencies. How thousands of OHCs, acting through the local cytoarchitecture, work to achieve cochlear amplification is not fully understood. The properties of individual OHCs measured in micro-chamber environments are low-pass with a corner frequency of, at most, a few kHz. OHC corner frequencies within the cochlea have been estimated to be ∼3 kHz. Since in-vivo cochlear motion measurements show amplified frequency responses that extend up to many tens of kHz, but OHCs have corner frequencies of a few kHz, it has been suggested that OHCs are not active at the highest frequencies. How to reconcile these measurements has been a challenge. To address this, we made transverse cochlear motion measurements using a high-resolution optical-coherence-tomography (OCT) system at approximately the two ends of the OHC axial length: at the OHC bottom near the OHC-Deiter-Cell junction and at the OHC top near the reticular lamina. From these in-vivo measurements near the 42 kHz best frequency (BF) region of the gerbil cochlea, we determined the transverse differential response of OHCs. To remove phase from the traveling wave, we calculated the ratio of the RL motion to the motion at the OHC-Deiter-Cell junction. The OHC top-to-bottom phase difference was almost exactly ½-cycle at frequencies up to 35-40 kHz. As frequency increased above 40 kHz, the phase difference decreased until near BF there was very little phase difference. The OHC top-to-bottom amplitude ratio transitioned from the OHC top moving less than the bottom at low frequencies, to the OHC top moving more than the bottom, at about 30 kH

关键词：

来源：评论

学校读者我要写书评

暂无评论

Towards simulating micromechanics of the organ of Corti induced by the traveling wave using a slice finite-element model of the mouse cochlea

引用

AIP Conference Proceedings 2024年第1期3062卷

作者： Yanli Wang Sunil Puria Dept. of Otolaryngology Harvard Medical School Boston MA USA Eaton-Peabody Laboratories Massachusetts Eye & Ear Boston MA USA Graduate Program in Speech and Hearing Bioscience and Technology Harvard University Cambridge MA USA

The intricate cellular structure within the organ of Corti (OoC) provides the structural basis for how the stereocilia bundles of the hair cells are stimulated, which triggers the mechanoelectrical transduction channels. This intricate process remains elusive, and the stimulation mechanism of the inner hair cell (IHC) stereocilia is under heated debate. Recent experimental data on micromechanics of the OoC and IHC stereocilia motion motivate modeling studies to understand the detailed cellular motion within OoC. However, such a model is not computationally feasible for a full-length model of the cochlea. Current work develops and examines a slice modeling technique that considers the stimulation input location and wavenumber at the slice location. This slice model is compared to results from a full-length box model.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Can machines learn continuous measures of speech severity from ordinal training labels? 33

Can machines learn continuous measures of speech severity fr...

引用

33rd International Florida Artificial Intelligence Research Society Conference, FLAIRS 2020

作者： Wisler, Alan Teplansky, Kristin Green, Jordan R. Austin, Sara G. Wang, Jun Department of Communication Sciences and Disorders University of Texas at Austin AustinTX United States Department of Communication Sciences and Disorders Mgh Institute of Health Professions BostonMA United States Speech and Hearing Bioscience and Technology Program Harvard University BostonMA United States Department of Neurology Dell Medical School University of Texas at Austin AustinTX United States

ISBN: (纸本)9781577358213

In this study we examined the efficacy of machine learning general regression algorithms for predicting ordinal variables based on the acoustic speech signal. We were specifically interested in whether predictions that fell between ordinal levels (e.g. a predicted score of 3.2 instead of a true score of 3) contained meaningful information about the outcome variable. As a test case, we explored speech-based estimation of the Amyotrophic Lateral Sclerosis Functional Rating Scale-Revised (ALSFRS-R), a clinical measure for speech severity. The ALSFRS-R is a diagnostic tool that measures individual components of motor function for patients with ALS along a 5-point ordinal scale. Using artificial neural networks (ANN) we can generate continuous estimates of the speech component in ALSFRS-R. However, the degree to which the improved resolution of these estimates contains useful information related to patients' motor control has not been thoroughly studied. In this paper, we sought to answer this question by comparing the residuals in machine learning estimates of the ALSFRS-R speech score with patients' intelligible speaking rate (ISR), which is a more granular measure of speech motor control. Experimental results using speech data from 45 patients with ALS confirmed that ANN regression is effective at learning granular information even when trained with coarse ordinal labels. © FLAIRS *** right reserved.

关键词： speech

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：