检索结果-内蒙古大学图书馆

Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

Visual Grounding Helps Learn Word Meanings in Low-Data Regim...

2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2024

作者： Zhuang, Chengxu Fedorenko, Evelina Andreas, Jacob McGovern Institute for Brain Research MIT United States Department of Brain and Cognitive Sciences MIT United States The Program in Speech and Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory MIT United States

ISBN: (纸本)9798891761148

Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with representations of language in the human brain. But to achieve these results, LMs must be trained in distinctly un-human-like ways-requiring orders of magnitude more language data than children receive during development, and without perceptual or social context. Do models trained more naturalistically-with grounded supervision-exhibit more human-like language learning? We investigate this question in the context of word learning, a key sub-task in language acquisition. We train a diverse set of LM architectures, with and without auxiliary visual supervision, on datasets of varying scales. We then evaluate these models’ learning of syntactic categories, lexical relations, semantic features, word similarity, and alignment with human neural representations. We find that visual supervision can indeed improve the efficiency of word learning. However, these improvements are limited: they are present almost exclusively in the low-data regime, and sometimes canceled out by the inclusion of rich distributional signals from text. The information conveyed by text and images is not redundant-models mainly driven by visual information yield qualitatively different from those mainly driven by word co-occurrences. However, our results suggest that current multimodal modeling approaches fail to effectively leverage visual information to build human-like word representations from human-scale data. © 2024 Association for Computational Linguistics.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Hering's opponent-colors theory fails a key test in a non-Western culture 44

Hering's opponent-colors theory fails a key test in a non-We...

引用

44th Annual Meeting of the Cognitive Science Society: Cognitive Diversity, CogSci 2022

作者： Conway, Bevil R. Malik-Moraleda, Saima Gibson, Edward Laboratory of Sensorimotor Research National Eye Institute National Institute of Mental Health Program in Speech and Hearing Bioscience and Technology Harvard University United States Department of Brain and Cognitive Sciences MIT

Opponent Colors Theory advances that four colors have special status and are yoked in opponent fashion (yellow-versus-blue, and red-versus-green). Classic hue cancelation studies provide evidence for this theory: people readily pick out colors that are neither red nor green, usually yellow. Here we conducted a version of a hue-cancelation experiment with the Tsimane' people, a nonindustrialized culture in the Amazon. Tsimane' speakers readily identified reddish and greenish color chips, but they showed idiosyncratic choices when asked to identify a color that is neither reddish nor greenish, unlike English speakers who consistently select focal yellow. The Tsimane' participants who also spoke Spanish and had a consistent label for English "yellow" ("amarillo"), performed similarly to the Tsimane' monolinguals, suggesting that simply having a label for "yellow" is not sufficient to explain the consistency of English speakers. The results add to a growing body of evidence that does not support Opponent Colors Theory. © 2022 The Author(s). This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY)

关键词： Color

来源：评论

学校读者我要写书评

暂无评论

Lexicon-Level Contrastive Visual-Grounding Improves Language Modeling

arXiv

引用

arXiv 2024年

Today’s most accurate language models are trained on orders of magnitude more language data than human language learners receive—but with no supervision from other sensory modalities that play a crucial role in human learning. Can we make LMs’ representations and predictions more accurate (and more human-like) with more ecologically plausible supervision? This paper describes LexiContrastive Grounding (LCG), a grounded language learning procedure that leverages visual supervision to improve textual representations. LexiContrastive Grounding combines a next-token prediction strategy with a contrastive visual grounding objective, focusing on early-layer representations that encode lexical information. Across multiple word-learning and sentence-understanding benchmarks, LexiContrastive Grounding not only outperforms standard language-only models in learning efficiency, but also improves upon vision-and-language learning procedures including CLIP, GIT, Flamingo, and Vokenization. Moreover, LexiContrastive Grounding improves perplexity by around 5% on multiple language modeling tasks. This work underscores the potential of incorporating visual grounding into language models, aligning more closely with the multimodal nature of human language acquisition. © 2024, CC BY.

关键词： Visual languages

来源：评论

学校读者我要写书评

暂无评论

Predicting Heart Activity from speech using Data-driven and Knowledge-based features

arXiv

引用

arXiv 2024年

作者： Elbanna, Gasser Mostaani, Zohreh Magimai.-Doss, Mathew Speech and Hearing Bioscience and Technology Harvard Medical School MA United States Idiap Research Institute Martigny Switzerland Ecole polytechnique fédérale de Lausanne Switzerland

Accurately predicting heart activity and other biological signals is crucial for diagnosis and monitoring. Given that speech is an outcome of multiple physiological systems, a significant body of work studied the acoustic correlates of heart activity. Recently, self-supervised models have excelled in speech-related tasks compared to traditional acoustic methods. However, the robustness of data-driven representations in predicting heart activity remained unexplored. In this study, we demonstrate that self-supervised speech models outperform acoustic features in predicting heart activity parameters. We also emphasize the impact of individual variability on model generalizability. These findings underscore the value of data-driven representations in such tasks and the need for more speech-based physiological data to mitigate speaker-related challenges. © 2024, CC BY-SA.

关键词： Electrocardiograms

来源：评论

学校读者我要写书评

暂无评论

Visual Grounding Helps Learn Word Meanings in Low-Data Regimes

arXiv

引用

arXiv 2023年

Modern neural language models (LMs) are powerful tools for modeling human sentence production and comprehension, and their internal representations are remarkably well-aligned with representations of language in the human brain. But to achieve these results, LMs must be trained in distinctly un-human-like ways—requiring orders of magnitude more language data than children receive during development, and without perceptual or social context. Do models trained more naturalistically—with grounded supervision—exhibit more human-like language learning? We investigate this question in the context of word learning, a key sub-task in language acquisition. We train a diverse set of LM architectures, with and without auxiliary visual supervision, on datasets of varying scales. We then evaluate these models’ learning of syntactic categories, lexical relations, semantic features, word similarity, and alignment with human neural representations. We find that visual supervision can indeed improve the efficiency of word learning. However, these improvements are limited: they are present almost exclusively in the low-data regime, and sometimes canceled out by the inclusion of rich distributional signals from text. The information conveyed by text and images is not redundant—models mainly driven by visual information yield qualitatively different from those mainly driven by word co-occurrences. However, our results suggest that current multimodal modeling approaches fail to effectively leverage visual information to build human-like word representations from human-scale data. © 2023, CC BY.

关键词： Semantics

来源：评论

学校读者我要写书评

暂无评论

Joint cortical registration of geometry and function using semi-supervised learning

arXiv

引用

arXiv 2023年

作者： Li, Jian Tuckute, Greta Fedorenko, Evelina Edlow, Brian L. Fischl, Bruce Dalca, Adrian V. A. A. Martinos Center for Biomedical Imaging Department of Radiology Massachusetts General Hospital Harvard Medical School United States Center for Neurotechnology and Neurorecovery Department of Neurology Massachusetts General Hospital Harvard Medical School United States Department of Brain and Cognitive Sciences Massachusetts Institute of Technology United States McGovern Institute for Brain Research Massachusetts Institute of Technology United States Program in Speech Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory Massachusetts Institute of Technology United States Harvard-MIT Program in Health Sciences and Technology United States

Brain surface-based image registration, an important component of brain image analysis, establishes spatial correspondence between cortical surfaces. Existing iterative and learning-based approaches focus on accurate registration of folding patterns of the cerebral cortex, and assume that geometry predicts function and thus functional areas will also be well aligned. However, structure/functional variability of anatomically corresponding areas across subjects has been widely reported. In this work, we introduce a learning-based cortical registration framework, JOSA, which jointly aligns folding patterns and functional maps while simultaneously learning an optimal atlas. We demonstrate that JOSA can substantially improve registration performance in both anatomical and functional domains over existing methods. By employing a semi-supervised training strategy, the proposed framework obviates the need for functional data during inference, enabling its use in broad neuroscientific domains where functional data may not be observed. The source code of JOSA will be released to the public at https://***. © 2023, CC BY.

关键词： Iterative methods

来源：评论

学校读者我要写书评

暂无评论

Preventing autosomal-dominant hearing loss in Bth mice with CRISPR/CasRx-based RNA editing

引用

Signal Transduction and Targeted Therapy 2022年第4期7卷 1258-1270页

作者： Ziwen Zheng Guo Li Chong Cui Fang Wang Xiaohan Wang Zhijiao Xu Huiping Guo Yuxin Chen Honghai Tang Daqi Wang Mingqian Huang Zheng-Yi Chen Xingxu Huang Huawei Li Geng-Lin Li Xiaoxiang Hu Yilai Shu ENT Institute and Department of Otorhinolaryngology Eye&ENT HospitalState Key Laboratory of Medical Neurobiology and MOE Frontiers Center for Brain ScienceFudan UniversityShanghai 200031China Institutes of Biomedical Sciences Fudan UniversityShanghai 200032China NHC Key Laboratory of Hearing Medicine(Fudan University) Shanghai 200032China State Key Laboratory of Agrobiotechnology China Agricultural UniversityBeijing 100193China College of Biological Sciences China Agricultural UniversityBeijing 100193China School of Life Science and Technology Southeast UniversityNanjing 210096China Department of Otolaryngology-Head and Neck Surgery Graduate Program in Speech and Hearing Bioscience and Technology and Program in NeuroscienceHarvard Medical SchoolBostonMA 02115USA Eaton-Peabody Laboratory Massachusetts Eye and Ear InfirmaryBostonMA 02114USA School of Life Science and Technology ShanghaiTech UniversityShanghai 200031China CAS Center for Excellence in Molecular Cell Science Shanghai Institute of Biochemistry and Cell BiologyChinese Academy of SciencesUniversity of Chinese Academy of SciencesShanghai 200031China Institutes of Brain Science and the Collaborative Innovation Center for Brain Science Fudan UniversityShanghai 200031China

CRISPR/RfxCas13d(CasRx)editing system can specifically and precisely cleave single-strand RNAs,which is a promising treatment for various disorders by downregulation of related gene ***,we tested this RNA-editing approach on Beethoven(Bth)mice,an animal model for human DFNA36 due to a point mutation in *** first screened 30 sgRNAs in cell cultures and found that CasRx with sgRNA3 reduced the Tmc1^(Bth)transcript by 90.8%,and the Tmc1 wild type transcript(Tmc1^(+))by 44.3%.We then injected a newly developed AAV vector(***)based CasRx into the inner ears of neonatal Bth mice,and we found that Tmc1^(Bth)was reduced by 70.2%in 2 weeks with few off-target effects in the whole ***,we found improved hair cell survival,rescued hair bundle degeneration,and reduced mechanoelectrical transduction ***,the hearing performance,measured in both ABR and DPOAE thresholds,was improved significantly in all ages over 8 ***,therefore,have validated the CRISPR/CasRx-based RNA editing strategy in treating autosomal-dominant hearing loss,paving way for its further application in many other hereditary diseases in hearing and beyond.

关键词： CRISPR/Cas editing hearing

来源：评论

学校读者我要写书评

暂无评论

Differential transverse motion of individual outer hair cells measured in gerbil high-frequency region

引用

AIP Conference Proceedings 2024年第1期3062卷

作者： Sunil Puria Nam Hyun Cho John Guinan Eaton Peabody Laboratories Massachusetts Eye and Ear Boston USA Harvard Medical School Boston MA USA Speech and Hearing & Bioscience and Technology Graduate Program at Harvard University Boston MA USA

The great sensitivity and frequency selectivity of mammalian hearing originates in the mechanical properties of the cochlea. Cochlear motions in response to sound are amplified using metabolic energy. The motor element of this cochlear amplification is presumed to be the outer hair cell (OHC), that expands and contracts lengthwise in response to a change in receptor potential at audio frequencies. How thousands of OHCs, acting through the local cytoarchitecture, work to achieve cochlear amplification is not fully understood. The properties of individual OHCs measured in micro-chamber environments are low-pass with a corner frequency of, at most, a few kHz. OHC corner frequencies within the cochlea have been estimated to be ∼3 kHz. Since in-vivo cochlear motion measurements show amplified frequency responses that extend up to many tens of kHz, but OHCs have corner frequencies of a few kHz, it has been suggested that OHCs are not active at the highest frequencies. How to reconcile these measurements has been a challenge. To address this, we made transverse cochlear motion measurements using a high-resolution optical-coherence-tomography (OCT) system at approximately the two ends of the OHC axial length: at the OHC bottom near the OHC-Deiter-Cell junction and at the OHC top near the reticular lamina. From these in-vivo measurements near the 42 kHz best frequency (BF) region of the gerbil cochlea, we determined the transverse differential response of OHCs. To remove phase from the traveling wave, we calculated the ratio of the RL motion to the motion at the OHC-Deiter-Cell junction. The OHC top-to-bottom phase difference was almost exactly ½-cycle at frequencies up to 35-40 kHz. As frequency increased above 40 kHz, the phase difference decreased until near BF there was very little phase difference. The OHC top-to-bottom amplitude ratio transitioned from the OHC top moving less than the bottom at low frequencies, to the OHC top moving more than the bottom, at about 30 kH

关键词：

来源：评论

学校读者我要写书评

暂无评论

JOSA: Joint surface-based registration and atlas construction of brain geometry and function

arXiv

引用

arXiv 2023年

作者： Li, Jian Tuckute, Greta Fedorenko, Evelina Edlow, Brian L. Dalca, Adrian V. Fischl, Bruce A. A. Martinos Center for Biomedical Imaging MGH & HMS United States Center for Neurotechnology and Neurorecovery MGH & HMS United States Department of Brain and Cognitive Sciences MIT United States McGovern Institute for Brain Research MIT United States Program in Speech Hearing Bioscience and Technology Harvard University United States Computer Science and Artificial Intelligence Laboratory MIT United States

Surface-based cortical registration is an important topic in medical image analysis and facilitates many downstream applications. Current approaches for cortical registration are mainly driven by geometric features, such as sulcal depth and curvature, and often assume that registration of folding patterns leads to alignment of brain function. However, functional variability of anatomically corresponding areas across subjects has been widely reported, particularly in higher-order cognitive areas. In this work, we present JOSA, a novel cortical registration framework that jointly models the mismatch between geometry and function while simultaneously learning an unbiased population-specific atlas. Using a semi-supervised training strategy, JOSA achieves superior registration performance in both geometry and function to the state-of-the-art methods but without requiring functional data at inference. This learning framework can be extended to any auxiliary data to guide spherical registration that is available during training but is difficult or impossible to obtain during inference, such as parcellations, architectonic identity, transcriptomic information, and molecular profiles. By recognizing the mismatch between geometry and function, JOSA provides new insights into the future development of registration methods using joint analysis of the brain structure and function. © 2023, CC BY.

关键词： Geometry

来源：评论

学校读者我要写书评

暂无评论

Effects of Recording Condition and Number of Monitored Days on Discriminative Power of the Daily Phonotrauma Index

arXiv

引用

arXiv 2024年

作者： Ghasemzadeh, Hamzeh Hillman, Robert E. Van Stan, Jarrad H. Mehta, Daryush D. Center for Laryngeal Surgery and Voice Rehabilitation Massachusetts General Hospital BostonMA United States Department of Surgery Harvard Medical School BostonMA United States Department of Communicative Sciences and Disorders Michigan State University East LansingMI United States Speech and Hearing Bioscience and Technology Division of Medical Sciences Harvard Medical School BostonMA United States MGH Institute of Health Professions BostonMA United States

Objective: The Daily Phonotrauma Index (DPI) can quantify pathophysiological mechanisms associated with daily voice use in individuals with phonotraumatic vocal hyperfunction (PVH). Since DPI was developed based on weeklong ambulatory voice monitoring, this study investigated if DPI can achieve comparable performance using (1) short laboratory speech tasks and (2) fewer than seven days of ambulatory data. Method: An ambulatory voice monitoring system recorded the vocal function/behavior of 134 females with PVH and vocally healthy matched controls in two different conditions. In the lab, the participants read the first paragraph of the Rainbow Passage and produced spontaneous speech (in-lab data). They were then monitored for seven days (in-field data). Separate DPI models were trained from in-lab and in-field data using the standard deviation of the difference between the magnitude of the first two harmonics (H1-H2) and the skewness of neck-surface acceleration magnitude. First, 10-fold cross-validation evaluated classification performance of in-lab and in-field DPIs. Second, the effect of the number of ambulatory monitoring days on the accuracy of in-field DPI classification was quantified. Results: The average in-lab DPI accuracy computed from the Rainbow passage and spontaneous speech were, respectively, 57.9% and 48.9%, which are close to chance performance. The average classification accuracy of infield DPI was significantly higher with a very large effect size (73.4%, Cohen’s D = 1.8). Second, the average in-field DPI accuracy increased from 66.5% for one day to 75.0% for seven days, with the gain of including an additional day on accuracy dropping below 1 percentage point after 4 days. Conclusion: The DPI requires ambulatory monitoring data as its discriminative power diminished significantly once computed from short in-lab recordings. Additionally, ambulatory monitoring should sample multiple days to achieve robust performance. The result of this paper can b

关键词： speech recognition

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：