检索结果-内蒙古大学图书馆

Bigger is not always better: The importance of human-scale language modeling for psycholinguistics

Journal of Memory and Language 2025年 144卷

作者： Ethan Gotlieb Wilcox Michael Y. Hu Aaron Mueller Alex Warstadt Leshem Choshen Chengxu Zhuang Adina Williams Ryan Cotterell Tal Linzen Department of Linguistics Georgetown University 1421 37th St NW Washington DC 20007 USA Department of Linguistics and Center for Data Science New York University 10 Washington Place NewYork NY 10003 USA Khoury College of Computer Sciences Northeastern University 440 Huntington Avenue Boston MA 02115 USA Department of Computer Science ETH Zürich Universitätstrasse 6 8092 Zürich Switzerland Electrical Engineering and Computer Science Department Massachusetts Institute of Technology 77 Massachusetts Avenue Cambridge MA 02139 USA FAIR Laboratories Meta Platforms Inc. 390 9th Ave NewYork NY 10001 USA

When trained to place high probability on a training corpus, neural network language models can learn a surprising amount about language. Recent work has demonstrated that large performance improvements can arise from simply increasing, i.e., scaling, the size of the corpora they are trained on and the number of parameters in those models. Accordingly, many contemporary systems are trained on trillions of words. While largely beneficial to performance on language applications, scaling has several downsides for both computational psycholinguistics and natural language processing research. We discuss the scientific challenges presented by the scaling paradigm, as well as the benefits that would result from language models that can learn from human-scale data. In the second half of this paper, we report on findings from a recent effort to bring about human-scale language model pretraining: the first iteration of the BabyLM Challenge, a shared task organized by the authors that invited participants to train a language model on 100 million words or less. The challenge produced several concrete best practices for practitioners interested in small-scale language modeling. For cognitive scientists, the challenge demonstrated that robust linguistic generalizations can be learned by models trained on a human-scale dataset, though this is not yet achieved through cognitively plausible mechanisms. Furthermore, it established a population of “BabyLMs” that are all effective at data-efficient language learning. Studying such models can help us identify hypotheses for the computational mechanisms that underlie human language acquisition.

关键词： Language modeling Scaling Cognitive modeling Connectionist networks Psycholinguistics Language acquisition

来源：评论

学校读者我要写书评

暂无评论

Validating daily social media macroscopes of emotions

arXiv

引用

arXiv 2021年

作者： Pellert, Max Metzler, Hannah Matzenberger, Michael Garcia, David Sony Computer Science Laboratories 6 Rue Amyot Paris75005 France Institute of Interactive Systems and Data Science Graz University of Technology Inffeldgasse 16C Graz8010 Austria Complexity Science Hub Vienna Josefstädter Straße 39 Vienna1080 Austria Section for Science of Complex Systems Center for Medical Statistics Informatics and Intelligent Systems Medical University of Vienna Spitalgasse 23 Vienna1090 Austria Institute of Globally Distributed Open Research and Education Sweden Der Standard Vordere Zollamtsstraße 13 Vienna1030 Austria

Measuring sentiment in social media text has become an important practice in studying emotions at the macroscopic level. However, this approach can suffer from methodological issues like sampling biases and measurement errors. To date, it has not been validated if social media sentiment can actually measure the temporal dynamics of mood and emotions aggregated at the level of communities. We ran a large-scale survey at an online newspaper to gather daily mood self-reports from its users, and compare these with aggregated results of sentiment analysis of user discussions. We find strong correlations between text analysis results and levels of self-reported feelings, as well as between inter-day changes of both measurements. We replicate these results using sentiment data from Twitter. We show that a combination of supervised text analysis methods based on novel deep learning architectures and unsupervised dictionary-based methods have high agreement with the time series of aggregated mood measured with self-reports. Our findings indicate that macro level dynamics of feelings expressed on an online platform can be tracked with social media text, especially in situations of high mood variability. © 2021, CC BY.

关键词： Sentiment analysis

来源：评论

学校读者我要写书评

暂无评论

Modeling, Capacity Studies, Antenna and System Designs for 6G/B6G 3D Continuous-Space Radio Channels Enabled by Electromagnetic Information Theory

引用

IEEE Communications Surveys and Tutorials 2025年

作者： Wang, Cheng-Xiang Li, Junling Huang, Jie Huang, Chen Zhang, Zhaoyang Liu, Ying Zhou, Shidong Chen, Yunfei You, Xiaohu Gao, Xiqi Cui, Tie Jun Renzo, Marco Di Thompson, John Haas, Harald Schober, Robert Zhang, Rui Southeast University National Mobile Communications Research Laboratory School of Information Science and Engineering Nanjing211189 China Purple Mountain Laboratories Nanjing211111 China Zhejiang University Zhejiang Provincial Key Laboratory of Information Processing Communication and Networking College of Information Science and Electronic Engineering Hangzhou310007 China Xidian University National Key Laboratory of Radar Detection and Sensing Xi'an710071 China Tsinghua University State Key Laboratory on Microwave and Digital Communications Beijing100084 China University of Durham Department of Engineering DurhamDH1 3LE United Kingdom Southeast University State Key Laboratory of Millimeter Waves School of Information Science and Engineering Nanjing211189 China UniversitèParis-Saclay Laboratoire des Signaux et Systèmes CNRS CentraleSupélec Gif-sur-Yvette91192 France Centre for Telecommunications Research - Department of Engineering King's College London LondonWC2R 2LS United Kingdom School of Engineering EdinburghEH9 3BF United Kingdom Cambridge University Department of Engineering CambridgeCB2 1PZ United Kingdom Institute for Digital Communications Erlangen91054 Germany The Chinese University of HongKong School of Science and Engineering Shenzhen Research Institute of Big Data Shenzhen518172 China National University of Singapore Department of Electrical and Computer Engineering 117583 Singapore

Channel theory is a fundamental theory of wireless communications. The sixth generation (6G) and beyond 6G (B6G) wireless communication networks are expected to provide space-air-ground-sea global coverage. Base stations and users tend to move in three dimensional (3D) continuous spaces, while antennas and propagation environments can be tightly coupled. The underlying channels show an evolutionary trend to 3D continuous-space radio channels that combine antennas and wireless propagation channels, in comparison to discrete local-space wireless propagation channels in previous generations. This introduces new challenges for channel modeling, channel capacity analysis, antenna design, system design, etc. To address these challenges, this paper performs a comprehensive study on 3D continuous-space radio channels in 6G/B6G with the aid of electromagnetic information theory (EIT) that integrates electromagnetic theory, information theory, wireless propagation channel modeling theory, and antenna theory. We start by revealing the connections and gaps between these four fundamental theories. Then, an in-depth investigation on the four major research thrusts of 3D continuous-space radio channels is provided: 1) channel measurements and modeling, 2) channel capacity analysis, 3) general antenna design, and 4) wireless system design. We aim to explore the intrinsic relationships between antenna parameters, channel parameters, channel characteristics, channel capacity, and communication system performance. Finally, future research directions and challenges for 3D continuous-space radio channels are outlined. Our study endeavors to establish a fundamental framework for 3D continuous-space radio channels, with the potential to catalyze breakthroughs in 6G/B6G theories. © 1998-2012 IEEE.

关键词： Antenna grounds

来源：评论

学校读者我要写书评

暂无评论

Perturbative quantum simulation

arXiv

引用

arXiv 2021年

作者： Sun, Jinzhao Endo, Suguru Lin, Huiping Hayden, Patrick Vedral, Vlatko Yuan, Xiao Center on Frontiers of Computing Studies Peking University Beijing100871 China Clarendon Laboratory University of Oxford Parks Road OxfordOX1 3PU United Kingdom Quantum Advantage Research Beijing100080 China NTT Computer & Data Science Laboratories NTT Corporation Musashino Tokyo180-8585 Japan School of Computer Science Peking University Beijing100871 China Stanford Institute for Theoretical Physics Stanford University StanfordCA94305 United States Centre for Quantum Technologies National University of Singapore Singapore117543 Singapore

Approximation based on perturbation theory is the foundation for most of the quantitative predictions of quantum mechanics, whether in quantum many-body physics, chemistry, quantum field theory or other domains. Quantum computing provides an alternative to the perturbation paradigm, yet state-of-the-art quantum processors with tens of noisy qubits are of limited practical utility. Here, we introduce perturbative quantum simulation, which combines the complementary strengths of the two approaches, enabling the solution of large practical quantum problems using limited noisy intermediate-scale quantum hardware. The use of a quantum processor eliminates the need to identify a solvable unperturbed Hamiltonian, while the introduction of perturbative coupling permits the quantum processor to simulate systems larger than the available number of physical qubits. We present an explicit perturbative expansion that mimics the Dyson series expansion and involves only local unitary operations, and show its optimality over other expansions under certain conditions. We numerically benchmark the method for interacting bosons, fermions, and quantum spins in different topologies, and study different physical phenomena, such as information propagation, charge-spin separation, and magnetism, on systems of up to 48 qubits only using an 8 + 1 qubit quantum hardware. We experimentally demonstrate our scheme on the IBM quantum cloud, verifying its noise robustness and illustrating its potential for benchmarking large quantum processors with smaller ones. Copyright © 2021, The Authors. All rights reserved.

关键词： Hamiltonians

来源：评论

学校读者我要写书评

暂无评论

Dual-GSE: Resource-efficient Generalized Quantum Subspace Expansion

arXiv

引用

arXiv 2023年

作者： Yang, Bo Yoshioka, Nobuyuki Harada, Hiroyuki Hakkaku, Shigeo Tokunaga, Yuuki Hakoshima, Hideaki Yamamoto, Kaoru Endo, Suguru LIP6 Sorbonne Université 4 Place Jussieu Paris75005 France Graduate School of Information Science and Technology The University of Tokyo 7-3-1 Hongo Bunkyo-ku Tokyo113-8656 Japan Department of Applied Physics School of Engineering University of Tokyo 7-3-1 Hongo Bunkyo-ku Tokyo113-8656 Japan Wako-shi Saitama351-0198 Japan JST PRESTO 4-1-8 Honcho Kawaguchi Saitama332-0012 Japan Department of Applied Physics and Physico-Informatics Keio University Hiyoshi 3-14-1 Kohoku Yokohama223-8522 Japan NTT Computer and Data Science Laboratories NTT Corporation 3-9-11 Midori-cho Musashino-shi Tokyo180-8585 Japan Center for Quantum Information and Quantum Biology Osaka University 1-3 Machikaneyama Toyonaka Osaka560-8531 Japan

There are considerable obstacles against realizing practical quantum computing: a significant amount of computation errors and the restricted qubit count. As a unified method of noise-agnostic quantum error mitigation (QEM) methods, i.e., the quantum subspace expansion and virtual purification, a generalized quantum subspace expansion (GSE) has recently been proposed that is significantly robust against stochastic and coherent errors. However, GSE requires entangled measurements between copies of the quantum states, which is a significant drawback under the current situation of the restricted number of qubits and their connectivity. In this work, we propose a resource-efficient implementation of GSE, which we name "Dual-GSE", circumventing significant overheads of state copies by constructing an ansatz of error-mitigated quantum states via dual-state purification. Remarkably, Dual-GSE can further simulate larger quantum systems beyond the size of available quantum hardware with a suitable ansatz construction inspired by divide-and-conquer methods that forge entanglement classically. This also significantly reduces the measurement overhead because we only need to measure subsystems’ Pauli operators. The proposed method is demonstrated by a numerical simulation of the eight-qubit transverse-field Ising model, showing that our method estimates the ground state energy with high precision under gate noise with low mitigation overhead and practical sampling cost. Copyright © 2023, The Authors. All rights reserved.

关键词： Ground state

来源：评论

学校读者我要写书评

暂无评论

Information-Geometric Set Embeddings (IGSE): From Sets to Probability Distributions

arXiv

引用

arXiv 2019年

作者： Sun, Ke Nielsen, Frank CSIRO Data61 Sydney Australia Sony Computer Science Laboratories Inc. Tokyo Japan

This letter introduces an abstract learning problem called the "set embedding": The objective is to map sets into probability distributions so as to lose less information. We relate set union and intersection operations with corresponding interpolations of probability distributions. We also demonstrate a preliminary solution with experimental results on toy set embedding examples. Copyright © 2019, The Authors. All rights reserved.

关键词： Probability distributions

来源：评论

学校读者我要写书评

暂无评论

Quantum error mitigation for rotation-symmetric bosonic codes with symmetry expansion

引用

Physical Review A 2025年第6期111卷 062402-062402页

作者： Suguru Endo Yasunari Suzuki Kento Tsubouchi Rui Asaoka Kaoru Yamamoto Yuichiro Matsuzaki Yuuki Tokunaga NTT Computer and Data Science Laboratories NTT Corporation Musashino 180-8585 Tokyo Japan Department of Applied Physics University of Tokyo 7-3-1 Hongo Bunkyo-ku Tokyo 113-8656 Japan Research Center for Emerging Computing Technologies National Institute of Advanced Industrial Science and Technology (AIST) Central2 1-1-1 Umezono Tsukuba Ibaraki 305-8568 Japan NEC-AIST Quantum Technology Cooperative Research Laboratory National Institute of Advanced Industrial Science and Technology (AIST) Tsukuba Ibaraki 305-8568 Japan

The rotation symmetric bosonic code (RSBC) is a unified framework of practical bosonic codes that have rotation symmetries, such as cat codes and binomial codes. While cat and binomial codes achieve the breakeven point in which the coherence time of the encoded qubits exceeds that of unencoded qubits, the state preparation fidelity needs to be improved for practical quantum computing. Concerning this problem, we investigate the framework of symmetry expansion, a class of quantum error mitigation that virtually projects the state onto the noise-free symmetric subspace by exploiting the system's intrinsic symmetries and postprocessing of measurement outcomes. Although symmetry expansion has been limited to error mitigation of quantum states immediately before measurement, we successfully generalize symmetry expansion for state preparation. Then, we consider two types of stabilization: photon number and phase stabilization. Photon number stabilization can be performed by leveraging the rotation operators. On the other hand, the phase errors can be suppressed via the extension towards phase direction by leveraging the recently proposed projective squeezing method. To implement our method, we use an ancilla qubit and a randomly generated controlled gate between the bosonic code states and the ancilla qubit. Our novel error mitigation method will significantly enhance computation accuracy in the near-term bosonic quantum computing.

关键词：

来源：评论

学校读者我要写书评

暂无评论

Applying the FAIR Principles to computational workflows

arXiv

引用

arXiv 2024年

作者： Wilkinson, Sean R. Aloqalaa, Meznah Belhajjame, Khalid Crusoe, Michael R. de Paula Kinoshita, Bruno Gadelha, Luiz Garijo, Daniel Ragnar Gustafsson, Ove Johan Juty, Nick Kanwal, Sehrish Khan, Farah Zaib Köster, Johannes Gehlen, Karsten Petersvon Pouchard, Line Rannow, Randy K. Soiland-Reyes, Stian Soranzo, Nicola Sufi, Shoaib Sun, Ziheng Vilne, Baiba Wouters, Merridee A. Yuen, Denis Goble, Carole Oak Ridge Leadership Computing Facility Oak Ridge National Laboratory Oak RidgeTN United States Department of Computer Science University of Manchester Manchester United Kingdom LAMSADE PSL Paris Dauphine University Paris France Berlin Germany Earth Sciences Barcelona Supercomputing Center Barcelona Spain Heidelberg Germany Ontology Engineering Group Universidad Politécnica de Madrid Madrid Spain Australian BioCommons University of Melbourne MelbourneVIC Australia ParkvilleVIC Australia University Medicine Essen University of Duisburg-Essen Essen Germany Data Management Department Deutsches Klimarechenzentrum GmbH Hamburg Germany Center for Computing Research Sandia National Laboratories AlbuquerqueNM United States Silverdraft Supercomputing BoiseID United States Informatics Institute University of Amsterdam Amsterdam Netherlands Earlham Institute Norwich United Kingdom Center for Spatial Information Science and Systems Department of Geography and Geoinformation Science George Mason University FairfaxVA United States Bioinformatics Group Riga Stradins University Riga Latvia School of Clinical Medicine University of New South Wales KensingtonNSW Australia Ontario Institute for Cancer Research TorontoON Canada

Recent trends within computational and data sciences show an increasing recognition and adoption of computational workflows as tools for productivity and reproducibility that also democratize access to platforms and processing know-how. As digital objects to be shared, discovered, and reused, computational workflows benefit from the FAIR principles, which stand for Findable, Accessible, Interoperable, and Reusable. The Workflows Community Initiative’s FAIR Workflows Working Group (WCI-FW), a global and open community of researchers and developers working with computational workflows across disciplines and domains, has systematically addressed the application of both FAIR data and software principles to computational workflows. We present recommendations with commentary that reflects our discussions and justifies our choices and adaptations. These are offered to workflow users and authors, workflow management system developers, and providers of workflow services as guidelines for adoption and fodder for discussion. The FAIR recommendations for workflows that we propose in this paper will maximize their value as research assets and facilitate their adoption by the wider community. © 2024, CC BY.

关键词： Application programs

来源：评论

学校读者我要写书评

暂无评论

Hunting for quantum-classical crossover in condensed matter problems

arXiv

引用

arXiv 2022年

作者： Yoshioka, Nobuyuki Okubo, Tsuyoshi Suzuki, Yasunari Koizumi, Yuki Mizukami, Wataru Department of Applied Physics University of Tokyo 7-3-1 Hongo Bunkyo-ku Tokyo113-8656 Japan Saitama Wako-shi351-0198 Japan JST PRESTO 4-1-8 Honcho Saitama Kawaguchi332-0012 Japan Institute for Physics of Intelligence University of Tokyo 7-3-1 Hongo Bunkyo-ku Tokyo113-0033 Japan NTT Computer and Data Science Laboratories Musashino180-8585 Japan Center for Quantum Information and Quantum Biology Osaka University 1-2 Machikaneyama Osaka Toyonaka560-0043 Japan Graduate School of Engineering Science Osaka University 1-3 Machikaneyama Osaka Toyonaka560-8531 Japan

The intensive pursuit for quantum advantage in terms of computational complexity has further led to a modernized crucial question: When and how will quantum computers outperform classical computers? The next milestone is undoubtedly the realization of quantum acceleration in practical problems. Here we provide a clear evidence and arguments that the primary target is likely to be condensed matter physics. Our primary contributions are summarized as follows: 1) Proposal of systematic error/runtime analysis on state-of-the-art classical algorithm based on tensor networks;2) Dedicated and high-resolution analysis on quantum resource performed at the level of executable logical instructions;3) Clarification of quantum-classical crosspoint for ground-state simulation to be within runtime of hours using only a few hundreds of thousand physical qubits for 2d Heisenberg and 2d Fermi-Hubbard models, assuming that logical qubits are encoded via the surface code with the physical error rate of p = 10−3. To our knowledge, we argue that condensed matter problems offer the earliest platform for demonstration of practical quantum advantage that is order-of-magnitude more feasible than ever known candidates, in terms of both qubit counts and total runtime. Copyright © 2022, The Authors. All rights reserved.

关键词： Condensed matter physics

来源：评论

学校读者我要写书评

暂无评论

Error-Mitigated Quantum Metrology via Virtual Purification

arXiv

引用

arXiv 2021年

作者： Yamamoto, Kaoru Endo, Suguru Hakoshima, Hideaki Matsuzaki, Yuichiro Tokunaga, Yuuki NTT Computer and Data Science Laboratories NTT Corporation Musashino180-8585 Japan JST PRESTO 4-1-8 Honcho Kawaguchi Saitama332-0012 Japan Central2 1-1-1 Umezono Ibaraki Tsukuba305-8568 Japan Center for Quantum Information and Quantum Biology Osaka University 1-2 Machikaneyama Toyonaka560-0043 Japan Ibaraki Tsukuba305-8568 Japan

Quantum metrology with entangled resources aims to achieve sensitivity beyond the standard quantum limit by harnessing quantum effects even in the presence of environmental noise. So far, sensitivity has been mainly discussed from the viewpoint of reducing statistical errors under the assumption of perfect knowledge of a noise model. However, we cannot always obtain complete information about a noise model due to coherence time fluctuations, which are frequently observed in experiments. Such unknown fluctuating noise leads to systematic errors and nullifies the quantum advantages. Here, we propose an error-mitigated quantum metrology that can filter out unknown fluctuating noise with the aid of purification-based quantum error mitigation. We demonstrate that our protocol mitigates systematic errors and recovers superclassical scaling in a practical situation with time-inhomogeneous bias-inducing noise. Our result is the first demonstration to reveal the usefulness of purification-based error mitigation for unknown fluctuating noise, thus paving the way not only for practical quantum metrology but also for quantum computation affected by such noise. Copyright © 2021, The Authors. All rights reserved.

关键词： Purification

来源：评论

学校读者我要写书评

暂无评论

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案：

请选择收藏分类：

通借通还

建议与咨询 留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

时间限定

文献类型

馆藏选择

核心期刊

语言

文献类型

帮助

文字说明：

检索规则说明：

检索范例：

分类表

所选分类

限定检索结果

文献类型

馆藏范围

日期分布

学科分类号

主题

机构

作者

语言

请选择保存的检索档案： 新增检索档案 确定 取消

请选择收藏分类： 新增自定义分类 确定 取消

通借通还

建议与咨询留下您的常用邮箱和电话号码，以便我们向您反馈解决方案和替代方法

请选择保存的检索档案：

请选择收藏分类：