DL techniques have increased the efficiency of decision making in different areas. However, in the case of the presence of uncertainties in the data or in the environment, decision-making requires the explainability o...
详细信息
Non-convex models, like deep neural networks, have been widely used in machine learning applications. Training non-convex models is a difficult task owing to the saddle points of models. Recently,stochastic normalized...
详细信息
Non-convex models, like deep neural networks, have been widely used in machine learning applications. Training non-convex models is a difficult task owing to the saddle points of models. Recently,stochastic normalized gradient descent(SNGD), which updates the model parameter by a normalized gradient in each iteration, has attracted much attention. Existing results show that SNGD can achieve better performance on escaping saddle points than classical training methods like stochastic gradient descent(SGD).However, none of the existing studies has provided theoretical proof about the convergence of SNGD for non-convex problems. In this paper, we firstly prove the convergence of SNGD for non-convex ***, we prove that SNGD can achieve the same computation complexity as SGD. In addition, based on our convergence proof of SNGD, we find that SNGD needs to adopt a small constant learning rate for convergence guarantee. This makes SNGD do not perform well on training large non-convex models in practice. Hence, we propose a new method, called stagewise SNGD(S-SNGD), to improve the performance of SNGD. Different from SNGD in which a small constant learning rate is necessary for convergence guarantee,S-SNGD can adopt a large initial learning rate and reduce the learning rate by stage. The convergence of S-SNGD can also be theoretically proved for non-convex problems. Empirical results on deep neural networks show that S-SNGD achieves better performance than SNGD in terms of both training loss and test accuracy.
Electrocardiogram (ECG) is an important non-invasive technique for diagnosing cardiovascular diseases (CVD). After acquiring the patients' raw ECG signal data, signal processing is essential for the diagnosis. Con...
详细信息
In global data analysis, the central server needs the global statistic of the user data stored in local clients. In such cases, an Honest-but-Curious central server might put user privacy at risk in trying to collect ...
详细信息
This paper presents a comprehensive framework for activity recognition and anomaly detection in smart home environments, targeting applications in convenience, efficiency, responsiveness, and healthcare. The proposed ...
详细信息
Online music platforms that include social networking features sometimes become supportive social communities where young people can disclose their emotional distress and receive support. However, few studies have exa...
详细信息
Recently,OpenAI released Chat Generative Pre-trained Transformer(ChatGPT)(Schulman et al.,2022)(https://***),which has attracted considerable attention from the industry and academia because of its impressive *** is t...
详细信息
Recently,OpenAI released Chat Generative Pre-trained Transformer(ChatGPT)(Schulman et al.,2022)(https://***),which has attracted considerable attention from the industry and academia because of its impressive *** is the first time that such a variety of open tasks can be well solved within one large language *** better understand ChatGPT,we briefly introduce its history,discuss its advantages and disadvantages,and point out several potential ***,we analyze its impact on the development of trustworthy artificial intelligence,conversational search engine,and artificial general intelligence.
As Large Language Models (LLMs) become more integrated into our daily lives, it is crucial to identify and mitigate their risks, especially when the risks can have profound impacts on human users and societies. Guardr...
详细信息
As Large Language Models (LLMs) become more integrated into our daily lives, it is crucial to identify and mitigate their risks, especially when the risks can have profound impacts on human users and societies. Guardrails, which filter the inputs or outputs of LLMs, have emerged as a core safeguarding technology. This position paper takes a deep look at current open-source solutions (Llama Guard, Nvidia NeMo, Guardrails AI), and discusses the challenges and the road towards building more complete solutions. Drawing on robust evidence from previous research, we advocate for a systematic approach to construct guardrails for LLMs, based on comprehensive consideration of diverse contexts across various LLMs applications. We propose employing sociotechnical methods through collaboration with a multi-disciplinary team to pinpoint precise technical requirements, exploring advanced neural-symbolic implementations to embrace the complexity of the requirements, and developing verification and testing to ensure the utmost quality of the final product. Copyright 2024 by the author(s)
A new spotted hyena intelligent optimizer (ISHO) algorithm incorporating multi-strategy improvement was proposed for the characteristics of the capacitated vehicle routing problem (CVRP).A combination of K-means clust...
详细信息
With the burgeon deployment of the fifth-generation new radio (5 G NR) networks, the codebook plays a crucial role in enabling the base station (BS) to acquire the channel state information (CSI). Different 5 G NR cod...
详细信息
暂无评论