Recently, many large language models (LLMs) have been proposed, showing advanced proficiency in code generation. Meanwhile, many efforts have been dedicated to evaluating LLMs on code generation benchmarks such as Hum...
详细信息
ISBN:
(纸本)9798400702174
Recently, many large language models (LLMs) have been proposed, showing advanced proficiency in code generation. Meanwhile, many efforts have been dedicated to evaluating LLMs on code generation benchmarks such as HumanEval. Although being very helpful for comparing different LLMs, existing evaluation focuses on a sim-ple code generation scenario (i.e., function-level or statement-level code generation), which mainly asks LLMs to generate one single code unit (e.g., a function or a statement) for the given natural language description. Such evaluation focuses on generating independent and often small-scale code units, thus leaving it unclear how LLMs perform in real-world software development scenarios. To fill this knowledge gap, we make the first attempt to evaluate LLMs in a more challenging code generation scenario, i.e., class-level code generation. Compared with existing code generation benchmarks, it better reflects real-world software development scenarios due to it comprising broader contextual dependencies and multiple, interdependent units of code. We first manually construct the first class-level code generation benchmark ClassEval of 100 class-level Python code generation tasks with approximately 500 person-hours. Based on the new benchmark ClassEval, we then perform the first study of 11 state-of-the-art LLMs on class-level code generation. Based on our results, we find that all LLMs perform much worse on class-level code generation compared to the method-level. While GPT models still dominate other LLMs on class-level code generation, the performance rankings of other models on method-level code generation no longer holds for class-level code generation. Besides, most models (except GPT models) perform better when generating the class method by method;and they have the limited ability of generating dependent code. Based on our findings, we call for software engineering (SE) researchers' expertise to build more LLM benchmarks based on practical and com
Area has become one of the main bottlenecks restricting the development of integrated circuits. The area optimization approaches of existing XNOR/OR-based mixed polarity Reed-Muller(MPRM) circuits have poor optimizati...
详细信息
Area has become one of the main bottlenecks restricting the development of integrated circuits. The area optimization approaches of existing XNOR/OR-based mixed polarity Reed-Muller(MPRM) circuits have poor optimization effect and efficiency. Given that the area optimization of MPRM logic circuits is a combinatorial optimization problem, we propose a whole annealing adaptive bacterial foraging algorithm(WAA-BFA), which includes individual evolution based on Markov chain and Metropolis acceptance criteria, and individual mutation based on adaptive probability. To address the issue of low conversion efficiency in existing polarity conversion approaches, we introduce a fast polarity conversion algorithm(FPCA). Moreover, we present an MPRM circuits area optimization approach that uses the FPCA and WAA-BFA to search for the best polarity corresponding to the minimum circuits area. Experimental results demonstrate that the proposed MPRM circuits area optimization approach is effective and can be used as a promising EDA tool.
Internet of Things (IoT) applications have recently been widely used in safety-critical scenarios. To prevent sensitive information leaks, IoT device vendors provide hardware-assisted protections, called Trusted Execu...
详细信息
Rtecently a lot of works have been investigating to find the tenuous groups,i.e.,groups with few social interactions and weak relationships among members,for reviewer selection and psycho-educational group ***,the met...
详细信息
Rtecently a lot of works have been investigating to find the tenuous groups,i.e.,groups with few social interactions and weak relationships among members,for reviewer selection and psycho-educational group ***,the metrics(e.g.,k-triangle,k-line,and k-tenuity)used to measure the tenuity,require a suitable k value to be specified which is difficult for users without background ***,in this paper we formulate the most tenuous group(MTG)query in terms of the group distance and average group distance of a group measuring the tenuity to eliminate the influence of parameter k on the tenuity of the *** address the MTG problem,we first propose an exact algorithm,namely MTGVDIS,which takes priority to selecting those vertices whose vertex distance is large,to generate the result group,and also utilizes effective filtering and pruning *** MTGVDIS is not fast enough,we design an efficient exact algorithm,called MTG-VDGE,which exploits the degree metric to sort the vertexes and proposes a new combination order,namely degree and reverse based branch and bound(DRBB).MTG-VDGE gives priority to those vertices with small *** a large p,we further develop an approximation algorithm,namely MTG-VDLT,which discards candidate attendees with high degree to reduce the number of vertices to be *** experimental results on real datasets manifest that the proposed algorithms outperform existing approaches on both efficiency and group tenuity.
Recent advancements in learning-based Multi-View Stereo (MVS) methods have prominently featured transformer-based models with attention mechanisms. However, existing approaches have not thoroughly investigated the pro...
详细信息
Online job advertisements on various job portals or websites have become the most popular way for people to find potential career opportunities ***,the majority of these job sites are limited to offering fundamental f...
详细信息
Online job advertisements on various job portals or websites have become the most popular way for people to find potential career opportunities ***,the majority of these job sites are limited to offering fundamental filters such as job titles,keywords,and compensation *** often poses a challenge for job seekers in efficiently identifying relevant job advertisements that align with their unique skill sets amidst a vast sea of ***,we propose well-coordinated visualizations to provide job seekers with three levels of details of job information:a skill-job overview visualizes skill sets,employment posts as well as relationships between them with a hierarchical visualization design;a post exploration view leverages an augmented radar-chart glyph to represent job posts and further facilitates users’swift comprehension of the pertinent skills necessitated by respective positions;a post detail view lists the specifics of selected job posts for profound analysis and *** using a real-world recruitment advertisement dataset collected from 51Job,one of the largest job websites in China,we conducted two case studies and user interviews to evaluate *** results demonstrated the usefulness and effectiveness of our approach.
This article presents an in-depth exploration of the acoustofluidic capabilities of guided flexural waves(GFWs)generated by a membrane acoustic waveguide actuator(MAWA).By harnessing the potential of GFWs,cavity-agnos...
详细信息
This article presents an in-depth exploration of the acoustofluidic capabilities of guided flexural waves(GFWs)generated by a membrane acoustic waveguide actuator(MAWA).By harnessing the potential of GFWs,cavity-agnostic advanced particle manipulation functions are achieved,unlocking new avenues for microfluidic systems and lab-on-a-chip *** localized acoustofluidic effects of GFWs arising from the evanescent nature of the acoustic fields they induce inside a liquid medium are numerically investigated to highlight their unique and promising *** traditional acoustofluidic technologies,the GFWs propagating on the MAWA’s membrane waveguide allow for cavity-agnostic particle manipulation,irrespective of the resonant properties of the fluidic ***,the acoustofluidic functions enabled by the device depend on the flexural mode populating the active region of the membrane *** demonstrations using two types of particles include in-sessile-droplet particle transport,mixing,and spatial separation based on particle diameter,along with streaming-induced counter-flow virtual channel generation in microfluidic PDMS *** experiments emphasize the versatility and potential applications of the MAWA as a microfluidic platform targeted at lab-on-a-chip development and showcase the MAWA’s compatibility with existing microfluidic systems.
With the emergence of the Metaverse concept, the rendering and transmission of 3D virtual scenes demand high-bandwidth, high-quality real-time rendering technology, as well as ultra-reliable low-latency communication ...
详细信息
Advancements in language models (LMs) have sparked interest in exploring their potential as knowledge bases (KBs) due to their high capability for storing huge amounts of factual knowledge and semantic understanding. ...
详细信息
暂无评论