CodeT: Code Generation with Generated Tests (Microsoft)
The work explores the use of pre-trained language models to automatically generate test cases. Method is titled CodeT: Code generation with generated Tests. CodeT executes the code solutions using the generated test cases, and then chooses the best solution based on a dual execution agreement with both the generated test cases and other generated solutions.
The work explores the use of pre-trained language models to automatically generate test cases. Method is titled CodeT: Code generation with generated Tests. CodeT executes the code solutions using the generated test cases, and then chooses the best solution based on a dual execution agreement with both the generated test cases and other generated solutions.
ESEC/FSE 2023
https://conf.researchr.org/home/fse-2023
Sat 11 - Fri 17 November 2023 San Francisco, California, United States
Thu 26 Jan 2023 Research Papers Paper registration
Thu 2 Feb 2023 Research Papers Full paper submission
Thu 4 May 2023 Research Papers Initial notification
Thu 29 Jun 2023 Research Papers Revised manuscript submissions (major revisions only)
Thu 27 Jul 2023 Research Papers Final notification for major revisions
Thu 24 Aug 2023 Research Papers Camera ready
https://conf.researchr.org/home/fse-2023
Sat 11 - Fri 17 November 2023 San Francisco, California, United States
Thu 26 Jan 2023 Research Papers Paper registration
Thu 2 Feb 2023 Research Papers Full paper submission
Thu 4 May 2023 Research Papers Initial notification
Thu 29 Jun 2023 Research Papers Revised manuscript submissions (major revisions only)
Thu 27 Jul 2023 Research Papers Final notification for major revisions
Thu 24 Aug 2023 Research Papers Camera ready
conf.researchr.org
ESEC/FSE 2023
The ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) is an internationally renowned forum for researchers, practitioners, and educators to present and discuss the most recent innovations…
Amazon CodeWhisperer is a machine learning (ML)–powered service that helps improve developer productivity by generating code recommendations based on their comments in natural language and code in the integrated development environment (IDE).
Two surveys on Text-to-SQL: datasets, algorithms, metrics.
- Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey
- Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect
- Deep Learning Driven Natural Languages Text to SQL Query Conversion: A Survey
- Recent Advances in Text-to-SQL: A Survey of What We Have and What We Expect
So Much in So Little: Creating Lightweight Embeddings of Python Libraries (JetBrains, Huawei)
- python library embeddings
- a prototype tool for suggesting relevant libraries to a given project
- python library embeddings
- a prototype tool for suggesting relevant libraries to a given project
Hardin and Taylor proved that any function of the real numbers can be correctly predicted based solely on its past behavior at almost any point in time. However, these results do not provide a practical means of predicting the future. For example, the definition of the \mu-strategy requires the Axiom of Choice. One might wonder if a different approach could yield similar results without using the Axiom of Choice. In nontrivial cases, the answer is no.
A peculiar connection between the axiom of choice and predicting the future
A peculiar connection between the axiom of choice and predicting the future
Category Theory for AI
Program
Week 1: Why Category Theory?
Week 2: Essential building blocks: Categories and Functors
Week 3: Categorical Dataflow: Optics and Lenses as data structures for backpropagation
Week 4: Geometric Deep Learning & Naturality
Week 5: Monoids, Monads, Mappings, and lstMs
Program
Week 1: Why Category Theory?
Week 2: Essential building blocks: Categories and Functors
Week 3: Categorical Dataflow: Optics and Lenses as data structures for backpropagation
Week 4: Geometric Deep Learning & Naturality
Week 5: Monoids, Monads, Mappings, and lstMs
cats.for.ai
Categories for Machine Learning
This seminar series seeks to promote
the learning and use of Category Theory by Machine Learning Researchers
the learning and use of Category Theory by Machine Learning Researchers
No More Fine-Tuning? An Experimental Evaluation of Prompt Tuning in Code Intelligence
The authors compare fine-tuning and prompt-tuning for different tasks in code understanding: defect prediction, code summarization and code translation. They come to the conclusion that prompt-tuning is more effective than fine-tuning on the code intelligence tasks, with respect to different pre-trained models and different programming languages. Besides, the advantage of prompt tuning is more obvious for smaller pre-trained models. Also prompt tuning is more effective in low-resource scenarios than fine-tuning. The fewer training instances, the larger the improvement achieved by prompt tuning. And prompt-tuning also shows superior performance on the cross-domain code intelligence task.
The authors compare fine-tuning and prompt-tuning for different tasks in code understanding: defect prediction, code summarization and code translation. They come to the conclusion that prompt-tuning is more effective than fine-tuning on the code intelligence tasks, with respect to different pre-trained models and different programming languages. Besides, the advantage of prompt tuning is more obvious for smaller pre-trained models. Also prompt tuning is more effective in low-resource scenarios than fine-tuning. The fewer training instances, the larger the improvement achieved by prompt tuning. And prompt-tuning also shows superior performance on the cross-domain code intelligence task.
Stanford, CS 349M: Machine Learning for Software Engineering
In recent years, tools based on machine learning have become increasingly prevalent in the software engineering field. The ubiquity of machine learning is an important factor, but just as important is the availability of software engineering data: there are billions of lines of code available in public repositories (e.g. on GitHub), there is the change history of that code, there are discussion fora (e.g. Stack Overflow) that contain a wealth of information for developers, companies have access tontelemetry on their apps from millions of users, and so on. The scale of software engineering data has permitted machine learning and statistical approaches to imagine tools that are beyond the capabilities of traditional, semantics-based approaches. In this graduate seminar, students will learn the various ways in which code and related artifacts can be treated as data, and how various developer tools can be built by applying machine learning over this data. The course will consist of discussion of a selection of research papers, as well as a hands-on project that can be done in small groups.nnPrerequisites: Familiarity with basic machine learning, and either CS143 or CS295.
In recent years, tools based on machine learning have become increasingly prevalent in the software engineering field. The ubiquity of machine learning is an important factor, but just as important is the availability of software engineering data: there are billions of lines of code available in public repositories (e.g. on GitHub), there is the change history of that code, there are discussion fora (e.g. Stack Overflow) that contain a wealth of information for developers, companies have access tontelemetry on their apps from millions of users, and so on. The scale of software engineering data has permitted machine learning and statistical approaches to imagine tools that are beyond the capabilities of traditional, semantics-based approaches. In this graduate seminar, students will learn the various ways in which code and related artifacts can be treated as data, and how various developer tools can be built by applying machine learning over this data. The course will consist of discussion of a selection of research papers, as well as a hands-on project that can be done in small groups.nnPrerequisites: Familiarity with basic machine learning, and either CS143 or CS295.
BigCode project
Large Language Models (LLMs) are fast becoming an essential tool for all fields of AI research. One striking feature of these large pre-trained models is that they can be adapted to a wide variety of language tasks, often with very little in-domain data.
BigCode is focused on developing state-of-the-art LLMs for code. Code LLMs enable the completion and synthesis of code, both from other code snippets and natural language descriptions, and work across a wide range of domains, tasks, and programming languages. These models can, for example, assist professional and citizen developers with coding new applications.
BigCode invites AI researchers to collaborate on the following topics:
* A representative evaluation suite for code LLMs, covering a diverse set of tasks and programming languages
* Responsible data governance and development for code LLMs
* Faster training and inference methods for LLMs
Large Language Models (LLMs) are fast becoming an essential tool for all fields of AI research. One striking feature of these large pre-trained models is that they can be adapted to a wide variety of language tasks, often with very little in-domain data.
BigCode is focused on developing state-of-the-art LLMs for code. Code LLMs enable the completion and synthesis of code, both from other code snippets and natural language descriptions, and work across a wide range of domains, tasks, and programming languages. These models can, for example, assist professional and citizen developers with coding new applications.
BigCode invites AI researchers to collaborate on the following topics:
* A representative evaluation suite for code LLMs, covering a diverse set of tasks and programming languages
* Responsible data governance and development for code LLMs
* Faster training and inference methods for LLMs
BigCode
The Mission
BigCode is an open scientific collaboration working on the responsible development and use of large language models for code (Code LLMs), empowering the machine learning and open source communities through open governance.
One of the challenges typically…
One of the challenges typically…
Anomaly Detection in Time Series: A Comprehensive Evaluation
The authors collected and re-implemented 71 anomaly detection algorithms from different domains and evaluated them on 976 time series datasets. Paper: https://vldb.org/pvldb/vol15/p1779-wenig.pdf
The authors collected and re-implemented 71 anomaly detection algorithms from different domains and evaluated them on 976 time series datasets. Paper: https://vldb.org/pvldb/vol15/p1779-wenig.pdf
RING: Repair Is Nearly Generation: Multilingual Program Repair with LLMs (Microsoft)
An LLMC-based approach to multilingual repair powered by Codex. It enables a flipped model for AI-assisted programming, in which the user writes code and the assistant suggests fixes for last-mile mistakes. The approach is built on the intuition that fixing such mistakes can be conceptualized into three tasks: fault localization, code transformation, and candidate ranking.
An LLMC-based approach to multilingual repair powered by Codex. It enables a flipped model for AI-assisted programming, in which the user writes code and the assistant suggests fixes for last-mile mistakes. The approach is built on the intuition that fixing such mistakes can be conceptualized into three tasks: fault localization, code transformation, and candidate ranking.
Transformers in Time Series: A Survey
The application of the Transformer architecture looks natural in the case of time series. In a sense, time series is even a more native area for sec2sec approaches than CV or NLP. What peculiarities of time series should be taken into account when using Transformers? A lot of them. Let's list some.
- it is often necessary to be able to process very long sequences (periodicity, seasonality)
- sequences in time series often have timestamps labels that contain more information than just a position number in the sequence from NLP or CV. For example, days of the week, holidays. - sometimes sequential generation is not efficient and the decoder needs to be able to generate subsequences at one time
- time series pose special tasks: spatio-temporal forecasting, event forecasting, anomaly detection, etc.
The survey systematizes approaches in the field of application of Transformers for time series. In addition, the authors test different algorithms on the ETTm2 dataset.
The application of the Transformer architecture looks natural in the case of time series. In a sense, time series is even a more native area for sec2sec approaches than CV or NLP. What peculiarities of time series should be taken into account when using Transformers? A lot of them. Let's list some.
- it is often necessary to be able to process very long sequences (periodicity, seasonality)
- sequences in time series often have timestamps labels that contain more information than just a position number in the sequence from NLP or CV. For example, days of the week, holidays. - sometimes sequential generation is not efficient and the decoder needs to be able to generate subsequences at one time
- time series pose special tasks: spatio-temporal forecasting, event forecasting, anomaly detection, etc.
The survey systematizes approaches in the field of application of Transformers for time series. In addition, the authors test different algorithms on the ETTm2 dataset.
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
As you know, there are several severe issues with Transformer that prevent it from being directly applicable to long sequence time-series forecasting including:
- quadratic time complexity
- high memory usage
- inherent limitation of the encoder-decoder architecture.
To address these issues, the authors design an efficient transformer-based model named Informer.
Informer includes:
- a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences’ dependency
alignment
- the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences
- the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference
speed of long-sequence predictions.
As you know, there are several severe issues with Transformer that prevent it from being directly applicable to long sequence time-series forecasting including:
- quadratic time complexity
- high memory usage
- inherent limitation of the encoder-decoder architecture.
To address these issues, the authors design an efficient transformer-based model named Informer.
Informer includes:
- a ProbSparse self-attention mechanism, which achieves O(L log L) in time complexity and memory usage, and has comparable performance on sequences’ dependency
alignment
- the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences
- the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference
speed of long-sequence predictions.
🔥2
COMBO Pre-training representations of binary code using contrastive learning
Binary code analysis is critical to applications in reverse engineering and computer security tasks where source code is not available.
In the paper authors propose a COntrastive learning Model for Binary cOde Analysis, or COMBO, that incorporates source code and comment information into binary code during representation learning. Specifically, there are three components in COMBO:
- a primary contrastive learning method for cold-start pre-training
- a simplex interpolation method to incorporate source code, comments, and binary code
- an intermediate representation learning algorithm to provide binary code embeddings.
The effectiveness of the pre-trained representations produced by COMBO is evaluated using three indicative downstream tasks relating to binary code:
- algorithmic functionality classification
- binary code similarity
- vulnerability detection.
Binary code analysis is critical to applications in reverse engineering and computer security tasks where source code is not available.
In the paper authors propose a COntrastive learning Model for Binary cOde Analysis, or COMBO, that incorporates source code and comment information into binary code during representation learning. Specifically, there are three components in COMBO:
- a primary contrastive learning method for cold-start pre-training
- a simplex interpolation method to incorporate source code, comments, and binary code
- an intermediate representation learning algorithm to provide binary code embeddings.
The effectiveness of the pre-trained representations produced by COMBO is evaluated using three indicative downstream tasks relating to binary code:
- algorithmic functionality classification
- binary code similarity
- vulnerability detection.