Earlier this year, GitHub introduced Copilot Chat, a programming-focused chatbot similar to ChatGPT, designed for enterprises that have subscribed to Copilot for Business. Recently, Copilot Chat was made available to individual Copilot customers that pay $10 per month, in beta. Now, GitHub is making Chat available to all users.
Starting today, Copilot Chat can be found in the sidebar of Microsoft’s IDEs, Visual Studio Code and Visual Studio. It is included as part of the subscription tiers of GitHub Copilot, but is also available for free to verified teachers, students, and maintainers of certain open source projects.
“As the global hub for developers, we have introduced to the market the AI developer tool that has gained the highest level of adoption in history,” said Shuyin Zhao, Vice President of Product Management at GitHub, in an email interview with TechCrunch. “And code complete was only the start.”
Not much else has changed about Copilot Chat since the beta.
The chatbot is still powered by GPT-4, which is OpenAI’s main generative AI model, specifically fine-tuned for development settings. Developers have the ability to use natural language to request real-time help from Copilot Chat. This can include requesting Copilot Chat to provide explanations, identify vulnerabilities, or generate unit tests.
Similar to other generative AI models, the underlying model of Copilot Chat, GPT-4, was trained using publicly accessible data, some of which may be copyrighted or subject to restricted licensing. Vendors such as GitHub assert that the fair use theory protects them from copyright disputes. However, this has not prevented programmers from initiating class action lawsuits against GitHub, Microsoft (the parent firm of GitHub), and OpenAI, claiming that there have been violations of open source licensing and intellectual property.
I inquired with Zhao if codebase owners will be given the opportunity to choose not to participate in training, if they so desire. She stated that there is no new process for this with the wider release of Copilot Chat and instead recommended that codebase owners set their repositories to private to avoid them being used in future training datasets.
I believe codebase owners may not appreciate your approach. There are several reasons for keeping copyrighted code public, one of which is the opportunity for crowdsourcing bug hunting. However, it seems that GitHub is currently unwilling to change its stance on allowing users to opt-out of training data.
Generative AI models, such as GPT-4, also tend to provide false information or confidently fabricate facts, which can be especially problematic in the field of coding. Based on a recent study conducted by Stanford, developers who utilize AI helpers for coding tend to generate code that is less secure in comparison to those who do not use AI assistants. This is partly due to the fact that the AI assistants present code snippets that are faulty or outdated.
Zhao mentioned that GPT-4 shows improved performance in dealing with hallucinations compared to the previous model that was used in Copilot. Zhao also highlighted the exploit-mitigating features of GPT-4, such as filters for insecure code patterns. These filters alert Copilot Chat users about vulnerabilities such as hardcoded credentials, SQL injections, and path injections. However, she emphasized the significance of thorough human examination of any code provided by AI.
“GitHub Copilot is driven by OpenAI’s models, which we have determined to be the most effective models for the services we currently provide,” Zhao stated. “We are in a highly advantageous position to further enable developers with the AI tools they require to construct superior, more secure software on a large scale — and to enjoy themselves in the process.”
In October, Satya Nadella, the CEO of Microsoft, informed investors that Copilot had one million users who paid for the service and approximately 37,000 clients from enterprises. However, it is essential for GitHub to enhance the appeal of Copilot in order to avoid losing market share to competitors and maybe losing revenue.
Based on an article in the Wall Street Journal, Copilot has an average monthly loss of $20 per user, while some customers cost GitHub as much as $80 per month. The expensive cost of operating the underlying AI models is said to be the reason for this, which is also what caused the GenAI coding startup Kite to close down in December.
While GitHub faces challenges in making Copilot viable, Amazon is actively improving CodeWhisperer, which is considered to be one of Copilot’s strongest competitors.
In April, Amazon made CodeWhisperer available to developers for free, with no limitations on usage. In addition, CodeWhisperer Professional Tier was introduced during that month. This update included the integration of single sign-on with AWS Identity and Access Management, as well as increased restrictions for scanning security vulnerabilities. A business strategy for CodeWhisperer was implemented in September. In early November, Amazon made changes to CodeWhisperer to improve the advice it provides for app development on MongoDB, the open source database management application.
In addition to CodeWhisperer, Copilot has competition from startups such as Magic, Tabnine, Codegen, and Laredo. There are also open source models like Meta’s Code Llama and Hugging Face’s and ServiceNow’s StarCoder.