Free trial

Docgen: Enhance your code with OpenAI’s GPT in your Jetbrains IDE, a cautionary tale

Innovation is at the core of Profiq’s mission. Arguably, the most innovative development in technology recently is ChatGPT, developed by OpenAI. In an earlier article, we explored OpenAI’s Codex model, which we used for code generation and editing. In this follow-up article, we share our experience leveraging OpenAI’s models programmatically via its API, for creating a plugin for Jetbrains IDEs.

Querying OpenAI from the IDE

We’ve been using ChatGPT from the browser in various ways to aid with our daily software development and QA. When the interaction with ChatGPT involves frequent code generation and editing, using the browser interface becomes cumbersome, as there’s a lot of copying and pasting between the browser and the editor. As many of us at profiq are avid JetBrains users, we wondered whether GPT models could help us directly in the editor with OpenAI’s API? A number of plugins already exist that aim to integrate ChatGPT. There’s Github Copilot, which is more of a completion tool than a prompt-response assistant. Besides Github Copilot, we could not find any plugin that inserts generated code directly in the editor buffer without needing to switch to the browser. So, we decided to write one.

Which useful daily task could a GPT model help with? It’s well known that most developers are not very fond of writing documentation for their code. Could GPT help with that? Of course! By generating code documentation, it could save time, and let developers focus on actual problem-solving. Thus, Docgen, a Jetbrains IDE plugin for generating documentation, was born.

Getting started with the plugin

When we started developing the plugin, OpenAI’s service of choice for code generation and code editing was Codex, so we used the Codex API. Since we did not have much Java expertise or prior experience with IntelliJ Platform plugin development, we turned to ChatGPT for help with writing the plugin itself. It guided us through development one step at a time, gave us useful code suggestions, and helped us write the plugin considerably faster. Our goal was to create a simple MVP that would generate documentation for code in the current editor buffer, including formatted docs for individual functions. Although this sounds like a complex task, Codex was up to it. After a few days, we had a working version of the plugin that, triggered by a keyboard shortcut, was able to generate code documentation for various languages known to Codex. Everything was bright and sunny.

Change is the only constant

Satisfied with our MVP, we submitted it for review on the Jetbrains Marketplace on March 21, and awaited an email response with feedback. We received a different email from OpenAI than we expected—effective March 23, all support for the Codex API would be discontinued, and customers would need to transition to a different model. Yes, OpenAI gave developers 3 days to transition from Codex to another model. Three days! Well, at least we got the notice before the plugin’s release.

Fortunately, since we were making only one request to the Codex API in the plugin’s source code, the implementation change needed was not too dramatic. First, we replaced the Codex model with GPT-3.5-turbo, the most sophisticated model at the time. As GPT models are conversational, the main programming challenge was to parse the documentation string from the response and insert it into the right position in the current buffer, instead of simply replacing the entire buffer content as before. A bit tricky, but doable. There was another challenge, though. While Codex always returned the generated documentation string in an appropriate format for the given programming language, GPT-3.5 did not. No matter how we wrote the prompt, it returned an explanation of what the inputted code does in plain English instead of the desired documentation string. Bummer.

Narrowing the scope

One consequence of the necessary model change was, we had to give up the idea of a plugin that would document source code in any programming language known to the model. Since Python is one of the two most popular languages at profiq (along with Javascript), we made the prompt Python-specific:

Generate high-quality docstring for the following 
Python function including function signature:

{{ function code }}

With this prompt, GPT-3.5 actually does return a valid docstring (along with an explanation in plain English), which we can parse and insert directly after the Python function’s signature. One could, of course, modify the prompt to generate documentation for a different language, such as, Java:

Generate Javadoc for the following code:

Unfortunately, since GPT models return a conversational message with an explanation in plain English in addition to the code itself, the documentation string always needs to be parsed from the message. Since each language has its own documentation format, we’d need to implement a custom parser for each language. Our goal, however, was to create an MVP, so the plugin does not include any additional parsers for other languages for now.

In summary, we ended up creating and publishing a plugin that can generate and insert documentation in docstring format into a selected Python function. Despite our initial ambitions for a more general documentation plugin for many languages, we are happy to have a tool we can use inside our IDE without switching to the browser and copy-pasting.

A word of caution

We’ve learned a few lessons while building on OpenAI’s services and APIs. This year has definitely been a turning point in the AI domain. Current AI models are evolving at a rapid pace, and we expect the models and their capabilities to significantly improve in the months to come. Governments worldwide are currently trying to catch up and limit and control the AI (r)evolution. As we’ve seen with Codex, AI platforms rise and fall. We recommend using caution when building on top of the existing APIs, as they may change unexpectedly.

If you’ve read this article until the end, you might be curious and want to try the plugin yourself. It’s available for download from Jetbrains Marketplace free of charge:

Leave a review if you like the plugin or run into any issues with it. We hope you find it useful!

ai api chatgpt openai plugin

Leave a Reply

Related articles


Let’s make LLMs generate JSON!

In this article, we are going to talk about three tools that can, at least in theory, force any local LLM to produce structured output: LM Format Enforcer, Outlines, and Guidance. After a short description of each tool, we will evaluate their performance on a few test cases ranging from book recommendations to extracting information from HTML. And the best for the end, we will show you how forcing LLMs to produce a structured output can be used to solve a very common problem in many businesses: extracting structured records from free-form text.

Notiondipity: What I learned about browser extension development

Me and many of my colleagues at profiq use Notion for note-taking and work organization. Our workspaces contain a lot of knowledge about our work, plans, or the articles or books we read. At some point, a thought came to my mind: couldn’t we use all this knowledge to come up with project ideas suited to our skills and interests?

From ChatGPT to Smart Agents: The Next Frontier in App Integration

It has been over a year since OpenAI introduced ChatGPT and brought the power of AI and large language models (LLMs) to the average consumer. But we could argue that introducing APIs for seamlessly integrating large language models into apps developed by companies and independent hackers all over the world can be the true game changer in the long term. Developers are having heated discussions about how we can utilize this technology to develop truly useful apps that provide real value instead of just copying what OpenAI does. We want to contribute to this discussion by showing you how we think about developing autonomous agents at profiq. But first a bit of background.