How Software Engineers Use GenAI Tools 

Through my conversations, to me it became fairly apparent that there were three schools of thought when it comes to how software engineers use Generative AI (GenAI): 

• I’m a purist, and I don’t use GenAI at all
• I use GenAI to create code
• I use GenAI to audit code

Through several information conversations with colleagues and friends who consider themselves to be a “Software Engineer” by their own definition, my interactions spanned career paths, years of experience and generational gaps. While by no means scientific in its application, my research showed me a few standout points, registered along interesting lines of division.

“Primates evolve over millions of years. I evolve in seconds.” ~ SkyNet, Terminator Genisys (2015) 

The most opinionated response to GenAI in software development was a loud thumbs down. For most, the reasons were mostly related to comfort; in that, they have established working practices that meet the requirements of their work, and that flow is comfortable for them. In addition, several people pointed out, either through their own testing or “through the grapevine,” that GenAI tools cannot be trusted to create the level of quality and code reusability that we have come to expect.   

While louder with their opinions, it was a relatively small number of actual people who I spoke with that say they do not use GenAI in any of their regular workflows or coding iterations.  Most people have found at least one use-case for GenAI that has become part of their workflow.   

Other notable points include, “most GenAI models currently available do not produce what I would need from them correctly, as emerging technologies and capabilities are not represented in the available training data,” and, “the company/client that I work for currently does not have a GenAI usage policy, so therefore, we do not use it at all.” 

“We can only see a short distance ahead, but we can see plenty there that needs to be done.” ~ Alan Turing 

Unsurprisingly to me, the most common response to my inquiry of, “How do you use GenAI in your Software Development workflow?” can be best summed up by using a GenAI tool to write the first iteration of a solution. From a logical perspective, this is how most people see GenAI, as a generative tool that will produce some output that can be consumed. People described different methods of asking their preferred GenAI tool for output, all of which I will summarize as “individualized prompt engineering,” with mention of many different approaches (although, almost everyone claimed their method to be “the best,” which shows that even of those asking GenAI tools to help them get started, everyone’s expectations on output are unique.   

What happens after that first iteration, however, seemed to differ along several different paths. For some, the code generated was immediately tested to see, “if it worked,” or to at least, “find an error to chase.” For others, however, the AI generated code was treated more as “stub code,” that the developer would then use as a starting point to solve their next problem. Regardless of the path chosen, most “Jump Starters,” claimed that GenAI tools always “saved them time” and helped them to avoid writing repetitive or potentially complex blocks of code.

“Pair programmers: Keep each other on task. Brainstorm refinements to the system. Clarify ideas. Take initiative when their partner is stuck, thus lowering frustration. Hold each other accountable to the team’s practices. Pairing.” ~ Kent Beck 

The last distinct group that emerged took a similar approach to the Jump Starters, but instead of using GenAI tools to create the first iteration of a solution, this group preferred to use GenAI tools to debug code that they had written themselves. On its face, this approach seems to follow a separate, yet equally logical approach; the developer is still using GenAI to help solve their problem, but instead of asking for a starting point, now the developer is asking the GenAI engine to build on an idea already established.    

At first, I saw the Pair Programmer as just the reciprocal of the Jump Starter, but after more thought, I noticed behavioral differences that pointed elsewhere. In my subjective interpretation, developers using GenAI tools as a Jump Start seemed to be those with less experience or confidence in the task(s) at hand, whereas the Pair Programmer knows, at least, where to begin and needs help iterating beyond a certain point. 

It’s that – the needing help iterating – that caused me to realize that this behavior was not the opposite approach to the Jump Starters, but a different approach all together. One that helped me see GenAI as the “pair programmer” for the disaggregated office; the “invisible double” that Developers use to help them solve problems, unblock progress, and iterate forward. 

Another way of framing these types of users is as follows:

Developer Alpha: Developer Alpha likes to utilize help from AI to refactor and reformat their code. They think the tools mostly do a good job, but GPT still messes up code around 15-20 percent of the time. They feel when they use it to create an API or a new project or template, it would speed up their time and would deliver the expected results. They added they would need to ask the GPTs inquisitive questions to get the expected responses, and they would have to refine their queries to extract the required responses. They said their interactions with GPTs are never abstract and has to provide more information and they feel it serves them best for most of the Front-End related code.

Tools Used: Copilot, ChatGPT

Tools they would like to integrate : Copilot with Git.

Do they think it speeds up the workloads : No.

Do they think this would replace google : Not yet

Do you trust the GPT Output : No. I will always re-test it.

Do you think it improved developer productivity: Not a lot, yet.

Additional Concerns: If it gets better, it will make the developer dumb and they will stop researching themselves.

Developer Beta: Thinks the tools integration has been helpful, but can currently only be useful for syntax corrections, etc., which can speed up a junior or mid-level developer, not the experienced. They mostly use AI help to parse through large amounts of documentation and to figure out the capabilities of the tools they are planning to use. For example, when they tried to understand the ag-grid capabilities or trying to render it in a specific order based on conditions, they were able to use the help from the models to understand the ag-grid capabilities, find the alternative options and how they can best approach this problem. They used it better as a data-miner to mine through vast amounts of data such as a new library or AWS functionality as information accessor.

Tools Used: VS code Extn, Copilot, ChatGPT, Gemini

Tools they would like to integrate : Copilot with Git. Vs Code IntelliSense generative code

Do they think it speeds up the workloads : Sometimes.

Do they think this would replace google : Sometimes it is easier to hit google than trying to ask the questions in a specific way.

Do you trust the GPT Output : Never.

Do you think it improved developer productivity: Depends on how you use it and if you know how to use it.

Additional Concerns: Lazy Dev vs Pure Dev phenomenon. With the level of handholding, they feel it might impact Jr Developers long term learning capabilities as they will not have to learn their basics.

Developer Delta: Felt it is useful to use GPT tools if you know what you are looking for. They feel, as they got better at using it, it is helping them with their search time. They used it for simple tasks such as designing a basic layout to provide them with HTML, as well as when they needed to create a complex LinQ query that references various tables or for fluent validations, etc. Developer Delta used it when they knew what to look for by asking specific questions to get specific code results. They also used it for documentation parsing and easily create the Front-End Templates.

Tools Used: Bard, ChatGPT, Gemini

Tools they would like to keep using : Gemini

Do they think it speeds up the workloads : Yes.

Do they think this would replace google : They go directly to Gemini most of the time.

Do you trust the GPT Output : Initially No, second guessing a lot. Even now, the output is a hit or miss.

Do you think it improved developer productivity: Overall it’s a productive tool, effectiveness depends on developer

Additional Concerns: Concerned about client information if people start using it extensively. They think GPTs won’t take your Job anytime soon and if they really advance more, maybe.

Developer Gamma: Hasn’t really used AI. They are not resistant to change but the work/client-related obligations have kept them strictly in the know-how as the client has specific rules and guidelines on how things need to be get done. They felt they would be doing the same work twice if they handed it over to GPTs as it would not be easier to explain the requirements, without divulging into the client/business details, to get the required output.

Developer Theta: Has never used GPTs and is not into the frenzy yet. They feel what they are doing (Google , other Tech blogs) is helpful for their day-to-day work and has not found the need to start using new tools.

Overall, there is a learning curve involved in using AI tools. The common theme has been, they don’t yet trust the output of the GPTs. The majority expressed their commitment to not use any client/business related information in any of their queries. Though the common misconception has been that it would simply emit the output for junior developers and would make them look knowledgeable, it's felt the GPTs are not that proficient yet to fake a professional. Also, there are still vast swaths of people who are not comfortable using GPTs for their regular work and they would need an external event to trigger their AI usage.

And so remains the looming question: What kind of GenAI software developer are you?