Generative AI made its mark on many skills, professions and businesses. Within the realm of software development, the emergence of GitHub Copilot has been a game-changer. Today, at In The Pocket, this funky tool has found its way into the hands of nearly a third of our workforce. But before we move to full adoption, we sought to quantify its impact through a comprehensive survey. Join us as we unveil the insights we've gathered.

Copilot is here to stay

As of now, an impressive 33% of our colleagues have Copilot in their coding arsenal. Considering that not everyone at In The Pocket is directly involved in coding, this translates to approximately 75% of our software engineers harnessing the power of Copilot. And they do use it frequently. Almost 80% of them use it on a daily basis and a whopping 95% on a weekly basis.

It’s difficult to objectively assess what impact Copilot has had on the organisation since we started using it, but we did probe how it was experienced by our software engineers - the true judges. According to our survey, a mere 12% perceived marginal improvements, while 21% believed Copilot was saving them over an hour each week.

Other notable numbers? We delved into the realm of GitHub code suggestions. A perfect split emerged; 50% embrace the suggestions most of the time, while the other half leans towards a more conservative approach.

Finally, we posed a rather compelling question: what would you miss more? Coffee or Copilot? A resounding 59% would willingly forget coffee to keep using Copilot. Now, that's a statistic that's hard to ignore.

What do we use Copilot for? 

Now, what exactly are we entrusting Copilot with? We quizzed our colleagues and discovered that smart code completion reigns supreme. It's the go-to feature for virtually everyone. Beyond that, 50% of our respondents mentioned they use it for Guided Code Generation, while a smaller percentage even uses it to learn a new coding language or to discover new ways to write certain lines of code.

“It removes all the cumbersome tasks, so I can focus on actually building the good stuff.”
- ITP respondent 

Copilot is being used mostly for business logic (85%), followed by clean code practices - like documentation and unit tests (55%) and even branching into infrastructure-as-code (25%). 

Who (should) use Copilot? 

We were also really curious about which profiles we think should use this tool to its full extent. Mainly, we believe that everyone can benefit from using Copilot. Of course, due to its nature and the tools we use it in (VS Code, NeoVim and other code editors) it is tailored to developing profiles, so we only promote it to these profiles.

“Copilot is mostly a good way to write stupid code faster. With a few adaptations here and there, I’ve been able to write quite some code quicker than I had to do it manually.” - ITP respondent

However, we emphasise that code should never be blindly copied and pasted without critical thinking. Copilot's suggestions can be vague about their origins and reasoning, making it crucial to educate new users about these pitfalls. Currently, we're divided on whether junior software engineers should fully embrace it. Nevertheless, we've made it accessible to everyone, regardless of seniority or experience.

In conclusion

For us it’s clear: Copilot is here to stay. Right now, we offer it to all engineering profiles, given they understand the limitations of the tool. While other non-developer profiles are welcome to explore it, we encourage a thoughtful assessment of whether it genuinely adds value to their tasks or if other tools might be more suitable.

It seems that in our studios, we’re really coding over coffee, powered by Copilot.