It seems the only thing bigger than AI is talking about AI. I’ve been playing around with some AI-related things, and reading up on various aspects of AI use, so I guess it’s my turn to write down my current thoughts. This will probably end up as a series of posts, if I can get a bit more disciplined about writing here.

NOTE: thoughts may change on a whim, depending on future usage and current caffeine consumption.

AI as software development tool⌗

In my day job at Itron, I am part of a group trialling GitHub Copilot. The demos using C#, Python, JavaScript, etc. look great, where it seems the AI is reading your mind, but most of my usage doesn’t quite live up to the demo.

Unlike most of the demos, the main languages I use daily are Ruby/Rails and Elixir/Phoenix, with UI primarily in CoffeeScript. For various reasons, I haven’t done much Elixir lately, but for Ruby, the AI suggestions are pretty good. If I’m adding some logging for debugging purposes, after the first one is manually created, I only need to start others and the suggested completions are useful about 90% of the time.

Similarly with actual new code, if I add a new method call before defining it, and the name is somewhat logical, when I start the actual method definition, about half the time the AI suggests an implementation that is exactly what I want; the rest of the time it requires some minor editing, but the initial suggestion saves me some typing.

For CoffeeScript, however, we run into one of the problems with AI that most never talk about. At least for these tools, they’re really only as good as what they’re trained on, and apparently there really isn’t very much CoffeeScript in the training set. Copilot hasn’t had many useful suggestions for any of the recent work I’ve needed to do, and ChatGPT similarly hasn’t given me much help on general questions.

For now, I think of these tools a a less-opinionated version of Stack Overflow. To get the most out of the AI, you need some understanding of what you’re actually trying to do, and you need to review what you get back.

There are some things I want to continue to explore for personal projects, if I can make time.