My experience with LLMs for coding has been similar. You have to be extremely vigilant, because they can produce very good code but will also miss important things that will cause disasters. It makes you very paranoid with their output, which is probably how you should approach it and is honestly how you should approach any code that you’re writing or getting from somewhere else.
I can’t bring my self to actually use them for generating code like he does in this blog post though. That seems infuriating. I find them useful as a way to query knowledge about stuff that I’m interested in which I then cross reference with documentation and other sources to make sure I understand it.
Sometimes you’re dealing with a particular issue or problem that is very hard to Google for or look up. LLMs are a good starting point to get an understanding of it; even if that understanding could be flawed. I found that it usually points me in the right direction. Though the environmental and ethical implications of using these tools also bother me. Is making my discovery phase for a topic a little bit easier worth the cost of these things?
Yeah, a lot of school bus stops on five lane highways… /s