My experience "vibe-coding" a take-home test
I just used LLMs for about 3 days to do an at-home challenge. My experience was that I "de-activated" my brain during most of it, I kept hammering at the prompt over and over, and ended up solving through mere persistence and increasing the context; not by skill.
I used Claude Opus 4, Gemini Pro 2.5, and Chat GPT o3 via the chat interface and Cursor using the MAX models. The chat-based models were useful for one-off queries on thorny issues, with o3 taking the lead in being able to diagnose the problem in a "clean" way. Gemini didn't seem as useful. Opus 4 was very powerful, but it almost did too much.
What worked best was actually adding the context of the documentation I needed to Cursor, to have it test its findings, and then to verify them myself. It was excellent at drafting the first code to solve the problem, but got worst as the context increased and the prompt grew longer. It eventually got confused with its own context and none of the solutions worked.
The best way to debug was to create a clean window of context and add just the code I needed to o3. I didn't trust Opus 4 since it would try to re-write the code from scratch. Gemini either.
Overall, I don't think I enjoyed prompting the LLMs over and over again. I found it frustrating at times, and amusing at best. I mostly learned how each model behaves, but not as much about the code or the problem I was solving.
I don't think this is a good approach to learn. Next time I would recommend reading the documentation of each API, drawing with Excalidraw how to solve it, and figuring out which concepts are best to apply.
It took me about three days of prompting the LLM at about 3-4 hours each day to solve it. I'd assume it would take me 2-3 hours to get acquainted with the problem manually, then one or two more days manually coding it with light code-completion activated.
I still think they're fantastic tools, but I think it would be nightmarish to run code like this in production. If you don't know how it works, you're going to get your head chopped.