Nov Product - Moviebot - Python Prototype
Published on: Nov 07, 2023<< Idea Validation and Prompt Engineering
- Cost: $0
- Time: Under 1 hour
- Outcome: Moderate Success
- Tools Used: VS Code, Github Copilot, gpt-3.5-turbo-instruct
Armed with my prompt and a production version of Github Copilot,I wanted to see if I could build a quick python script that would call the GPT 3.5 endpoint, deliver a Few Shot Training Prompt request, and return a robot-generated movie pitch.
I was only in my second hour, as I noted at the time, this was the first point in the process where I was glad that I had the technical skill to do this without Copilot, if only to see where and when Copilot was making mistakes.
This is an area where a product person attempting to solo using AI needs a bit of technical and domain knowledge, or else the product will not get prototype successfully.
This specific error? OpenAI recently replaced the older text-davinci-003 endpoint to the new gpt-3.5-turbo.
I used the Chat function in Copilot to craft what felt like a clear product requirement and specified the newer endpoint. Copilot created a working a API request, but consistently used the old endpoint.
This was not a huge error - easily fixable in the code itself - but it does show the shortcomings of Copilot, the nature of the model means it’s beholden to patterns, and patterns take time to update. The specificity of my request ( `using the gpt-3.5turbo endpoint` ), however, is something that I’m sure the Github team will fix.
Still, I was able to correct the Copilot oversight, and get a working request to OpenAI in under an hour. This is roughly the same amount of time it would have taken me to do this without Copilot, largely because I am out of practice and because I’d never used Copilot before.