← Graph

Generate code once vs instruct LLM per request

question 2 connections

Audience question: in the demo, instructions are sent to the LLM per incoming request — could you instead ask the LLM once to generate Ruby code that performs those instructions, and save it? Andrei's answer: that's an interesting angle many companies are pursuing. He is curious about the inverse: generating the execution itself rather than saving code. As Rails developers we are essentially stringing APIs together; his demo attempts to illustrate writing business logic in prompts and letting execution be inferred — and inference is literally what the LLM does as it predicts next tokens.

answer_summary
Interesting angle. Many companies pursue code generation. Andrei is more interested in generating the execution directly — inference is what LLMs do natively — rather than persisting generated code.
question Generate code once vs instruct LLM per request
about
AI Agent concept
Question concerns alternative architectures for agent-style LLM apps.
question Generate code once vs instruct LLM per request
asked_at
Audience question during the Q&A.

Provenance

Read by
10 extractions