AI sandwich as a defacto principle for anything agentic engineering related

The AI sandwich technique outlines a process where certain phases of product development are led by AI agents, while others require human intervention. The phases led by AI agents include planning, coding, reviewing, testing, and creating pull requests (PR). These tasks are automated to enhance efficiency and consistency.

In contrast, the phases that should be human-led are brainstorming (deciding what to build) and polishing (ensuring the product is of high quality). These stages require human creativity, judgment, and critical thinking, which AI agents cannot fully replicate.

The AI sandwich technique outlines a structured approach where humans and AI agents collaborate effectively.

In this model, the top layer involves human input, where goals and instructions are clearly defined by humans. This ensures that the desired outcomes are aligned with human intentions. The middle layer is where AI agents take over, handling the orchestration, execution, and processing tasks. This allows for efficient and automated handling of complex operations. Finally, the bottom layer involves human evaluation, where the output is assessed based on subjective human taste and feedback. This ensures that the final result meets human standards and expectations.

Think of the end game polishing done by humans, akin to how pilots and co-pilots still have the final call on the airliner they’re operating in, despite all the automations at place that could technically automate the role of the pilot, but not in principle.

The technique is particularly effective in scenarios where outcomes can be verified, such as in RLHF (Reinforcement Learning from Human Feedback) environments. Initially, human feedback is crucial, but as the AI system becomes more proficient, the need for human intervention decreases. The analogy of a sandwich or an airliner illustrates this process, where humans set the course and make critical decisions, while AI agents manage the routine operations. This collaborative approach leverages the strengths of both humans and AI, ensuring optimal results.