Synthesize and compare multiple LLM responses with OpenRouter council

This template adapts Andrej Karpathy’s LLM Council concept for use in n8n, creating a workflow that collects, evaluates, and synthesizes multiple large language model (LLM) responses to reduce individual model bias and improve answer quality.

🎯 The gist

This LLM Council workflow acts as a moderation board for multiple LLM “opinions”:

- The same question is answered independently by several models. - All answers are anonymized. - Each model then evaluates and ranks all responses. - A designated Council Chairman model synthesizes a final verdict based on these evaluations. - The final output includes: - The original query - The Chairman’s verdict - The ranking of each response by each model - The original responses from all models

The goal is to reduce single‑model bias and arrive at more balanced, objective answers.

🧰 Use cases

This workflow enables several practical applications:

- Receiving more balanced answers by combining multiple model perspectives - Benchmarking and comparing LLM responses - Exploring diverse viewpoints on complex or controversial questions