Model Testing - Mode for Prompt Evaluation Across Models

I’d love a “Model Testing” feature in AI Flow Chat that enables me to run parallel model comparisons directly within an automation node. The idea is to split a single prompt across 2–5 selected models and see the output side by side.

This would be particularly useful when fine-tuning automations, as it would allow me to:

  • Compare model outputs instantly

  • Spot quality differences and consistency

  • Identify the best-fitting model for a specific task

Key UX request:
This should ideally be surfaced as a dedicated modal or popup, where I can:

  • Select which models to test

  • Run the test multiple times (e.g., 3–5) for consistency checks

  • Review and compare outputs in a clear, side-by-side view

  • Finalize my model choice with confidence

This would turn model selection from guesswork into an empirical, workflow-integrated process.

Please authenticate to join the conversation.

Upvoters
Status

In Review

Board

Bugs or Features

Date

11 months ago

Author

Frederik Beier

Subscribe to post

Get notified by email when there are changes.