Same image job. Same constraints. Show the misses.
This page fixes the test before the outputs exist. We will not publish a winner, numeric score or claimed benchmark result until the controlled runs are completed and the evidence is shown.
Tools in the first run
ChatGPT Image, Midjourney, Ideogram, Adobe Firefly and FLUX are the planned comparison set. If access or product capabilities differ at run time, that limitation will be recorded rather than silently changing the test.
Five controlled tasks
Premium bottle on a controlled surface with defined lighting, negative space and no unrequested objects.
Exact headline and short supporting copy with hierarchy, spelling and layout constraints.
Natural materials, believable geometry, lighting and perspective under a fixed composition brief.
Preserve defining identity, geometry and requested details from the same supplied reference.
Change one requested element while preserving already-approved composition and details.
What we will measure
| Dimension | Evidence we inspect |
|---|---|
| Instruction adherence | Requested objects, omissions, composition and explicit constraints |
| Text handling | Exact wording, spelling, legibility and unintended text |
| Visual coherence | Geometry, anatomy where relevant, materials, perspective, lighting and artifacts |
| Reference fidelity | Whether defining identity and geometry survive generation |
| Revision control | Whether the requested change occurs without unnecessary collateral changes |
| Cleanup burden | Visible defects or corrections needed before practical use |
Run rules
- The task requirement is frozen before comparing tools.
- The same semantic requirement is used for every tool; syntax-only adaptation is documented.
- Default/current production model settings are recorded at test time.
- The number of first-pass generations and revision attempts is kept equal wherever product capabilities permit.
- No output is quietly regenerated until a preferred example appears.
- Every published conclusion links back to visible output evidence and known limitations.
Results
This is deliberate. The page establishes the methodology first so later recommendations cannot be reverse-engineered around a preferred tool.