The open models caught up. That fact alone tells you nothing about what your product should run on. Here is the checklist we actually use at Dinimiciuil Labs when picking the model behind a build, six checks, in the order that eliminates options fastest
Context for the moment: in July 2026, open weight releases hit benchmark parity with the closed frontier, we covered the month that changed it, and hosted open inference now undercuts closed APIs by multiples, with GLM 5.2 at $1.40 per million input tokens, per devFlokers' roundup. The temptation is to default to open everything. Resist defaults. Run the checks.
The six checks, in elimination order
- Quality fit on YOUR task, not the arena. Benchmarks measure the average of everything. Your product needs one thing done excellently. Build a 20 example evaluation from your real use case and run it against two open and two closed candidates. An afternoon of testing beats a month of leaderboard reading.
- License compatibility. Apache 2.0 and MIT weights are safely commercial. Modified licenses need reading for scale triggers and output conditions, the open weight trap we detailed here. If the license is unclear, the model is eliminated, quality does not matter.
- Cost at YOUR volume curve. Under about 5 million tokens a month, API pricing differences barely matter, pick on quality. Above it, the open model hosting discount compounds, per SitePoint's 2026 cost analysis. Model the cost at 10x your current volume, because that is the bill you are actually choosing.
- Data rules. If customer data cannot leave your infrastructure, open weights are not a preference, they are the only option, and you accept the $3,000 to $6,000 monthly staffing reality of running them, per AI Superior's deployment breakdown. If data can flow to a processor, hosted open inference gives you the economics without the ops.
- Supplier risk. Closed model risk: price rises, deprecations, policy shifts. Open model risk in 2026: geopolitics, with Reuters reporting China may restrict foreign access to its open models. The open hedge is structural: mirror the weights on adoption day and no policy can un-release them.
- Switching cost. Whatever you pick, build behind a thin abstraction so the model is a config value, not an architecture. The teams hurting today are not the ones who picked wrong, they are the ones who welded the pick into the product.
This per job discipline is no longer a contrarian take. On 24 July 2026, Microsoft built its entire open weight case around the same idea, that open models let organizations "match the right model to the right job at the right cost", per Microsoft's open weight brief. The checklist above is how you actually do that matching, one build at a time.
The deeper principle is the one behind all our Ai native product thinking: the model is a component, chosen per job, replaceable by design. Open weight parity did not change that principle. It just made the component market dramatically more competitive, which is the best thing that has happened to small builders in years.
Frequently asked questions
Should a startup use open source or closed AI models?
Run six checks: task quality, license, cost at 10x volume, data residency, supplier risk, switching cost. Early products with modest volume usually start on hosted APIs and migrate text workloads to hosted open models as volume grows. Data residency requirements force open weights immediately.
How do I evaluate an LLM for my specific product?
Build a small evaluation set of about 20 real examples from your actual use case with known good answers, then run every candidate model against it. Task specific testing beats public benchmarks, which measure averages across everything except your product.
How do I avoid lock in to one model?
Put every model call behind one thin internal interface, keep prompts in versioned files rather than scattered through code, and maintain your evaluation set so any new model can be trialled in a day. The model should be a config change, not a rebuild.
© 2026 Dinimiciuil Labs. All rights reserved. Written on the build floor in Dublin. You are welcome to quote a short excerpt with a link back; please do not republish the full article without permission.