Why AI alone can't price your flips

Short answer: AI cannot reliably tell you what an item will sell for — and the more confident the number, the more expensive the mistake. We say that as heavy AI users: an LLM writes more than half the price estimates in GavelGap's scraped feed. But after 630 pull requests and 2,000+ automated pipeline runs spent building a resale-pricing engine for government surplus, the biggest lesson is this: resale value is not a fact a model remembers. It's a measurement you take from real sold transactions — and the raw data fights you the whole way down.

If you follow the big resellers, you've heard the advice: get an AI browser or assistant — Perplexity's Comet, ChatGPT, Claude, take your pick — and let it research your niche, summarize listings, draft your posts. It's good advice. We agree with it.

Then you ask it what the lot in front of you is worth, it answers instantly, in a complete sentence, with a dollar figure — and that's where it gets expensive. Because sourcing lives and dies on one number: what this item actually sells for, versus what it costs you to own it. Get that number wrong by 2× in either direction and you either bought a money pit or walked past a paycheck.

We've spent the better part of a year teaching software to get that number right on GovDeals lots. Here is what we learned, with receipts.

The hall of fame of confidently wrong prices

Every row below is a real government-surplus lot that some part of our own pipeline priced wrong at some point — caught by a backtest, an eval harness, or an unlucky afternoon of reading the feed. Some never escaped a shadow test; some sat on our live feed until a guard caught them. This is what "just have AI price it" looks like at scale:

None of these are exotic failures. They are what happens, by default, when any system — a language model, a keyword search, or a spreadsheet — produces a resale number without being forced to prove it against reality.

We're not AI skeptics. AI writes half of our estimates.

Let's be clear about the thesis, because it isn't "AI bad." GavelGap runs a tuned version of Gemini in production — not stock Gemini, but one we've calibrated against real auction outcomes, category by category. It values the comp-less tail of government surplus — the used lab cart, the pallet of municipal odds and ends, the fire-department air compressor — where no clean comparable sale exists anywhere. On those lots, a guarded LLM genuinely beats every alternative, because the alternative is nothing: of the lots our AI prices today, 96.7% have no reputable market ceiling available at all — no clean comp, no retail band. That absence is why the AI prices them.

But we also backtest those estimates against what the lots later sold for, and the failure modes are consistent enough that we can name them:

• It prices from memory, not from a market. Measured against realized outcomes, our production LLM ran 3–5× high on used office equipment — it anchors on what things cost new — and about 0.48× on bulk clothing lots, because it can't quite believe 400 shirts are worth real money. No flat multiplier fixes that; we had to rewrite its instructions per category and outrank it with real data wherever real data exists.
• It can't see, and doesn't know it can't see. That F-350 in the table had its wreck in the first photo and "non-op" in the description. The text-only estimate said $14,500 at 85% confidence. Even after we taught the comp engine to condemn non-runners, the cached AI value kept the "Strong Buy" alive through a second release. It took a third fix — condition rules injected straight into the model's prompt — to kill it.
• It does confident arithmetic on absurd premises. Multiply a plausible per-unit price by 766,371 units and you get $1.53M "resale" on a $10 bid. The math was fine. The premise — that you can flip three-quarters of a million radio modules at eBay unit prices — was insane. We now have a guard specifically for confident arithmetic at extreme quantities.
• It never says "I don't know" unless you force it. A model will always complete the sentence. "Unknown" — the single most protective output a pricing tool can give you — had to be engineered in as a feature, with confidence gates that throw away answers the model itself isn't sure of.

All of that led to a house rule, adopted the day we found a lot showing +$39,435 of "profit" off a $62,000 AI estimate while twelve real auction sales of the same machine sat next to it averaging $34,500: an unverified AI guess never outranks a real sold comp. Ever.

And yet — the reverse rule would be wrong too. A non-running Case 580B backhoe: the AI said $3,500; the sold comps said $30,000, because every comp was a working machine. The comps were real and the AI was right. Sold data isn't magic either. It has to be matched — which brings us to the actual hard part.

The auction price is a trap, too

The obvious shortcut, once you distrust AI: price from auction results. They're public, they're real transactions, there are millions of them. We tried. Our first live backtest was humiliating in a very specific way: 99 of the 100 most-bid-on lots that week were rated "hard pass" or "unknown" by our own scorer, and 84.6% of the lots we'd called "hard pass" sold for more than our estimate of their full resale value. Laptop lots were closing at an average of $1,702 against our $68 average estimate — 25× off.

The estimates weren't random — they were anti-correlated with demand, because the pipeline leaned on GovDeals' own price history. An auction price is your acquisition cost. Pricing resale from it predicts what you'll pay, not what you'll make — it defines the gap to be zero, which is exactly the number a reseller doesn't need. Across 106 verified lots we later tracked from the gavel to the open market, the median lot resold for 2.4× its winning bid — a median gap of $11,500. That gap is the entire business. A "score" built on auction prices grades the wrong number.

The trap is subtle, because it hides inside "real sold data" too. Our heavy-equipment comp pool is built from auction results. A Komatsu WA380 wheel loader was estimated at $69,000 while its own comp window contained a $131,000 sale of the same-year machine — conservative percentile math, tuned for dirty keyword data, was double-discounting an already-clean comp set. Sold data has a basis: auction hammer prices sit near dealer trade-in money, not retail-lot money. You have to know which one your comps are made of, or your "real data" quietly grades you against the wrong market.

Real sold data fights back

So the answer is real sold comps, matched to your item. Simple to say. Here is what "matched" turned out to mean, one painful lesson at a time:

Identity. Search engines match words; markets price things. "Misc. RAM" pulled Dodge RAM trucks. A lot titled "Massive Truckload of Electric Motors" got keyword-shortened to "Massive Truckload" and matched Lionel toy train truck lots. A DYNAPAC road roller — real machine, ~$65,000 — was valued at $15 off eBay listings for parts, manuals, and scale models, and confidently flagged as an $18,050 loss. Every one of those needed its own filter: vehicle-title detection, hype-word stripping, accessory and toy exclusion, parts-listing rejection.

Agreement isn't identity. The scariest contamination doesn't look noisy — it looks unanimous. A commercial ab-crunch machine (real resale $400–800) came back at ~$30 from a tight, consistent cluster of comps… all of which were $25–40 knockoff look-alikes of the thing, not the thing. Low variance feels like truth. It isn't; it's just consensus among the wrong products.

Quantity. About 94% of sold comps are single units, but surplus sells in lots. Miss the multiplier one way and a "Lot of 444" desktops is valued at $51 — one desktop. Miss it the other way and 25 two-dollar bills become a $2,500 "strong buy." Miss the parsing entirely — "13 Laptops," no "lot of," no parentheses — and the auction itself out-bids your estimate. And the parser can't be greedy, or "Lot of 1919 Coins" multiplies by a year and "925 Silver" by a purity mark. When we finally audited one week honestly, a high-side sanity guard demoted 149 fantasy buy-verdicts — a hand-checked sample of them was 100% junk.

Condition. The same laptop is three different products: BIOS-locked with no drive, clean used, or upgraded with a new SSD — and government surplus is routinely stripped while eBay comps are routinely loaded. Eight no-hard-drive laptops once read "$480, strong buy" off fully-loaded comps. Vehicles are worse: "engine fire," "motor knocks," "no title" each need their own rule, and each rule needs a negation guard so "NO engine issues" and "knock sensor replaced" don't condemn a healthy truck.

Fees and friction. Value still isn't profit. GovDeals' buyer's premium (typically 7.5–12.5%) appears in no machine-readable field anywhere — we sampled 240 lots and zero stated the percentage outside the logged-in bid box, so software has to estimate it or silently overstate every margin by ~12%. Shipping assumptions bite harder: a $100 Ford Focus seven miles away once graded out at −$97 because an unknown-distance default billed it $1,460 of auto transport. And some "deals" aren't deals at all: a strong-buy police Interceptor turned out to be restricted to "Police and Fire Departments Only." You can't profit on a lot you're not allowed to buy — our friction engine now runs 50 distinct checks for exactly this class of margin-killer.

Freshness. Prices move and listings mutate. One lot sat mis-scored for three weeks because nothing about it had "changed" enough to trigger a re-score — a green pipeline, quietly serving a stale answer. Closing lots now re-score hourly, caches expire on schedules, and a health check audits the whole thing every 12 hours. Sold data isn't a dataset you download; it's a garden you weed forever.

What it actually took

This is the part that made us want to write this post. None of the fixes above is clever. There's no genius insight in "don't comp a tractor against its manual." It is pure, grinding volume — finding each failure against reality, fixing it, and proving the fix didn't break something else:

The vendor graveyard deserves its own line. Most "pricing APIs" turn out to sell catalog data — specs, MSRPs, VIN decodes — which tells you what something cost new, not what it flips for used. eBay's official sold-history API is a limited release that is, in practice, enterprise-only (we've applied; we wait). So real sold data gets assembled the hard way: a paid VIN-level vehicle valuation here, a third-party eBay-sold feed there, an equipment sold-comp pool built from auction archives, the extension's own eBay-sold lookups running in your browser — each source trust-ranked, guarded, and backtested weekly against what lots actually closed for.

And when a number still can't clear the bar, the honest output is the humble one. We once built 13 public "deals" pages powered by the feed's estimates — and unpublished them within days, because the numbers didn't meet the standard we're describing in this post. Our public receipts page runs the other way: it only admits lots priced by trusted sources — no AI estimates allowed on the proof — and its contamination ceiling is strict enough that it throws away half the wins we could brag about. In its first month, 92% of trusted buy-rated lots closed below their estimated resale.

What this means if you flip

Keep the AI browser. Honestly. Let it research brands, summarize a 40-photo listing, draft your eBay description. (An AI helped draft this post — we let it write; we don't let it price.) But when a dollar figure decides your bid, insist on knowing where the number came from:

• Sold, not asking. An asking price is a wish. A sold price is a transaction.
• The item, not its words. Watch for parts, manuals, toys, and look-alikes wearing your item's keywords.
• The lot, not the unit — multiplied the right direction, by the right count.
• Its condition, not the comp's. Stripped vs. loaded, running vs. "non-op."
• Profit, not value. Subtract the premium, tax, shipping — landed cost — before you feel anything.
• A source that can say "unknown." A tool that always has a number is a tool that's sometimes lying.

Next: What a "flip score" really measures: auction price vs. resale value →