SIG-P4B 2026-08-13

Session Recording Summary · 38m 43s · Full notes ↗

Reading: Not clearly identified in the discussion. This session was a working/scoping conversation rather than a discussion of a shared text. A prior "water project" report is referenced as a comparable prior case study, but no reading was analyzed here.

This was a one-on-one working session between rafa and Andre Comeau to scope a potential "Protocols for Business" case study focused on the construction bidding/estimating pipeline. rafa presented findings from experiments run with his father-in-law (Virgil), a former heavy-civil construction CEO, showing that an LLM could reproduce a bid estimate within ~10% and even flag design errors from solicitation PDFs. The conversation examined why this analysis isn't already automated, identified the core bottleneck (canonical drawings live in non-machine-readable PDFs), and mapped the industry incentive structure that keeps the status quo in place. They ended by discussing how to structure the project, funding, and roles.

• **The experiment (rafa):** rafa fed a solicitation bid PDF to an LLM (referenced as "Opus five or whatever") and, in ~30 min of runtime (~1 hour with probing/adversarial analysis and blind agent reviews), produced numbers within 10% of a full organizational bid produced by Virgil's team. He also had the model generate a "takeoff analysis." Andre confirmed "takeoff analysis" is a real, standard industry term (not just Michigan-specific).
• **Takeoff analysis defined:** A table comparing the design engineer's requested materials vs. the actual planned/likely materials, and both against the contractor's read of the visual plans and real-world conditions.
• **Where value really sits (rafa relaying Virgil):** The final bid number is largely deterministic (driven by subcontractor and material/equipment prices where risk is passed to others). The estimator's real leverage is in the accuracy of quantities and pricing for maximum profit opportunity.
• **Finding design errors = profit (rafa):** When given a different spec, the model found a design error. This is commercially valuable because contractors often make money via change orders when they catch design errors.
• **The four core questions rafa raised:** (1) Why isn't someone in the office already running PDFs through ChatGPT? (2) Why doesn't existing bidding software do this analysis? (3) Why doesn't solicitation-bid software (e.g. BidNet) do it? (4) Why is this analysis done off PDFs instead of raw CAD data?
• **The root-cause diagnosis (rafa):** The real fix is upstream — getting machine-readable (vector) data from design firms rather than parsing PDFs. LLM PDF-parsing is viable only as an MVP / quality-control suggestion tool. rafa argued you can process a PDF faster than a $200/hr estimator to get a starting point, but you can't *remove* the estimator until you have vector-format drawings, because confidence won't be high enough. He also tied this to autonomous robot deployment: you likely can't get there without fixing the drawing/information pipeline first.

Questions & Disagreements: - **Is this real / does the MVP work?** rafa flagged the MVP might not pan out if LLM reliability is too low to remove the estimator. Andre said the case "looks great" but, being "heavier on the data side," he couldn't personally judge whether it's "real or not," while noting the production value/professionalism was high. - **Can you actually get the CAD/drawings?** rafa's specific ask — is there

Participants: rafa (UTC+1), Andre Comeau