About the AI model

Potholes, overflowing bins, illegal dumping, clogged drains and broken public assets often go unreported or are not prioritized. Photos are easy for citizens to capture, so visual analysis can turn them into structured, triaged reports.

These problems are visible. A vision model can recognize them from pixels in seconds, consistently, and at the scale of a whole city — without the reporter needing technical vocabulary.

Inference uses openai/gpt-6-astra, a pretrained multimodal vision-language model (a transformer that encodes the image into visual tokens and reasons over them with text), accessed server-side through the Lovable AI Gateway. API credentials never reach the browser.

It is used zero-shot: it has not been fine-tuned or trained on a civic dataset by this project. The model returns structured JSON validated against a strict schema: category, self-reported confidence, image quality, visual evidence, severity indicators and approximate bounding boxes.

The inference module is isolated so a dedicated detector (e.g. a fine-tuned YOLO or segmentation model) can replace it while keeping the same output contract.

• Pothole / damaged road
• Overflowing garbage / illegal dumping
• Blocked or garbage-filled drain
• Damaged public infrastructure
• Plus two non-issue outputs: no issue and uncertain

No project-specific labeled dataset has been used.

Files are checked for type (JPG/PNG/WebP) and size (≤ 8 MB), decoded in the browser, downscaled so the longest side is at most 1600 px, and re-encoded as JPEG. The image is sent to the server as base64 for analysis. Saved reports and their photos remain in this browser, not in a cloud account.

The server sends the image with a fixed inspection prompt. The model reports visible indicators; severity is then computed by a documented rule in code, not by the model:

score = extent (small 0 · moderate 1 · large 2)
      + 2 if visible safety hazard
      + 1 if it blocks road / footpath / drain flow
      + 1 if visible health risk
0–1 Low · 2–3 Medium · 4 High · 5–6 Critical

A prediction is flagged for manual review when the category is uncertain, self-reported confidence is below 60%, or image quality is poor.

Not yet evaluated. No accuracy, precision, recall or F1 figures are claimed. To evaluate, collect a held-out labeled test set (e.g. 100+ images per class), run the pipeline, and compute per-class precision = TP/(TP+FP), recall = TP/(TP+FN) and F1 = 2·P·R/(P+R).

• Confidence is the model's self-reported estimate, not a calibrated probability.
• Bounding boxes are approximate; this is not a dedicated object detector.
• False positives: shadows, puddles or road markings mistaken for potholes; tidy bagged waste mistaken for dumping.
• False negatives: small, distant, night-time or partially occluded issues.
• Severity cannot measure true depth, size or legal priority from a single photo.

Collect geo-tagged, labeled images (e.g. public pothole and litter datasets plus local photos), annotate boxes in a tool like CVAT or Label Studio, fine-tune a detector such as YOLOv8/v11 on the four classes, evaluate on a held-out split, and host it behind an HTTPS endpoint. Then point the server inference module at it. Reports flagged for review and later corrected can become new training labels.