The need
Accident reports arrive as free text, in different languages, with fields often missing. Insurers need that information as structured data. The real question is which model does this reliably, including when it has to admit that a value is missing.
What I owned
The pipeline, the prompts, the model integration and the comparison, in a team of two.
The solution
Input
Accident report PDF
Processing
Text extractionStrict system prompt
Models
Gemini 2.5 FlashCroissantLLMChatGemma 2 2B
Output
JSON parsing and validationOne saved file per report
How it was built
Each model faced three scenarios: an English report with all fields, a French report with all fields, and an English report with several fields missing.
The first open-source baseline, CroissantLLM, returned template-based outputs that did not reflect the reports, so we replaced it with Gemma. We also removed all example values from the system prompt, so the model could not copy them.
Results
| Scenario | Gemini 2.5 Flash | CroissantLLMChat | Gemma 2 2B |
|---|---|---|---|
| English, all fields | 8.3 s | 20.0 s | 12.3 s |
| French, all fields | 5.5 s | 2.1 s | 12.7 s |
| English, missing fields | 5.9 s | 10.2 s | 9.9 s |
- Gemini 2.5 Flash gave the most accurate and consistent extractions and was the fastest overall.
- Gemma 2 2B proved a solid open-source option, with correct JSON and controlled handling of missing fields.
- CroissantLLMChat did not meet the requirements, and its fast French time came with wrong output.