Skip to content

Commit

Permalink
Update report.md for ITA 2025
Browse files Browse the repository at this point in the history
  • Loading branch information
lgabs authored Nov 15, 2024
1 parent 5538ba3 commit f15e569
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions exams/ita_2025/report.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ See more details in the [official exam announcemnt](https://vestibular.ita.br/in

## Results

Preliminary solutions can be found by widely known high schools like [Poliedro](https://poliedroresolve.sistemapoliedro.com.br/vestibulares/ita?year=2025). Official solutions will be released soon.
Preliminary solutions can be found by widely known high schools like [Poliedro](https://poliedroresolve.sistemapoliedro.com.br/vestibulares/ita?year=2025) or [Elite](https://www.colegioecursoelite.com.br/confira-o-gabarito-extraoficial-da-2a-fase-do-ita-2024-2025/). Official solutions will be released soon.

| Exam | Phase | Section | Type | Model | Status | Score | PDF Compiled |
|------|-------|---------|------|-------|--------|-------|--------------|
Expand All @@ -40,4 +40,4 @@ For almost of all questions, `max_tokens_question_description=400` and `max_toke

Again, `o1-preview` got all questions correct in the Chemistry essay exam. The only wrong question was question 3, where it got parts (a) and (b) correct, but (c) and (d) wrong. This is probably because parts (c) and (d) involved reasoning about what was represented in the picture of the question, which is hard for LLMs to do, so gpt-4o did some limited description of the image but it was not enough for o1-preview. This is why, for this question, we considered 0.5 points for the score (2 parts out of 4).

`max_tokens_question_answer=5000` worked for most questions; only question 8 could not and so `10000` was used. Also, for some questions the confusion about how to use decimal separators or thousands separators in portuguese or english induced some errors, even when the development was clearly correct. To fix that, we updated the o1-preview prompt to use `,` for decimal separators and avoid thousands separators.
`max_tokens_question_answer=5000` worked for most questions; only question 8 could not and so `10000` was used. Also, for some questions the confusion about how to use decimal separators or thousands separators in portuguese or english induced some errors, even when the development was clearly correct. To fix that, we updated the o1-preview prompt to use `,` for decimal separators and avoid thousands separators.

0 comments on commit f15e569

Please sign in to comment.