[HW3] prob 2.1

[emjay73]

In the referenced example (The annotated transformer), I realize that the number 1 is used as a start symbol.
Is it ok to use the number 1 as the start symbol same as the referenced example, fixing one among 32 tokens as start symbol 1? (or possibly other than the numbers between 0 to 9? like 10?)

[seominjoon]

number 1 used in Transformer is ID, while "1" here is the token string, which can have any ID you want.

[KimSSung]

In 'The annotated transformer', generated batched data looks like this:

1. Are you saying that those yellow colored '1's are Transformer ID?
2. Should I fix the first number of all data like this example?

thanks.

[seominjoon]

1. They are vocab ids, not actual string numbers.
2. You don't have to fix it. Rather, you can use a different vocab id for the string numbers (e.g. you can use 2 for "0", 3 for "1", etc.