Evaluation problem

[andypinxinliu]

It seems like doing training is fine, but while doing evaluation, for some of documents it works but it does not work and generate the following error:

====> (nw/wsj/00/wsj_0049); part 000:
File (nw/wsj/00/wsj_0049); part 000:
Entity 0: (0,3) (73,73) (221,222) (237,238) (281,282) (326,326) (402,403) (456,456) (524,525) (578,578) (605,605) (636,637) (645,645) (653,654) (699,701) (770,771)
Entity 1: (5,5) (28,47) (72,74)
Entity 2: (14,26) (62,63) (619,621) (714,716) (718,719) (757,758)
Entity 3: (44,46) (704,706)
Entity 4: (67,70) (228,231)
Entity 5: (77,121) (191,192)
Entity 6: (81,81) (455,458)
Entity 7: (92,95) (130,132)
Entity 8: (113,114) (148,149)
Entity 9: (116,121) (123,124) (154,155) (247,248)
Entity 10: (151,155) (170,171)
Entity 11: (167,167) (874,874) (934,934)
Entity 12: (184,184) (197,197)
Entity 13: (188,194) (453,453) (648,648)
Entity 14: (218,219) (242,244)
Entity 15: (250,251) (261,261)
Entity 16: (266,278) (299,331) (333,334) (367,368) (412,412)
Entity 17: (272,278) (415,419)
Entity 18: (294,295) (382,383) (1738,1739)
Entity 19: (343,352) (354,355) (363,363)
Entity 20: (378,378) (386,386) (398,399) (409,410)
Entity 21: (429,430) (464,465) (1270,1271) (1444,1445) (1918,1919)
Entity 22: (438,439) (542,543) (588,589) (778,779)
Entity 23: (441,445) (469,471) (482,484)
Entity 24: (460,475) (516,517) (519,519) (551,551) (558,558) (575,576)
Entity 25: (489,489) (499,499)
Entity 26: (492,493) (503,504)
Entity 27: (520,520) (548,549)
Entity 28: (527,539) (554,556) (581,584) (685,688) (692,692)
Entity 29: (593,598) (661,665) (674,675) (774,775)
Entity 30: (650,654) (669,669)
Entity 31: (708,708) (712,712)
Entity 32: (730,730) (747,747)
Entity 33: (815,816) (833,841) (846,847) (854,854) (886,886) (893,893) (901,901) (904,904) (917,918)
Entity 34: (862,866) (1026,1030)
Entity 35: (883,888) (911,911)
Entity 36: (924,925) (942,942)
Entity 37: (948,952) (1010,1010) (1032,1032) (1039,1040) (1044,1044) (1053,1053)
Entity 38: (958,958) (962,962)
Entity 39: (966,966) (971,971)
Entity 40: (1065,1066) (1074,1076) (1101,1101) (1123,1124) (1129,1136) (1173,1173) (1176,1177) (1186,1187) (1229,1230) (1232,1234) (1320,1321)
Entity 41: (1065,1068) (1110,1111)
Entity 42: (1079,1084) (1091,1091) (1133,1134) (1157,1158) (1200,1201) (1203,1205) (1247,1249) (1294,1295) (1313,1314)
Entity 43: (1086,1088) (1303,1304)
Entity 44: (1097,1097) (1242,1243)
Entity 45: (1113,1127) (1144,1155) (1183,1183) (1194,1195)
Entity 46: (1160,1167) (1179,1180)
Entity 47: (1173,1174) (1209,1210) (1306,1307)
Entity 48: (1197,1201) (1212,1212)
Entity 49: (1232,1240) (1258,1259) (1273,1283)
Entity 50: (1255,1256) (1291,1295)
Entity 51: (1323,1324) (1332,1356)
Entity 52: (1325,1325) (1331,1331)
Entity 53: (1327,1330) (1436,1445) (1447,1448)
Entity 54: (1344,1345) (1373,1378) (1414,1416)
Entity 55: (1358,1368) (1397,1398)
Entity 56: (1480,1480) (1502,1504)
Entity 57: (1482,1482) (1608,1610)
Entity 58: (1482,1483) (1493,1497) (1506,1506) (1541,1542) (1593,1594) (1605,1605) (1661,1663)
Entity 59: (1485,1485) (1630,1630)
Entity 60: (1489,1491) (1533,1535) (1548,1549) (1556,1556) (1564,1564)
Entity 61: (1506,1510) (1661,1664)
Entity 62: (1578,1591) (1635,1636) (1657,1658)
Entity 63: (1623,1624) (1639,1639)
Entity 64: (1697,1697) (1713,1714) (1742,1752)
Entity 65: (1701,1702) (1706,1714) (1801,1802)
Entity 66: (1721,1721) (1923,1923)
Entity 67: (1742,1759) (1774,1775) (1792,1792)
Entity 68: (1764,1770) (1783,1784)
Entity 69: (1829,1837) (1873,1877)
Entity 70: (1839,1861) (1879,1879) (1890,1894)
====> (nw/wsj/00/wsj_0049); part 000:
Detected the end of a mention 30 without begin (?,1136) at reference-coreference-scorers/lib/CorScorer.pm line 292, line 78226.

Do you know what causes that?

[vdobrovolskii]

Hi!
I'm not sure why this happens.
Could you please share everything I might need to reproduce the error?

[andypinxinliu]

Hi, The files were too large, so I can also share the google drive <https://drive.google.com/drive/folders/1l0xC-_WYRu7C-FYnC2k8BWGA2SdBkBoR?usp=sharing> with you. Best, Andy Liu liu Andy ***@***.***> 于2023年2月10日周五 04:25写道：

…

Hi, I was trying using Roberta-large proposed in your paper, and I also tried deberta-large-v3. For some epochs, the evaluation is unsuccessful. I am using the same training code. I have also attached the conll.py which I looked at the previous responses and issues. I did not modify any other parts of the code. For epoch 12, the evaluation is successful while for epochs 13 -20, the evaluation is unsuccessful. I am not sure what might cause this problem. I would really appreciate it if you are willing to have a look at the attached files. And if you would like to see any others, please let me know. Best, Andy Liu vdobrovolskii ***@***.***> 于2023年2月3日周五 09:28写道： > Hi! > I'm not sure why this happens. > Could you please share everything I might need to reproduce the error? > > — > Reply to this email directly, view it on GitHub > <#40 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AT3IT6LFBGDX6ZKWOZU2YFTWVUIZHANCNFSM6AAAAAAUF3E7GI> > . > You are receiving this because you authored the thread.Message ID: > ***@***.***> >

[vdobrovolskii]

Hi
I'm sorry I disappeared and lost the track of time. Could you kindly confirm seeking the solution to the problem is still relevant to you?