Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I'm probably messing something up #3

Open
funtimes-ninja opened this issue Oct 14, 2018 · 8 comments
Open

I'm probably messing something up #3

funtimes-ninja opened this issue Oct 14, 2018 · 8 comments

Comments

@funtimes-ninja
Copy link

funtimes-ninja commented Oct 14, 2018 •
edited
Loading

Hello,

I'd really like to get this project working, but I've seemed to run into an issue.

After cloning the repo, and filling out the config.py I get an error.

$ python get_grades.py
setting up...
logging in...
Traceback (most recent call last):
  File "get_grades.py", line 172, in <module>
    main()
  File "get_grades.py", line 20, in main
    login()
  File "get_grades.py", line 127, in login
    br.select_form(nr=0)  # select the first form
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_mechanize.py", line 668, in select_form
    raise FormNotFoundError("no form matching " + description)
mechanize._mechanize.FormNotFoundError: no form matching nr 0

Any ideas on what I'm doing wrong?
@funtimes-ninja
Copy link
Author

So I fixed my login issue, I didn't have the entire url in the config.py so it was never finding that form.

But now onto the next error.

$ python get_grades.py
setting up...
logging in...
getting grades...
Traceback (most recent call last):
  File "get_grades.py", line 172, in <module>
    main()
  File "get_grades.py", line 26, in main
    class_links = get_class_links()
  File "get_grades.py", line 87, in get_class_links
    page = br.open(get_schedule_page_url())
  File "get_grades.py", line 141, in get_schedule_page_url
    ) + 'portal/portalOutlineWrapper.xsl?x=portal.PortalOutline&contentType=text/xml&lang=en')
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_mechanize.py", line 253, in open
    return self._mech_open(url_or_request, data, timeout=timeout)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_mechanize.py", line 283, in _mech_open
    response = UserAgentBase.open(self, request, data)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_opener.py", line 193, in open
    response = urlopen(self, req, data)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 353, in _open
    '_open', req)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 341, in _call_chain
    result = func(*args)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 1209, in https_open
    return self.do_open(conn_factory, req)
  File "/home/pi/.local/lib/python2.7/site-packages/mechanize/_urllib2_fork.py", line 1156, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno -2] Name or service not known>

@funtimes-ninja
Copy link
Author

So figured out that error, because the school link i'm trying to view has campus in the url title it didn't get the proper url. So for now i've harded the school's url value.

But now onto the next problem.

$ python get_grades.py
setting up...
logging in...
getting grades...
Traceback (most recent call last):
  File "get_grades.py", line 179, in <module>
    main()
  File "get_grades.py", line 34, in main
    class_info = info_boxes[-1]
IndexError: list index out of range

I've thrown in a debug print to try and see what's going on, not sure I quite understand the issue. I also put in a length check to move on as well, but not getting any data written to grades_db.json

            print "INFO_BOXES: " + str(info_boxes)
            if len(info_boxes) == 0:
                continue
            print "INFO_BOXES[-1]: " + str(info_boxes[-1])

Here is what the output looks like

$ python get_grades.py
setting up...
logging in...
getting grades...
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER1
Comments: " class="gridNotPartOfTermGPA">90<br /><span class="grayText">89.86%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER1" class="gridNotPartOfTermGPA">87<br /><span class="grayText">87.3%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER1" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER1" class="gridNotPartOfTermGPA"></a>, <a title="Task:  EOCT/SE - Sem Exam
Teacher: TEACHER1" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task:  EOCT/SE - Sem Exam
Teacher: TEACHER1" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER2
Comments: " class="gridNotPartOfTermGPA">94<br /><span class="grayText">94%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER2" class="gridNotPartOfTermGPA">93<br /><span class="grayText">93.22%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: P3 - Progress 3
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q3 - Report Card
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: P4 - Progress 4
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q4 - Report Card
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER2" class="gridNotPartOfTermGPA">93<br /><span class="grayText">93.22%</span></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task: SA - Sem Avg
Teacher: TEACHER2" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER3
Comments: " class="gridNotPartOfTermGPA">100<br /><span class="grayText">100%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER3
Comments: " class="gridNotPartOfTermGPA">100<br /><span class="grayText">100%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER3" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER3" class="gridNotPartOfTermGPA"></a>, <a title="Task:  EOCT/SE - Sem Exam
Teacher: TEACHER3" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task:  EOCT/SE - Sem Exam
Teacher: TEACHER3" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER4
Comments: " class="gridNotPartOfTermGPA">82<br /><span class="grayText">82%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER4" class="gridNotPartOfTermGPA">83<br /><span class="grayText">83.3%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: P3 - Progress 3
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q3 - Report Card
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: P4 - Progress 4
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q4 - Report Card
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task: SA - Sem Avg
Teacher: TEACHER4" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER5
Comments: " class="gridNotPartOfTermGPA">85<br /><span class="grayText">84.5%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER5" class="gridNotPartOfTermGPA">83<br /><span class="grayText">83.2%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: P3 - Progress 3
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q3 - Report Card
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: P4 - Progress 4
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q4 - Report Card
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER5" class="gridNotPartOfTermGPA">83<br /><span class="grayText">83.2%</span></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task: SA - Sem Avg
Teacher: TEACHER5" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: []
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER6
Comments: " class="gridNotPartOfTermGPA">87<br /><span class="grayText">86.8%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER6
Comments: " class="gridNotPartOfTermGPA">83<br /><span class="grayText">82.55%</span></a>, <a title="Task: P2 - Progress 2
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q2 - Report Card
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: P3 - Progress 3
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q3 - Report Card
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: P4 - Progress 4
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: Q4 - Report Card
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>, <a title="Task: SA - Sem Avg
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task: SA - Sem Avg
Teacher: TEACHER6" class="gridNotPartOfTermGPA"></a>
INFO_BOXES: [<a title="Task: P1 - Progress 1
Teacher: TEACHER7
Comments: Your child is progressing through the course nicely!  Keep working hard.  CodeHS Module 3 due Sept 14.  Vocabulary Check 1 and Skills Check 1 upcoming mid-September.  Please check class website and Remind for updates." class="gridNotPartOfTermGPA">94<br /><span class="grayText">93.57%</span></a>, <a title="Task: Q1 - Report Card
Teacher: TEACHER7" class="gridNotPartOfTermGPA">95<br /><span class="grayText">94.6%</span></a>, <a title="Task: *FG - FINAL GRADE
Teacher: TEACHER7" class="gridNotPartOfTermGPA"></a>]
INFO_BOXES[-1]: <a title="Task: *FG - FINAL GRADE
Teacher: TEACHER7" class="gridNotPartOfTermGPA"></a>

$ ls -la grades_db.json
-rw-r--r-- 1 pi pi 2 Oct 14 18:50 grades_db.json
$ cat grades_db.json
{}

Any ideas?

@aaronjanse
Copy link
Owner

It seems like your school has a different formatting than mine did.

I'd start debugging by printing out the value of page after get_grades.py line 29:

https://github.com/aaronduino/better-ic-grade-viewer/blob/e670cedba1c9bf25d42a2dbe83694618a922667f/get_grades.py#L26-L34

From there, there should be some tutorials online for how to do the parsing.
I'd be happy to help you out if you are able to wait till later this week.

@funtimes-ninja
Copy link
Author

I'll keep attempting to chug along and pound my head against the keyboard. Web stuff if def not my strong suit by any means.. :( But if you'd like to help I would greatly appreciate it, whenever is convenient for you.

@aaronjanse
Copy link
Owner

aaronjanse commented Oct 23, 2018 via email

@funtimes-ninja
Copy link
Author

funtimes-ninja commented Oct 23, 2018 •
edited
Loading

I've managed to get some of the data I needed to scrape properly and have it displaying on a pi3+ RGB Matrix. I'd still really like to get this repo working for my sons school site, just can't quite figure out what/where is going wrong.

@aaronjanse
Copy link
Owner

@funtimes-ninja sounds like a fun project. And I'd love to help.

Since debugging the issue may end up giving away clues regarding where your son goes to school, feel free to privately send me an email with the specifics of the issues you are having (my email is in my github profile).

I'd be happy to help you figure out the issues you are having in particular, fixing the scraping code of this repo to work with your school, getting it to work on the RGB matrix, etc

Cheers

@funtimes-ninja
Copy link
Author

email/hangout sent

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@aaronjanse @funtimes-ninja