Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bulk scan dies when one process raises an uncaught exception #25

Open
krayzpipes opened this issue Aug 1, 2019 · 1 comment
Open

Bulk scan dies when one process raises an uncaught exception #25

krayzpipes opened this issue Aug 1, 2019 · 1 comment
Labels
bug Something isn't working

Comments

@krayzpipes
Copy link
Owner

When a subprocess throws an uncaught exception, the entire bulk scan stops instead of just skipping and letting the others finish.

Traceback (most recent call last):
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "/home/some_user/venv/lib64/python3.6/site-packages/txtferret/cli.py", line 69, in bootstrap
    ferret.scan_file()
  File "/home/some_user/venv/lib64/python3.6/site-packages/txtferret/core.py", line 315, in scan_file
    for index, line in enumerate(rf):
  File "/usr/lib64/python3.6/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position 1471: invalid continuation byte
"""

Traceback (most recent call last):
  File "/home/some_user/venv/bin/txtferret", line 11, in <module>
    sys.exit(main())
  File "/home/some_user/venv/lib64/python3.6/site-packages/txtferret/__init__.py", line 7, in main
    exit(cli())
  File "/home/some_user/venv/lib64/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/home/some_user/venv/lib64/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/home/some_user/venv/lib64/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/some_user/venv/lib64/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/some_user/venv/lib64/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/home/some_user/venv/lib64/python3.6/site-packages/txtferret/cli.py", line 190, in scan
    results = p.map(bootstrap, configs)
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 266, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/usr/lib64/python3.6/multiprocessing/pool.py", line 644, in get
    raise self._value
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position 1471: invalid continuation byte

Need to fix the UnicodeDecoding issues in another issue.

@krayzpipes krayzpipes added the bug Something isn't working label Aug 1, 2019
@krayzpipes
Copy link
Owner Author

Codec / encoding support addressed in Issue #26

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant