Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use a simple dictionary-based compression on the filenames #37

Open
ShaheedHaque opened this issue Sep 28, 2018 · 2 comments
Open

Use a simple dictionary-based compression on the filenames #37

ShaheedHaque opened this issue Sep 28, 2018 · 2 comments

Comments

@ShaheedHaque
Copy link

Thanks for this library. My interest in it is in sending exceptions across a Celery-to-parent multiprocess/billard process boundary, and also in storing the result in the Celery result backend. Some of this is moderately expensive/limited and it would be very useful to reduce the data involved. Given that a typical to_dict() representation is dominated by the filenames:

 'tb_frame': {
    'f_globals': {
      '__file__': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
      '__name__': '?'
    },
    'f_code': {
      'co_filename': '/main/srhaque/redacted/testdjango/redacted/tasks.py',
      'co_name': 'function_run'
    }
  },
  'tb_lineno': 601,
  'tb_next': {

I implemented a simple dict-based "compression" scheme that eliminates the redundant filenames. Would an API-compatible patch to to_dict()/from_dict() for this purpose be of interest? If so, I'd be happy to provide a PR.

@ionelmc
Copy link
Owner

ionelmc commented Sep 28, 2018

So my concern is that if we have this in tblib it would draw people to make it more and more efficient (and more complicated at the same time). I would rather have this sort of code as a documentation/cookbook example, provided it's not very big.

@ShaheedHaque
Copy link
Author

ShaheedHaque commented Sep 29, 2018

I see. I'm thinking the change is not very big, but it is intrusive. Perhaps I should make a PR, and then you can see what you think either about taking it, or not...at least its available if needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants