Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lossless parsing for structs and ToTokens implementation. #53

Open
vldm opened this issue Apr 17, 2023 · 0 comments
Open

Lossless parsing for structs and ToTokens implementation. #53

vldm opened this issue Apr 17, 2023 · 0 comments

Comments

@vldm
Copy link

vldm commented Apr 17, 2023

Hi! First of all thanks for a great project!.

In my pet project, i am parsing some context using syn-rsx, and trying to recreate part of token stream as input for other library. To recreate tokens i using quote!()\quote_spanned!() macros and found few problems:

  1. Only few elements implement ToTokens, so i need to implement it manually in my wrappers.
  2. Tokens that i hardcode and tokens from NodeElement semantically have different sites (call_site vs def_site vs mixed_site), and source file info, and if i understand correctly cannot be joined.

What do you think about modifying your implementation, so it will save original token stream inside struct

For example, instead of

pub struct NodeElement {
    /// Name of the element.
    pub name: NodeName,
    /// Attributes of the element node.
    pub attributes: Vec<Node>,
    /// Children of the element node.
    pub children: Vec<Node>,
    /// Source span of the element for error reporting.
    ///
    /// Note: This should cover the entire node in nightly, but is a "close
    /// enough" approximation in stable until [Span::join] is stabilized.
    pub span: Span,
}

You could have:

pub struct StartTag {
    pub less_sign: Token![<],
    pub tag_name: NodeName,
    pub attributes: Vec<Attribute>,
    pub solidus: Option<Token![/]>,
    pub great_sign: Token![>],
}
impl StartTag {
    pub fn is_self_closing(&self) -> bool {
        self.solidus.is_some()
    }
}

pub struct EndTag {
    pub less_sign: Token![<],
    pub solidus: Token![/],
    pub tag_name: NodeName,
    pub great_sign: Token![>],
}

pub struct NodeElement {
     pub start_tag: StartTag,
     pub end_tag: Option<EndTag>,
}

Benefits:

  • The same approach is used in syn library itself. So the code will look more idiomatically.
  • It allows you to implement parser fully on type - TypeDD aproach makes contracts more visible.
  • You can implement syn::Parse just by going and init every field in struct
  • Allows you to implement ToTokens (Spanned will be derived from this method) without introducing mixed site.
  • Also, you don't need to implement your own TryFrom<Struct> for String because TokenStream could be converted to text directly. But you still be able to implement fn safe_html_text for important types

Downside:

  • Its a breaking change.
  • ?Anything else
@vldm vldm mentioned this issue Apr 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant