From 59f9d4b1cb81223f56db5f96c6ce126001d4ae06 Mon Sep 17 00:00:00 2001 From: pedrobslisboa Date: Wed, 11 Sep 2024 15:44:48 +0100 Subject: [PATCH 1/3] feat: improve extender and attributes explanation --- examples/1 - AST/README.md | 26 ++++++++++++++++--- .../a - Context Free/README.md | 12 ++++++--- 2 files changed, 30 insertions(+), 8 deletions(-) diff --git a/examples/1 - AST/README.md b/examples/1 - AST/README.md index d61a870..f46d528 100644 --- a/examples/1 - AST/README.md +++ b/examples/1 - AST/README.md @@ -223,14 +223,32 @@ I'm not going to be able to cover all structure items, but you can find more abo As the AST represents the structure of the source code in a tree-like format, it also represents the Extension nodes and Attributes. It is mostly from the extension and attributes that the PPXs are built, so it's important to understand that they are part of the AST and have their own structure. -- **Extension nodes** are generic placeholders in the syntax tree. They are rejected by the type-checker and are intended to be “expanded” by external tools such as -ppx rewriters. On AST, it is represented as `string Ast_414.Asttypes.loc * payload`. +- Extension nodes are generic placeholders in the syntax tree. They are rejected by the type-checker and are intended to be “expanded” by external tools such as -ppx rewriters. On AST, it is represented as `string Ast_414.Asttypes.loc * payload`. - For example, in the code `let name = [%name "John Doe"]`, `[%name "John Doe"]` is the extension node, where **name** is the extension name (`string Ast_414.Asttypes.loc`) and **"John Doe"** is the `payload`. + So, as extension nodes are placeholders for a code to be added, adding a new extension node with no extender declared should break the compilation. For example, in the code `let name = [%name "John Doe"]`. See a demo [here](https://sketch.sh/s/6DxhTCXYpOkI0G8k9keD0d/) -- **Attributes** are “decorations” of the syntax tree, which are mostly ignored by the type-checker but can be used by external tools. Decorators must be attached to a specific node in the syntax tree. (Check it breaking on this [AST sample](https://astexplorer.net/#/gist/c2f77c38bd5b855775e7ea6513230775/95bbbedaf54dd6daadb278b9d5ed7b28718331f2)) + There are 2 forms of extension nodes: - For example, in the code `let name = "John Doe" [@print expr]`, `[@print expr]` is the attribute of the `"John Doe"` node, where **print** is the attribute name (`string Ast_414.Asttypes.loc`) and **expr** is the `payload`. To be an attribute of the entire item `let name = "John Doe"`, you must use `@@`: `[@@print]`. + - **For “algebraic” categories**: `[%name "John Doe"]` + - **For structures and signatures**: `[%%name "John Doe"]` + > In the code `let name = [%name "John Doe"]`, `[%name "John Doe"]` is the extension node, where **name** is the extension name (`string Ast_414.Asttypes.loc`) and **"John Doe"** is the `payload`. For the entire item `let name = "John Doe"`, you must use `%%`: `[%%name "John Doe]`. + + Don't worry much about creating a new extension node; we'll cover it in the [Wrinting PPXs section](../2%20-%20Writing%20PPXs/README.md). + +- Attributes are “decorations” of the syntax tree, which are mostly ignored by the type-checker but can be used by external tools. Decorators must be attached to a specific node in the syntax tree. (Check it breaking on this [AST sample](https://astexplorer.net/#/gist/c2f77c38bd5b855775e7ea6513230775/95bbbedaf54dd6daadb278b9d5ed7b28718331f2)) + + As attributes are just “decorations”, you can add a new attribute without breaking the compilation. For example, in the code, `let name = "John Doe" [@print]`. See a demo [here](https://sketch.sh/s/6DxhTCXYpOkI0G8k9keD0d/) + + There are 3 forms of attributes: + + - **Attached to on “algebraic” categories**: `[@name]` + - **Attached to “blocks”**: `[@@name]` + - **Stand-alone of signatures or structures modules**: `[@@@name]` + + > In the code `let name = "John Doe" [@print expr]`, `[@print expr]` is the attribute of the `"John Doe"` node, where **print** is the attribute name (`string Ast_414.Asttypes.loc`) and **expr** is the `payload`. To be an attribute of the entire item `let name = "John Doe"`, you must use `@@`: `[@@print]`. If it is an stand-alone attribute of a module, you must use `@@@`: `[@@@print]`. + + Don't worry much about creating a new attributes node; we'll cover it in the [Wrinting PPXs section](../2%20-%20Writing%20PPXs/README.md).
I know that it can be a lot, but don't worry; we are going step by step, and you are going to understand it. diff --git a/examples/2 - Writing PPXs/a - Context Free/README.md b/examples/2 - Writing PPXs/a - Context Free/README.md index 0fa10ad..f976e03 100644 --- a/examples/2 - Writing PPXs/a - Context Free/README.md +++ b/examples/2 - Writing PPXs/a - Context Free/README.md @@ -30,9 +30,12 @@ There are two main types of context-free transformations: - **[Derivers](#derivers)**: These append code after the item without changing the original item. ## Extenders -:page_facing_up: [Doc](https://ocaml-ppx.github.io/ppxlib/ppxlib/driver.html#def_extenders) +:page_facing_up: [Doc](https://ocaml-ppx.github.io/ppxlib/ppxlib/driver.html#def_extenders)
+⬅️ Extenders work with extension nodes. If you have any doubts about attributes, please review the [AST Extension Node section](../../1%20-%20AST/README.md#ast_extension_node). -Extenders allow you to replace an extension node with new content. However, they do not have direct access to the surrounding code context, so they cannot change the surrounding code. +Extenders allow you to replace an extension node with new content. However, they do not have direct access to the surrounding code context, so they cannot modify the surrounding code. + +If an extender is broken or missing, the code will not compile. Therefore, it is important to ensure that the extender is correctly implemented. An extension node is a node in the AST that represents an extension point. For example, in the code `let x = [%foo]`, `[%foo]` is an extension node. @@ -169,9 +172,10 @@ let grin = [%emoji "grin"] --- ## Derivers -:page_facing_up: [Doc](https://ocaml-ppx.github.io/ppxlib/ppxlib/driver.html#def_derivers) +:page_facing_up: [Doc](https://ocaml-ppx.github.io/ppxlib/ppxlib/driver.html#def_derivers)
+⬅️ A deriver is a custom attribute provided by PPXlib. If you have any doubts about attributes, please review the AST Attributes section. -Derivers are different from extenders in that they append new code after an existing item rather than replacing parts of it. The new code works in conjunction with the original item, or independently, depending on the transformation needed. +Derivers differ from extenders in that they append new code after an existing item rather than replacing parts of it. The new code can work in conjunction with the original item or independently, depending on the transformation needed. They are specified using the [@@deriving] attribute. A simple and common example of a deriver is the `enum` deriver: From 0667941ff4579cbcc4686bdba18cc3c084ddcf55 Mon Sep 17 00:00:00 2001 From: pedrobslisboa Date: Wed, 11 Sep 2024 15:50:35 +0100 Subject: [PATCH 2/3] feat: add more details of why learn ast --- examples/1 - AST/README.md | 49 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/examples/1 - AST/README.md b/examples/1 - AST/README.md index f46d528..d58f21c 100644 --- a/examples/1 - AST/README.md +++ b/examples/1 - AST/README.md @@ -40,6 +40,55 @@ OCaml's Parsetree can be confusing, verbose, and hard to understand, but it's a You don't need to be an expert on it knowing all the tree possibilities, but you should know how to read it. For this, I'm going to use the [AST Explorer](https://astexplorer.net/) throughout the repository to help you understand the AST. +A simple example of learning more about the OCaml compiler is that types are recursive by default, while values are non-recursive. +With the AST, we can see this clearly: +```ocaml +type name = string +let name = "John Doe" +``` +```json5 +// AST Tree +{ + "type": "structure", + "structure": [ + // type name = string + { + "type": "structure_item", + "pstr_desc": { + "type": "Pstr_type", + "rec_flag": { + "type": "Recursive" + }, + "type_declarations": [ + { + "type": "type_declaration", + "ptype_name": {/* ... */}, + } + ] + }, + "pstr_loc": {/* ... */}, + }, + // let name = "John Doe" + { + "type": "structure_item", + "pstr_desc": { + "type": "Pstr_value", + "rec_flag": { + "type": "Nonrecursive" + }, + "value_bindings": [ + { + "type": "value_binding", + "pvb_pat": {/* ... */}, + } + ] + }, + "pstr_loc": {/* ... */}, + } + ] +} +``` + ### First Look By comparing code snippets with their AST representations, you'll better understand how OCaml interprets your code, which is essential for working with PPXs or delving into the compiler's internals. The [AST Explorer](https://astexplorer.net/) tool will help make these concepts clearer and more accessible. From e73807088d5ee985c96993577c0b266d6111604b Mon Sep 17 00:00:00 2001 From: pedrobslisboa Date: Wed, 11 Sep 2024 16:11:27 +0100 Subject: [PATCH 3/3] feat: add spaces between callouts --- examples/1 - AST/README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/examples/1 - AST/README.md b/examples/1 - AST/README.md index d58f21c..51d26dc 100644 --- a/examples/1 - AST/README.md +++ b/examples/1 - AST/README.md @@ -279,7 +279,8 @@ As the AST represents the structure of the source code in a tree-like format, it There are 2 forms of extension nodes: - **For “algebraic” categories**: `[%name "John Doe"]` - - **For structures and signatures**: `[%%name "John Doe"]` + - **For structures and signatures**: `[%%name "John Doe"]` +
> In the code `let name = [%name "John Doe"]`, `[%name "John Doe"]` is the extension node, where **name** is the extension name (`string Ast_414.Asttypes.loc`) and **"John Doe"** is the `payload`. For the entire item `let name = "John Doe"`, you must use `%%`: `[%%name "John Doe]`. @@ -294,6 +295,7 @@ As the AST represents the structure of the source code in a tree-like format, it - **Attached to on “algebraic” categories**: `[@name]` - **Attached to “blocks”**: `[@@name]` - **Stand-alone of signatures or structures modules**: `[@@@name]` +
> In the code `let name = "John Doe" [@print expr]`, `[@print expr]` is the attribute of the `"John Doe"` node, where **print** is the attribute name (`string Ast_414.Asttypes.loc`) and **expr** is the `payload`. To be an attribute of the entire item `let name = "John Doe"`, you must use `@@`: `[@@print]`. If it is an stand-alone attribute of a module, you must use `@@@`: `[@@@print]`.