Potential integration with Ecto?

[cblage]

I think it would be great if we could use the typedstruct macro in conjunction with Ecto. Right now Ecto does not create @type t for the struct, but it defines other struct-related things which collde with typedstruct.

So maybe add a typedschema macro to integrate with Ecto? Right now, one declares Ecto schemas like this:

defmodule Aruki.Database.Schema.User do
  use Ecto.Schema

  @primary_key {:id, :id, autogenerate: true}

  @timestamps_opts [
    inserted_at: "creation_date",
    updated_at: "status_date",
    type: :utc_datetime,
    usec: false
  ]

  schema "t_user" do
    field :id_type,       :integer
    field :id_status,     :integer
    field :creation_date, :utc_datetime
    field :status_date,   :utc_datetime
    field :login_date,    :utc_datetime
    field :email,         :string
    field :first_name,    :string
    field :last_name,     :string
    field :telephone_nr,  :string
    field :fiscal_id,     :string
    field :reference,     :string
    field :mobileos_id,   :integer
    field :birth_day,     :integer
    field :birth_month,   :integer
    field :birth_year,    :integer
    field :national_id,   :string
    field :country,       :string
    field :rating,        :integer
    field :version,       :integer
  end
end

I suggest supporting something like:

defmodule Aruki.Database.Schema.User do
  use TypedStruct.EctoSchema

  @primary_key {:id, :id, autogenerate: true}

  @timestamps_opts [
    inserted_at: "creation_date",
    updated_at: "status_date",
    type: :utc_datetime,
    usec: false
  ]

  typedschema "t_user" do
    field :id_type,       :integer
    field :id_status,     :integer
    field :creation_date, :utc_datetime
    field :status_date,   :utc_datetime
    field :login_date,    :utc_datetime
    field :email,         :string
    field :first_name,    :string
    field :last_name,     :string
    field :telephone_nr,  :string
    field :fiscal_id,     :string
    field :reference,     :string
    field :mobileos_id,   :integer
    field :birth_day,     :integer
    field :birth_month,   :integer
    field :birth_year,    :integer
    field :national_id,   :string
    field :country,       :string
    field :rating,        :integer
    field :version,       :integer
  end
end

What do you think?

[cblage]

elixir-ecto/ecto#2594
I mean, they've been debating for ages adding it to Ecto, a lot of people would use this.

I can open a PR with a potential patch if you want.

[ejpcmac]

Hi @cblage!

This feature is actually on my roadmap from the beginning. I haven’t implemented yet for two reasons:

1. After a quick thinking about it, it seems there are non-obvious edge cases to handle;
2. Right after I published my Medium article about TypedStruct, Wojtek Mach mentioned it in the PR you mention here. I assumed they would do something directly in Ecto.

As you’ve just stated, they continue to debate and things seem to move very slowly, so why not implementing something here.

A few reflections

Edge cases

You example is quite straightforward: each Ecto type can be mapped to a nullable typespec and other “hidden” fields can be added. However, how would you handle associations? Maybe there are other corner cases too, we must think about them.

Design

• Any Ecto integration should be compiled only if Ecto is in the project. Otherwise it adds useless code. Use of a dedicated module like in your example seems to be the good direction. Your names are good 👍
• It should be easily maintainable. I mean, it shouldn’t contain copy-pasted code from Ecto that could evolve. I don’t exactly recall how schemas are built, I should take a look on this at some point.
• A first version may not add refinements like required: true, but we could add some interesting features in the future.

If you do a PR (on develop), please test your code like for the TypedStruct module. I currently use a private Elixir API to do so, that’s not perfect but it is only for testing purpose. Also, please run Dialyzer on some sample modules to ensure it’s all good. I’ll review it depending on my bandwidth—which can be quite limited these two next weeks since I’m on a few other projects, I’ll do my best!

Last but not least: thank you for helping! ❤️

[cblage]

Well, I have actually decided to make my domain-specific models separate from my DB Schema "models", and so my immediate need for this is no longer there, but I'll still try and see if I could make the PR because it sounds fun to do

[ejpcmac]

@cblage No problem! And yeah, Elixir macros are fun to play with :)

[bamorim]

I'd like to know if there is still interest.
I started to play around with the idea. As you can see here.

I think integrating with Ecto is a whole new thing. This is because there are a lot of ecto-specific things.

I have also changed some of the logic for enforcing/default/etc.

My problem was that I may want to enforce but still enable people to set it to nil.

[ejpcmac]

Hi @bamorim !

Yes, I am still interested in this actually. I have uncommitted code locally about a plugin system you could build upon (see #9, API not contractual), but I am rather slow to work on it for a few months now since I am on other subjects in the mean time—will be on a scientific base with low internet access in Kerguelen Islands from next November, so lots of things to get done before. TypedStruct 0.2.0 with plugins is my the pipeline.

The code I have seems to be functional (I’ve tried it briefly with a Lens integration) and really flexible (more than the current state in the develop branch). I had a train today and wrote quickly some docs, will continue shortly in the times to come. As soon as I have some valid docs / code, I’ll commit so you can see if that interests you.

[bamorim]

@ejpcmac after going through integrating with Ecto (I'm almost finished). I don't think it is possible to create a one-size-fits-all solution.

The reason is that this lib defines it's own DSL while when integrating with ecto, ideally, I just want to add some syntax sugar on top of Ecto's own DSL.

The way I solved this was creating a "struct type builder" that is called by my macro.
That way, we could have just one shared interface:

defmodule MyStruct do
  StructTypeBuilder.__init__()
  StructTypeBuilder.__add_field__(:field_name, String.t(), enforce: true, null: false)
  @enforce_keys StructTypeBuilder.__enforced_fields__()
  defstruct StructTypeBuilder.__fields__()
  StructTypeBuilder.__define_type__()
end

Where in Ecto, instead of calling defstruct, I'd just call Ecto's field/3 everytime I add a new field.

However, I think there is one "core" component to be reused between my Ecto Implementation and a general struct implementation.
The reason is that I can't just say whether or not something will be nil or not based on the enforced thing. For example, when we are talking about associations, so I changed some stuff.

That being said, I think the best is for me to create a new lib focused on Ecto and later we try to see how to fit things together. What do you say?

Have any idea?

[bamorim]

Also, since Ecto.Schema but doesn't only use field for defining fields, instead of defining my own fields, I made a "syntax sugar" that transforms all

field(:my_field, :string) into:

field(:my_field, :string, [])
__add_field__(:my_field, :string, [])

(Of course, with more stuff and options)

[ejpcmac]

@ejpcmac after going through integrating with Ecto (I'm almost finished). I don't think it is possible to create a one-size-fits-all solution. The reason is that this lib defines it's own DSL while when integrating with ecto, ideally, I just want to add some syntax sugar on top of Ecto's own DSL.

You’re right. Ideally a common solution would use the TypedStruct DSL for raw structs, and the Ecto one for Ecto schemas. Don’t know if we can do this properly ATM.

That being said, I think the best is for me to create a new lib focused on Ecto and later we try to see how to fit things together. What do you say?

Let’s try this in a second time, once both TypedStruct 0.2.0 and your library are live. I don’t have any more bandwidth to allow on new projects before November.

[bamorim]

Just FYI, I released the lib:
https://github.com/bamorim/typed_ecto_schema

[tino415]

Today I needed to use Ecto.Changeset on project without database, so I created plugin for it https://github.com/tino415/typed_struct_ecto_changeset, for now I lack motivation to integrate Ecto.Repo but I can imagine integrate embeds (I'm using typed struct for non database data as alternative to Ecto.Schema)