MDX v3 - Upgrade Support

[slorber]

Docusaurus v3 will use MDX v3.

This is a breaking change that might require you to adjust your docs.

This GitHub discussion is a place to ask for help in case our migration notes are not enough, and you can't find a solution.

Other useful support links

• Docusaurus v3 - Upgrade Support
• Docusaurus v3 - Discord channel #migration-v2-to-v3
• MDX v3 - Upgrade Support
• MDX v3 - Remark/Rehype Plugins Support
• MDX v3 - Discord channel #migration-mdx-v3

Before asking for support

Before asking for support, make sure you:

• have read carefully the Docusaurus MDX v2 PR instructions and understood the various compatibility options we provide
• have read carefully the official MDX v2 and MDX v3 release blog posts
• have read carefully the MDX v2 and MDX v3 migration guides
• have tested your Markdown code in the MDX playground: this might help you figure out the problem on your own

To ask for support, please:

• Check if someone else already asked a similar question
• Create a minimal repro using docusaurus.new
• Give us the fs path of the problematic Markdown file, and the URL pathname to access it
• Spend some time explaining your problem clearly in a structured and concise way
• Paste full, clear error messages and provide screenshots if appropriate

It's not scalable to provide done-for-you support for all Docusaurus users looking to upgrade. Any support request not respecting those constraints will possibly be ignored.

If you want help, you need to invest your own time so that helping you is easy.

Alternatively, you can look for a paid Docusaurus Service Provider to help you.

[ntucker]

Is there a script to easily rename all our .md files to .mdx since this is required in the upgrade? This will also need to rename all links since docusaurus has recommended we use filenames for links

[ntucker]

Or perhaps an option to disable commonmark completely?

[slorber]

Renaming all files from .md to .mdx is not required for the upgrade.

But it is recommended to use the .mdx extension when you use MDX / JSX.
Not by Docusaurus, but by the MDX creator.
Using the correct extension allows related tools to work better (IDE, Prettier, ESLint, syntax highlighting, GitHub preview...)

As part of #9097, there is now an option to customize how Docusaurus process md/mdx files by default. For better compatibility with Docusaurus v2, Docusaurus v3 will parse everything with MDX by default, but you can opt-in for CommonMark parsing.

You can opt-in for CommonMark parsing by using siteConfig.markdown.format: 'md' | 'mdx' | 'detect', and on a per-file basis using format: 'md' | 'mdx' | 'detect'.

---

There is no automated script to migrate extensions to the .mdx, but that could be nice if someone in the community worked on one.

I think I have used that locally for the Docusaurus site itself. No guarantee it will work for you, but you can use it as an inspiration

CLI to rename files:

find website/docs -type f -name "*.md" -exec rename 's/\.md$/.mdx/' '{}' \;`

IDE/Intellij replaceAll regexp to replace markdown link paths:

\(https[^ ]*?\.mdx(#[^ ]+)?\)


From: \.md(?<hash>#(.*?))?\)
To: .mdx${hash})


From: \((?<url>(?!http)[^ ]*?)(?<ext>\.md)(?<hash>#[^ ]+)?\)
To: (${url}.mdx${hash})


Find .md refs in docs: \.md[^ x)#]

Again these are rough notes that I took while doing the migration and are just an inspiration. If someone can come up with proper instructions that work for all OS/IDEs that can be nice. I will try to see if I can write a process myself.

[ntucker]

Anyone get

ERROR in ../docs/core/concepts/normalization.md
Module build failed (from ./node_modules/@docusaurus/mdx-loader/lib/index.js):
Error: MDX compilation failed for file "/home/ntucker/src/rest-hooks/docs/core/concepts/normalization.md"
Cause: chunks[startIndex].slice is not a function
Details:
{}
    at Object.mdxLoader (/home/ntucker/src/rest-hooks/website/node_modules/@docusaurus/mdx-loader/lib/loader.js:111:25)

normalization.md

---

title: Entity and Data Normalization
sidebar_label: Data Normalization

import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
import LanguageTabs from '@site/src/components/LanguageTabs';

Entities have a primary key. This enables easy access via a lookup table.
This makes it easy to find, update, create, or delete the same data - no matter what
endpoint it was used in.

<Tabs
defaultValue="State"
values={[
{ label: 'State', value: 'State' },
{ label: 'Response', value: 'Response' },
{ label: 'Endpoint', value: 'Endpoint' },
{ label: 'Entity', value: 'Entity' },
{ label: 'React', value: 'React' },
]}>

[
  { "id": 1, "title": "this is an entity" },
  { "id": 2, "title": "this is the second entity" }
]

const PresentationList = new Endpoint(
  () => fetch(`/presentations`).then(res => res.json()),
  { schema: [PresentationEntity] },
);

class PresentationEntity extends Entity {
  readonly id: string = '';
  readonly title: string = '';

  pk() {
    return this.id;
  }
}

export function PresentationsPage() {
  const presentation = useSuspense(PresentationList, {});
  return presentation.map(presentation => (
    <div key={presentation.pk()}>{presentation.title}</div>
  ));
}

Extracting entities from a response is known as normalization. Accessing a response reverses
the process via denormalization.

:::info Global Referential Equality

Using entities expands Rest Hooks' global referential equality guarantee beyond the granularity of
an entire endpoint response.

:::

Mutations and Dynamic Data

When an endpoint changes data, this is known as a side effect. Marking an endpoint with sideEffect: true
tells Rest Hooks that this endpoint is not idempotent, and thus should not be allowed in hooks
that may call the endpoint an arbitrary number of times like useSuspense() or useFetch()

By including the changed data in the endpoint's response, Rest Hooks is able to able to update
any entities it extracts by specifying the schema.

<Tabs
defaultValue="Create"
values={[
{ label: 'Create', value: 'Create' },
{ label: 'Update', value: 'Update' },
{ label: 'Delete', value: 'Delete' },
]}>

import { RestEndpoint } from '@rest-hooks/rest';

const todoCreate = new RestEndpoint({
  urlPrefix: 'https://jsonplaceholder.typicode.com',
  path: '/todos',
  method: 'POST',
  schema: Todo,
});

Example Usage

import { useController } from '@rest-hooks/react';

export default function NewTodoForm() {
  const ctrl = useController();
  return (
    <Form onSubmit={e => ctrl.fetch(todoCreate, new FormData(e.target))}>
      <FormField name="title" />
    </Form>
  );
}

import { RestEndpoint } from '@rest-hooks/rest';

const todoUpdate = new RestEndpoint({
  urlPrefix: 'https://jsonplaceholder.typicode.com',
  path: '/todos/:id',
  method: 'PUT',
  schema: Todo,
});

Example Usage

import { useController } from '@rest-hooks/react';

export default function UpdateTodoForm({ id }: { id: number }) {
  const todo = useSuspense(todoDetail, { id });
  const ctrl = useController();
  return (
    <Form
      onSubmit={e => ctrl.fetch(todoUpdate, { id }, new FormData(e.target))}
      initialValues={todo}
    >
      <FormField name="title" />
    </Form>
  );
}

import { schema, RestEndpoint } from '@rest-hooks/rest';

const todoDelete = new RestEndpoint({
  urlPrefix: 'https://jsonplaceholder.typicode.com',
  path: '/todos/:id',
  method: 'DELETE',
  schema: new schema.Delete(Todo),
});

Example Usage

import { useController } from '@rest-hooks/react';

export default function TodoWithDelete({ todo }: { todo: Todo }) {
  const ctrl = useController();
  return (
    <div>
      {todo.title}
      <button onClick={() => ctrl.fetch(todoDelete, { id: todo.id })}>
        Delete
      </button>
    </div>
  );
}

:::info

Mutations automatically update the normalized cache, resulting in consistent and fresh data.

:::

Schema

Schemas are a declarative definition of how to process responses

• where to expect Entities
• Classes to deserialize fields

import { RestEndpoint } from '@rest-hooks/rest';

const getTodoList = new RestEndpoint({
  urlPrefix: 'https://jsonplaceholder.typicode.com',
  path: '/todos',
  // highlight-next-line
  schema: [Todo],
});

Placing our Entity Todo in an array, tells Rest Hooks to expect
an array of Todos.

Aside from array, there are a few more 'schemas' provided for various patterns. The first two (Object and Array)
have shorthands of using object and array literals.

• Object: maps with known keys
• Array: variably sized arrays
• Union: select from many different types
• Values: maps with any keys - variably sized
• Invalidate: remove an entity

Learn more

Nesting

Additionally, Entities themselves can specify nested schemas
by specifying a static schema member.

<Tabs
defaultValue="Entity"
values={[
{ label: 'Entity', value: 'Entity' },
{ label: 'Response', value: 'Response' },
]}>

import { Entity } from '@rest-hooks/endpoint';

class Todo extends Entity {
  readonly id: number = 0;
  readonly user: User = User.fromJS({});
  readonly title: string = '';
  readonly completed: boolean = false;

  pk() {
    return `${this.id}`;
  }

  // highlight-start
  static schema = {
    user: User,
  };
  // highlight-end
}

class User extends Entity {
  readonly id: number = 0;
  readonly username: string = '';

  pk() {
    return `${this.id}`;
  }
}

{
  "id": 5,
  "user": {
    "id": 10,
    "username": "bob"
  },
  "title": "Write some Entities",
  "completed": false
}

Learn more

Data Representations

Additionally, any newable class that has a toJSON() method, can be used as a schema. This will simply construct the object during denormalization.
This might be useful with representations like bignumber

import { Entity } from '@rest-hooks/endpoint';

class Todo extends Entity {
  readonly id: number = 0;
  readonly user: User = User.fromJS({});
  readonly title: string = '';
  readonly completed: boolean = false;
  // highlight-next-line
  readonly dueDate: Date = new Date(0);

  pk() {
    return `${this.id}`;
  }

  static schema = {
    user: User,
    // highlight-next-line
    dueDate: Date,
  };
}

:::info

Due to the global referential equality guarantee - construction of members only occurs once
per update.

:::

with the canary build?

Is this due to it using commonmark?

[slorber]

This is a bug with multi-line HTML comments.

Reported here: #9084

Will be fixed soon, and there's a temporary workaround
Edit: now fixed in latest canary

[johnnyreilly]

Just linking in this issue here: #9081 (comment)

I'd love to see a guide on how to migrate remark / rehype plugins to Docusaurus v3 / MDX v2

[slorber]

It's not super easy to write a generic guide for that.

You have to read the MDX v2 docs and understand how the whole Unified ecosystem works.

I created a dedicated issue to help Remark/Rehype plugin authors, with some useful notes to put you on track.

MDX v2 - Remark/Rehype Plugins Support

[ntucker]

Error: MDX compilation failed for file "/home/ntucker/src/rest-hooks/docs/graphql/api/validateRequired.md"
Cause: Expected a closing tag for `</summary>` (57:1-57:20) before the end of `paragraph`
Details:
{
  "name": "57:1-57:55",
  "message": "Expected a closing tag for `</summary>` (57:1-57:20) before the end of `paragraph`",
  "reason": "Expected a closing tag for `</summary>` (57:1-57:20) before the end of `paragraph`",
  "line": 57,
  "column": 1,
  "position": {
    "start": {
      "line": 57,
      "column": 1,
      "offset": 1389,
      "_index": 0,
      "_bufferIndex": 0
    },
    "end": {
      "line": 57,
      "column": 55,
      "offset": 1443,
      "_index": 1,
      "_bufferIndex": -1
    }
  },
  "source": "mdast-util-mdx-jsx",
  "ruleId": "end-tag-mismatch"
}

However the only 'summary' shows up as <details collapsed><summary><b>exclude()</b></summary>. There is no paragraph and it is correctly closed...

Also this is on line 61...so the line numbers are off?

Site: https://github.com/data-client/rest-hooks in /website (using 'canary' to replace existing version of docusaurus)

Full file below

validateRequired.md

---

title: validateRequired

function validateRequired(processedEntity: any, requiredDefaults: Record<string, unknown>): string | undefined;

Returns a string message if any keys of requiredDefaults are missing in processedEntity. This
can be used to validate fields that must be provided.

class CustomBaseEntity extends Entity {
  static validate(processedEntity) {
    return validateRequired(processedEntity, this.defaults) || super.validate(processedEntity);
  }
}

Partial/full results

This can be useful to automatically validate for partial results

class SummaryAnalysis extends Entity {
  readonly id: string = '';
  readonly createdAt: Date = new Date(0);
  readonly meanValue: number = 0;
  readonly title: string = '';

  pk() {
    return this.id;
  }
}

class FullAnalysis extends SummaryAnalysis {
  readonly graph: number[] = [];
}

Optional fields

In case we have a field that won't always be present (like lastRun here), we can simply
'exclude' it from the fields we require.

class FullAnalysis extends SummaryAnalysis {
  readonly graph: number[] = [];
  readonly lastRun?: Date = new Date(0);

  static schema = {
    lastRun: Date,
  }

  static validate(processedEntity) {
    return validateRequired(processedEntity, exclude(this.defaults, ['lastRun']));
  }
}

exclude()

function exclude<O extends Record<string, unknown>>(
  obj: O,
  keys: string[],
): Partial<O> {
  const r: any = {};
  Object.keys(obj).forEach(k => {
    if (!keys.includes(k)) r[k] = obj[k];
  });
  return r;
}

Full results only have optional fields

In case every field of the 'full' resource was optional:

class FullAnalysis extends SummaryAnalysis {
  readonly graph?: number[] = [];
  readonly lastRun?: Date = new Date(0);

  static schema = {
    lastRun: Date,
  }

  static validate(processedEntity) {
    return validateRequired(processedEntity, exclude(this.defaults, ['graph', 'lastRun']));
  }
}

This code would not successfully know to fetch the 'full' resource if the summary is already provided.
There would be no way of knowing whether the fields simply don't exist for that data, or were not fetched.
In this case, it is best to provide a null default for at least one field.

class FullAnalysis extends SummaryAnalysis {
  readonly graph: number[] = null;
  readonly lastRun?: Date = new Date(0);

  static schema = {
    lastRun: Date,
  }

  static validate(processedEntity) {
    return validateRequired(processedEntity, exclude(this.defaults, ['lastRun']));
  }
}

This enables the client to understand whether the 'full' resource has been fetched at all.

[ntucker]

I reduced the entire file to just the details element and it still complained:

<details collapsed><summary><b>exclude()</b></summary>

hi

</details>

[ntucker]

Figured this out on my own, but I still cannot find this in the migration docs....

<details collapsed>
<summary><b>exclude()</b></summary>

hi

</details>

Needs a newline now for some reason

[slorber]

@ntucker if you can reproduce this in the MDX Playground (it's the case here) and don't understand the MDX behavior, then the best is to search MDX issues and eventually open a question on that repository directly: if MDX has a certain behavior, Docusaurus and other tools using it will usually have it too.

In this case, I have no idea why it doesn't compile 😅 The minimal case seems:

<details><summary>summary</summary>

</details>

Any idea @wooorm ?

[wooorm]

It’s unclear. The details element here wants to have blocks (flow content). The summary element wants to have inline (text content). It can’t combine like that.

Put text and tags on the same line, or not, don’t mix things:

<x><y>z</y></x>

<x>
  <y>z</y>
</x>

<x>
  <y>
    z
  </y>
</x>

[ntucker]

Is a code block metastring not parsed anymore? For

```typescript title="api/Feed.ts"
const a = 1;
```

I get props of:

{
className:"language-typescript"
metastring: "title=\"api/Feed.ts\""
}

[slorber]

The props you get are expected and afaik didn't change.

Is a code block metastring not parsed anymore? For

In Docusaurus v3.0 the metastring remains parsed inside the CodeBlock theme component, like it used to be in v2.

Your code snippet should display as before with a title despite not having a title prop:

I don't understand what lead you to think anything changed recently.

[ntucker]

This is what I get when inspecting a component created by block comments in the current version:

{
  children:
    "//the javascript code is here",
  className: 'language-tsx',
  mdxType: 'code',
  metastring: 'title="api/Feed.ts"',
  originalType: 'code',
  parentName: 'pre',
  title: 'api/Feed.ts',
}

I use this to enhance the block comments turning them into a live preview rather than just the code itself. See the live previews at https://dataclient.io/docs/api/useSuspense as example mdx source

Obviously nothing is stopping me from figuring out how to parse it myself, etc. However, I am wondering if that code was moved somewhere and if this is intentional? This will also be obvious when people inspect things using the react dev tools in the browser.

[MilenkoActiveViam]

I have problem

Error while compiling file ../docs/concepts/data_versioning.mdx (Line=157 Column=2)
Details: Could not parse expression with acorn

line
{@codeBlock: TestEpochManager.defineCustomEpochPolicy}

How to change the line?