using Croatian chars in text breaks plugin

[gour]

Hello,
thanks a lot for smiley-plugin, but have problem with emoji - as soon as text contain some of Croatian characters, plugin fails...

Here is test text:

This is a test for smiley plugin... :smile:

Let me try to add another smiley. :smile:

and it renders both, but if I change 2nd line into:

Let me try to add another šmiley. :smile:

it just shows :smile: instead. Any hint?

[Sommerregen]

@gour That is a bug. Smileys don't work atm with UTF-8 texts. However you can either proposed a PR or use https://github.com/yoshikin/grav-plugin-markdown-fontawesome as a skeleton to try something on your own. Don't have time atm.

[gour]

Smileys don't work atm with UTF-8 texts.

Isn't that strange considering that Grav's doc says:

Ensure you save your .md files as UTF8 files. This will ensure they work with language-specific special characters.

I could understand that something like :šmiley: does not work, but one would expect that plugin should not fail with regular UTF-8 content ?

However you can either proposed a PR or use https://github.com/yoshikin/grav-plugin-markdown-fontawesome as a skeleton to try something on your own.

I'll take a look to see if I can come up with something...

[gour]

@gour That is a bug.

Actually, now I see that it does not work even with pure English text which means there is some other issue here...will try to inspect those regexps...

[Sommerregen]

@gour This plugin was written before Grav even supported multi-languages. The reason why it does not work in your case (with UTF-8 characters) is that internally this plugin tries to detect utf-8 characters and convert them, because DOMDocument does this internally. But some are converted differently. That is why the preg_replace L134 fails.

In other words, this plugin only replaces those characters, which can be recognized. Otherwise content might get broken and I rather wanted to be on the safe side. The algorithm is falsy and obsolete. One can do better without these drawbacks.

The link I mentioned is a simple way of doing it. However it is syntax limited. If you don't want to be limited by the syntax and ensure that e.g.smilieys in code environments don't get replaced, you have to think about it how to do it. AFAIK there is no good algorithm I no in the wild (even Wordpress just does a simple replacement), but I have something in mind, which could work...

BUT I don't have much time actually. I program plugins for Grav and provide support in my free time and I don't get any money. When I don't have time and have to care about from where I get money to live, then I have to prioritize things.

You can help with the development though and any other. I always appreciate PRs 😃 !

[gour]

This plugin was written before Grav even supported multi-languages. The reason why it does not work in your case (with UTF-8 characters) is that internally this plugin tries to detect utf-8 characters and convert them, because DOMDocument does this internally. But some are converted differently. That is why the preg_replace L134 fails.

I see...

The link I mentioned is a simple way of doing it. However it is syntax limited. If you don't want to be limited by the syntax and ensure that e.g.smilieys in code environments don't get replaced, you have to think about it how to do it. AFAIK there is no good algorithm I no in the wild (even Wordpress just does a simple replacement), but I have something in mind, which could work...

For now, I believe I'll simply cp emojis in /images folder and reference them via markdown-extra syntax.

Next step would be to create shortcodes...

When I don't have time and have to care about from where I get money to live, then I have to prioritize things.

Sure. ;)

You can help with the development though and any other. I always appreciate PRs 😃 !

As I wrote above, the first workaround is to simply reference images and then I'll consider to provide shortcodes for the most used ones...atm I'm converting site to Grav and there are not so many emojis used in the content.