Today's update includes a fix for Issue #169.

Warn user before saving a story which contains unsupported unicode characters

There was an issue whereby any text after one of those extended unicode characters would be truncated when saving the story... admittedly quite frustrating experience for the user. It's never fun to see your edits disappear and have no feedback.

Now you'll see an error message when saving a story that contains any characters (including some emojis) that can't be saved properly.

For the technically inclined, this means any characters whose unicode value is greather than 0xFFFF (or 65535) which is too large for UCS-2.

In an ideal world the database should support all unicode characters, including CJK Unified Ideographs Extension B and above.

As of today in practice all kanji and hanzi from RTK Volume 1/3, as well as RTH and RSH (Remembering the Hanzi books) are supported, both in Study pages and referencing in stories.

However there are many rare forms of kanji as well as certain symbols and punctuation marks that can be found sometimes in Japanese text, which unfortunately can not be saved into a story (a typical example would be when you want to reference "primitives" aka "radicals", some of them have a valid UCS-2 code, but many can't be saved into the story as they are part of those CJK Extended ranges).

Changes under the hood

From today the live site is now updated with all the refactoring I have done in feature/use-vite. In a nutshell, all legacy Javascript and Css was unified under a new Vite build. These are really good changes, that give a new lifespan to the "front end" side of the code, allowing me to do things I was completely stuck before. There should be no impact on the site's performance... in theory it should even perform a little better though likely not noticeable for most.

With so many structural changes under the hood I hope I didn't introduce any new issues... if I did it will be small things I can take care of fairly quickly so please let me know if you notice any new issues from today.